Advertisement

Cray's Jaguar supercomputer upgraded with NVIDIA Tesla GPUs, renamed Titan

Alexis Santos
Updated ·9 min read
0

Cray's Jaguar (or XK7) supercomputer at Oak Ridge National Laboratory has been loaded up with the first shipping NVIDIA Tesla K20 GPUs and renamed Titan. Loaded with 18,688 of the Kepler-based K20s, Titan's peak performance is more than 20 petaflops. Sure, the machine has an equal number of 16-core AMD Opteron 6274 processors as it does GPUs, but the Tesla hardware packs 90 percent of the entire processing punch. Titan is roughly ten times faster and five times more energy efficient than it was before the name change, yet it fits into the same 200 cabinets as its predecessor. Now that it's complete, the rig will analyze data and create simulations for scientific projects ranging from topics including climate change to nuclear energy. The hardware behind Titan isn't meant to power your gaming sessions, but the NVIDIA says lessons learned from supercomputer GPU development trickle back down to consumer-grade cards. For the full lowdown on the beefed-up supercomputer, hit the jump for a pair of press releases.

Show full PR text

NVIDIA Powers Titan, World's Fastest Supercomputer For Open Scientific Research

Oak Ridge National Lab's 20+ Petaflops System – Powered by 18,000-Plus

NVIDIA Tesla K20 GPUs – Paves Path to Efficient, Affordable Exascale Computing
SANTA CLARA, Calif.-Oct. 29, 2012-Titan, the world's fastest open-science supercomputer,(1) was completed this month at Oak Ridge National Laboratory in Tennessee, opening new windows of opportunity into the exploration of some of the world's toughest scientific challenges.

Titan's peak performance is more than 20 petaflops – or 20 million billion floating-point operations per second – about 90 percent of which comes from 18,688 NVIDIA® Tesla® K20 GPU accelerators. These are based on the NVIDIA KeplerTM architecture, the fastest, most efficient, highest-performance computing architecture ever built.

Researchers use ever faster supercomputers to accelerate the pace of discovery and innovation across a range of scientific fields of inquiry – from developing more efficient engines and higher capacity, lighter weight batteries, to studying climate change and finding cures for disease. Titan is a milestone on the path to exascale computing, which targets building a 1,000 petaflops supercomputer.

Titan is operated by Oak Ridge National Laboratory, part of the U.S. Department of Energy's network of research labs, as an open-science system. This means it is available to researchers from academia, government laboratories, and a broad range of industries, who will use Titan to model physical and biological phenomena and seek breakthroughs faster than possible by experimentation alone.

Supported by the energy efficiency and cost-effectiveness of the Tesla K20 GPU, Titan is more than 10 times faster and five times more energy efficient than its predecessor, the 2.3-petaflops(2) Jaguar system, while occupying the same floor space. Had Oak Ridge upgraded Jaguar by simply expanding its CPU-based architecture, the system would be more than four times its current size and consume more than 30 megawatts of power.(3)

"Basing Titan on Tesla GPUs allows Oak Ridge to run phenomenally complex applications at scale, and validates the use of accelerated computing to address our most pressing scientific problems," said Steve Scott, chief technology officer of the GPU Accelerated Computing business at NVIDIA. "You simply can't get these levels of performance, power- and cost- efficiency with conventional CPU-based architectures. Accelerated computing is the best and most realistic approach to enable exascale performance levels within the next decade."

Titan development began three years ago with Oak Ridge's decision to upgrade Jaguar, the previous open science system leader and a former world No. 1 most powerful supercomputer. The upgrade includes the Tesla K20 GPU accelerators, a replacement of the compute modules to convert the system's 200 cabinets to a Cray XK7 supercomputer, and 710 terabytes of memory.

"Science and technology have always been our primary goal, and Titan is a groundbreaking tool that will allow researchers worldwide to leverage GPU-accelerated computing to make unparalleled breakthroughs," said Jeff Nichols, associate laboratory director for computing and computational sciences at Oak Ridge National Laboratory. "The new Tesla GPU accelerators offer the performance and energy efficiency that enable Titan to scale to unprecedented performance levels without consuming the energy equivalent of a small city."

(1) Based on June 2012 Top500 list

(2) November 2011 Top500 List (http://top500.org/list/2011/11/100) – Jaguar = 2.3 petaflops peak
performance @ 7 megawatts; Titan's peak performance will be in excess of 20 petaflops, consuming
approximately 9 megawatts of electrical power.

(3) Assuming the same peak performance with a system composed of dual-CPU nodes, using the same
CPU as in Titan.

Show full PR text

ORNL DEBUTS TITAN SUPERCOMPUTER

Supercomputer combines gaming and traditional computing technologies to provide unprecedented power for research

OAK RIDGE, Tenn. – The U.S. Department of Energy's (DOE) Oak Ridge National Laboratory launched a new era of scientific supercomputing today with Titan, a system capable of churning through more than 20,000 trillion calculations each second-or 20 petaflops-by employing a family of processors called graphic processing units first created for computer gaming. Titan will be 10 times more powerful than ORNL's last world-leading system, Jaguar, while overcoming power and space limitations inherent in the previous generation of high-performance computers.

Titan, which is supported by the Department of Energy, will provide unprecedented computing power for research in energy, climate change, efficient engines, materials and other disciplines and pave the way for a wide range of achievements in science and technology.

The Cray XK7 system contains 18,688 nodes, with each holding a 16-core AMD Opteron 6274 processor and an NVIDIA Tesla K20 graphics processing unit (GPU) accelerator. Titan also has more than 700 terabytes of memory. The combination of central processing units, the traditional foundation of high- performance computers, and more recent GPUs will allow Titan to occupy the same space as its Jaguar predecessor while using only marginally more electricity.

"One challenge in supercomputers today is power consumption," said Jeff Nichols, associate laboratory director for computing and computational sciences. "Combining GPUs and CPUs in a single system requires less power than CPUs alone and is a responsible move toward lowering our carbon footprint. Titan will provide unprecedented computing power for research in energy, climate change, materials and other disciplines to enable scientific leadership."

Because they handle hundreds of calculations simultaneously, GPUs can go through many more than CPUs in a given time. By relying on its 299,008 CPU cores to guide simulations and allowing its new NVIDIA GPUs to do the heavy lifting, Titan will enable researchers to run scientific calculations with greater speed and accuracy.

"Titan will allow scientists to simulate physical systems more realistically and in far greater detail," said James Hack, director of ORNL's National Center for Computational Sciences. "The improvements in simulation fidelity will accelerate progress in a wide range of research areas such as alternative energy and energy efficiency, the identification and development of novel and useful materials and the opportunity for more advanced climate projections"

Titan will be open to select projects while ORNL and Cray work through the process for final system acceptance. The lion's share of access to Titan in the coming year will come from the Department of Energy's Innovative and Novel Computational Impact on Theory and Experiment program, better known as INCITE.

Researchers have been preparing for Titan and its hybrid architecture for the past two years, with many ready to make the most of the system on day one. Among the flagship scientific applications on Titan:

Materials Science
The magnetic properties of materials hold the key to major advances in technology. The application WL- LSMS provides a nanoscale analysis of important materials such as steels, iron-nickel alloys and advanced permanent magnets that will help drive future electric motors and generators. Titan will allow researchers to improve the calculations of a material's magnetic states as they vary by temperature.

"The order-of-magnitude increase in computational power available with Titan will allow us to investigate even more realistic models with better accuracy" noted ORNL researcher and WL-LSMS developer Markus Eisenbach.
Combustion

The S3D application models the underlying turbulent combustion of fuels in an internal combustion engine. This line of research is critical to the American energy economy, given that three-quarters of the fossil fuel used in the United States goes to powering cars and trucks, which produce one-quarter of the country's greenhouse gases.

Titan will allow researchers to model large-molecule hydrocarbon fuels such as the gasoline surrogate isooctane; commercially important oxygenated alcohols such as ethanol and butanol; and biofuel surrogates that blend methyl butanoate, methyl decanoate and n-heptane.

"In particular, these simulations will enable us to understand the complexities associated with strong coupling between fuel chemistry and turbulence at low preignition temperatures," noted team member Jacqueline Chen of Sandia National Laboratories. "These complexities pose challenges, but also opportunities, as the strong sensitivities to both the fuel chemistry and to the fluid flows provide multiple control options which may lead to the design of a high-efficiency, low-emission, optimally combined engine-fuel system"

Nuclear Energy
Nuclear researchers use the Denovo application to, among other things, model the behavior of neutrons in a nuclear power reactor. America's aging nuclear power plants provide about a fifth of the country's electricity, and Denovo will help them extend their operating lives while ensuring safety. Titan will allow Denovo to simulate a fuel rod through one round of use in a reactor core in 13 hours; this job took
60 hours on the Jaguar system.

Climate Change
The Community Atmosphere Model–Spectral Element simulates long-term global climate. Improved atmospheric modeling under Titan will help researchers better understand future air quality as well as the effect of particles suspended in the air.

Using a grid of 14-kilometer cells, the new system will be able to simulate from one to five years per day of computing time, up from the three months or so that Jaguar was able to churn through in a day.

"As scientists are asked to answer not only whether the climate is changing but where and how, the workload for global climate models must grow dramatically," noted CAM-SE team member Kate Evans of ORNL. "Titan will help us address the complexity that will be required in such models."

ORNL is managed by UT-Battelle for the Department of Energy. The Department of Energy is the single largest supporter of basic research in the physical sciences in the United States, and is working to address some of the most pressing challenges of our time. For more information, please visit http://science.energy.gov.