September 30, 2019
A Supercomputer’s Final Mission Will Help Us Understand the Universe
The supercomputer Mira is retiring to make room for the fastest computer in the country — but first, it’s finishing its biggest project yet
By Sarah Kessler
4 min read
What will soon be America's fastest supercomputer is expected to log online in 2021. Named Aurora, it will be capable of a quintillion (a billion billion) calculations per second, making it about 50 times swifter than today's most powerful computers and fast enough to start mapping the human brain's one million billion or so connections. Scientists also hope the supercomputer, which will be housed at the Argonne National Laboratory in Lemont, Illinois, will supercharge A.I. to aid in the development of new cancer treatments, pinpoint places to look for dark matter, and enable the discovery of novel materials that could improve everything from batteries to medicines.
Aurora, in short, is an extremely exciting and highly anticipated development.
But this story is not about Aurora, the hotshot new machine that all of the computational scientists are buzzing about. It's about Mira, the seven-year-old and still very capable supercomputer that is shutting down this year in order to make room for the newbie.
Mira is a little more than two months from being phased out, but it isn't easing into retirement. Instead, Mira is quietly hard at work on the biggest task of its career: simulating how a large portion of our universe evolved over billions of years — a project that will help answer fundamental questions about our existence.
Mira is having a last hurrah.
"We don't have a plan for cake or champagne — and technically we can't have champagne on the premises," says Katherine Riley, the director of science at Argonne Leadership Computing Facility, who has worked on Mira and its predecessors of the same architecture for the last 15 years. "This may sound like a really nerdy version of cake, but it's really our cake." Scientists at the computing facility have dubbed the project "The Last Journey."
Mira shares the Argonne computing science building's 25,000 square feet of raised computer floor space with several other giant machines. From the outside, it looks a lot like an oversized shipping container. Behind its outer wrapping, it resembles a set of library stacks, albeit ones filled with 48 racks, 786,432 processors, and 768 terabytes of memory instead of books.
Only the most powerful supercomputers in the world would be capable of running a simulation and analysis as complex as The Last Journey at the necessary high resolution. And few such simulations exist because time on machines like Mira is usually in very high demand, and simulating evolution in a large portion of the universe requires an awful lot of it. The Last Journey will require 800 million core-hours (about six weeks of nonstop work for Mira), which is the largest amount of time ever allocated to a project by the Argonne Leadership Computing Facility.
"We don't have a plan for cake or champagne — and technically we can't have champagne on the premises."
Because the computing facility is part of a national laboratory run by the Department of Energy, any researcher can apply to a peer-reviewed allocation program for time on its incredibly expensive tools—Aurora will cost $500 million—with the only requirement that they publish the research when it's finished. But winning time is a competitive process, and awards are usually given for several projects to run on Mira at the same time, each using only a portion of the machine's processing power. The Last Journey, in contrast, has all of Mira's attention.
This unusually large allocation is possible partly because Mira has an established retirement date, and only projects that can definitely be completed in its remaining time can be started.
Here's what the Last Journey involves: On August 20, Mira began with a model of how the universe looked 50 million years after the Big Bang, according to the latest cosmological theories. Mira then started simulating the billions of years of evolution that passed between that point and the present in order to create a high-resolution model of what a large portion of the universe would look like today under that theory. This model will be compared to data from telescopes about what a similar portion of the universe actually does look like. By comparing the simulation results to what can actually be observed, scientists will be able to evaluate how accurate the theory likely is.
Simulating the lifespan of the universe could help answer some of science's most intriguing questions: Dark matter and dark energy make up 95% of the universe, but where do they come from? Why does the universe appear to be expanding at an accelerating pace, and what does it mean for our future?
"We are really hoping to shed some light on something we found and don't understand that's very fundamental to the makeup of our universe," says Katrin Heitmann, a physicist and computational scientist at Argonne National Laboratory who is the principal investigator for The Last Journey.
But is an aging supercomputer — elderly, really, among most of its peers — a good choice for such a big, complicated task? Oh, you bet it is.
Mira's many parts are still connected by what is still one of the most efficient networks in the world. And the supercomputer is unusually reliable. "Nothing fails," says Riley, " it just keeps going." That's especially helpful for projects that, like The Last Journey, will use every node in the infrastructure, because if one were to fail, the whole simulation would crash. Mira also has enough memory to handle massive amount of data the simulation will create.
"Nothing fails. It just keeps going."
Simulating how a large portion of the universe evolved will be the final major achievement in Mira's distinguished career. The supercomputer's capabilities have pushed the boundaries of both the very small and very large. Its been used to help figure out how to stitch together nanofabrics (layers of single atoms that enhance or separate other materials), a feat that scientists weren't even sure they could simulate at the time; to improve safety in materials used in extreme environments; and to reduce the carbon impact of concrete. On the very large scale, it has contributed to the understanding of our global climate and simulated blowing up quite a few stars, which is, like the current simulation, useful for understanding the raw materials of our universe.
After Mira finishes The Last Journey, Argonne plans to shut it off on December 31. Mira's parts will most likely be recycled.
"I'm heartbroken," says Riley of Mira's retirement. Part of the feeling is personal — Mira and its predecessors have been a big part of her career — but part of it is also technical. While Argonne needs to save space and energy to make room for Aurora, Mira remains impressive. "Right near its end, it's running these huge simulations," Riley says. "And it's really rocking them — it's not even blinking."