7/22/2020 Argonne National Laboratory
Written by Argonne National Laboratory
"This is one of the key milestones, and there will be more such milestones from Argonne," said Sibendu Som, manager of Argonne's Computational Multi-Physics group in the Energy Systems (ES) division, of the groundbreaking simulation.
About a year and a half ago, Som and Muhsin Ameen, a research scientist at the Center for Transportation Research in ES, came up with the idea of conducting a direct numerical simulation (DNS), meant to accurately resolve all the turbulent flow scales inside an internal combustion engine. Before that simulation could be performed, however, smaller simulations were necessary to ensure that the largest-ever would go as planned, said Ameen.
As simulations can provide a more detailed view of turbulent flow, automotive manufacturers rely on them to evaluate several potential engine designs and determine the best ones, but their resources are limited.
Performing simulations at such a grand scale requires bigger and better resources, such as the Theta supercomputer at the Argonne Leadership Computing Facility (ALCF), a DOE Office of Science User Facility.
Ameen and Som collaborated with Saumil Patel, an assistant computational scientist in Argonne's Computational Science division, who helped with pre- and post-processing, as well as developing the algorithms.
In the summer of 2019, with the help of Patel, Ameen was awarded computing time on Theta through DOE's Advanced Scientific Computing Research (ASCR) Leadership Computing Challenge.
The calculations on Theta were performed with Argonne's fluid-thermal simulation code, Nek5000, which was recognized with the Gordon Bell prize for its outstanding scalability on high-performance parallel computers in 1999.
The present-day Nek5000, which scales to millions of processors, was developed primarily at Argonne. A new version, NekRS, is under development to target accelerator-based machines and has been supported in the Center for Efficient Exascale Discretizations, part of the DOE's Exascale Computing Project.
The chief architect of the Nek5000, Paul Fischer, was consulted in the early stages of the development of the present calculations. Fischer is a senior scientist in Argonne's Mathematics and Computer Science division and a professor in the Departments of Computer Science and Mechanical Science & Engineering at the University of Illinois at Urbana-Champaign.
After years of work to adapt Nek5000 for improved combustion modeling, the scientists performed the DNS of flow inside an internal combustion engine this spring.
"The current simulation effort is the first-ever direct numerical simulation of the flow and heat transfer inside an internal combustion engine for a real engine geometry and operating conditions," Ameen said.
This simulation required the solution of 2 billion degrees of freedom—which track things such as velocity, pressure and temperature—on 51,328 cores of the Theta supercomputer.
"This is one of the most detailed simulations ever of the flow in an internal combustion engine," Ameen said.
The DNS dataset generated from the current work will be useful to automotive manufacturers in several ways. The detailed information about the velocity, pressure and temperature distribution within the engine will illuminate in-cylinder processes that are inaccessible to experimentation or low-fidelity simulations. Additionally, the dataset will serve as a simulation benchmark that engine modelers can use to assess and improve the accuracy of the engineering submodels.
The research might also benefit heavy-duty engine companies.