AI is hot and NCSA research is making it faster and more accessible

The age of artificial intelligence is upon us, a fact that means huge opportunities to glean knowledge from massive datasets in businesses, science, engineering and medicine. But it also means new challenges to make AI and machine learning (ML) – a branch of AI that uses data and algorithms to help machines learn in a way that imitates the human brain – live up to their promise. 

Seid Koric is the technical associate director in NCSA’s Research Consulting directorate and research associate professor in the Department of Mechanical Science and Engineering (MechSE) at the University of Illinois. His work involves leading a team of researchers at NCSA and The Grainger College of Engineering that stands at the forefront of AI and ML advances. With collaborators from Khalifa University of Science and Technology in Abu Dhabi, UAE, they’ve used high-performance computing to generate data and train novel AI models. The high-throughput and parallel capabilities of HPC  decrease the time and expense needed for data generation and ML training by orders of magnitude. Their novel research, which they explain in a paper published in May in the prestigious journal  Engineering with Computers, applies a recently introduced deep learning operator network, or DeepONet, in nonlinear and irreversible plastic behavior exhibited by many engineering materials under large deformation. Due to the outstanding scientific value of the article, the publisher (Springer Nature) has given free open access to the article for two months. 

Typically, training an AI model involves using a method called an artificial neural network, an ML-based deep-learning process that uses interconnected nodes, or neurons, in a layered structure that resembles the human brain. By feeding neural networks different values of training data, the researchers help them learn by example and train them to recognize patterns in data, find connections, predict outcomes and predict solutions in a wide range of fields, including image recognition, natural language processing, fluid dynamics, material science, robotics, health and life sciences and in the case presented in the paper, computational (structural) mechanics. 

Once a model is trained, it can ensure accurate predictions and instant analysis, based on the new inputs never before seen by the network. However, in most existing approaches, neural networks in computational mechanics can only provide solutions for a fixed set of input parameters, such as material properties, source terms, loads, boundaries and initial conditions. If any changes to those parameters are necessary, the network needs to be retrained, which can be time-consuming, especially when dealing with temporal inputs, such as complex loading histories when predicting nonlinear stress and deformation evolution in structures.   

Faster solutions and the chance to solve larger problems 

DeepONet revolutionizes this training process. Due to its dual network nature, DeepONet can learn to map linear and nonlinear parametric input functions to the output solution functions. Once DeepONet is properly trained, it can speed up the process of accurately finding solutions in the entire domain for a new parametric function by as much as 10,000 times compared to classical numerical methods, such as nonlinear finite element analysis, said Diab Abueidda, Ph.D., an NCSA research scientist who works with Koric. 

“If you use classical numerical methods and you need to design and optimize your model, you need to run many forward analyses to get to the optimal design,” said Abueidda. “If you need a million of these simulations, that can take months, which may be impractical even on the newest HPC machines.” 

DeepONet cuts that single simulation time to fractions of a second.  

“It means you can look at larger problems and previously unsolvable problems,” said Koric. “Once you train the DeepONet model, you can move their trainable parameters to a laptop, introduce new sets of inputs and get instant results.” 

For the past five years, Koric and his team have been working to improve the training of deep neural network-based ML models using NCSA supercomputers, including Delta, the center’s newest system that combines advanced GPU and CPU computing architectures from NVIDIA and AMD. Their success in devising and applying novel ML methods in engineering and science has been noticed in the scientific community; Koric estimates the team has published 17 journal papers and been cited by other researchers more than 1,000 times. 

In their latest published work, Koric and colleagues from Kalifa University used the DeepONet process to solve problems related to predicting highly nonlinear stress solutions during non-reversible plastic deformation in the entire domain based on variable loads and material properties Their results showed that when the DeepONet framework is adequately trained on high-end computers, it can predict stress solutions accurately and many orders of magnitude faster than traditional numerical solvers for any combination of input parameters and without any additional training. 

“In the field of non-linear structural mechanics, DeepONet was a revelation,”  said Asha Viswanath, Ph.D., a researcher at Kalifa University who is a co-author of the paper. “It was stimulating to discover the potential of these operator networks and exciting to see how they can be applied in this domain. Developing this paper under Dr. Khan’s guidance (Kamran Khan, associate professor of aerospace engineering at KU) and in close collaboration with Prof. Seid Koric and Dr. Diab Abueidda of NCSA was a truly rewarding experience. I am proud to be part of this wonderful collaboration between Khalifa University and NCSA.” 

New funding for a new paradigm 

The University of Illinois has also recognized NCSA’s support for cutting-edge computational research. The combined $50-million investment in the Illinois Computes program from the University of Illinois at Urbana-Champaign and the U of I System is designed to help Illinois researchers use advanced computing and new techniques such as ML as part of their research. Seid said his groups’ breakthroughs have also led to new funding from the National Science Foundation and from NCSA and Illinois industry partners. 

“Companies have always come to NCSA to tackle their greatest data-driven challenges,” said Brendan McGinty, industry program director at NCSA. “Supercomputers have allowed them to realize solutions faster than anywhere else. Now, adding AI provides even greater speed to solutions. But perhaps more importantly, advancements in machine, deep learning and AI acceleration provide greater depth of solutions and foster our own growth by adding AI to our domain and supercomputing expertise.” 

According to Koric, as advanced artificial neural networks such as DeepONet prove themselves    as a new paradigm in scientific and engineering computational methods, Illinois, national and international researchers from a wide range of disciplines have approached his team and expressed interest in collaborating. Additionally, NCSA is home to the Center for Artificial Intelligence Innovation (CAII), which spearheads interdisciplinary research and applications of AI and where Koric and Abueidda are affiliates. Koric said these state and national investments are expected to keep AI and ML at the forefront of new and promising methods of analyzing massive amounts of data, whether collected from observations and experiments or generated by classical numerical methods, such as in the DeepONet research. 

“This work is very inspiring as it demonstrates new and emerging applications of deep neural networks for solving computationally intensive problems at a much higher speed compared to classical modeling simulation approaches,” said Volodymyr Kindratenko, PhD, director of NCSA’s Center for AI Innovation. “Dr. Koric and his team once again obtained impressive results by taking advantage of the latest developments in the field of neural nets. This cutting-edge work paves the way to enable rapid modeling and analysis of new materials by utilizing advanced computational capabilities and expertise provided by NCSA and the Center for AI Innovation.” 

Laura Herriott, NCSA’s associate director of research consulting, said the breakthrough research will enable more effective use of CPU and GPU computer systems by NCSA partners, especially those who are interested in ML methods and need to solve nonlinear problems, such as computational fluid dynamics (CFD) and material science problems. 

“We are so fortunate at NCSA to employ staff who have the deep technical knowledge in specific domains and work modalities that has inspired the type of AI confluence work that is being done with faculty in bioengineering, aerospace engineering, mechanical engineering and more,” she said. “Our staff works tirelessly to enable effective use of CPUs and in this case, the NVIDIA A100 GPUs that make Delta special.” 

Currently, Koric’s team is working with MechSE professor Iwona Jasiuk and her students through the Illinois Computes program, as well as with their collaborators at KU, on separate projects that extend their previous work with deep learning operator neural networks. Koric expects more requests from research and industry partners looking to use ML models to solve problems in manufacturing design, digital twins (a virtual model of a physical object), material science, life sciences and more. 

He also was quick to cite George Em Karniadakis, a renowned professor of applied mathematics and his research team at Brown University for inventing the DeepONet method. 

“We are building on his ideas,” said Koric. “That is how science works.”