SEARCH · Engineering Papers

Results for “scientific deep learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Hybrid learning techniques for scientific data reduction with performance guarantees

The research initiatives supported by the U.S. Department of Energy (DOE) Grant DE-SC0022265 are fundamentally aimed at pioneering advanced machine learning (ML) techniques for scientific data compression within high-performance computing (HPC) environments. This comprehensive body of work addresses the critical challenge posed by the exponential growth of data generated by scientific simulations in domains such as fusion energy, climate modeling, and computational fluid dynamics (CFD). A core objective is to develop compression algorithms that achieve substantial data reduction—often by orders of magnitude—while rigorously ensuring the fidelity of both the primary data (PD) and scientifically crucial derived quantities of interest (QoI). The methodologies deployed under this grant integrate sophisticated deep learning architectures, prominently featuring autoencoders, advanced generative models like conditional diffusion, and hybrid learning techniques. Key innovations include the development of Guaranteed Autoencoders (GAE) and the Guaranteed Conditional Diffusion with Tensor Correction (GCDTC) framework, which provide explicit, instance-level error bounds on reconstructed data. Furthermore, specialized strategies such as nonlinear constraint satisfaction are employed to preserve the integrity of QoI, a vital requirement for the trustworthiness of downstream scientific analyses. This research also focuses on the design and implementation of scalable, GPU-accelerated software pipelines that seamlessly integrate into existing HPC workflows, ensuring both computational efficiency and practical applicability. The CAESAR framework, for example, unifies foundation and generative models to create an adaptive and efficient compression solution for spatio-temporal scientific data. Collectively, these efforts represent a significant advancement in mitigating the scientific data deluge, enabling more effective data management, accelerated scientific discovery, and optimized utilization of HPC resources.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Final report- UFL - RAPIDS2: A SciDAC Institute for Computer Science, Data, and Artificial Intelligence

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

A non-cooperative meta-modeling game for automated third-party calibrating, validating and falsifying constitutive laws with parallelized adversarial attacks

The evaluation of constitutive models, especially for high-risk and high-regret engineering applications, requires efficient and rigorous third-party calibration, validation and falsification. While there are numerous efforts to develop paradigms and standard procedures to validate models, difficulties may arise due to the sequential, manual, and often biased nature of the commonly adopted calibration and validation processes, thus slowing down data collections, hampering the progress towards discovering new physics, increasing expenses and possibly leading to misinterpretations of the credibility and application ranges of proposed models. This work attempts to introduce concepts from game theory and machine learning techniques to overcome many of these existing difficulties. Here, we introduce an automated meta-modeling game where two competing AI agents systematically generate experimental data to calibrate a given constitutive model and to explore its weakness such that the experiment design and model robustness can be improved through competitions. The two agents automatically search for the Nash equilibrium of the meta-modeling game in an adversarial reinforcement learning framework without human intervention. In particular, a protagonist agent seeks to find the more effective ways to generate data for model calibrations, while an adversary agent tries to find the most devastating test scenarios that expose the weaknesses of the constitutive model calibrated by the protagonist. By capturing all possible design options of the laboratory experiments into a single decision tree, we recast the design of experiments as a game of combinatorial moves that can be resolved through deep reinforcement learning by the two competing players. Our adversarial framework emulates idealized scientific collaborations and competitions among researchers to achieve a better understanding of the application range of the learned material laws and prevent misinterpretations caused by conventional AI-based third-party validation. Numerical examples are given to demonstrate the wide applicability of the proposed meta-modeling game with adversarial attacks on both human-crafted constitutive models and machine learning models.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Midland College Requirements Analysis Report

EPOC uses the Deep Dive process to discuss and analyze current and planned science, research, or education activities and the anticipated data output of a particular use case, site, or project to help inform the strategic planning of a campus or regional networking environment. This includes understanding future needs related to network operations, network capacity upgrades, and other technological service investments. A Deep Dive comprehensively surveys major research stakeholders’ plans and processes in order to investigate data management requirements over the next 5–10 years. In April of 2022, staff members from the Engagement and Performance Operations Center (EPOC) met with researchers and staff from LEARN and Midland College for the purpose of a Deep Dive into scientific and research drivers. The goal of this activity was to help characterize the requirements for a number of campus use cases, and to enable cyberinfrastructure support staff to better understand the needs of the researchers within the community.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Advances in Machine and Deep Learning for Modeling and Real-time Detection of Multi-Messenger Sources

We live in momentous times. The science community is empowered with an arsenal of cosmic messengers to study the universe in unprecedented detail. Gravitational waves, electromagnetic waves, neutrinos, and cosmic rays cover a wide range of wavelengths and timescales. Combining and processing these datasets that vary in volume, speed, and dimensionality requires new modes of instrument coordination, funding, and international collaboration with a specialized human and technological infrastructure. In tandem with the advent of large-scale scientific facilities, the last decade has experienced an unprecedented transformation in computing and signal-processing algorithms. The combination of graphics processing units, deep learning, and the availability of open source, high-quality datasets has powered the rise of artificial intelligence. This digital revolution now powers a multibillion dollar industry, with far-reaching implications in technology and society. In this chapter, we describe pioneering efforts to adapt artificial intelligence algorithms to address computational grand challenges in multi-messenger astrophysics. We review the rapid evolution of these disruptive algorithms, from the first class of algorithms introduced in early 2017 to the sophisticated algorithms that now incorporate domain expertise in their architectural design and optimization schemes. We discuss the importance of scientific visualization and extreme-scale computing in reducing time-to-insight and obtaining new knowledge from the interplay between models and data.

Artificial Intelligence↗

DOE OSTI2021

A data-driven global soil heterotrophic respiration dataset and the drivers of its inter-annual variability

Soil heterotrophic respiration (SHR), one of the primary carbon fluxes from terrestrial ecosystems to the atmosphere, is important for carbon-climate feedbacks because of its sensitivity to available litter and soil carbon, climatic conditions, and nutrient availability. However, until recently limited SHR data were available, and most published global SHR estimates have either a short time span, coarse spatial resolution, or reply on overly-simple model formulations. To better understand and quantify the global distribution of SHR and its sensitivity to climate variability, we produced a new global SHR dataset using Random Forest algorithms, up-scaling 455 point data from the Global Soil Respiration Database (SRDB 4.0) with gridded fields of climatic, edaphic and productivity as explanatory variables. We estimated a global total SHR of 46.8 Pg C yr-1 over 1985-2013 (95% confidence interval: 38.6-56.3 Pg C yr-1), with a significant increasing trend of 0.03 Pg C yr-2 during this period. We found that the choice of soil moisture datasets contributes more to the difference among these data-driven SHR members rather than that of productivity, temperature and precipitation data sources. We also analyzed the influence of climatic variables on the inter-annual variability (IAV) of our SHR product. Water availability was the dominant driver of IAV at global scales, although the inferred sensitivity depends on the choice of the soil moisture gridded dataset. At the ecosystem scale, temperature strongly controls the IAV of SHR in tropical forests, while water availability dominates in extra-tropical forest and semi-arid regions. Our machine-learning gridded SHR dataset and outputs from process-based land surface models (TRENDYv6) show agreement for a strong association between water variability and SHR IAV at the global scale, but the two approaches lead to different temporal trend globally and different controlling variables for IAV at the ecosystem scale. Our study provides evidence for the pervasive and important role of water availability in driving SHR, indicating both a direct effect limiting decomposition rates and an indirect effect through the amount of fresh organic matter made available to SHR from productivity. In consideration of potential limitations and uncertainties remaining in our data-driven SHR datasets, we call for a more scientifically designed observation network for SHR, more observation data compilation, and increased use of deep learning methods making maximum use of observation data in hand. This will benefit process-based models, and improve our understanding of SHR response to future anomalous environmental conditions.

Yao, Yitong↗

DOE OSTI2021

Physics-informed machine learning

Despite great progress in simulating multiphysics problems using the numerical discretization of partial differential equations (PDEs), one still cannot seamlessly incorporate noisy data into existing algorithms, mesh generation remains complex, and high-dimensional problems governed by parameterized PDEs cannot be tackled. Moreover, solving inverse problems with hidden physics is often prohibitively expensive and requires different formulations and elaborate computer codes. Machine learning has emerged as a promising alternative, but training deep neural networks requires big data, not always available for scientific problems. Instead, such networks can be trained from additional information obtained by enforcing the physical laws (for example, at random points in the continuous space-time domain). Such physics-informed learning integrates (noisy) data and mathematical models, and implements them through neural networks or other kernel-based regression networks. Moreover, it may be possible to design specialized network architectures that automatically satisfy some of the physical invariants for better accuracy, faster training and improved generalization. Furthermore, we review some of the prevailing trends in embedding physics into machine learning, present some of the current capabilities and limitations and discuss diverse applications of physics-informed learning both for forward and inverse problems, including discovering hidden physics and tackling high-dimensional problems.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Integrating HPC, AI, and Workflows for Scientific Data Analysis: Report from Dagstuhl Seminar 23352

The Dagstuhl Seminar 23352, titled “Integrating HPC, AI, and Workflows for Scientific Data Analysis,” held from August 27 to September 1, 2023, was a significant event focusing on the synergy between High-Performance Computing (HPC), Artificial Intelligence (AI), and scientific workflow technologies. The seminar recognized that modern Big Data analysis in science rests on three pillars: workflow technologies for reproducibility and steering, AI and Machine Learning (ML) for versatile analysis, and HPC for handling large data sets. These elements, while crucial, have traditionally been researched separately, leading to gaps in their integration. The seminar aimed to bridge these gaps, acknowledging the challenges and opportunities at the intersection of these technologies. The event highlighted the complex interplay between HPC, workflows, and ML, noting how ML has increasingly been integrated into scientific workflows, thereby enhancing resource demands and bringing new requirements to HPC architectures, like support for GPUs and iterative computations. The seminar also addressed the challenges in adapting HPC for large-scale ML tasks, including in areas like deep learning, and the need for workflow systems to evolve to leverage ML in data analysis fully. Moreover, the seminar explored how ML could optimize scientific workflow systems and HPC operations, such as through improved scheduling and fault tolerance. A key focus was on identifying prestigious use cases of ML in HPC and understanding their unique, unmet requirements. The stochastic nature of ML and its impact on the reproducibility of data analysis on HPC systems was also a topic of discussion.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

McLennan Community College Requirements Analysis Report

EPOC uses the Deep Dive process to discuss and analyze current and planned science, research, or education activities and the anticipated data output of a particular use case, site, or project to help inform the strategic planning of a campus or regional networking environment. This includes understanding future needs related to network operations, network capacity upgrades, and other technological service investments. A Deep Dive comprehensively surveys major research stakeholders’ plans and processes in order to investigate data management requirements over the next 5–10 years. In March of 2022, staff members from the Engagement and Performance Operations Center (EPOC) met with researchers and staff from LEARN and McLennan Community College (MCC) for the purpose of a Deep Dive into scientific and research drivers. The goal of this activity was to help characterize the requirements for a number of campus use cases, and to enable cyberinfrastructure support staff to better understand the needs of the researchers within the community.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2022

South Plains College Requirements Analysis Report

EPOC uses the Deep Dive process to discuss and analyze current and planned science, research, or education activities and the anticipated data output of a particular use case, site, or project to help inform the strategic planning of a campus or regional networking environment. This includes understanding future needs related to network operations, network capacity upgrades, and other technological service investments. A Deep Dive comprehensively surveys major research stakeholders’ plans and processes in order to investigate data management requirements over the next 5–10 years. Between October 2021 and January 2022, staff members from the Engagement and Performance Operations Center (EPOC) met with researchers and staff from LEARN and South Plains College (SPC) for the purpose of a Deep Dive into scientific and research drivers. The goal of this activity was to help characterize the requirements for a number of campus use cases, and to enable cyberinfrastructure support staff to better understand the needs of the researchers within the community.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2023

St. Mary’s University Requirements Analysis Report

EPOC uses the Deep Dive process to discuss and analyze current and planned science, research, or education activities and the anticipated data output of a particular use case, site, or project to help inform the strategic planning of a campus or regional networking environment. This includes understanding future needs related to network operations, network capacity upgrades, and other technological service investments. A Deep Dive comprehensively surveys major research stakeholders’ plans and processes in order to investigate data management requirements over the next 5–10 years. Between September and December 2022, staff members from the Engagement and Performance Operations Center (EPOC) met with researchers and staff from LEARN and St. Mary’s University for the purpose of a Deep Dive into scientific and research drivers. The goal of this activity was to help characterize the requirements for a number of campus use cases, and to enable cyberinfrastructure support staff to better understand the needs of the researchers within the community.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2022

Green AI: Insights Into Deep Learning's Looming Energy Efficiency Crisis

As demands grow to integrate artificial intelligence into every aspect of industry, commerce, and life, deep learning's exploding energy cost has become a looming crisis, making AI systems a salient energy-efficiency challenge. One might expect that doubling a neural network's size would halve its error rate, or at least allow it to achieve greater performance given the same amount of time and energy. I will present clear and substantial scientific evidence which indicates that not only is this intuition wildly wrong, but that neural networks scale so poorly that to increase deep learning performance by only a small fraction can easily require an order of magnitude or more increase in computational resources and energy. Further, the marginal trade-off price of to increase model performance rapidly explodes as performance targets are increased. To address this challenge, I will provide a toolkit of techniques that can be applied today to mitigate the inefficiency of modern deep learning. And, I will conclude by illuminating a practical path forward towards efficient, Green AI.

artificial intelligence↗

DOE OSTI2021

Deep Learning the Electromagnetic Properties of Metamaterials—A Comprehensive Review

Deep neural networks (DNNs) are empirically derived systems that have transformed traditional research methods, and are driving scientific discovery. Here, artificial electromagnetic materials (AEMs)—including electromagnetic metamaterials, photonic crystals, and plasmonics—are research fields where DNN results valorize the data driven approach; especially in cases where conventional methods have failed. In view of the great potential of deep learning for the future of artificial electromagnetic materials research, the status of the field with a focus on recent advances, key limitations, and future directions is reviewed. Strategies, guidance, evaluation, and limits of using deep networks for both forward and inverse AEM problems are presented.

14 SOLAR ENERGY↗

DOE OSTI2022

Multi-fidelity information fusion with concatenated neural networks

Recently, computational modeling has shifted towards the use of statistical inference, deep learning, and other data-driven modeling frameworks. Although this shift in modeling holds promise in many applications like design optimization and real-time control by lowering the computational burden, training deep learning models needs a huge amount of data. This big data is not always available for scientific problems and leads to poorly generalizable data-driven models. This gap can be furnished by leveraging information from physics-based models. Exploiting prior knowledge about the problem at hand, this study puts forth a physics-guided machine learning (PGML) approach to build more tailored, effective, and efficient surrogate models. For our analysis, without losing its generalizability and modularity, we focus on the development of predictive models for laminar and turbulent boundary layer flows. In particular, we combine the self-similarity solution and power-law velocity profile (low-fidelity models) with the noisy data obtained either from experiments or computational fluid dynamics simulations (high-fidelity models) through a concatenated neural network. We illustrate how the knowledge from these simplified models results in reducing uncertainties associated with deep learning models applied to boundary layer flow prediction problems. The proposed multi-fidelity information fusion framework produces physically consistent models that attempt to achieve better generalization than data-driven models obtained purely based on data. While we demonstrate our framework for a problem relevant to fluid mechanics, its workflow and principles can be adopted for many scientific problems where empirical, analytical, or simplified models are prevalent. In line with grand demands in novel PGML principles, this work builds a bridge between extensive physics-based theories and data-driven modeling paradigms and paves the way for using hybrid physics and machine learning modeling approaches for next-generation digital twin technologies.

42 ENGINEERING↗

DOE OSTI2026

A Benchmark Suite for Evaluating Scientific AI Workloads on GPUs

AI applications have been steadily increasing in the allocation portfolio among leadership computing facilities. These applications depend on deep learning frameworks with hardware acceleration and underlying software systems. With the rapid development of applications, software stacks, and hardware devices, it is essential to evaluate the performance of core operations in AI workloads for direction of optimizations and procurement of next-generation high-performance computing (HPC) infrastructures. Currently, most benchmarks lack scientific AI workloads. So, we present DeepKernelBench and the experimental results of evaluating the benchmark suite for early observations and performance comparisons on datacenter GPUs using representative workloads for scientific AI, including Attentions, General matrix multiplications, Geometrics and Fourier neural operations.

Jin, Zheming [Advanced Micro Devices (AMD)]↗

DOE OSTI2021

On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks

Physics-informed neural networks (PINNs) are demonstrating remarkable promise in integrating physical models with gappy and noisy observational data, but they still struggle in cases where the target functions to be approximated exhibit high-frequency or multi-scale features. Here in this work we investigate this limitation through the lens of Neural Tangent Kernel (NTK) theory and elucidate how PINNs are biased towards learning functions along the dominant eigen-directions of their limiting NTK. Using this observation, we construct novel architectures that employ spatio-temporal and multi-scale random Fourier features, and justify how such coordinate embedding layers can lead to robust and accurate PINN models. Numerical examples are presented for several challenging cases where conventional PINN models fail, including wave propagation and reaction–diffusion dynamics, illustrating how the proposed methods can be used to effectively tackle both forward and inverse problems involving partial differential equations with multi-scale behavior.

42 ENGINEERING↗

DOE OSTI2022

Autoencoder Neural Network for chemically reacting systems

Incorporating detailed chemical kinetic models is critical for accurate simulations of reacting flows. However, detailed models involve a large number of thermochemical (TC) state variables. Solving the governing equations to evolve these TC variables becomes impractical for real-world applications. In this work, we propose an autoencoder (AE) neural network (NN)-based reduced model to accelerate such simulations. The AE NN is first trained to find a low-dimensional latent representation of the TC states. Then, the evolving state of a chemical system can be tracked by solving the equations of the latent variables instead of the original TC equations. We demonstrate the reduced model in a syngas CO/H 2 combustion system, using training data collected from canonical perfectly stirred reactors (PSRs). It is found that the AE model can reduce the dimension of the combustion system from 12 to 2 while maintaining low reconstruction error and excellent elemental mass conservation for the test dataset. In the a posteriori test, the combustion states obtained from solving the two latent equations are compared to those from solving the 12 equations of the full model. The AE reduced method is found to be able to capture the diverse combustion states on the top two branches of the S-curve well including the extinction turning point, but with higher prediction errors for states near the ignition turning point.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Global field reconstruction from sparse sensors with Veronoi tessellation-assisted deep learning

Achieving accurate and robust global situational awareness of a complex time-evolving field from a limited number of sensors has been a longstanding challenge. This reconstruction problem is especially difficult when sensors are sparsely positioned in a seemingly random or unorganized manner, which is often encountered in a range of scientific and engineering problems. Moreover, these sensors can be in motion and can become online or offline over time. The key leverage in addressing this scientific issue is the wealth of data accumulated from the sensors. As a solution to this problem, we propose a data-driven spatial field recovery technique founded on a structured grid-based deep-learning approach for arbitrary positioned sensors of any numbers. It should be noted that the naïve use of machine learning becomes prohibitively expensive for global field reconstruction and is furthermore not adaptable to an arbitrary number of sensors. In the present work, we consider the use of Voronoi tessellation to obtain a structured-grid representation from sensor locations enabling the computationally tractable use of convolutional neural networks. One of the central features of the present method is its compatibility with deep-learning based super-resolution reconstruction techniques for structured sensor data that are established for image processing. The proposed reconstruction technique is demonstrated for unsteady wake flow, geophysical data, and three-dimensional turbulence. The current framework is able to handle an arbitrary number of moving sensors, and thereby overcomes a major limitation with existing reconstruction methods. The presented technique opens a new pathway towards the practical use of neural networks for real-time global field estimation.

Fukami, Kai↗