Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “LEARNING SYSTEM”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

I/O in Machine Learning Applications on HPC Systems: A 360-degree Survey

Growing interest in Artificial Intelligence (AI) has resulted in a surge in demand for faster methods of Machine Learning (ML) model training and inference. This demand for speed has prompted the use of high performance computing (HPC) systems that excel in managing distributed workloads. Because data is the main fuel for AI applications, the performance of the storage and I/O subsystem of HPC systems is critical. In the past, HPC applications accessed large portions of data written by simulations or experiments or ingested data for visualizations or analysis tasks. ML workloads perform small reads spread across a large number of random files. This shift of I/O access patterns poses several challenges to modern parallel storage systems. In this paper, we survey I/O in ML applications on HPC systems, and target literature within a 6-year time window from 2019 to 2024. We define the scope of the survey, provide an overview of the common phases of ML, review available profilers and benchmarks, examine the I/O patterns encountered during offline data preparation, training, and inference, and explore I/O optimizations utilized in modern ML frameworks and proposed in recent literature. Lastly, we seek to expose research gaps that could spawn further R&D.

97 MATHEMATICS AND COMPUTING↗

System and Machine Learning-Guided Materials Design for High-Pressure Hydrogen Compression

Cost-effective and reliable hydrogen compression remains a challenging barrier in the widespread adoption of hydrogen as an energy carrier. The prevailing technology of mechanical compression suffers from several drawbacks, some of which can be addressed by nonmechanical compression strategies (e.g., electrochemical or metal hydride-based thermal compression). Thermally driven metal hydride compression strategies typically rely on multistage metal hydride-based compressors; however, discovering or optimizing low-stability metal hydrides that can pressurize hydrogen upward of 1000 bar is difficult, both with respect to computational predictions and experimental validation. Here, in this study, we (1) demonstrate that simple machine learning-derived design rules can inform the rational design of alloying strategies yielding low-stability hydrides, (2) validate their experimental pressure–composition–temperature (PCT) isotherms up to 875 bar, and (3) utilize a dynamic system-level model of a metal hydride compressor design to evaluate their performance under realistic operating conditions. Importantly, this analysis yields predicted operational efficiencies of both 2-stage (90–875 bar) and 3-stage (20–875 bar) metal hydride compressors to enable further evaluation of this technology and its techno-economic outlook.

alloy optimization↗

A Fortran–Python interface for integrating machine learning parameterization into earth system models

Abstract. Parameterizations in earth system models (ESMs) are subject to biases and uncertainties arising from subjective empirical assumptions and incomplete understanding of the underlying physical processes. Recently, the growing representational capability of machine learning (ML) in solving complex problems has spawned immense interests in climate science applications. Specifically, ML-based parameterizations have been developed to represent convection, radiation, and microphysics processes in ESMs by learning from observations or high-resolution simulations, which have the potential to improve the accuracies and alleviate the uncertainties. Previous works have developed some surrogate models for these processes using ML. These surrogate models need to be coupled with the dynamical core of ESMs to investigate the effectiveness and their performance in a coupled system. In this study, we present a novel Fortran–Python interface designed to seamlessly integrate ML parameterizations into ESMs. This interface showcases high versatility by supporting popular ML frameworks like PyTorch, TensorFlow, and scikit-learn. We demonstrate the interface's modularity and reusability through two cases: an ML trigger function for convection parameterization and an ML wildfire model. We conduct a comprehensive evaluation of memory usage and computational overhead resulting from the integration of Python codes into the Fortran ESMs. By leveraging this flexible interface, ML parameterizations can be effectively developed, tested, and integrated into ESMs.

54 ENVIRONMENTAL SCIENCES↗

A Fortran-Python Interface for Integrating Machine Learning Parameterization into Earth System Models

Parameterizations in Earth System Models (ESMs) are subject to biases and uncertainties arising from subjective empirical assumptions and incomplete understanding of the underlying physical processes. Recently, the growing representational capability of machine learning (ML) in solving complex problems has spawned immense interests in climate science applications. Specifically, ML-based parameterizations have been developed to represent convection, radiation and microphysics processes in ESMs by learning from observations or high-resolution simulations, which have the potential to improve the accuracies and alleviate the uncertainties. Previous works have developed some surrogate models for these processes using ML. These surrogate models need to be coupled with the dynamical core of ESMs to investigate the effectiveness and their performance in a coupled system. In this study, we present a novel Fortran-Python interface designed to seamlessly integrate ML parameterizations into ESMs. This interface showcases high versatility by supporting popular ML frameworks like PyTorch, TensorFlow, and Scikit-learn. We demonstrate the interface's modularity and reusability through two cases: a ML trigger function for convection parameterization and a ML wildfire model. We conduct a comprehensive evaluation of memory usage and computational overhead resulting from the integration of Python codes into the Fortran ESMs. By leveraging this flexible interface, ML parameterizations can be effectively developed, tested, and integrated into ESMs.

54 ENVIRONMENTAL SCIENCES↗

Controlling optical-cavity locking using reinforcement learning

Abstract This study applies an effective methodology based on Reinforcement Learning to a control system. Using the Pound–Drever–Hall locking scheme, we match the wavelength of a controlled laser to the length of a Fabry-Pérot cavity such that the cavity length is an exact integer multiple of the laser wavelength. Typically, long-term drift of the cavity length and laser wavelength exceeds the dynamic range of this control if only the laser’s piezoelectric transducer is actuated, so the same error signal also controls the temperature of the laser crystal. In this work, we instead implement this feedback control grounded on Q-Learning. Our system learns in real-time, eschewing reliance on historical data, and exhibits adaptability to system variations post-training. This adaptive quality ensures continuous updates to the learning agent. This innovative approach maintains lock for eight days on average.

47 OTHER INSTRUMENTATION↗

AEOLUS: Advances in Experimental Design, Optimal Control, and Learning for Uncertain Complex Systems

Sustained advances in the mathematics of modeling and simulation have resulted in the capability today for routine simulation of a number of large scale complex DOE-relevant systems. As remarkable as this capability for solving the so-called forward problem is, it is typically only the first step-an inner loop within an outer loop that explores the simulation model's parameter space and decision space to characterize uncertainty in the model's predictions, learn unknown model parameters from data, design the most informative experiments, determine optimal control strategies, and create optimal designs. Broadly, what unifies all of these outer loop problems is that they are, in one form or another, optimization problems over parameter/control/design space that are constrained by complex uncertain models. To fully realize the power of scientific simulation as a basis for scientific discovery, technological innovation, and rational decision-making, it is imperative to move beyond simulation to tackle the outer loop of optimization for learning from data, experimental design, and control with complex uncertain models. When the models under consideration are large-scale and complex, and when the optimization variable and uncertain parameter spaces are high (or infinite) dimensional, this constitutes a grand challenge of the highest order, and is intractable with conventional methods. To overcome these challenges, the AEOLUS Center was established to develop a unified mathematical, computational, and statistical framework for (1) Learning predictive models from complex data via Bayesian inference and optimization, and (2) Optimizing experiments, processes, and designs using the resulting uncertain models. These problems are intractable with conventional methods, for several reasons: (1) The simulation problems that govern the inner loops of the optimization problems are expensive to execute (due to severe nonlinearity, heterogeneity, multiphysics/multiscale coupling); (2) The optimization variable and uncertain parameter spaces are high dimensional, often stemming from discretizations of infinite dimensional fields such as initial conditions, sources, or material properties. We argue that the key to overcoming these challenges is to develop new mathematical, computational, and statistical methods that exploit the structure of the Bayesian inference and optimization problems mediated by their underlying complex uncertain models. This structure includes the regularity, sparsity, geometry, low intrinsic dimensionality, and multifidelity nature of the maps from uncertain parameter/optimization variable spaces to the specific objectives targeted: Bayesian inference, optimal experimental design, and optimal control design. Black box methods developed as generic tools are incapable of exploiting this structure. To be successful, we must create, integrate, and cross-fertilize ideas across multiple areas of applied math--including approximation theory, Bayesian inference, data science, experimental design, information theory, machine learning, model reduction, optimal control theory, parallel algorithms, PDE-constrained optimization, randomized algorithms, stochastic optimization, and uncertainty quantification--all while exploiting the structure of the problems at hand. With this goal in mind, we have marshaled a team of leading authorities in these areas. While the methods we develop will be broadly applicable across a wide spectrum of DOE problems in which experiments inform models and the systems those models describe must be optimized under uncertainty, we have chosen a specific area, advanced manufacturing and materials, to drive our work. AMM is characterized by complex models across multiple scales, and is a rich source of challenging problems in inference, experimental design, and optimal control, requiring multifaceted and integrated advances in applied mathematics. As such, AMM serves as an excellent vehicle to motivate and demonstrate the advances in applied mathematics developed by our center.

97 MATHEMATICS AND COMPUTING↗

An Advanced Machine Learning and Artificial Intelligence System for Demonstrating Radiation Regulatory Compliance in DOE Accelerator Facilities

In this Phase II proposal, Applied Research LLC (ARLLC), Thomas Jefferson National Accelerator Facility (Jefferson Lab), and Old Dominion University (ODU) propose the combination of domain knowledge (beam characteristics, fixed structural shielding, earthen burden (the soil and foliage added to the dome of the experimental halls as additional shielding), etc.), machine learning (ML) and/or artificial intelligence (AI) to correlate a variety of multi-modal onsite signals and the radiation fields seen in accessible areas of the accelerator site and the site boundary. The ML/AI will consider the complex influence of environmental parameters affecting the radon contribution of the measurements, focusing on actual data obtained from Jefferson Lab. In Phase I, the coded beam and location data were fed into a deep learning model to predict doses at several designated locations in Jefferson Lab’s facility. Moreover, a dense radiation map was generated using only a sparse collection of the samples in a facility. In Phase II, we will develop a software prototype containing a radiation prediction algorithm, dense radiation map algorithms, and background noise prediction algorithms, with actual data used to evaluate the prototype. This work will provide a framework for evaluation of radiation measurement results around the site based on learned responses. In addition, the proposed approach allows more granular mapping of radiation levels. Better understanding and communication of these levels is related to the overall approach in keeping doses to personnel ALARA.

43 PARTICLE ACCELERATORS↗

An Efficient Checkpointing System for Large Machine Learning Model Training

As machine learning models increase in size and complexity rapidly, the cost of checkpointing in ML training became a bottleneck in storage and performance (time). For example, the latest GPT-4 model has massive parameters at the scale of 1.76 trillion. It is highly time and storage consuming to frequently writes the model to checkpoints with more than 1 trillion floating point values to storage. This work aims to understand and attempt to mitigate this problem. First, we characterize the checkpointing interface in a collection of representative large machine learning/language models with respect to storage consumption and performance overhead. Second, we propose the two optimizations: i) A periodic cleaning strategy that periodically cleans up outdated checkpoints to reduce the storage burden; ii) A data staging optimization that coordinates checkpoints between local and shared file systems for performance improvement.

machine learning, artificial intelligence↗

Does practice make perfect? Lessons learned from full-scale power system incident response exercise

While threats to the energy sector occur daily, few utilities get the opportunity to fully test out their detection and response mechanisms to advanced threats in the real world. With the high demand for reliability, few grid operators would allow execution of simulated cyber-attacks on their live systems. The DOE-funded Liberty Eclipse project offers a unique opportunity for small and large utilities and coops to practice their combined IT/OT responses to a live red team executing attacks against an isolated power system on an island in New York. Both cyber teams and power operations teams must work together to detect and respond to attacks, even restoring the power system against extreme impacts. Lessons learned from these exercises reveal key takeaways for understanding what a real attack against the electric sector will look like, gaps in execution of the best-laid plans when the pressure of a real event is bearing down, and how organizations can better prepare for advanced attacks by optimizing participation in exercises. This presentation will discuss successes and opportunities for improvement both in how utilities can prepare for and respond to events, as well as how full-scale IT/OT exercises can be coordinated.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Resilience Assessment for Distribution Systems during Hurricanes: A Learning-Based Framework

This paper presents a proactive strategy for hurricane-resilient distribution systems. It proposes a Bayesian Neural Network-based outage prediction model considering various parameters, including electrical components, and weather and environmental factors. Addressing challenges in imbalanced outage datasets, a Bias-Variance Tradeoff method is proposed. A resilience assessment model quantifies resilience indices, providing insights into system weaknesses. The approach identifies weak points and serves as a planning benchmark. Numerical results on the modified IEEE 123-node test system demonstrate effectiveness in realistic hurricane scenarios.

Vahedi, Soroush↗

Influence of initial conditions on data-driven model identification and information entropy for ideal mhd problems

Data-driven methods of model identification are able to discern governing dynamics of a system from data. Such methods are well suited to help us learn about systems with unpredictable evolution or systems with ambiguous governing dynamics given our current understanding. Many plasma problems of interest fall into these categories as there are a wide range of models that exist, however each model is only useful in a certain regime and often limited by computational complexity. To ensure data-driven methods align with theory, they must be consistent and predictable when acting on data whose governing dynamics are known. Weak Sparse Identification of Nonlinear Dynamics (WSINDy) is a recently developed data-driven method that has shown promise in learning governing dynamics from data with high noise levels [1]. This work examines how WSINDy acts on ideal MHD test problems as the initial conditions are varied and specifies limiting requirements for successful equation identification. Furthermore, it is hard to recover the governing dynamics from data that emphasize a single dominant behavior. In these low information cases, Shannon information entropy is able to pick up on the redundancies in the data that affect recoverability.

97 MATHEMATICS AND COMPUTING↗

PowerModelsGAT-AI: Physics-Informed Graph Attention for Multi-System Power Flow With Continual Learning

Solving the alternating current power flow equations in real time is essential for secure grid operation, yet classical Newton–Raphson solvers can be slow under stressed conditions. Existing graph neural networks for power flow are typically trained on a single system and often degrade on different systems. We present PowerModelsGAT-AI, a physics-informed graph attention network that predicts bus voltages and generator injections. The model uses bus-type-aware masking to handle different bus types and balances multiple loss terms, including a power-mismatch penalty, using learned weights. We evaluate the model on 14 benchmark systems (4 to 6,470 buses) and train a unified model on 13 of these under contingency conditions with up to two branch outages, achieving an average normalized mean absolute error of 0.89% for voltage magnitudes and R 2 >0.99 for voltage angles. We also show continual learning: when adapting a base model to a new 1,354-bus system, standard fine-tuning causes severe forgetting with error increases exceeding 1000% on base systems, while our experience replay and elastic weight consolidation strategy keeps error increases below 2% and in some cases improves base-system performance. Interpretability analysis shows that learned attention weights correlate with physical branch parameters (susceptance: r=0.38 ; thermal limits: r=0.22 ), and feature importance analysis supports that the model captures established power flow relationships.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Space‐Time Causal Discovery in Earth System Science: A Local Stencil Learning Approach

Causal discovery tools enable scientists to infer meaningful relationships from observational data, spurring advances in fields as diverse as biology, economics, and climate science. Despite these successes, the application of causal discovery to space-time systems remains immensely challenging due to the high-dimensional nature of the data. For example, in climate sciences, modern observational temperature records over the past few decades regularly measure thousands of locations around the globe. To address these challenges, we introduce Causal Space-Time Stencil Learning (CaStLe), a novel meta-algorithm for discovering causal structures in complex space-time systems. CaStLe leverages regularities in local space-time dependencies to learn governing global dynamics. This local perspective eliminates spurious confounding and drastically reduces sample complexity, making space-time causal discovery practical and effective. For causal discovery, CaStLe flexibly accepts any appropriately adapted time series causal discovery algorithm to recover local causal structures. These advances enable causal discovery of geophysical phenomena that were previously unapproachable, including non-periodic, transient phenomena such as volcanic eruption plumes. Regularities in local space-time dependencies are transformed into informative spatial replicates, which actually improve CaStLe's performance when applied to ever-larger spatial grids. We successfully apply CaStLe to discover the atmospheric dynamics governing the climate response to the 1991 Mount Pinatubo volcanic eruption. We provide validation experiments to demonstrate the effectiveness of CaStLe over existing causal-discovery frameworks on a range of geophysics-inspired benchmarks while identifying the method's limitations and domains where its assumptions may not hold.

Nichol, J. Jake [Univ. of New Mexico, Albuquerque,↗

Bayesian stability and force modeling for uncertain machining processes

Accurately simulating machining operations requires knowledge of the cutting force model and system frequency response. However, this data is collected using specialized instruments in an ex-situ manner. Bayesian statistical methods instead learn the system parameters using cutting test data, but to date, these approaches have only considered milling stability. This paper presents a physics-based Bayesian framework which incorporates both spindle power and milling stability. Initial probabilistic descriptions of the system parameters are propagated through a set of physics functions to form probabilistic predictions about the milling process. The system parameters are then updated using automatically selected cutting tests to reduce parameter uncertainty and identify more productive cutting conditions, where spindle power measurements are used to learn the cutting force model. The framework is demonstrated through both numerical and experimental case studies. Results show that the approach accurately identifies both the system natural frequency and cutting force model.

42 ENGINEERING↗

Enabling Secure and Resilient XFC: A Software/Hardware-Security Co-Design Approach

Extremely fast charging (XFC) has the potential to reduce the charging time of battery electric vehicles (BEV) to be equivalent to the filling time of internal combustion engine vehicles (ICEV), thus eliminating one of the few advantages ICEV still poses for light- and heavy-duty vehicles. Enabling XFC will, however, require coordination and cooperation between the grid, charging stations, and the vehicles themselves, which leads to an inevitable increase in the attack surface for all systems combined. In securing the overall system, we must not only embrace traditional cybersecurity, which is chiefly concerned with communications and the operation of digital systems, but also cyber-physical systems security as the proper operation of XFC is critically dependent on systems’ abilities to know about (sense) and interact with (actuate) the physical world. The project team consists of academic and industry researchers with backgrounds in cybersecurity, cyber-physical systems security, learning in adversarial environments, transportation security, grid security and resilience, wireless power transfer, converter design, and battery management systems.

33 ADVANCED PROPULSION SYSTEMS↗

Advancing Cyber-Attack Detection in Power Systems: A Comparative Study of Machine Learning and Graph Neural Network Approaches

This paper explores the detection and localization of cyber-attacks on power systems, focusing on comparing conventional machine learning (ML) and deep learning methods, and graph neural network (GNN)-based techniques. We assess the detection accuracy of these approaches and their potential to pinpoint the locations of specific buses under attack. Given the demonstrated success of GNNs in other time series anomaly detection applications, we aim to evaluate their performance within the context of power systems cyber-attack. Utilizing the IEEE 68-bus system, we simulated four types of attacks to test the selected approaches. Our results indicate that GNN-based methods outperform conventional machine learning and deep learning models in detection. Additionally, GNNs show promise in accurately localizing attacks for simple scenarios, although they still face challenges in more complex cases.

artificial intelligence↗