Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Machine Learning (ML)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

Deep learning of dynamically responsive chemical Hamiltonians with semiempirical quantum mechanics

Conventional machine-learning (ML) models in computational chemistry learn to directly predict molecular properties using quantum chemistry only for reference data. While these heuristic ML methods show quantum-level accuracy with speeds several orders of magnitude faster than traditional quantum chemistry methods, they suffer from poor extensibility and transferability; i.e., their accuracy degrades on large or new chemical systems. Incorporating quantum chemistry frameworks into the ML models directly solves this problem. Here we take the structure of semiempirical quantum mechanics (SEQM) methods to construct dynamically responsive Hamiltonians. SEQM methods use empirical parameters fitted to experimental properties to construct reduced-order Hamiltonians, facilitating much faster calculations than ab initio methods but with compromised accuracy. By replacing these static parameters with machine-learned dynamic values inferred from the local environment, we greatly improve the accuracy of the SEQM methods. Trained on molecular energies and atomic forces, these dynamically generated Hamiltonian parameters show a strong correlation with atomic hybridization and bonding. Trained with only about 60,000 small organic molecular conformers, the resulting model retains interpretability, extensibility, and transferability when testing on much larger chemical systems and predicting various molecular properties. Overall, this work demonstrates the virtues of incorporating physics-based descriptions with ML to develop models that are simultaneously accurate, transferable, and interpretable.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Data for: A hybrid biophysical-machine learning framework for diurnal surface energy flux estimation using proximal sensing

Thermal-based remote sensing of surface energy fluxes has traditionally relied on high spatial resolution satellite data with revisit frequencies on the order of weeks. In this study, we evaluate a biophysics-based analytical surface energy balance model for predicting latent energy (LE) and sensible heat (H) fluxes using proximal sensing observations. The Surface Temperature Initiated Closure (STIC1.2) model has been extensively validated across a wide range of spatial and temporal scales using various satellite-derived thermal datasets. Here we extend this validation by applying STIC at sub-hourly temporal resolution over multiple growing seasons for four distinct agricultural systems. We further develop and evaluate novel STIC variants that incorporate machine learning (ML) techniques to eliminate the need for specific surface energy balance observations, specifically net radiation and soil heat flux, thereby enhancing model applicability in data-sparse settings. The integration of an ML component to estimate surface available energy is shown to have strong predictive performance for both LE (R2 = 0.81-0.94) and H (R2 = 0.46-0.72) across all agricultural systems examined here, demonstrating the potential of hybrid biophysical – machine learning approaches for surface energy balance modeling with minimal data requirements. This study concludes with a novel application of explainable machine learning (exML) to diagnose sources of model error. This exML framework attributes residual prediction errors to both model input variables and environmental drivers not explicitly included in the simulation experiments. This approach provides a new pathway for improving model design and integrating previously overlooked yet influential variables into future model iterations.

Agricultural Sciences↗

ML Track Fitting in Nuclear Physics

Charged particle tracking represents the largest consumer of CPU resources in high data volume Nuclear Physics (NP) experiments. An effort is underway to develop machine learning (ML) networks that will reduce the resources required for charged particle tracking. Tracking in NP experiments represent some unique challenges compared to high energy physics (HEP). In particular, track finding typically represents only a small fraction of the overall tracking problem in NP. This presentation will outline the differences and similarities between NP and HEP charged particle tracking and areas where ML learning may provide a benefit. The status of the specific effort taking place at Jefferson Lab will also be shown.

Britton, Thomas↗

Reification of latent microstructures: On supervised unsupervised and semi-supervised deep learning applications for microstructures in materials informatics

Machine learning (ML), including deep learning (DL), has become increasingly popular in the last few years due to its continually outstanding performance. In this context, we apply machine learning techniques to "learn" the microstructure using both supervised and unsupervised DL techniques. In particular, we focus (1) on the localization problem bridging (micro)structure (localized) property using supervised DL and (2) on the microstructure reconstruction problem in latent space using unsupervised DL. The goal of supervised and semi-supervised DL is to replace crystal plasticity finite element model (CPFEM) that maps from (micro)structure (localized) property, and implicitly the (micro)structure (homogenized) property relationships, while the goal of unsupervised DL is (1) to represent high-dimensional microstructure images in a non-linear low-dimensional manifold, and (2) to discover a way to interpolate microstructures via latent space associating with latent microstructure variables. At the heart of this report is the applications of several common DL architectures, including convolutional neural networks (CNN), autoencoder (AE), and generative adversarial network (GAN), to multiple microstructure datasets, and the quest of neural architecture search for optimal DL architectures.

36 MATERIALS SCIENCE↗

Machine Learning in Power System Operations: Training Data

Reliability and stability of the electric grid today has depended upon operations of the grid which include the protective relay. Today, the electricity sector faces new challenges with the shift of generation resource characteristics away from the traditional “big iron” generation to inverter-based resources (IBR) which shift the physics and assumption used in grid operation and protection. These changing conditions represent new challenges for protective relays (identification of faults) and increased challenges for protection engineers (correct settings and configuration, reduction of mis-operations), both issues recognized in research and industry. Finding new approaches to reduce mis-operations in relaying and new approaches to fault identification is critical to grid operations. Using today’s modern technology of embedded systems, edge computing, machine learning (ML), and communications we can help address challenges and augment and improve on existing power system operations methodologies.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Cyber risk assessment and investment optimization using game theory and ML-based anomaly detection and mitigation for wide-area control in smart grids

The electric power grid is increasingly becoming susceptible to cyber attacks that exploit vulnerabilities in the smart grid control, information, and physical layers. Successful cyber attacks can have catastrophic impacts on the social and economic well-being of any nation all over the globe. It has, thus, become imperative to secure the smart grid against such adversarial actions to ensure stable, secure, and reliable operation of the grid. The existing research and industry practices prove to be inadequate in terms of providing pragmatic and effective defense methodologies and measures for long-term cybersecurity planning and real-time cybersecurity for grid operation. For example, existing works lack models that incorporate uncertain behavior of cyber-attackers and pragmatic defense measures for cyber risk assessment and cybersecurity investment optimization which often provide unreliable and strictly qualitative solutions to these problems. At the same time, with the growing number of cyber incidents in the grid, there still exists a need to develop attack-resilient algorithms for wide-area monitoring, protection, and control (WAMPAC) applications like the wide-area voltage control systems (WAVCS) for Flexible AC Transmissions Systems (FACTS) that lack in scalable and feasible solutions from the cybersecurity perspective. This dissertation proposes novel models and methodologies for: (1) Cybersecurity planning, and (2) Cybersecurity for system operation. The cybersecurity planning is achieved through cyber risk assessment and cybersecurity resource investment optimization for long-term cybersecurity of the grid using game theory and attack-defense trees. Cybersecurity for system operation consists of development of cyber anomaly detection and mitigation algorithms for flexible AC transmission system (FACTS) controller-based wide-area voltage control systems (WAVCS) using machine learning (ML), and software defined networking-based moving target defense network routing for achieving real-time cyber-physical security for grid operations. This is followed by hardware-in-the-loop (HIL) implementation and evaluation of these attack prevention, detection, and mitigation algorithms and methodologies showcasing their feasibility in a close to real-world environment. For cybersecurity planning, a novel approach involving a combination of game theory and attack defense trees (ADT) for optimal cybersecurity resource allocation in the smart grid is proposed. This methodology involves modeling of the cyber-physical smart grid substations as ADTs, defining attacker costs, defense costs, and attack probabilities for attack access points. Using game theoretical formulation, optimal defense strategies for the defender of the system to invest cybersecurity resources in the grid are obtained. Additionally, a game-theoretic framework is developed for quantitative cyber-physical risk assessment of the grid under a dynamically changing cyber threat space and uncertain behavior of cyber attackers which is further used to optimize investments in the smart grid's cybersecurity resources. The attacker, defender, and the smart grid system are modeled while incorporating attacker-stochasticity and federal guidelines for smart grid cybersecurity. This allows quantification of threat, vulnerabilities, and attack impact of the grid for quantitative risk assessment. The defender's budget to invest in the security resources in the grid is optimized based on the strategies leading to minimum system risk. The evaluation of the proposed solutions highlight the feasibility for practical implementation of these methodologies and algorithms in the smart grid, while taking the federal requirements and guidelines for smart grid security into consideration. For achieving cybersecurity for system operation, attack prevention, detection, and mitigation algorithms and methodologies are developed specifically for FACTS-based WAVCS. Anomaly detection and mitigation in the WAVCS are achieved using algorithms based on machine learning which involves offline training and testing of ML models with CPS datasets incorporating physics-based features that allow accurate distinction between system faults and cyber attacks. For attack prevention, a methodology based on software defined network (SDN)-based moving target defense (MTD) network routing is proposed that enables prevention of Denial of Service (DoS) type attacks on the smart grid communication system. Subsequently, these methodologies and algorithms are implemented and evaluated on an HIL testbed that allows for real-time attack prevention, detection, and mitigation of emulated cyber attacks on the WAVCS in a close to real-world environment. The results show highly accurate and efficient performance of the implemented algorithms and methodologies with the smart grid system operating within the NERC's system operation limits even in the presence of DoS and data integrity cyber attacks. This work opens up future research opportunities in other directions such as (1) Expanding cybersecurity planning methodologies to real-time cyber contingency analysis with different game formulations; and (2) Applying the cybersecurity for system operation algorithms to broader categories of wide-area control applications.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Toward Machine Learning-Driven Mass Spectrometric Identification of Trichothecenes in the Absence of Standard Reference Materials

While a significant body of work exists on the detection of commonly known trichothecene toxins, biological, environmental, and other transformational processes can generate many under-characterized and unknown modified trichothecenes. Lacking both analytical reference standards and associated mass spectral databases, identification of these modified compounds reflects both a challenge and a critical gap from forensic and public health perspectives. Here we report here the application of machine learning (ML) techniques toward identification of discriminative fragment ions from mass spectrometric data that can be exploited to detect evidence of type A and B trichothecenes. The goal of this work is to establish a new method for the identification of unknown, though structurally similar trichothecenes, by leveraging objective ML techniques. Discriminative fragments derived from a series of gradient-boosted machine learners are then used to develop ML-driven precursor ion scan (PIS) methods on a triple quadrupole mass spectrometer (QQQ) for screening of “unknown unknown” trichothecenes. Specifically, we apply the PIS method to a laboratory-synthesized trichothecene, a first step in demonstrating the power of alternative, machine learning-driven mass spectrometric methods.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Using Machine Learning to Predict MCNP Bias

For many real-world applications in radiation transport where simulations are compared to experimental measurements, like in nuclear criticality safety, the bias (simulated - experimental k eff ) in the calculation is an extremely important quantity used for code validation. The objective of this project is to accurately predict the bias of MCNP6 [1] criticality calculations using machine learning (ML) algorithms, with the intention of creating a tool that can complement the current nuclear criticality safety methods. In the latest release of MCNP6, the Whisper tool is available for criticality safety analysts and includes a large catalogue of experimental benchmarks, sensitivity profiles, and nuclear data covariance matrices. This data, coming from 1100+ benchmark cases, is used in this study of ML algorithms for criticality safety bias predictions.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Cyber threat assessment of machine learning driven autonomous control systems of nuclear power plants

We report advanced cyber-attacks against critical infrastructure and the energy sector are becoming more common. With the invention of autonomous control systems (ACS) within advanced nuclear reactor designs, system designers, reactor operators, and regulators must consider cybersecurity during the design and operational phases. This article provides a cyber threat assessment of machine learning (ML)-based digital twinning (DT) technologies in the context of advanced reactor ACS. A cyber–physical testbed was created to emulate nuclear reactor digital instrumentation and controls (I&C) and act as a basis for the ACS. The ACS was designed as two plant-level DTs predicting reactor malfunctions and determining control actions and two component-level DTs responsible for classifying component states and forecasting component inputs and outputs (I/O). Two duplicate ACS designs– one using a traditional ML framework and one using an automated ML (AutoML) framework– were created and tested against cyber-attacks on training data, real-time process data, and ML model architectures to determine their respective qualitative cyber-risk in terms of likelihood and impact. Both frameworks showed similar cyber-resilience against training, real-time, and ML architecture attacks, proving that neither is inherently more secure. Recommended safeguard and security measures are posed to system designers, reactor operators, and regulators to maintain the cybersecurity of ML-based DT technologies such as ACS, prompting a holistic view of shared responsibility for maintaining cyber-secure ML-based systems.

99 GENERAL AND MISCELLANEOUS↗

Report for 2.2.1 Task 5: Develop and Document a State-Based Alarm System for a Nuclear Power Plant Control Room Using Machine Learning. Light Water Reactor Sustainability Program report

In legacy control rooms the operators might have 20 to 30 alarms to deal with on 12 control boards, during a reactor trip. The new digital alarm system in the upgraded control room will display between 150 and 200 alarm points on a single screen. This overload of information prevents the operators from identifying any abnormal alarms quickly without having to scroll through the alarm list looking for the abnormal alarms. One way to tackle this problem could be to attempt to create a state-based alarm system which would identify plant states and would identify which alarms are expected in those states and suppress them from the alarm screen so that any unexpected alarms would be displayed and easily identified to the operators. This can be done today but is very labor intensive in that an operator or training instructor would be required to run scenarios and log the alarms to be suppressed. The hope is that Machine Learning (ML) can provide an alternative solution with less manual effort required.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

ClimSim: A large multi-scale dataset for hybrid physics-ML climate emulation

Modern climate projections lack adequate spatial and temporal resolution due to computational constraints. A consequence is inaccurate and imprecise predictions of critical processes such as storms. Hybrid methods that combine physics with machine learning (ML) have introduced a new generation of higher fidelity climate simulators that can sidestep Moore’s Law by outsourcing compute-hungry, short, high-resolution simulations to ML emulators. However, this hybrid ML-physics simulation approach requires domain-specific treatment and has been inaccessible to MLexperts because of lack of training data and relevant, easy-to-use workflows. Wepresent ClimSim, the largest-ever dataset designed for hybrid ML-physics research. It comprises multi-scale climate simulations, developed by a consortium of climate scientists and ML researchers. It consists of 5.7 billion pairs of multivariate input and output vectors that isolate the influence of locally-nested, high-resolution, high-fidelity physics on a host climate simulator’s macro-scale physical state. The dataset is global in coverage, spans multiple years at high sampling frequency, and is designed such that resulting emulators are compatible with downstream coupling into operational climate simulators. We implement a range of deterministic and stochastic regression baselines to highlight the ML challenges and their scoring. The data (https://huggingface.co/datasets/LEAP/ClimSim_high-res2) and code(https://leap-stc.github.io/ClimSim)arereleasedopenlytosupport the development of hybrid ML-physics and high-fidelity climate simulations for the benefit of science and society.

artificial intelligence, machine learning↗

A Hybrid Biophysical‐Machine Learning Framework for Diurnal Surface Energy Flux Estimation Using Proximal Sensing

Thermal infrared-based remote sensing of surface energy fluxes has traditionally relied on high spatial resolution satellite data with revisit frequencies on the order of weeks. In this study, we evaluate a biophysics-based analytical surface energy balance model for predicting latent energy ( LE ) and sensible heat ( H ) fluxes using proximal sensing observations. The Surface Temperature Initiated Closure (STIC1.2) model has been extensively validated across a wide range of spatial and temporal scales using various satellite-derived thermal infrared data sets. Here we extend this validation by applying STIC at sub-hourly temporal resolution over multiple growing seasons for four distinct agricultural systems. We further develop and evaluate novel STIC variants that incorporate machine learning (ML) techniques to eliminate the need for surface energy balance observations, specifically net radiation and soil heat flux, thereby enhancing model applicability in data-sparse settings. The integration of a ML component to estimate surface available energy is shown to have strong predictive performance for both LE (R 2 = 0.81–0.94) and H (R 2 = 0.46–0.72) across all agricultural systems examined here, demonstrating the potential of hybrid biophysical-machine learning approaches for surface energy balance modeling with minimal data requirements. This study concludes with a novel application of explainable machine learning (exML) to diagnose sources of model error. This exML framework attributes residual prediction errors to both model input variables and environmental drivers not explicitly included in the simulation experiments. This approach provides a new pathway for improving model design and integrating previously overlooked yet influential variables into future model iterations.

evapotranspiration↗

HLS4ML Integration with QICK

The QICK (Quantum Instrumentation Control Kit) integration aims to enhance the readout of superconducting qubits by leveraging machine learning (ML) techniques. These techniques offer high accuracy, increased speed, and better state preservation for qubit readouts. The integration process employs neural network algorithms to optimize system performance and achieve high accuracy rates. The QICK board uses an FPGA, allowing for efficient real-time processing and high-performance execution of machine learning algorithms.

Ali, Mohamud↗

Application of an automated machine learning-genetic algorithm (AutoML-GA) coupled with computational fluid dynamics simulations for rapid engine design optimization

In recent years, the use of machine learning-based surrogate models for computational fluid dynamics (CFD) simulations has emerged as a promising technique for reducing the computational cost associated with engine design optimization. However, such methods still suffer from drawbacks. One main disadvantage is that the default machine learning (ML) hyperparameters are often severely suboptimal for a given problem. This has often been addressed by manually trying out different hyperparameter settings, but this solution is ineffective in case of a high-dimensional hyperparameter space. Besides this problem, the amount of data needed for training is also not known a priori. In response to these issues that need to be addressed, the present work describes and validates an automated active learning approach, AutoML-GA, for surrogate-based optimization of internal combustion engines. In this approach, a Bayesian optimization technique is used to find the best machine learning hyperparameters based on an initial dataset obtained from a small number of CFD simulations. Subsequently, a genetic algorithm is employed to locate the design optimum on the ML surrogate surface. In the vicinity of the design optimum, the solution is refined by repeatedly running CFD simulations at the projected optima and adding the newly obtained data to the training dataset. It is demonstrated that AutoML-GA leads to a better optimum with a lower number of CFD simulations, compared to the use of default hyperparameters. The proposed framework offers the advantage of being a more hands-off approach that can be readily utilized by researchers and engineers in industry who do not have extensive machine learning expertise.

Owoyele, Opeoluwa↗

Networks and interfaces as catalysts for polymer materials innovation

Autonomous experimental systems offer a compelling glimpse into a future where closed-loop, iterative cycles—performed by machines and guided by artificial intelligence (AI) and machine learning (ML)—play a foundational role in materials research and development. This perspective draws attention to the roles of networks and interfaces—of and between humans and machines—for the purpose of generating knowledge and accelerating innovation. Polymers, a class of materials with massive global impact, present a unique opportunity for the application of informatics and automation to pressing societal challenges. To develop these networks and interfaces in polymer science, the Community Resource for Innovation in Polymer Technology (CRIPT)—a polymer data ecosystem based on novel polymer data model, representation, search, and visualization technologies—is introduced. The ongoing co-design efforts engage stakeholders in industry, academia, and government to uncover rapidly actionable, high-impact opportunities to build networks, bridge interfaces, and catalyze innovation in polymer technology.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Continual Learning for Particle Accelerators

Particle accelerators operate under dynamically changing conditions, which often lead to data distribution drifts. These drifts pose significant challenges for Machine Learning (ML) models, which typically fail to maintain performance when faced with such non-stationary data. In particle accelerators, the primary sources of these data drifts include changes in accelerator settings and non-measured parameters such as machine degradation and environmental factors. Previous research has proposed conditional models to handle multiple beam configurations effectively; however, it is challenging to train the ML models on all possible configuration settings. Additionally, conditional models alone can not address performance degradation caused by drifts due to non-measured factors. These limitations contribute to a significant gap between ML development and its deployment in real-world operational settings. To bridge this gap, in this paper, we identify some of the key areas within particle accelerators where continual learning can help mitigate drift-induced performance degradation. In addition, we present a practical use case where a conditional Auto-Encoder model coupled with memory-based continual learning has been employed to demonstrate stable performance even when underlying data drifts.

Schram, Malachi [Thomas Jefferson National Acceler↗

Continual Learning for Particle Accelerators

Particle accelerators operate under dynamically changing conditions, which often lead to data distribution drifts. These drifts pose significant challenges for Machine Learning (ML) models, which typically fail to maintain performance when faced with such non-stationary data. In particle accelerators, the primary sources of these data drifts include changes in accelerator settings and non-measured parameters such as machine degradation and environmental factors. Previous research has proposed conditional models to handle multiple beam configurations effectively; however, it is challenging to train the ML models on all possible configuration settings. Additionally, conditional models alone can not address performance degradation caused by drifts due to non-measured factors. These limitations contribute to a significant gap between ML development and its deployment in real-world operational settings. To bridge this gap, in this paper, we identify some of the key areas within particle accelerators where continual learning can help mitigate drift-induced performance degradation. In addition, we present a practical use case where a conditional Auto-Encoder model coupled with memory-based continual learning has been employed to demonstrate stable performance even when underlying data drifts

Rajput, Kishansingh [Thomas Jefferson National Acc↗

Prediction of Power Measurements Using Adaptive Filters

With the advent of smart grid concept, Internet of Things (IoT) and the deployment of smart meters, the cyberattack threats on power networks have increased due to the use of communication systems that can be accessed by adversaries. Attackers will have the ability to manipulate the outcomes of smart meters which in turn influence the core application of Energy Management System 9EMS): State Estimation (SE). Bad data analytic tools may fail to detect some attacks into measurements. Meanwhile, Machine Learning (ML) solutions have been proposed for detecting False Data Injection (FDI) attacks. However, there is a lack of ML time-series solutions presented in the state-of-the-art that is yet to be complex. In signal processing, time-series solutions do not only consider the signal, but also the statistics of the signal over time. Therefore, in this paper, a machine learning for time-series solutions is presented as an application to model the measurements of the power grid that are used in SE. The presented model takes into account adaptive linear and non-linear filters: Finite Impulse Response (FIR), and Infinite Impulse Response (IIR). The presented models are implemented and performed on the IEEE-118 bus system. The results indicate the advantage of applying those filters over the state-of-the-art machine learning solutions.

Hamad, Khaled↗