Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “dynamic model learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Biochemical Process Modeling and Simulation (BPMS)

The Biochemical Process Modeling and Simulation project aims to reduce the cost and time of research by applying theory, modeling, and simulation to the most relevant bottlenecks in the biochemical process. We use molecular modeling, quantum mechanics, metabolic modeling, fluid dynamics, and reaction-diffusion methods in close collaboration with pretreatment, hydrolysis, upgrading, and TEA. The project's outcomes are increased yields and efficiency of the biochemical process, added value to products, and reduced price of fuels by specifically targeting catalytic efficiency, reactor design, enzyme efficiency, and microbial design. We work closely with experimental projects to identify problems and iterate with experiments to find and refine solutions. By working with experimentalists, we decide on problems that can be solved with simulation that could otherwise not be solved or would take too long with experiment alone to reach BETO's targets. Over the years, we have produced solutions that have resulted in determining the most likely fatty-acid derivative for passive transport out of bacteria that upgrade biomass, and we have also designed enzyme mutations for enhanced lignin upgrading. Metabolic models have been developed to tune the activity of 2,3 butanediol production for the 2030 target. A computational method to deliver understanding of how complex omics data can be interpreted in the metabolic pathways of organisms used in the Agile Biofoundry. We have found methods to overcome specific barriers and continue to develop those methods. Our reactor studies have guided the design of both the microbes and reactors for aerobic and micro-aerobic production at all scales and have been instrumental in improving the accuracy of techno-economic analysis models. This project is essential in the process of selecting the final processes for 2030 SAF production targets. More specifically, recently, we have: 1) Predicted the strength of the basic structural interactions in commodity plastics to provide guidance for plastics upcycling strategies. 2) Developed computational tool to improve the characterization of lignin-derived compounds 3) Developed new methodologies to enable Machine Learning-based Directed Evolution for protein engineering. 4) Developed Machine Learning methods to predict protein promiscuity and mutations to further improve microbial and enzymatic driven processes and demonstrated the utility of ML approaches to engineering proteins from sparse experimental datasets. 5) Developed new methods to enable high-fidelity simulation of aerobic fermentation at industrial scale and resolving mismatch of time scales through subcycling/operator splitting 7) Identified the difficulty in preventing local high-oxygen conditions in industrial bubble columns, which leads to less-desirable acetoin production, suggesting future research directions in alternative reactor configurations (e.g loop reactors, shallow-channel reactors).

BIOMASS FUELS↗
DOE OSTI2023

Uncertainty-driven dynamics for active learning of interatomic potentials

Machine learning (ML) models, if trained to data sets of high-fidelity quantum simulations, produce accurate and efficient interatomic potentials. Active learning (AL) is a powerful tool to iteratively generate diverse data sets. In this approach, the ML model provides an uncertainty estimate along with its prediction for each new atomic configuration. If the uncertainty estimate passes a certain threshold, then the configuration is included in the data set. Here we develop a strategy to more rapidly discover configurations that meaningfully augment the training data set. The approach, uncertainty-driven dynamics for active learning (UDD-AL), modifies the potential energy surface used in molecular dynamics simulations to favor regions of configuration space for which there is large model uncertainty. The performance of UDD-AL is demonstrated for two AL tasks: sampling the conformational space of glycine and sampling the promotion of proton transfer in acetylacetone. The method is shown to efficiently explore the chemically relevant configuration space, which may be inaccessible using regular dynamical sampling at target temperature conditions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Dynamic energy system modeling using hybrid physics-based and machine learning encoder–decoder models

Three model configurations are presented for multi-step time series predictions of the heat absorbed by the water and steam in a thermal power plant. The models predict over horizons of 2, 4, and 6 steps into the future, where each step is a 5-minute increment. The evaluated models are a pure machine learning model, a novel hybrid machine learning and physics-based model, and the hybrid model with an incomplete dataset. The hybrid model deconstructs the machine learning into individual boiler heat absorption units: economizer, water wall, superheater, and reheater. Each configuration uses a gated recurrent unit (GRU) or a GRU-based encoder–decoder as the deep learning architecture. Mean squared error is used to evaluate the models compared to target values. The encoder–decoder architecture is over 11% more accurate than the GRU only models. The hybrid model with the incomplete dataset highlights the importance of the manipulated variables to the system. The hybrid model, compared to the pure machine learning model, is over 10% more accurate on average over 20 iterations of each model. Automatic differentiation is applied to the hybrid model to perform a local sensitivity analysis to identify the most impactful of the 72 manipulated variables on the heat absorbed in the boiler. The models and sensitivity analyses are used in a discussion about optimizing the thermal power plant.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Dissolving salt is not equivalent to applying a pressure on water

Abstract Salt water is ubiquitous, playing crucial roles in geological and physiological processes. Despite centuries of investigations, whether or not water’s structure is drastically changed by dissolved ions is still debated. Based on density functional theory, we employ machine learning based molecular dynamics to model sodium chloride, potassium chloride, and sodium bromide solutions at different concentrations. The resulting reciprocal-space structure factors agree quantitatively with neutron diffraction data. Here we provide clear evidence that the ions in salt water do not distort the structure of water in the same way as neat water responds to elevated pressure. Rather, the computed structural changes are restricted to the ionic first solvation shells intruding into the hydrogen bond network, beyond which the oxygen radial-distribution function does not undergo major change relative to neat water. Our findings suggest that the widely cited pressure-like effect on the solvent in Hofmeister series ionic solutions should be carefully revisited.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

pnnl/deps_arXiv2020

This software implements data-driven control and dynamic systems models using deep learning methods. The particular application is building temperature control.

Tuor, Aaron↗
DOE OSTI2024

FTL: Transfer Learning Nonlinear Plasma Dynamic Transitions in Low Dimensional Embeddings (FTL) v1.0

Fusion Transfer Learning (FTL) model provides a new paradigm to study high-dimensional dynamical behaviors, such as those in fusion plasma systems. The knowledge transfer process leverages a pre-trained neural encoder-decoder network, initially trained on linear simulations, to effectively capture nonlinear dynamics. The low-dimensional embeddings extract the coherent structures of interest, while preserving the inherent dynamics of the complex system. Experimental results highlight FTL's capacity to capture transitional behaviors and dynamical features in plasma dynamics -- a task often challenging for conventional methods. The model developed in this study is generalizable and can be extended broadly through transfer learning to address various magnetohydrodynamics (MHD) modes.

Bai, Zhe↗
DOE OSTI2025

Detecting Unclassified Electromagnetic Signals for Secure Wireless Communication Using Open Set Recognition

We developed multiple machine learning methods for the detection and classification of new wireless communication waveforms, which is critical for targeted attacks in wireless networks and electronic warfare. Our machine learning models are capable of dynamically detecting security threats in near real time through our advanced open set recognition (OSR) approach. This model has demonstrated significant improvements in the detection of unknown waveforms, thereby enhancing the security and reliability of mission critical communications. Our approach to detecting uncertain security threats is novel; we advanced OSR techniques by incorporating domain knowledge of wireless signals. Specifically, we combined time and frequency domain model features to enhance the model’s performance. Utilizing an OSR approach eliminates the need for training data to be distributed similarly to the deployment environment and removes the requirement for the training set to contains all possible threat classes. This is crucial because it is often infeasible to determine and characterize all potential security threats in advance. Our model were trained on simulated data, generated in partnership with the University at Albany, State of New York. The data set contained a diverse array of wireless signals, including those with additive white Gaussian noise and multipath signals, with and without line of sight. This comprehensive training set allowed us to optimize our models to detect unknown waveforms under various challenging scenarios, such as low signal-to-noise ratios. By training on various waveforms, varying signal-to-noise ratio, and different sample sizes under normal conditions, our models were fine tuned to perform effectively in challenging environments.

99 - GENERAL AND MISCELLANEOUS↗
DOE OSTI2022

Dynamic calibration of differential equations using machine learning, with application to turbulence models

We present a methodology for calibration of parametric ordinary and partial differential equation models, using off-the-shelf software for back-propagation in Neural Networks (NN). As a prototypical example, we consider calibration of a Reynolds-averaged Navier-Stokes (RANS) turbulence closure model, against ground truth data from direct numerical simulations (DNS) of two different turbulent flows. Numerical time integration is represented as a custom NN, where only the RANS model parameters are trainable. A loss function is defined to quantify the mismatch between the NN prediction and the ground truth over a predefined, finite time integration window. This loss function is then minimized using a gradient descent method utilizing the back-propagation algorithm. Furthermore, this dynamic approach to training is to be contrasted with a static approach, wherein a least square regression estimate for parameters is obtained in the limit of an infinitesimal time integration window. In a first test of static and dynamic approaches against ground truth data generated by the model, the former proves to be significantly faster and more accurate than the latter at recovering the parameters. When both calibration approaches are tested against DNS data, for which it is known that the model cannot achieve a perfect fit, the static approach yields a good prediction only for short times, while the dynamic approach results in physical and stable predictions over the entire integration window. After optimization of the dynamic approach for time step, spatial resolution, stability, and physics-based constraints, we obtain a 50% improvement of outcomes over those obtained from the existing, manually calibrated set of parameters, demonstrating the merits of this systematic and automated procedure.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Dynamic Parameter Estimation with Physics-based Neural Ordinary Differential Equations

Accurate estimation of dynamic parameters of gen-erators is crucial to building a reliable model for dynamical studies and reliable operation of the power system. This paper develops a physics-based neural ordinary differential equations (ODE) approach to learn the parameters of generator dynamic model using phasor measurement units (PMU) data. We design a physics-based neural network to represent the swing equations of the power system dynamics. A loss function is defined as the difference between dynamic simulation results from the physics-based neural networks and pseudo PMU measurements. The parameters of generator dynamic model are iteratively updated using the neural ODEs and the adjoint method. By exploiting the mini-batch scheme in neural ODE training, the parameter estimation performance is significantly improved. Numerical study results on a 3-machine 9-bus system show that the proposed algorithm outperforms state-of-the-art baseline method in both computation time and dynamic parameter estimation accuracy.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Linking Threat Agents to Targeted Organizations: A Pipeline for Enhanced Cybersecurity Risk Metrics

In this study, we present a methodology leveraging Large Language Models (LLMs) to transform Cybersecurity Threat Intelligence (CTI) narratives into actionable insights for individual organizations. Our approach automates the extraction of machine-readable adversary SKRAM (Skills, Knowledge, Resources, Authorities, and Motivation) attributes from open-source reports, extending LLM utility beyond typical interactions. This innovation enables precise, automated assessments of cybersecurity risks posed by various adversaries. Using a chain-of-thought and multi-shot prompting strategy, our methodology advances the automation of cybersecurity feature extraction for new machine-learning models that predict the risk of adversary targeting. This approach is refined using a substantial dataset of over 150 analyst-validated threat reports and synthetic organizational data from 900 companies. Here, by bootstrapping the training data with a rule-based heuristic over synthetic data, we have developed a high-accuracy machine-learning model that allows entities to dynamically prioritize threats and defensive actions.

Cyber Threat Intelligence↗
DOE OSTI2021

Arrested Phase Separation in Double-Exchange Models: Large-Scale Simulation Enabled by Machine Learning

In this work, we present large-scale dynamical simulations of electronic phase separation in the single-band double-exchange model based on deep-learning neural-network potentials trained from small-size exact diagonalization solutions. We uncover an intriguing correlation-induced freezing behavior as doped holes are segregated from half filled insulating background during equilibration. While the aggregation of holes is stabilized by the formation of ferromagnetic clusters through Hund’s coupling between charge carriers and local magnetic moments, this stabilization also creates confining potentials for holes when antiferromagnetic spin-spin correlation is well developed in the background. The dramatically reduced mobility of the self-trapped holes prematurely disrupts further growth of the ferromagnetic clusters, leading to an arrested phase separation. Implications of our findings for phase separation dynamics in materials that exhibit colossal magnetoresistance effect are discussed.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2023

Physics-informed machine-learning model of temperature evolution under solid phase processes

We model temperature dynamics during Shear Assisted Proccess Extrusion (ShAPE), a solid phase process that plasticizes feedstock with a rotating tool and subsequently extrudes it into a consolidated tube, rod, or wire. Control of temperature is critical during ShAPE processing to avoid liquefaction, ensure smooth extrusion, and develop desired material properties in the extruded products. Accurate modeling of the complicated thermo-mechanical feedbacks between process inputs, material temperature, and heat generation presents a significant barrier to predictive modeling and process design. In particular, connecting micro-structural scale mechanisms of heat generation to macro-scale predictions of temperature can become computationally intractable. In this work we use a neural network (NN) model of heat generation to bridge this gap, by combining it with a simplified model of the temperature dynamics due to conduction and convection to capture the macro scale evolution of temperature. We inform the construction of the NN heat generation model using crystal plasticity simulations at the micro-structural scale to model the effects of process inputs on generation of heat. We achieved close fits of the temperature dynamics model to a diverse experimental data-set. Further, the relationships learned by the NN model between process inputs and heat generation showed qualitative agreement with those predicted by crystal plasticity simulations.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Machine Learning for Anomaly Detection in Neural Network Security and SRF Cavities

This dissertation explores the development and deployment of machine learning approaches to address critical challenges in anomaly detection across two distinct domains: neural network security in federated learning settings and cavity behavior analysis in particle accelerator operations at Jefferson Lab in Newport News, Virginia. Anomaly detection identifies deviations from expected patterns, safeguarding systems in cybersecurity, industry, and research against malicious activities and failures. This dissertation demonstrates how our machine learning approaches enhance detection accuracy and efficiency in both neural network security and industrial applications. First, we investigate vulnerabilities in deep neural networks deployed in federated learning. Although federated learning preserves user privacy by training models locally, it remains vulnerable to backdoor attacks, in which malicious participants embed hidden triggers that induce targeted misbehavior. We propose a self-supervised contrastive learning framework to detect and mitigate such backdoor attacks. In our experiments, this method achieves higher detection accuracy and lower false positive rates than existing defenses, while operating without access to local model updates or original training data and thus preserving the privacy guarantees of the federated setting. Second, we address the operational reliability of superconducting radio-frequency (SRF) cavities at the Continuous Electron Beam Accelerator Facility (CEBAF). Our research leverages an unsupervised learning approach, combined with Principal Component Analysis (PCA) and k-means clustering, to identify anomalous behaviors in SRF cavities. Our method detects subtle anomalous behavior by analyzing SRF signal data. This knowledge allows for the early detection and resolution of potential faults, significantly improving the efficiency and reliability of operations. Third, we extend these insights to time-series anomaly detection more broadly. We design a contrastive-learning based model tailored to increasingly dynamic environments and academic research. This model improves detection accuracy in settings that require real-time monitoring and predictive maintenance. Our research underscores the broader applicability and impact of advanced machine learning techniques in anomaly detection. By extracting meaningful patterns from complex data, machine learning can significantly enhance security in distributed neural networks and improve the efficiency of particle accelerator operations. This dissertation serves as a stepping stone for future investigations into the vast possibilities of anomaly detection, inspiring further exploration and development of machine learning techniques in this field.

Ferguson, Hal [Old Dominion University]↗
DOE OSTI2023

Enhancing SRF cavity stability and minimizing detuning with data-driven resonance control based on dynamic mode decomposition

Effective resonance control of superconducting radio frequency (SRF) cavities is critical for large machines like LCLS-II, as failure to achieve proper control can result in increased RF power consumption, higher cryogenic heat loads, and increased costs. To address this challenge, we have developed a machine learning (ML) model based on the dynamic mode decomposition method to represent the forced cavity dynamics. Using this model, we designed a model predictive controller (MPC) and demonstrated through simulation that the MPC can effectively stabilize the amplitude and phase of SRF cavities using only a frequency actuator, even in the presence of multiple mechanical modes. The lightweight and explicit ML model makes the controller suitable for direct implementation on field-programmable gate arrays, unlocking the full potential of SRF linacs like LCLS-II, enabling higher beam power and energy, and also serving as an advanced motion controller for various applications, such as photon beamlines and storage rings.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2025

From sequence to protein structure and conformational dynamics with artificial intelligence/machine learning

The 2024 Nobel Prize in Chemistry was awarded in part for de novo protein structure prediction using AlphaFold2, an artificial intelligence/machine learning (AI/ML) model trained on vast amounts of sequence and three-dimensional structure data. AlphaFold2 and related models, including RoseTTAFold and ESMFold, employ specialized neural network architectures driven by attention mechanisms to infer relationships between sequence and structure. At a fundamental level, these AI/ML models operate on the long-standing hypothesis that the structure of a protein is determined by its amino acid sequence. More recently, AlphaFold2 has been adapted for the prediction of multiple protein conformations by subsampling multiple sequence alignments. Herein, we provide an overview of the deterministic relationship between sequence and structure, which was hypothesized over half a century ago with profound implications for the biological sciences ever since. We postulate that protein conformational dynamics are also determined, at least in part, by amino acid sequence and that this relationship may be leveraged for construction of AI/ML models dedicated to predicting protein conformational ensembles. Accordingly, we describe a conceptual model architecture, which may be trained on sequence data in combination with conformationally sensitive structural information, coming primarily from nuclear magnetic resonance (NMR) spectroscopy. Notwithstanding certain limitations in this context, NMR offers abundant structural heterogeneity conducive to conformational ensemble prediction. As NMR and other data continue to accumulate, sequence-informed prediction of protein structural dynamics with AI/ML has the potential to emerge as a transformative capability across the biological sciences.

Artificial intelligence↗