Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “learning dynamics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,765 records · Page 98
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Deep Kronecker neural networks: A general framework for neural networks with adaptive activation functions

Here we propose a new type of neural networks, Kronecker neural networks (KNNs), that form a general framework for neural networks with adaptive activation functions. KNNs employ the Kronecker product, which provides an efficient way of constructing a very wide network while keeping the number of parameters low. Our theoretical analysis reveals that under suitable conditions, KNNs induce a faster decay of the loss than that by the feed-forward networks. This is also empirically verified through a set of computational examples. Furthermore, under certain technical assumptions, we establish global convergence of gradient descent for KNNs. As a specific case, we propose the Rowdy activation function that is designed to get rid of any saturation region by injecting sinusoidal fluctuations, which include trainable parameters. The proposed Rowdy activation function can be employed in any neural network architecture like feed-forward neural networks, Recurrent neural networks, Convolutional neural networks etc. The effectiveness of KNNs with Rowdy activation is demonstrated through various computational experiments including function approximation using feed-forward neural networks, solution inference of partial differential equations using the physics-informed neural networks, and standard deep learning benchmark problems using convolutional and fully-connected neural networks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Low-index mesoscopic surface reconstructions of Au surfaces using Bayesian force fields

Metal surfaces have long been known to reconstruct, significantly influencing their structural and catalytic properties. Many key mechanistic aspects of these subtle transformations remain poorly understood due to limitations of previous simulation approaches. Using active learning of Bayesian machine-learned force fields trained from ab initio calculations, we enable large-scale molecular dynamics simulations to describe the thermodynamics and time evolution of the low-index mesoscopic surface reconstructions of Au (e.g., the Au(111)-‘Herringbone,’ Au(110)-(1 × 2)-‘Missing-Row,’ and Au(100)-‘Quasi-Hexagonal’ reconstructions). This capability yields direct atomistic understanding of the dynamic emergence of these surface states from their initial facets, providing previously inaccessible information such as nucleation kinetics and a complete mechanistic interpretation of reconstruction under the effects of strain and local deviations from the original stoichiometry. We successfully reproduce previous experimental observations of reconstructions on pristine surfaces and provide quantitative predictions of the emergence of spinodal decomposition and localized reconstruction in response to strain at non-ideal stoichiometries. A unified mechanistic explanation is presented of the kinetic and thermodynamic factors driving surface reconstruction. Furthermore, we study surface reconstructions on Au nanoparticles, where characteristic (111) and (100) reconstructions spontaneously appear on a variety of high-symmetry particle morphologies.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Deep learning workflow for the inverse design of molecules with specific optoelectronic properties

The inverse design of novel molecules with a desirable optoelectronic property requires consideration of the vast chemical spaces associated with varying chemical composition and molecular size. First principles-based property predictions have become increasingly helpful for assisting the selection of promising candidate chemical species for subsequent experimental validation. However, a brute-force computational screening of the entire chemical space is decidedly impossible. To alleviate the computational burden and accelerate rational molecular design, we here present an iterative deep learning workflow that combines (i) the density-functional tight-binding method for dynamic generation of property training data, (ii) a graph convolutional neural network surrogate model for rapid and reliable predictions of chemical and physical properties, and (iii) a masked language model. As proof of principle, we employ our workflow in the iterative generation of novel molecules with a target energy gap between the highest occupied molecular orbital (HOMO) and the lowest unoccupied molecular orbital (LUMO).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

Experiments and computations of microfluidic liquid–liquid flow patterns

We study two-phase liquid–liquid flow patterns in a 500 μm capillary microchannel for four biphasic systems: ethyl acetate/water, 2-pentanol/water, methyl isobutyl ketone/water, and heptane/water. Flow visualization experiments using laser induced fluorescence (LIF) reveal a total of 7 different flow patterns for all solvent pairs, namely slug flow, droplet flow, slug-droplet flow, parallel, annular, dispersed, and irregular flow. A map of different flow patterns was built to delineate the origin of their formation. We find conventional dimensionless groups are insufficient to uniquely identify the flow patterns. Computational fluid dynamics (CFD) modeling in OpenFOAM shows agreement with the experimental flow patterns for most of the two-phase flows. Principal component analysis reduces the dimensionality of potential descriptors of flow patterns and, unlike prior work using two dimensionless numbers, determines six important features that describe >95% of the variance of the experimental flow patterns. These include the total flow rate, the flow rate ratio between the two phases, the capillary and Ohnesorge numbers of the aqueous phase, and the Weber number and velocity of the organic phase. Lastly, we build a decision-tree model to further regress the data and identify the critical features and demonstrate an accuracy in predicting the flow patterns of up to 93%.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Bayesian differential programming for robust systems identification under uncertainty

This paper presents a machine learning framework for Bayesian systems identification from noisy, sparse and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of recent developments in differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling. This allows an efficient inference of the posterior distributions over plausible models with quantified uncertainty, while the use of sparsity-promoting priors enables the discovery of interpretable and parsimonious representations for the underlying latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed methods, including nonlinear oscillators, predator–prey systems and examples from systems biology. Taken together, our findings put forth a flexible and robust workflow for data-driven model discovery under uncertainty. All codes and data accompanying this article are available at https://bit.ly/34FOJMj .

Science & Technology - Other Topics↗
DOE OSTI2021

pnnl/FlexyAirDeepMPC

This is a trained machine learning model written in Pytorch for automated control of the FlexyAir (https://ocl.sk/hardware/flexyair/) dynamical system control educational device. This software is to be released to support our demonstration of our deepMPC methods on embedded hardware to control a real (albeit tutorial) physical system with control actuators.

Tuor, Aaron↗
DOE OSTI2024

Decision-making based on Markov decision process in integrated artificial reasoning framework—Part I: Theory

This paper presents a decision-making framework based on an integrated artificial reasoning framework and Markov decision process (MDP). The integrated artificial reasoning framework provides a physics-based approach that converts system information into state transition models, and the analysis result will be represented by the transition probabilities that can be used with an MDP to find a traceable and explainable optimal pathway. A dynamic Bayesian network (DBN) is well suited for representing the structure of an MDP. The causality information among process variables (or among subsystems) is mathematically represented in a DBN by the conditional probabilities of the node’s states provided different probabilities of the parent node’s states. To define node states in a physically understandable manner, we used multilevel flow modeling (MFM). An MFM follows the fundamental energy and mass conservation laws and supports the selection of process variables that represent the system of interest so that causal relations among process variables are properly captured. An MFM-based DBN supports developing state transition models in an MDP to capture the effect of process variables of system having physical relations. The operators of the target system can capture stochastic system dynamics as multiple subsystem state transitions based on their physical relations and uncertainties coming from component degradation or random failures. We analyzed a simplified exemplary system to illustrate an optimal operational policy using the suggested approach.

Markov decision process↗
DOE OSTI2013

Review of July 2013 Nuclear Security Insider Threat Exercise November 2013

This document is a review of the Nuclear Security Insider Threat Exercise which was hosted at ORNL in July 2013. Nuclear security culture and the insider threat are best learned through experience. Culture is inherently difficult to teach, and as such is best learned through modeled behaviors and learning exercise. This TTX, NSITE, is a tool that strives to aid students in learning what an effective (and ineffective) nuclear security culture might look like by simulating dynamic events that strengthen or weaken the nuclear security regime. The goals of NSITE are to stimulate complex thought and discussion and assist decision makers and management in determining the most effective policies and procedures for their country or facility.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2024

OPEN-Augmented Reality GUI for Bioenergy Crop Phenotyping and Precision Agriculture (Donald Danforth Plant Science Center Final Scientific Technical Report)

The project led by the Donald Danforth Plant Science Center, in collaboration with Arizona State University, George Washington University, and Saint Louis University, has made significant strides in advancing the phenotypic analysis of bioenergy crops through the development of an innovative AI processing pipeline. This initiative was primarily funded by ARPA-E, with additional cost-sharing provided by the participating institutions. The project successfully utilized a variety of sensors—3D scanners, thermal, RGB, and hyperspectral—to refine algorithms for data-driven trait signature identification and improve the classification and visualization of plant traits. The developed AI processing pipeline is capable of handling the complex, multidimensional data characteristic of dynamic agricultural environments. 1) Contributions to understanding: The research has advanced the field of plant phenomics by showcasing the synergistic use of various sensor data to enhance the precision of trait analysis in bioenergy crops. Through the integration of 3D scanners, thermal, RGB, and hyperspectral sensors, the project has developed robust data-driven trait signature algorithms and visualization techniques. These innovations have facilitated detailed monitoring and management of plant traits, providing vital insights into plant growth dynamics and stress responses. Further, the project has broadened our understanding of how machine learning can be effectively applied in multi-sensor environments to refine trait analysis. By leveraging diverse datasets, the research has not only improved the accuracy of phenotypic assessments but also established a versatile methodological framework that can be extended beyond agriculture to other fields requiring detailed phenotypic analysis. 2) Technical effectiveness and economic feasibility: The AI processing pipeline developed in this project demonstrated significant technical effectiveness, achieving high throughput analysis of extensive phenotypic data and meeting targeted accuracies. This system exemplified the capability of advanced machine learning technologies to efficiently manage and analyze large, complex datasets. Economically, the implementation of the project-developed pipelines may offer substantial cost savings across multiple sectors. It enhances data analysis processes and significantly reduces the need for manual data interpretation, thereby decreasing both the time and resources required. 3) Public benefit: The project has significantly broadened the scope of agricultural methodologies to enhance phenotypic analysis, with potential applications in various sectors beyond agriculture. Additionally, the initiative fostered an enriching educational and collaborative environment, significantly enhancing the technical skills of participants. It also made substantial contributions to the scientific community by providing open-access data sets and tools, encouraging ongoing research and development across various disciplines. Overall, the project not only met its scientific goals but also showcased the extensive utility of integrating advanced machine learning and sensor data analysis technologies. These advancements have proven instrumental in driving forward both theoretical research and practical applications, setting a strong foundation for future explorations and innovations in data-driven science.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Novel experimental probes of QCD in SIDIS and e + e - annihilation (Final Technical Report)

The research addressed with this award seeks to advance our understanding of the structure and dynamics underlying the properties of visible matter. In our current understanding the nucleons (protons and neutrons) are not fundamental but are comprised of quarks and gluons, which are collectively called partons. A static quark picture fails to explain the properties of the nucleons, such as their mass and intrinsic spin, which are thought to emerge dynamically from the quark-gluon interactions via the strong force. In this work, novel observables employing correlations of particles produced in the scattering of high energy electrons off protons at the CLAS12 experiment at Jefferson Lab were analyzed to probe quark-gluon interactions. The ultimate goal of this line of inquiry is to be able to describe the properties of protons and neutrons from first principles, similar to how studying the hydrogen atom has led to the formulation of the theory of Quantum Electrodynamics. Because quarks cannot be observed directly but only as part of more complex composite particles, a smaller, complimentary part of this work was the analysis of particle production in electron-positron annihilation at the Belle II experiment to understand the production of particles from initial quarks. This takes advantage of the fact that in e + e - annihilation the initial quark dynamics is known, unlike in the scattering of nucleons. Several new applications using Machine Learning algorithms for event tagging and reconstruction to support this program were developed as part of this award. In addition, we had a significant role in the development of the physics program for the future Electron-Ion Collider, which is a new collider to be build in the US within the next decade.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Transformer Neural Networks with Spatiotemporal Attention for Predictive Control and Optimization of Industrial Processes

In the context of real-time optimization and model predictive control of industrial systems, machine learning, and neural networks represent cutting-edge tools that hold promise for enhancing dynamic modeling. This work presents a novel transformer neural network architecture for real-time optimization and model predictive control. This network design includes a modified attention mechanism inspired by positional embedding attention from vision transformers and task-specific modifications to the input-output structure of the transformer’s decoder stack. Experiments were conducted using data from a 450 MW coal-fired power plant to evaluate this approach's effectiveness. The transformer neural network was compared with conventional recurrent models, including GRU and LSTM. The transformer exhibited a 6% increase in the R-squared (R2) value of predictions and an 83% reduction in mean squared error (MSE). Computation time was also reduced by 84% compared to conventional recurrent models.

Gallup, Ethan R.↗
NASA NTRS1992

Neural network error correction for solving coupled ordinary differential equations

A neural network is presented to learn errors generated by a numerical algorithm for solving coupled nonlinear differential equations. The method is based on using a neural network to correctly learn the error generated by, for example, Runge-Kutta on a model molecular dynamics (MD) problem. The neural network programs used in this study were developed by NASA. Comparisons are made for training the neural network using backpropagation and a new method which was found to converge with fewer iterations. The neural net programs, the MD model and the calculations are discussed.

Shelton, R. O.↗
NASA NTRS1999

A Strategy for Integrating a Large Finite Element Model Using MSC NASTRAN/PATRAN: X-33 Lessons Learned

The X-33 vehicle is an advanced technology demonstrator sponsored by NASA. For the past 3 years the Structural Dynamics and Loads Branch of NASA's Marshall Space Flight Center has had the task of integrating the X-33 vehicle structural finite element model. In that time, five versions of the integrated vehicle model have been produced and a strategy has evolved that would benefit anyone given the task of integrating structural finite element models that have been generated by various modelers and companies. The strategy that has been presented here consists of six decisions that need to be made: purpose of models, units, common materials list, model numbering, interface control, and archive format. This strategy has been proven and expanded from experience on the X-33 vehicle.

McGhee, D. S.↗
NASA NTRS2000

A Strategy for Integrating a Large Finite Element Model: X-33 Lessons Learned

The X-33 vehicle is an advanced technology demonstrator sponsored by NASA. For the past three years the Structural Dynamics & Loads Group of NASA's Marshall Space Flight Center has had the task of integrating the X-33 vehicle structural finite element model. In that time, five versions of the integrated vehicle model have been produced and a strategy has evolved that would benefit anyone given the task of integrating structural finite element models that have been generated by various modelers and companies. The strategy that has been presented here consists of six decisions that need to be made. These six decisions are: purpose of model, units, common material list, model numbering, interface control, and archive format. This strategy has been proved and expanded from experience on the X-33 vehicle.

McGhee, David S.↗
NASA NTRS2003

Learning and planning for Mars Rover science

In this paper, we describe the OASIS system, which provides autonomous capabilities for dynamically pursuing these science-collection opportunities during long-range rover traverses.

data analysis planning autonomous rover operations↗
NASA NTRS2012

Application of the Semi-Empirical Force-Limiting Approach for the CoNNeCT SCAN Testbed

The semi-empirical force-limiting vibration method was developed and implemented for payload testing to limit the structural impedance mismatch (high force) that occurs during shaker vibration testing. The method has since been extended for use in analytical models. The Space Communications and Navigation Testbed (SCAN Testbed), known at NASA as, the Communications, Navigation, and Networking re-Configurable Testbed (CoNNeCT), project utilized force-limiting testing and analysis following the semi-empirical approach. This paper presents the steps in performing a force-limiting analysis and then compares the results to test data recovered during the CoNNeCT force-limiting random vibration qualification test that took place at NASA Glenn Research Center (GRC) in the Structural Dynamics Laboratory (SDL) December 19, 2010 to January 7, 2011. A compilation of lessons learned and considerations for future force-limiting tests is also included.

Staab, Lucas D.↗
NASA NTRS2024

Integrated Systems Engineering, Safety, Reliability and Risk Management – Minimizing Black Swan Events

This paper examines key barriers that can possibly inhibit safe and reliable mission execution and, in the worst case, result in loss of human life due to many unknown contributory factors that can lead to Black Swan events. Some of the representative contributory factors include decision errors, overconfidence and a host of common causes including cultural and human factors. Decisions are always easy to criticize in hindsight when more information is available after a major accident. Depending on the type and complexity of the project and/or mission, the catastrophic risks of drifting into failure can be alleviated by implementing uniquely and strategically tailored Integrated-System-of-Systems, dynamic, risk-informed decision management processes. This paper presents some of the lessons learned from James Webb Space Telescope (JWST), NASA’s Human Space Flight program, and industry that provide motivation to organizations working on mega-complex missions to prudently accomplish targeted mission success. These lessons are important for future human Lunar, Mars and Beyond missions planned to be pursued by NASA through a public-private partnership using nimble but effective safety-conscious, proven sound engineering practices including implementation of integrated risk mitigation practices.

SLS↗