Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Learning Theory”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 847 records · Page 47

Proceedings of the Second Joint Technology Workshop on Neural Networks and Fuzzy Logic, volume 1

Documented here are papers presented at the Neural Networks and Fuzzy Logic Workshop sponsored by NASA and the University of Houston, Clear Lake. The workshop was held April 11 to 13 at the Johnson Space Flight Center. Technical topics addressed included adaptive systems, learning algorithms, network architectures, vision, robotics, neurobiological connections, speech recognition and synthesis, fuzzy set theory and application, control and dynamics processing, space applications, fuzzy logic and neural network computers, approximate reasoning, and multiobject decision making.

Lea, Robert N.↗

Sparse distributed memory

Sparse distributed memory was proposed be Pentti Kanerva as a realizable architecture that could store large patterns and retrieve them based on partial matches with patterns representing current sensory inputs. This memory exhibits behaviors, both in theory and in experiment, that resemble those previously unapproached by machines - e.g., rapid recognition of faces or odors, discovery of new connections between seemingly unrelated ideas, continuation of a sequence of events when given a cue from the middle, knowing that one doesn't know, or getting stuck with an answer on the tip of one's tongue. These behaviors are now within reach of machines that can be incorporated into the computing systems of robots capable of seeing, talking, and manipulating. Kanerva's theory is a break with the Western rationalistic tradition, allowing a new interpretation of learning and cognition that respects biology and the mysteries of individual human beings.

Denning, Peter J.↗

Proceedings of the Third International Workshop on Neural Networks and Fuzzy Logic, volume 2

Papers presented at the Neural Networks and Fuzzy Logic Workshop sponsored by the National Aeronautics and Space Administration and cosponsored by the University of Houston, Clear Lake, held 1-3 Jun. 1992 at the Lyndon B. Johnson Space Center in Houston, Texas are included. During the three days approximately 50 papers were presented. Technical topics addressed included adaptive systems; learning algorithms; network architectures; vision; robotics; neurobiological connections; speech recognition and synthesis; fuzzy set theory and application, control and dynamics processing; space applications; fuzzy logic and neural network computers; approximate reasoning; and multiobject decision making.

Culbert, Christopher J.↗

Proceedings of the Third International Workshop on Neural Networks and Fuzzy Logic, volume 1

Documented here are papers presented at the Neural Networks and Fuzzy Logic Workshop sponsored by the National Aeronautics and Space Administration and cosponsored by the University of Houston, Clear Lake. The workshop was held June 1-3, 1992 at the Lyndon B. Johnson Space Center in Houston, Texas. During the three days approximately 50 papers were presented. Technical topics addressed included adaptive systems; learning algorithms; network architectures; vision; robotics; neurobiological connections; speech recognition and synthesis; fuzzy set theory and application, control, and dynamics processing; space applications; fuzzy logic and neural network computers; approximate reasoning; and multiobject decision making.

Culbert, Christopher J.↗

What Might We Learn About Magnetospheric Substorms at the Earth from the MESSENGER Measurements at Mercury?

Satellite observations at the Earth, supported by theory and modeling, have established a close connection between the episodes of intense magnetospheric convection termed substorms and the occurrence of magnetic reconnection. Magnetic reconnection at the dayside magnetopause results in strong energy input to the magnetosphere. This energy can either be stored or used immediately to power the magnetospheric convection that produces the phenomena that collectively define the 'substorm.' However, many aspects of magnetic reconnection and the dynamic response of the coupled solar wind - magnetosphere - ionosphere system at the Earth during substorms remain poorly understood. For example, the rate of magnetic reconnection is thought to be proportional to the local Alfven speed, but the limited range of changes in this solar wind parameter at 1 AU have made it difficult to detect its influence over energy input to the Earth's magnetosphere. In addition, the electrical conductance of the ionosphere and how it changes in response to auroral charged particle precipitation are hypothesized to play a critical role in the development of substorms, but the nature of this electrodynamic interaction remain difficult to deduce from Earth observations alone. The amount of energy the terrestrial magnetosphere can store in its tail, the duration of the storage, and the trigger(s) for its dissipation are all thought to be determined by not only the microphysics of the cross-tail current layer, but also the properties of the coupled magnetosphere - ionosphere system. Again, the separation of microphysics effects from system response has proved very difficult using measurements taken only at the Earth. If MESSENGER'S charged particle and magnetic field measurements confirm the occurrence of terrestrial-style substorms in Mercury's miniature magnetosphere, then it may be possible to determine how magnetospheric convection, field-aligned currents, charged particle acceleration, reconnection, and tail energy storage are influenced by the intense magnetic reconnection expected to be associated with solar wind conditions at 0.3 - 0.5 AU from the Sun and the simplified electrodynamic feed-back anticipated for a planet lacking an ionosphere. MESSENGER observations from its 14 January 2008 Mercury flyby relevant to the occurrence of magnetic reconnection and substorms at Mercury will be discussed.

Slavin, James A.↗

Representations and strategies for transferable machine learning improve model performance in chemical discovery

Strategies for machine-learning (ML)-accelerated discovery that are general across material composition spaces are essential, but demonstrations of ML have been primarily limited to narrow composition variations. By addressing the scarcity of data in promising regions of chemical space for challenging targets such as open-shell transition-metal complexes, general representations and transferable ML models that leverage known relationships in existing data will accelerate discovery. Over a large set (~1000) of isovalent transition-metal complexes, we quantify evident relationships for different properties (i.e., spin-splitting and ligand dissociation) between rows of the Periodic Table (i.e., 3d/4d metals and 2p/3p ligands). We demonstrate an extension to the graph-based revised autocorrelation (RAC) representation (i.e., eRAC) that incorporates the group number alongside the nuclear charge heuristic that otherwise overestimates dissimilarity of isovalent complexes. To address the common challenge of discovery in a new space where data are limited, we introduce a transfer learning approach in which we seed models trained on a large amount of data from one row of the Periodic Table with a small number of data points from the additional row. We demonstrate the synergistic value of the eRACs alongside this transfer learning strategy to consistently improve model performance. Analysis of these models highlights how the approach succeeds by reordering the distances between complexes to be more consistent with the Periodic Table, a property we expect to be broadly useful for other material domains.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Integrated machine learning-molecular dynamics framework for electrolyte property prediction

Electrochemical stability windows determine the operating range of battery electrolytes, yet accurate prediction remains challenging because stability emerges from statistical ensembles of local solvation environments rather than single ground-state molecular structures. Traditional density functional theory calculations on energy-minimized clusters cannot capture the thermal variations in local coordination environments and geometries that govern decomposition, while SMILES-based machine learning methods lack explicit representation of three-dimensional solvation structure and ion pairing. Here, we introduce a structure-aware machine learning framework that predicts frontier orbital energies (HOMO and LUMO) directly from molecular dynamics-sampled solvation configurations, achieving sub-0.6 eV accuracy at computational costs 3–4 orders of magnitude lower than first-principles methods. Across twelve representative battery electrolytes, we demonstrate that solvent-separated and contact ion pairs exhibit strong size- and local chemistry dependent electronic stability, with variations in coordination shifts of HOMO or LUMO level by 2–3 eV, and that extended solvation structure and partially desolvated environment further modulate stability by up to 3 eV. By encoding the statistical nature of electrochemical failure through ensemble sampling of explicit solvation geometries, our approach enables high-throughput screening and rational design of next-generation battery electrolytes with mechanistic understanding of structure–property relationships.

Energy - Storage↗

Machine Learning Accelerated First-Principles Study of the Hydrodeoxygenation of Propanoic Acid

The complex reaction network of catalytic biomass conversions often involves hundreds of surface intermediates and thousands of reaction steps, greatly hindering the rational design of metal catalysts for these conversions. Here, we present a framework of machine learning (ML)-accelerated first-principles studies for the hydrodeoxygenation (HDO) of propanoic acid over transition metal surfaces. The microkinetic model (MKM) is initially parametrized by ML-predicted energies and iteratively improved by identifying the rate-determining species and steps (RDS), computing their energies by density functional theory (DFT), and reparameterizing the MKM until all the RDS are computed by DFT. The Gaussian process (GP) model performs significantly better than the linear ridge regression model for predicting both the adsorption free energies and transition state free energies. Parameterized with energies from the GP model, only 5–20% of the full reaction network has to be computed by DFT for the MKM to possess DFT-level accuracy for the TOF and dominant reaction pathway. While the linear ridge regression model performs worse than the GP model, its performance is greatly improved when only transition states are predicted by the regression model and adsorption energies are computed by DFT. Overall, we find that a high accuracy in adsorption free energies is more important for a reliable MKM than a high accuracy in TS free energies. Lastly, based on the GP model with GOH and GCHCHCO as catalyst descriptors, we build two-dimensional volcano plots in activity and selectivity that can help design promising alloy catalysts for HDO reactions of organic acids.

adsorption↗

Multi-fidelity information fusion with concatenated neural networks

Recently, computational modeling has shifted towards the use of statistical inference, deep learning, and other data-driven modeling frameworks. Although this shift in modeling holds promise in many applications like design optimization and real-time control by lowering the computational burden, training deep learning models needs a huge amount of data. This big data is not always available for scientific problems and leads to poorly generalizable data-driven models. This gap can be furnished by leveraging information from physics-based models. Exploiting prior knowledge about the problem at hand, this study puts forth a physics-guided machine learning (PGML) approach to build more tailored, effective, and efficient surrogate models. For our analysis, without losing its generalizability and modularity, we focus on the development of predictive models for laminar and turbulent boundary layer flows. In particular, we combine the self-similarity solution and power-law velocity profile (low-fidelity models) with the noisy data obtained either from experiments or computational fluid dynamics simulations (high-fidelity models) through a concatenated neural network. We illustrate how the knowledge from these simplified models results in reducing uncertainties associated with deep learning models applied to boundary layer flow prediction problems. The proposed multi-fidelity information fusion framework produces physically consistent models that attempt to achieve better generalization than data-driven models obtained purely based on data. While we demonstrate our framework for a problem relevant to fluid mechanics, its workflow and principles can be adopted for many scientific problems where empirical, analytical, or simplified models are prevalent. In line with grand demands in novel PGML principles, this work builds a bridge between extensive physics-based theories and data-driven modeling paradigms and paves the way for using hybrid physics and machine learning modeling approaches for next-generation digital twin technologies.

42 ENGINEERING↗

Accurate and efficient parameterization of an atomic cluster expansion (ACE) potential for ammonia under extreme conditions

We present a machine learning interatomic potential for ammonia designed to capture its complex multiphase behavior, including both molecular and superionic phases. The potential is based on the atomic cluster expansion (ACE) formulation and has been parameterized to facilitate high-fidelity molecular dynamics simulations of ammonia under extreme conditions, for pressures up to 100 GPa and for temperatures above 500 K and up to 6000 K. A diverse range of configurations was generated through high-quality ab initio molecular dynamics simulations, covering insulating and superionic ice phases, liquid ammonia, molecular nitrogen (N 2 ) and hydrogen (H 2 ), and metastable compounds that form upon dissociation, including $NH^{+}_{4}$, $H^{+}_{3}$, N 2 H 4 , and N 3 H. We demonstrate that the ammonia ACE potential accurately reproduces experimental and density functional theory predicted isotherms and Hugoniots. Crucially, the potential is able to capture the intricate phase behavior of ammonia, including the transition from insulating molecular fluid to the superionic phase. This work provides a robust interatomic potential that can be used for large-scale, accurate simulations of ammonia under extreme thermodynamic conditions, offering a powerful tool for investigating its behavior in various phases and applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Cyber risk assessment and investment optimization using game theory and ML-based anomaly detection and mitigation for wide-area control in smart grids

The electric power grid is increasingly becoming susceptible to cyber attacks that exploit vulnerabilities in the smart grid control, information, and physical layers. Successful cyber attacks can have catastrophic impacts on the social and economic well-being of any nation all over the globe. It has, thus, become imperative to secure the smart grid against such adversarial actions to ensure stable, secure, and reliable operation of the grid. The existing research and industry practices prove to be inadequate in terms of providing pragmatic and effective defense methodologies and measures for long-term cybersecurity planning and real-time cybersecurity for grid operation. For example, existing works lack models that incorporate uncertain behavior of cyber-attackers and pragmatic defense measures for cyber risk assessment and cybersecurity investment optimization which often provide unreliable and strictly qualitative solutions to these problems. At the same time, with the growing number of cyber incidents in the grid, there still exists a need to develop attack-resilient algorithms for wide-area monitoring, protection, and control (WAMPAC) applications like the wide-area voltage control systems (WAVCS) for Flexible AC Transmissions Systems (FACTS) that lack in scalable and feasible solutions from the cybersecurity perspective. This dissertation proposes novel models and methodologies for: (1) Cybersecurity planning, and (2) Cybersecurity for system operation. The cybersecurity planning is achieved through cyber risk assessment and cybersecurity resource investment optimization for long-term cybersecurity of the grid using game theory and attack-defense trees. Cybersecurity for system operation consists of development of cyber anomaly detection and mitigation algorithms for flexible AC transmission system (FACTS) controller-based wide-area voltage control systems (WAVCS) using machine learning (ML), and software defined networking-based moving target defense network routing for achieving real-time cyber-physical security for grid operations. This is followed by hardware-in-the-loop (HIL) implementation and evaluation of these attack prevention, detection, and mitigation algorithms and methodologies showcasing their feasibility in a close to real-world environment. For cybersecurity planning, a novel approach involving a combination of game theory and attack defense trees (ADT) for optimal cybersecurity resource allocation in the smart grid is proposed. This methodology involves modeling of the cyber-physical smart grid substations as ADTs, defining attacker costs, defense costs, and attack probabilities for attack access points. Using game theoretical formulation, optimal defense strategies for the defender of the system to invest cybersecurity resources in the grid are obtained. Additionally, a game-theoretic framework is developed for quantitative cyber-physical risk assessment of the grid under a dynamically changing cyber threat space and uncertain behavior of cyber attackers which is further used to optimize investments in the smart grid's cybersecurity resources. The attacker, defender, and the smart grid system are modeled while incorporating attacker-stochasticity and federal guidelines for smart grid cybersecurity. This allows quantification of threat, vulnerabilities, and attack impact of the grid for quantitative risk assessment. The defender's budget to invest in the security resources in the grid is optimized based on the strategies leading to minimum system risk. The evaluation of the proposed solutions highlight the feasibility for practical implementation of these methodologies and algorithms in the smart grid, while taking the federal requirements and guidelines for smart grid security into consideration. For achieving cybersecurity for system operation, attack prevention, detection, and mitigation algorithms and methodologies are developed specifically for FACTS-based WAVCS. Anomaly detection and mitigation in the WAVCS are achieved using algorithms based on machine learning which involves offline training and testing of ML models with CPS datasets incorporating physics-based features that allow accurate distinction between system faults and cyber attacks. For attack prevention, a methodology based on software defined network (SDN)-based moving target defense (MTD) network routing is proposed that enables prevention of Denial of Service (DoS) type attacks on the smart grid communication system. Subsequently, these methodologies and algorithms are implemented and evaluated on an HIL testbed that allows for real-time attack prevention, detection, and mitigation of emulated cyber attacks on the WAVCS in a close to real-world environment. The results show highly accurate and efficient performance of the implemented algorithms and methodologies with the smart grid system operating within the NERC's system operation limits even in the presence of DoS and data integrity cyber attacks. This work opens up future research opportunities in other directions such as (1) Expanding cybersecurity planning methodologies to real-time cyber contingency analysis with different game formulations; and (2) Applying the cybersecurity for system operation algorithms to broader categories of wide-area control applications.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Quantum Optics, Diffraction Theory, and Elementary Particle Physics

Physical optics has expanded greatly in recent years. Though it remains part of the ancestry of elementary particle physics, there are once again lessons to be learned from it. I shall discuss several of these, including some that have emerged at CERN and Brookhaven.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Machine learning interatomic potential for silicon-nitride (Si 3 N 4 ) by active learning

Silicon nitride (Si 3 N 4 ) is an extensively used material in the automotive, aerospace, and semiconductor industries. However, its widespread use is in contrast to the scarce availability of reliable interatomic potentials that can be employed to study various aspects of this material on an atomistic scale, particularly its amorphous phase. In this work, we developed a machine learning interatomic potential, using an efficient active learning technique, combined with the Gaussian approximation potential (GAP) method. Our strategy is based on using an inexpensive empirical potential to generate an initial dataset of atomic configurations, for which energies and forces were recalculated with density functional theory (DFT); thereafter, a GAP was trained on these data and an iterative re-training algorithm was used to improve it by learning on-the-fly. When compared to DFT, our potential yielded a mean absolute error of 8 meV/atom in energy calculations for a variety of liquid and amorphous structures and a speed-up of molecular dynamics simulations by 3–4 orders of magnitude, while achieving a first-rate agreement with experimental results. Our potential is publicly available in an open-access repository.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Application of machine learning to discover new intermetallic catalysts for the hydrogen evolution and the oxygen reduction reactions

The adsorption energies for hydrogen, oxygen, and hydroxyl were calculated by means of density functional theory on the lowest energy surface of 24 pure metals and 332 binary intermetallic compounds with stoichiometries AB, A 2 B, and A 3 B taking into account the effect of biaxial elastic strains. This information was used to train two random forest regression models, one for the hydrogen adsorption and another for the oxygen and hydroxyl adsorption, based on 9 descriptors that characterized the geometrical and chemical features of the adsorption site as well as the applied strain. All the descriptors for each compound in the models could be obtained from physico-chemical databases. The random forest models were used to predict the adsorption energy for hydrogen, oxygen, and hydroxyl of ≈2700 binary intermetallic compounds with stoichiometries AB, A 2 B, and A 3 B made of metallic elements, excluding those that were environmentally hazardous, radioactive, or toxic. This information was used to search for potential good catalysts for the HER and ORR from the criteria that their adsorption energy for H and O/OH, respectively, should be close to that of Pt. Further, this investigation shows that the suitably trained machine learning models can predict adsorption energies with an accuracy not far away from density functional theory calculations with minimum computational cost from descriptors that are readily available in physico-chemical databases for any compound. Moreover, the strategy presented in this paper can be easily extended to other compounds and catalytic reactions, and is expected to foster the use of ML methods in catalysis.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Structure-aware Initialization via Numerical Continuation and Informed Priors

Scientific machine learning (SciML) often operates in ill-conditioned, weakly identifiable regimes due to limited data or indirect observations. In such settings, optimization and inference are highly sensitive to the starting point, making initialization--often under-reported--a consequential degree of freedom. Random initialization is not a neutral default as it induces an implicit prior over candidate solutions and can systematically bias the result, producing large run-to-run variability. Here, we formalize this view by treating initialization as a hidden confounder in SciML and develop a unifying theory for structure-aware initialization via numerical continuation, constructing warm starts from related problem instances. Across representative tasks, including physics-informed neural networks, maximum likelihood estimation, and variational inference, warm starts have been shown to consistently reduce optimization effort and improve reliability.

Data integrity↗

Implementation Challenges for Multivariable Control: What You Did Not Learn in School

Multivariable control allows controller designs that can provide decoupled command tracking and robust performance in the presence of modeling uncertainties. Although the last two decades have seen extensive development of multivariable control theory and example applications to complex systems in software/hardware simulations, there are no production flying systems aircraft or spacecraft, that use multivariable control. This is because of the tremendous challenges associated with implementation of such multivariable control designs. Unfortunately, the curriculum in schools does not provide sufficient time to be able to provide an exposure to the students in such implementation challenges. The objective of this paper is to share the lessons learned by a practitioner of multivariable control in the process of applying some of the modern control theory to the Integrated Flight Propulsion Control (IFPC) design for an advanced Short Take-Off Vertical Landing (STOVL) aircraft simulation.

Garg, Sanjay↗

Searching for Physics Beyond the Standard Model: Strongly-Coupled Field Theories at the Intensity and Energy Frontiers

This proposal is to develop the software and algorithmic infrastructure needed for the numerical study of quantum chromodynamics (QCD), and of theories that have been proposed to describe physics beyond the Standard Model (BSM) of high energy physics, on current and future computers. This infrastructure will enable users (1) to improve the accuracy of QCD calculations to the point where they no longer limit what can be learned from high-precision experiments that seek to test the Standard Model, and (2) to determine the predictions of BSM theories in order to understand which of them are consistent with the data that will soon be available from the LHC. Work will include the extension and optimizations of community codes for the next generation of leadership class computers, the IBM Blue Gene/Q and the Cray XE/XK, and for the dedicated hardware funded for our field by the Department of Energy. Members of our collaboration at Brookhaven National Laboratory and Columbia University worked on the design of the Blue Gene/Q, and have begun to develop software for it. Under this grant we will build upon their experience to produce high-efficiency production codes for this machine. Cray XE/XK computers with many thousands of GPU accelerators will soon be available, and the dedicated commodity clusters we obtain with DOE funding include growing numbers of GPUs. We will work with our partners in NVIDIA's Emerging Technology group to scale our existing software to thousands of GPUs, and to produce highly efficient production codes for these machines. Work under this grant will also include the development of new algorithms for the effective use of heterogeneous computers, and their integration into our codes. It will include improvements of Krylov solvers and the development of new multigrid methods in collaboration with members of the FASTMath SciDAC Institute, using their HYPRE framework, as well as work on improved symplectic integrators.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Computational catalyst discovery: Active classification through myopic multiscale sampling

We report the recent boom in computational chemistry has enabled several projects aimed at discovering useful materials or catalysts. We acknowledge and address two recurring issues in the field of computational catalyst discovery. First, calculating macro-scale catalyst properties is not straightforward when using ensembles of atomic-scale calculations [e.g., density functional theory (DFT)]. We attempt to address this issue by creating a multi-scale model that estimates bulk catalyst activity using adsorption energy predictions from both DFT and machine learning models. The second issue is that many catalyst discovery efforts seek to optimize catalyst properties, but optimization is an inherently exploitative objective that is in tension with the explorative nature of early-stage discovery projects. In other words, why invest so much time finding a “best” catalyst when it is likely to fail for some other, unforeseen problem? We address this issue by relaxing the catalyst discovery goal into a classification problem: “What is the set of catalysts that is worth testing experimentally?” Here, we present a catalyst discovery method called myopic multiscale sampling, which combines multiscale modeling with automated selection of DFT calculations. It is an active classification strategy that seeks to classify catalysts as “worth investigating” or “not worth investigating” experimentally. Our results show an ~7–16 times speedup in catalyst classification relative to random sampling. These results were based on offline simulations of our algorithm on two different datasets: a larger, synthesized dataset and a smaller, real dataset.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗