Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “empirical machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Synergy of semiempirical models and machine learning in computational chemistry

Catalyzed by enormous success in the industrial sector, many research programs have been exploring data-driven, machine learning approaches. Performance can be poor when the model is extrapolated to new regions of chemical space, e.g., new bonding types, new many-body interactions. Another important limitation is the spatial locality assumption in model architecture, and this limitation cannot be overcome with larger or more diverse datasets. The outlined challenges are primarily associated with the lack of electronic structure information in surrogate models such as interatomic potentials. Given the fast development of machine learning and computational chemistry methods, we expect some limitations of surrogate models to be addressed in the near future; nevertheless spatial locality assumption will likely remain a limiting factor for their transferability. Here, we suggest focusing on an equally important effort—design of physics-informed models that leverage the domain knowledge and employ machine learning only as a corrective tool. In the context of material science, we will focus on semi-empirical quantum mechanics, using machine learning to predict corrections to the reduced-order Hamiltonian model parameters. The resulting models are broadly applicable, retain the speed of semiempirical chemistry, and frequently achieve accuracy on par with much more expensive ab initio calculations. These early results indicate that future work, in which machine learning and quantum chemistry methods are developed jointly, may provide the best of all worlds for chemistry applications that demand both high accuracy and high numerical efficiency.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Machine Learned Hückel Theory: Interfacing Physics and Deep Neural Networks

The Hückel Hamiltonian is an incredibly simple tight-binding model known for its ability to capture qualitative physics phenomena arising from electron interactions in molecules and materials. Part of its simplicity arises from using only two types of empirically fit physics-motivated parameters: the first describes the orbital energies on each atom and the second describes electronic interactions and bonding between atoms. By replacing these empirical parameters with machine-learned dynamic values, we vastly increase the accuracy of the extended Hückel model. The dynamic values are generated with a deep neural network, which is trained to reproduce orbital energies and densities derived from density functional theory. The resulting model retains interpretability, while the deep neural network parameterization is smooth and accurate and reproduces insightful features of the original empirical parameterization. Altogether, this work shows the promise of utilizing machine learning to formulate simple, accurate, and dynamically parameterized physics models.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Inexact Newton-CG algorithms with complexity guarantees

Abstract We consider variants of a recently developed Newton-CG algorithm for nonconvex problems (Royer, C. W. & Wright, S. J. (2018) Complexity analysis of second-order line-search algorithms for smooth nonconvex optimization. SIAM J. Optim., 28, 1448–1477) in which inexact estimates of the gradient and the Hessian information are used for various steps. Under certain conditions on the inexactness measures, we derive iteration complexity bounds for achieving $\epsilon $-approximate second-order optimality that match best-known lower bounds. Our inexactness condition on the gradient is adaptive, allowing for crude accuracy in regions with large gradients. We describe two variants of our approach, one in which the step size along the computed search direction is chosen adaptively, and another in which the step size is pre-defined. To obtain second-order optimality, our algorithms will make use of a negative curvature direction on some steps. These directions can be obtained, with high probability, using the randomized Lanczos algorithm. In this sense, all of our results hold with high probability over the run of the algorithm. We evaluate the performance of our proposed algorithms empirically on several machine learning models. Our approach is a first attempt to introduce inexact Hessian and/or gradient information into the Newton-CG algorithm of Royer & Wright (2018, Complexity analysis of second-order line-search algorithms for smooth nonconvex optimization. SIAM J. Optim., 28, 1448–1477).

Mathematics↗
DOE OSTI2025

Future Building Archetypes for Los Angeles (2100 Projection)

This dataset (Data.zip) includes empirical and machine learning-generated building information for the Los Angeles urban region. The MAv1_LA.csv file provides the baseline 2015 building data while Final_IECC_LO_2100_GAN.csv represents generative adversarial network-projected urban morphologies for the year 2100. Building archetypes were created for both datasets (Basecase_LA_Archetype.csv and LA_Simulation_2100_GAN_Archetype.csv) using footprint area as the key aggregation variable. More details about the dataset are provided in the attached readme file (README_LA_Archetype_MAv1.txt)

AutoBEM↗
DOE OSTI2025

Machine learning models for volumetric swelling in uranium nitride

Machine learning methods are applied to predict the volumetric swelling rate of the nuclear fuel uranium nitride (UN) over various temperatures, irradiation conditions, and power densities. Both kernel-based methods and symbolic regression models for UN swelling are developed and compared with multiple experimental datasets. We find that the UN pellet geometry and dimensions must be taken into account to accurately model swelling behavior. Strong agreement is observed between the developed machine learning models and the data. The predictive error generated by the machine learning models improves on empirical models taken from the literature. Sensitivity analysis is performed to determine which properties such as temperature, burnup, and power density, are most important in the swelling process. We find that machine learning can be used to quickly develop accurate swelling models for nuclear materials. In conclusion, the presented results illustrate the potential of machine learning to determine volumetric swelling in UN.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

Empirical relationships between environmental factors and soil organic carbon produce comparable prediction accuracy as the Machine Learning

Accurate representation of environmental controllers of soil organic carbon (SOC) stocks in Earth System Model (ESM) land models could reduce uncertainties in future carbon-climate feedback projections. Using empirical relationships between environmental factors and SOC stocks to evaluate land models can help modelers understand prediction biases beyond what can be achieved with the observed SOC stocks alone. In this study, we used 31 observed environmental factors, field SOC observations (n = 6,213) from the continental US, and two Machine Learning approaches [Random Forest (RF) and Generalized Additive Modeling (GAM)] to (1) select important environmental predictors of SOC stocks, (2) derive empirical relationships between environmental factors and SOC stocks, and (3) use the derived relationships to predict SOC stocks and compare the prediction accuracy of simpler model developed with the machine learning predictions. Out of the 31 environmental factors we investigated, 12 were identified as important predictors of SOC stocks by the RF approach. In contrast, the GAM approach identified six (of those 12) environmental factors as important controllers of SOC stocks: potential evapotranspiration, normalized difference vegetation index, soil drainage condition, precipitation, elevation, and net primary productivity. The GAM approach showed minimal SOC predictive importance of the remaining six environmental factors identified by the RF approach. Our derived empirical relations produced comparable prediction accuracy as the GAM and RF approach using only a subset of environmental factors. The empirical relationships we derived using the GAM approach can serve as important benchmarks to evaluate environmental control representations of SOC stocks in ESMs, which could reduce uncertainty in predicting future carbon-climate feedbacks.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Quantum circuit fidelity estimation using machine learning

The computational power of real-world quantum computers is limited by errors. When using quantum computers to perform algorithms which cannot be efficiently simulated classically, it is important to quantify the accuracy with which the computation has been performed. In this work, we introduce a machine learning-based technique to estimate the fidelity between the state produced by a noisy quantum circuit and the target state corresponding to ideal noise-free computation. Our machine learning model is trained in a supervised manner, using smaller or simpler circuits for which the fidelity can be estimated using other techniques like direct fidelity estimation and quantum state tomography. Here we demonstrate that, for simulated random quantum circuits with a realistic noise model, the trained model can predict the fidelities of more complicated circuits for which such methods are infeasible. In particular, we show that the trained model may make predictions for circuits with higher degrees of entanglement than were available in the training set and that the model may make predictions for non-Clifford circuits even when the training set included only Clifford-reducible circuits. This empirical demonstration suggests classical machine learning may be useful for making predictions about beyond-classical quantum circuits for some non-trivial problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Predictability and empirical dynamics of fisheries time series in the North Pacific

Previous studies have documented a strong relationship between marine ecosystems and large-scale modes of sea surface height (SSH) and sea surface temperature (SST) variability in the North Pacific such as the Pacific Decadal Oscillation and the North Pacific Gyre Oscillation. In the central and western North Pacific along the Kuroshio-Oyashio Extension (KOE), the expression of these modes in SSH and SST is linked to the propagation of long oceanic Rossby waves, which extend the predictability of the climate system to ~3 years. Using a multivariate physical-biological linear inverse model (LIM) we explore the extent to which this physical predictability leads to multi-year prediction of dominant fishery indicators inferred from three datasets (i.e., estimated biomasses, landings, and catches). We find that despite the strong autocorrelation in the fish indicators, the LIM adds dynamical forecast skill beyond persistence up to 5-6 years. By performing a sensitivity analysis of the LIM forecast model, we find that two main factors are essential for extending the dynamical predictability of the fishery indicators beyond persistence. The first is the interaction of the fishery indicators with the SST/SSH of the North and tropical Pacific. The second is the empirical relationship among the fisheries time series. This latter component reflects stock-stock interactions as well as common technological and human socioeconomic factors that may influence multiple fisheries and are captured in the training of the LIM. These results suggest that empirical dynamical models and machine learning algorithms, such as the LIM, provide an alternative and promising approach for forecasting key ecological indicators beyond the skill of persistence.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2020

Medium Energy Electron Flux in Earth's Outer Radiation Belt (MERLIN): A Machine Learning Model

The radiation belts of the Earth, filled with energetic electrons, comprise complex and dynamic systems that pose a significant threat to satellite operation. While various models of electron flux both for low and relativistic energies have been developed, the behavior of medium energy (120–600 keV) electrons, especially in the MEO region, remains poorly quantified. At these energies, electrons are driven by both convective and diffusive transport, and their prediction usually requires sophisticated 4D modeling codes. In this paper, we present an alternative approach using the Light Gradient Boosting (LightGBM) machine learning algorithm. The Medium Energy electRon fLux In Earth's outer radiatioN belt (MERLIN) model takes as input the satellite position, a combination of geomagnetic indices and solar wind parameters including the time history of velocity, and does not use persistence. MERLIN is trained on >15 years of the GPS electron flux data and tested on more than 1.5 years of measurements. Tenfold cross validation yields that the model predicts the MEO radiation environment well, both in terms of dynamics and amplitudes o f flux. Evaluation on the test set shows high correlation between the predicted and observed electron flux (0.8) and low values of absolute error. The MERLIN model can have wide space weather applications, providing information for the scientific community in the form of radiation belts reconstructions, as well as industry for satellite mission design, nowcast of the MEO environment, and surface charging analysis.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

tenzing

SAND2022-3576 O tenzing provides techniques for improving the performance of key applications and libraries. The program is specified as a directed acyclic graph of operations. It uses Monte-Carlo tree search to explore the design space of possible implementations and applies machine learning to the observed empirical performance to deliver design guidance to users. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

SciDAC↗
DOE OSTI2022

A Qualitative Strategy for Fusion of Physics into Empirical Models for Process Anomaly Detection

To facilitate the automated online monitoring of power plants, a systematic and qualitative strategy for anomaly detection is presented. This strategy is essential to provide credible reasoning on why and when an empirical versus hybrid (i.e., physics-supported) approach should be used and to determine the ideal mix of these two approaches for a defined anomaly detection scope. Empirical methods are usually based on pattern, statistical, and causal inference. Hybrid methods include the use of physics models to train and test data methods, reduce data dimensionality, reduce data-model complexity, augment data, and reduce empirical uncertainty; hybrid methods also include the use of data to tune physics models. The presented strategy is driven by key decision points related to data relevance, simple modeling feasibility, data inference, physics-modeling value, data dimensionality, physics knowledge, method of validation, performance, data availability, and suitability for training and testing, cause-effect, entropy inference, and model fitting. The strategy is demonstrated through a pilot use case for the application of anomaly detection to capture a valve packing leak at the high-pressure coolant injection system of a nuclear power plant.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

The seventh blind test of crystal structure prediction: structure ranking methods

A seventh blind test of crystal structure prediction has been organized by the Cambridge Crystallographic Data Centre. The results are presented in two parts, with this second part focusing on methods for ranking crystal structures in order of stability. The exercise involved standardized sets of structures seeded from a range of structure generation methods. Participants from 22 groups applied several periodic DFT-D methods, machine learned potentials, force fields derived from empirical data or quantum chemical calculations, and various combinations of the above. In addition, one non-energy-based scoring function was used. Results showed that periodic DFT-D methods overall agreed with experimental data within expected error margins, while one machine learned model, applying system-specific AIMnet potentials, agreed with experiment in many cases demonstrating promise as an efficient alternative to DFT-based methods. For target XXXII, a consensus was reached across periodic DFT methods, with consistently high predicted energies of experimental forms relative to the global minimum (above 4 kJ mol −1 at both low and ambient temperatures) suggesting a more stable polymorph is likely not yet observed. The calculation of free energies at ambient temperatures offered improvement of predictions only in some cases (for targets XXVII and XXXI). Several avenues for future research have been suggested, highlighting the need for greater efficiency considering the vast amounts of resources utilized in many cases.

Chemistry↗
DOE OSTI2022

Incorporation of Physics Phenomenology into an Adaptive Algorithm Framework (Final Report)

We attempt to incorporate prior physics knowledge into a machine learning architecture at an applied application level (empirical) as opposed to the level of fundamental physics (first principles). The purpose of this work is to allow for the application of methods of physics-informed machine learning to a broad range of national security problems, while enhancing the trust in machine-learned models by decreasing the “black box” nature of such methods.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Ultra-fast interpretable machine-learning potentials

Abstract All-atom dynamics simulations are an indispensable quantitative tool in physics, chemistry, and materials science, but large systems and long simulation times remain challenging due to the trade-off between computational efficiency and predictive accuracy. To address this challenge, we combine effective two- and three-body potentials in a cubic B-spline basis with regularized linear regression to obtain machine-learning potentials that are physically interpretable, sufficiently accurate for applications, as fast as the fastest traditional empirical potentials, and two to four orders of magnitude faster than state-of-the-art machine-learning potentials. For data from empirical potentials, we demonstrate the exact retrieval of the potential. For data from density functional theory, the predicted energies, forces, and derived properties, including phonon spectra, elastic constants, and melting points, closely match those of the reference method. The introduced potentials might contribute towards accurate all-atom dynamics simulations of large atomistic systems over long-time scales.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Measuring the Energy Consumption and Efficiency of Deep Neural Networks: An Empirical Analysis and Design Recommendations

Addressing the "Red-AI" trend of rising energy consumption by large-scale neural networks, this study investigates the measured energy consumption of training various fully connected neural network architectures. We introduce the BUTTER-E dataset, an augmentation to the BUTTER Empirical Deep Learning dataset, containing energy consumption and performance data from 41,129 individual experimental runs spanning 30,582 distinct configurations: 13 datasets, 20 sizes (trainable parameters), 8 "shapes", and 14 depths on both CPUs and GPUs using node-level watt-meters. This dataset reveals the complex relationship between dataset size, network structure, and energy use. Our analysis uncovers a surprising, hardware-mediated non-linear relationship between energy efficiency and network design, challenging the assumption that reducing the number of parameters or FLOPs is the best way to achieve greater energy efficiency. We propose a straightforward and effective energy model that accounts for network size, computing, and memory hierarchy. Highlighting the need for cache-considerate algorithm development, we suggest a codesign approach to energy efficient network, algorithm, and hardware design. This work contributes to the fields of sustainable computing and Green AI, offering practical guidance for creating more energy-efficient neural networks and promoting sustainable AI.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Measuring Equality in Machine Learning Security Defenses: A Case Study in Speech Recognition

Over the past decade, the machine learning security community has developed a myriad of defenses for evasion attacks. An understudied question in that community is: for whom do these defenses defend? This work considers common approaches to defending learned systems and how security defenses result in performance inequities across different sub-populations. We outline appropriate parity metrics for analysis and begin to answer this question through empirical results of the fairness implications of machine learning security methods. We find that many methods that have been proposed can cause direct harm, like false rejection and unequal benefits from robustness training. The framework we propose for measuring defense equality can be applied to robustly trained models, preprocessing-based defenses, and rejection methods. We identify a set of datasets with a user-centered application and a reasonable computational cost suitable for case studies in measuring the equality of defenses. In our case study of speech command recognition, we show how such adversarial training and augmentation have non-equal but complex protections for social subgroups across gender, accent, and age in relation to user coverage. We present a comparison of equality between two rejection-based defenses: randomized smoothing and neural rejection, finding randomized smoothing more equitable due to the sampling mechanism for minority groups. This represents the first work examining the disparity in the adversarial robustness in the speech domain and the fairness evaluation of rejection-based defenses.

• Artificial intelligence (AI) / machine learning ↗
DOE OSTI2025

A Fortran-Python Interface for Integrating Machine Learning Parameterization into Earth System Models

Parameterizations in Earth System Models (ESMs) are subject to biases and uncertainties arising from subjective empirical assumptions and incomplete understanding of the underlying physical processes. Recently, the growing representational capability of machine learning (ML) in solving complex problems has spawned immense interests in climate science applications. Specifically, ML-based parameterizations have been developed to represent convection, radiation and microphysics processes in ESMs by learning from observations or high-resolution simulations, which have the potential to improve the accuracies and alleviate the uncertainties. Previous works have developed some surrogate models for these processes using ML. These surrogate models need to be coupled with the dynamical core of ESMs to investigate the effectiveness and their performance in a coupled system. In this study, we present a novel Fortran-Python interface designed to seamlessly integrate ML parameterizations into ESMs. This interface showcases high versatility by supporting popular ML frameworks like PyTorch, TensorFlow, and Scikit-learn. We demonstrate the interface's modularity and reusability through two cases: a ML trigger function for convection parameterization and a ML wildfire model. We conduct a comprehensive evaluation of memory usage and computational overhead resulting from the integration of Python codes into the Fortran ESMs. By leveraging this flexible interface, ML parameterizations can be effectively developed, tested, and integrated into ESMs.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Physics-informed hybrid modeling methodology for building infiltration

Infiltration is responsible for one-third to one-half of the space conditioning load of a typical residential home, but the modeling of infiltration for building energy modeling is either represented by over-simplified equations or dependent on over-generalized rules of thumb. Here, this paper develops a physics-informed data-driven methodology for modeling infiltration using building-specific empirical measurements. The developed hybrid methodology combines machine-learning categorization and grey-box sub-modeling to improve the accuracy and generalization of commonly used grey-box infiltration models. The developed methodology excels at predicting infiltration by improving the ability to predict infiltration under unseen environmental conditions using machine learning algorithms with physical significance. In a case study conducted using the iUnit, a modular studio apartment experimental test facility located at the National Renewable Energy Laboratory, we use empirical airtightness measurements to fit an infiltration model using the developed methodology. We find that the developed methodology can improve the overall model accuracy by 43% and improve extrapolation by 38%, compared with the model based on the common grey-box infiltration equation. We also notice that the selected features can improve the performance of a pure machine-learning model, indicating that our methodology identifies the features with the most physical significance to infiltration modeling.

97 MATHEMATICS AND COMPUTING↗