Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Traditional Machine Learning Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Enhancing the hunt for new phenomena in dijet final states using anomaly detection filters at the high-luminosity large Hadron Collider

In the realm of dijet searches in high-energy physics, a significant challenge has emerged: with experiments producing more and more data, the traditional methods of using analytic functions to describe dijet mass spectra start to fail. Here, to address this, we suggest the application of an anomaly detection approach to eliminate less interesting background events based on event final states. This method not only bypasses the limitations of conventional background models but also significantly enhances our ability to detect potential signals of new physics. Through simulations that mimic the conditions of the upcoming high-luminosity large Hadron collider, we demonstrate the strength and efficiency of this approach in dealing with large data volumes. The integration of unsupervised machine learning into our experimental framework paves the way for a promising avenue to unveil hidden physics discoveries within the overwhelming influx of data.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Data-Driven Tailoring Optimization of Thermoset Polymers Using Ultrasonics and Machine Learning

Thermoset polymers are highly demanded for their structural robustness, thermal stability, and chemical resistance. Tailoring the properties of these polymers for high-performance applications is often preferred to designing brand-new polymers. However, the traditional destructive techniques used to characterize their properties as a function of manufacturing parameters are expensive and time-consuming. A novel non-destructive, data-driven method leveraging ultrasonics and machine learning techniques to tailor the properties of thermosets as a function of the manufacturing parameters is demonstrated. Thermoset epoxy samples with varying curing temperatures (15–40 °C) and curing agent amounts (±40%) were manufactured and tested. Their curing kinetics were monitored by determining the sound speed in the material in real time, while the longitudinal modulus of the samples was determined post-cure. Machine learning models were developed using a k-nearest neighbors algorithm. These models were implemented to predict the curing and final elastic properties using the manufacturing parameters, i.e., stoichiometry and curing temperature, and vice versa. Understanding and modeling how these parameters affect the cure kinetics and final properties will allow for efficient and reliable optimization of thermoset tailoring and manufacturing.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Real-Time Regional PV Spinning Reserve Estimator with AGC Look-Ahead Windows: Preprint

Curtailed PV generation is a zero-marginal cost spinning reserve that can be used for a number of active power control services. However, unlike the traditional spinning reserve providers, i.e., fossil-fueled generators, who have well-defined operating characteristics, e.g., available headroom or potential high limit (PHL), PV plants have by nature variable and uncertain operating characteristics. To ensure the effective coordination between PV plants and the system operator during an active power control event, accurate forecasts of the PV PHL are essential. A novel reference-control grouping based scaling method has been proposed by NREL to estimate the PV PHL in real-time. This work further enhances the methodology by: 1) improving the model accuracy through machine learning; 2) considering look-ahead windows introduced by the computation and communication latencies; 3) applying the method to regional spinning reserve estimation. A significant performance improvement, over 1.6% of estimation error reduction, has been observed based on real-world data collected by CAISO and PV plant operators.

potential high limit↗
DOE OSTI2022

A reforecasting-based dynamic reserve estimation for variable renewable generation and demand uncertainty

The installed capacity of renewables-based energy sources has been increasing in traditional power systems. In order to accommodate the increased variability and uncertainty associated with the deeper penetration of renewable sources like solar and wind, adjusted amounts of dynamic reserve are needed. Although probabilistic dynamic reserve estimation methods have been previously developed, most of them consider the uncertainty to be represented by parametric density functions that tend to perform poorly under extreme events and, moreover, neglect uncertainty introduced by the forecasting model itself. Toward addressing these limitations, this work presents, for the first time, a dynamic reserve estimation method for flexibility that incorporates nonparametric density estimation and a machine learning based reforecasting to provide a day-ahead prediction of the mean and spread of uncertainty around the base forecast. The prediction is, in turn, used to estimate the up and down reserve relative to the base forecast. Here, the present method takes various endogenous and exogenous features, including the calendar variables, as input to estimate the day-ahead reserve. Using a combination of reforecasting and dynamic reserve estimation techniques, the method is shown to adjust better to the dynamic nature of reserve requirements providing only what is needed to accommodate the expected deviations. Considering California Independent System Operator (CAISO) solar, wind and load data over an 18 month period, up to 67% reduction in the amount of reserve capacity needed for a one day reserve and reserve penalty for solar uncertainty is demonstrated. Additionally, the risk of reserve insufficiency in meeting the net demand is reduced by 20% with the proposed method.

14 SOLAR ENERGY↗
DOE OSTI2023

Toward DMC Accuracy Across Chemical Space with Scalable Δ-QML

In the past decade, quantum diffusion Monte Carlo (DMC) has been demonstrated to successfully predict the energetics and properties of a wide range of molecules and solids by numerically solving the electronic many-body Schrödinger equation. With O(N 3 ) scaling with the number of electrons N, DMC has the potential to be a reference method for larger systems that are not accessible to more traditional methods such as CCSD(T). Assessing the accuracy of DMC for smaller molecules becomes the stepping stone in making the method a reference for larger systems. We show that when coupled with quantum machine learning (QML)-based surrogate methods, the computational burden can be alleviated such that quantum Monte Carlo (QMC) shows clear potential to undergird the formation of high-quality descriptions across chemical space. We discuss three crucial approximations necessary to accomplish this: the fixed-node approximation, universal and accurate references for chemical bond dissociation energies, and scalable minimal amons-set-based QML (AQML) models. Numerical evidence presented includes converged DMC results for over 1000 small organic molecules with up to five heavy atoms used as amons and 50 medium-sized organic molecules with nine heavy atoms to validate the AQML predictions. Finally, numerical evidence collected for Δ-AQML models suggests that already modestly sized QMC training data sets of amons suffice to predict total energies with near chemical accuracy throughout chemical space.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Accelerating manufacturing for biomass conversion via integrated process and bench digitalization: a perspective

We present a perspective for accelerating biomass manufacturing via digitalization. We summarize the challenges for manufacturing and identify areas where digitalization can help. A profound potential in using lignocellulosic biomass and renewable feedstocks, in general, is to produce new molecules and products with unmatched properties that have no analog in traditional refineries. Discovering such performance-advantaged molecules and the paths and processes to make them rapidly and systematically can transform manufacturing practices. Furthermore, we discuss retrosynthetic approaches, text mining, natural language processing, and modern machine learning methods to enable digitalization. Laboratory and multiscale computation automation via active learning are crucial to complement existing literature and expedite discovery and valuable data collection without a human in the loop. Such data can help process simulation and optimization select the most promising processes and molecules according to economic, environmental, and societal metrics. We propose the close integration between bench and process scale models and data to exploit the low dimensionality of the data and transform the manufacturing for renewable feedstocks.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Unified Language Frontend for Physic-Informed AI/ML

Artificial intelligence and machine learning (AI/ML) are becoming important tools for scientific modeling and simulation as in several other fields such as image analysis and natural language processing. ML techniques can leverage the computing power available in modern systems and reduce the human effort needed to configure experiments, interpret and visualize results, draw conclusions from huge quantities of raw data, and build surrogates for physics based models. Domain scientists in fields like fluid dynamics, microelectronics and chemistry can automate many of their most difficult and repetitive tasks or improve the design times by use of the faster ML-surrogates. However, modern ML and traditional scientific highperformance computing (HPC) tend to use completely different software ecosystems. While ML frameworks like PyTorch and TensorFlow provide Python APIs, most HPC applications and libraries are written in C++. Direct interoperability between the two languages is possible but is tedious and error-prone. In this work, we show that a compiler-based approach can bridge the gap between ML frameworks and scientific software with less developer effort and better efficiency. We use the MLIR (multi-level intermediate representation) ecosystem to compile a pre-trained convolutional neural network (CNN) in PyTorch to freestanding C++ source code in the Kokkos programming model. Kokkos is a programming model widely used in HPC to write portable, shared-memory parallel code that can natively target a variety of CPU and GPU architectures. Our compiler-generated source code can be directly integrated into any Kokkosbased application with no dependencies on Python or cross-language interfaces.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

A Comparison of Infectious Disease Forecasting Methods across Locations, Diseases, and Time

Accurate infectious disease forecasting can inform efforts to prevent outbreaks and mitigate adverse impacts. This study compares the performance of statistical, machine learning (ML), and deep learning (DL) approaches in forecasting infectious disease incidences across different countries and time intervals. We forecasted three diverse diseases: campylobacteriosis, typhoid, and Q-fever, using a wide variety of features (n = 46) from public datasets, e.g., landscape, climate, and socioeconomic factors. We compared autoregressive statistical models to two tree-based ML models (extreme gradient boosted trees [XGB] and random forest [RF]) and two DL models (multi-layer perceptron and encoder–decoder model). The disease models were trained on data from seven different countries at the region-level between 2009–2017. Forecasting performance of all models was assessed using mean absolute error, root mean square error, and Poisson deviance across Australia, Israel, and the United States for the months of January through August of 2018. The overall model results were compared across diseases as well as various data splits, including country, regions with highest and lowest cases, and the forecasted months out (i.e., nowcasting, short-term, and long-term forecasting). Overall, the XGB models performed the best for all diseases and, in general, tree-based ML models performed the best when looking at data splits. There were a few instances where the statistical or DL models had minutely smaller error metrics for specific subsets of typhoid, which is a disease with very low case counts. Feature importance per disease was measured by using four tree-based ML models (i.e., XGB and RF with and without region name as a feature). The most important feature groups included previous case counts, region name, population counts and density, mortality causes of neonatal to under 5 years of age, sanitation factors, and elevation. This study demonstrates the power of ML approaches to incorporate a wide range of factors to forecast various diseases, regardless of location, more accurately than traditional statistical approaches.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Improving the Concrete Crack Detection Process via a Hybrid Visual Transformer Algorithm

Inspections of concrete bridges across the United States represent a significant commitment of resources, given their biannual mandate for many structures. With a notable number of aging bridges, there is an imperative need to enhance the efficiency of these inspections. This study harnessed the power of computer vision to streamline the inspection process. Our experiment examined the efficacy of a state-of-the-art Visual Transformer (ViT) model combined with distinct image enhancement detector algorithms. We benchmarked against a deep learning Convolutional Neural Network (CNN) model. These models were applied to over 20,000 high-quality images from the Concrete Images for Classification dataset. Traditional crack detection methods often fall short due to their heavy reliance on time and resources. This research pioneers bridge inspection by integrating ViT with diverse image enhancement detectors, significantly improving concrete crack detection accuracy. Notably, a custom-built CNN achieves over 99% accuracy with substantially lower training time than ViT, making it an efficient solution for enhancing safety and resource conservation in infrastructure management. These advancements enhance safety by enabling reliable detection and timely maintenance, but they also align with Industry 4.0 objectives, automating manual inspections, reducing costs, and advancing technological integration in public infrastructure management.

42 ENGINEERING↗
DOE OSTI2026

Conditional diffusion machine-learning framework for mapping valence electron distribution from convergent beam electron diffraction

Quantitative convergent beam electron diffraction (CBED) enables determination of aspherical valence electron distributions through refinement of low-order structure factors, which are highly sensitive to chemical bonding and charge density variations. However, conventional quantitative CBED (QCBED) requires solving a highly nonlinear inverse problem with many coupled parameters, and computationally intensive dynamical diffraction calculations, making it time-consuming and difficult to apply to complex systems. More broadly, reconstructing charge density and orbital electron distribution from diffraction data has long been a central challenge in both x-ray and electron crystallography. Here, in this study, we introduce an artificial-intelligence (AI)-based framework that replaces traditional refinement with a data-driven inverse solver. Using a large synthetic CBED dataset generated by Bloch-wave simulations, we train a conditional diffusion model to directly infer crystal structural parameters and multipole density formalism parameters, and hence valence electron distributions, from CBED patterns alone. By learning from forward simulations across realistic parameter space, the model effectively solves the inverse problem. Compared with direct regression approaches, the diffusion-based framework provides posterior parameter distributions for rigorous uncertainty quantification while preserving quantitative fidelity and reducing analysis time by orders of magnitude. By eliminating the need for external single-crystal x-ray diffraction data and complex nonlinear refinement, this approach enables practical, high-throughput, and in situ quantitative CBED, enabling real-time mapping of valence electron distributions and their correlation with functional responses in quantum and energy materials.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2026

Creep in multi-principal element materials –– A review

The ongoing push towards enhanced energy efficiency and reduced emissions has necessitated the creation of materials with superior performance, especially under extreme conditions. Modern industries, such as aerospace, energy production, and nuclear power, rely heavily on materials that can withstand elevated temperatures without compromising structural integrity. At these heightened temperatures, materials, even when subjected to mechanical stresses well below their yield strength, may experience slow deformation leading to eventual rupture — a phenomenon known as creep. With the expansive design space that comes with the high entropy concept and their reported excellent high temperature strength, multi-principal element materials (MPEMs) have attracted interest in the scientific community for high-temperature applications. Here, this review offers a comprehensive examination of existing studies on creep in MPEMs, which includes multi-principal element−alloys, −bulk metallic glasses, −ceramics, and −superalloys, comparing published findings on MPEMs with pure elements, traditional alloys, bulk metallic glasses, and superalloys. The sub-topics covered include a comparison among different creep-testing methods, creep mechanisms, creep exponents, creep strain rates, activation volume, and creep-activation energy. Modeling efforts for predicting creep behavior of MPEMs are also reviewed. Methods for improving creep resistance by performing heat treatments and/or modifying microstructures are discussed. Overall, the current state of MPEMs has not yet surpassed the creep performance of commercial alloys. Finally, directions for future efforts are suggested, such as experimenting in various controlled environments, expanding the number of compositions tested, exploring advanced manufacturing techniques, and using machine-learning to predict creep properties based on compositions and microstructures.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Teaching a neural network to attach and detach electrons from molecules

Abstract Interatomic potentials derived with Machine Learning algorithms such as Deep-Neural Networks (DNNs), achieve the accuracy of high-fidelity quantum mechanical (QM) methods in areas traditionally dominated by empirical force fields and allow performing massive simulations. Most DNN potentials were parametrized for neutral molecules or closed-shell ions due to architectural limitations. In this work, we propose an improved machine learning framework for simulating open-shell anions and cations. We introduce the AIMNet-NSE (Neural Spin Equilibration) architecture, which can predict molecular energies for an arbitrary combination of molecular charge and spin multiplicity with errors of about 2–3 kcal/mol and spin-charges with error errors ~0.01e for small and medium-sized organic molecules, compared to the reference QM simulations. The AIMNet-NSE model allows to fully bypass QM calculations and derive the ionization potential, electron affinity, and conceptual Density Functional Theory quantities like electronegativity, hardness, and condensed Fukui functions. We show that these descriptors, along with learned atomic representations, could be used to model chemical reactivity through an example of regioselectivity in electrophilic aromatic substitution reactions.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Leveraging intermediate resonances to probe CP violation at colliders

We explore the phenomenological impact of interference in tree-level contributions to three-body final states in $2\rightarrow 3$ scattering processes. This work introduces a novel search strategy leveraging asymmetries to enable sensitivity to CP-violating effects in less well-explored regions of phase space. Analytically, we demonstrate the effectiveness of this observable in probing interference between Standard Model charged-current decays and effective left-handed vector interactions, illustrated in a toy model featuring a scalar leptoquark, $S_1 \sim (3, 1, -\,1/3)$. Numerically, we apply this framework to studying the process $pp\rightarrow b \tau \nu $; unlike traditional high-$p_T$ searches or “bump hunts”, this approach utilizes an intermediate energy regime – where new physics is neither light enough to be produced on shell or heavy enough to justify an effective field theory treatment. A proof-of-principle analysis at parton level demonstrates a percent-level asymmetry, with sensitivity also to BSM weak-CP phase. While the specific phase sensitivity is diminished at particle level due to showering and detector effects, a machine learning classifier can recover sensitively to the presence of SM-BSM interference, significantly outperforming standard analysis methods. Notably discrimination between BSM signal and SM background could be achieved at the 2$\sigma $ level for the current LHC dataset and 8$\sigma $ at the High-Luminosity LHC. Moreover, this asymmetry observable as defined can also be more broadly applied to other searches for CP-violation in $2\rightarrow 3$ processes in present and future collider environments.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Comparison of multifidelity machine learning models for potential energy surfaces

Multifidelity modeling is a technique for fusing the information from two or more datasets into one model. It is particularly advantageous when one dataset contains few accurate results and the other contains many less accurate results. Within the context of modeling potential energy surfaces, the low-fidelity dataset can be made up of a large number of inexpensive energy computations that provide adequate coverage of the N-dimensional space spanned by the molecular internal coordinates. The high-fidelity dataset can provide fewer but more accurate electronic energies for the molecule in question. Here, we compare the performance of several neural network-based approaches to multifidelity modeling. We show that the four methods (dual, Δ-learning, weight transfer, and Meng–Karniadakis neural networks) outperform a traditional implementation of a neural network, given the same amount of training data. We also show that the Δ-learning approach is the most practical and tends to provide the most accurate model.

Chemistry↗
DOE OSTI2021

Efficient Learning of Accurate Surrogates for Simulations of Complex Systems

Machine learning methods are increasingly deployed to construct surrogate models for complex physical systems at a reduced computational cost. However, the predictive capability of these surrogates degrades in the presence of noisy, sparse, or dynamic data. We introduce an online learning method empowered by optimizer-driven sampling that has two advantages over current approaches: it ensures that all local extrema (including endpoints) of the model response surface are included in the training data and it employs a continuous validation and update process in which surrogates undergo retraining when their performance falls below a validity threshold. We find, using benchmark functions, that optimizer-directed sampling generally outperforms traditional sampling methods in terms of accuracy around local extrema even when the scoring metric is biased towards assessing overall accuracy. Finally, the application to dense nuclear matter demonstrates that highly accurate surrogates for a nuclear equation of state model can be reliably auto-generated from expensive calculations using few model evaluations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Estimating Watershed Subsurface Permeability From Stream Discharge Data Using Deep Neural Networks

Subsurface permeability is a key parameter in watershed models that controls the contribution from the subsurface flow to stream flows. Since the permeability is difficult and expensive to measure directly at the spatial extent and resolution required by fully distributed watershed models, estimation through inverse modeling has had a long history in subsurface hydrology. The wide availability of stream surface flow data, compared to groundwater monitoring data, provides a new data source to infer soil and geologic properties using integrated surface and subsurface hydrologic models. As most of the existing methods have shown difficulty in dealing with highly nonlinear inverse problems, we explore the use of deep neural networks for inversion owing to their successes in mapping complex, highly nonlinear relationships. We train various deep neural network (DNN) models with different architectures to predict subsurface permeability from stream discharge hydrograph at the watershed outlet. The training data are obtained from ensemble simulations of hydrographs corresponding to an permeability ensemble using a fully-distributed, integrated surface-subsurface hydrologic model. The trained model is then applied to estimate the permeability of the real watershed using its observed hydrograph at the outlet. Our study demonstrates that the permeabilities of the soil and geologic facies that make significant contributions to the outlet discharge can be more accurately estimated from the discharge data. Their estimations are also more robust with observation errors. Compared to the traditional ensemble smoother method, DNNs show stronger performance in capturing the nonlinear relationship between permeability and stream hydrograph to accurately estimate permeability. Our study sheds new light on the value of the emerging deep learning methods in assisting integrated watershed modeling by improving parameter estimation, which will eventually reduce the uncertainty in predictive watershed models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Efficient learning of accurate surrogates for simulations of complex systems

Machine learning methods are increasingly deployed to construct surrogate models for complex physical systems at a reduced computational cost. However, the predictive capability of these surrogates degrades in the presence of noisy, sparse or dynamic data. Here, we introduce an online learning method empowered by optimizer-driven sampling that has two advantages over current approaches: it ensures that all local extrema (including endpoints) of the model response surface are included in the training data, and it employs a continuous validation and update process in which surrogates undergo retraining when their performance falls below a validity threshold. We find, using benchmark functions, that optimizer-directed sampling generally outperforms traditional sampling methods in terms of accuracy around local extrema even when the scoring metric is biased towards assessing overall accuracy. Finally, the application to dense nuclear matter demonstrates that highly accurate surrogates for a nuclear equation-of-state model can be reliably autogenerated from expensive calculations using few model evaluations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

LandScan Global 2023: Silver Edition

For a quarter of a century, the LandScan Global (LSG) project has annually released a global, high-resolution gridded population dataset representing the ambient or unwarned population at a 30 arcsecond resolution. LSG supports a range of applications such as emergency management, disaster response, and human health and security for understanding populations at risk. The 2023 release of LSG, the LandScan Silver Edition, represents a major methodological leap forward while also leveraging previous knowledge—the previous year was the baseline for the current annual update carrying forward valuable knowledge of the built environment for the past quarter century—to train the machine learning models. Compared with annual releases over the past 24years, multiple advancements were made to different aspects of the methodology to achieve reproducibility, transparency, and consistent global propagation of solutions to modeling or population distribution issues identified during the review process. These novel changes include incorporation of the latest available geospatial inputs across the globe, machine learning models instead of manual modifications, population feature importance analysis, open-source solutions vs. proprietary software, generation of multiple global versions, analytic validations, and human-in-the-loop revisions to produce the final version. Additionally, algorithms—such as anomaly detection—were introduced to quickly identify areas of focus to develop a new and robust systematic review. Significant changes in modeled population distributions were observed between the 2022 and 2023 releases, largely attributable to improvements in data and methods and discussed thoroughly within this report. In summation, the LandScan Silver Edition leverages the best of the past quarter century of LSG legacy knowledge and continues a tradition of applying cutting-edge enhancements to serve as a new benchmark for accurate, actionable gridded population data

Lebakula, Viswadeep↗