Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “machine learning algorithms”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,711 records · Page 95
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Discovering hidden geothermal signatures using non-negative matrix factorization with customized k-means clustering

Discovery of hidden geothermal resources is challenging. It requires the mining of large datasets with diverse data attributes representing subsurface hydrogeological and geothermal conditions. The commonly used play fairway analysis approach typically incorporates subject-matter expertise to analyze regional data to estimate geothermal characteristics and favorability. We demonstrate an alternative approach based on machine learning (ML) to process a geothermal dataset from southwest New Mexico (SWNM). The study region includes low- and medium-temperature hydrothermal systems. Several of these systems are not well characterized because of insufficient existing data and limited past explorative work. This study discovers hidden patterns and relations in the SWNM geothermal dataset to improve our understanding of the regional hydrothermal conditions and energy-production favorability. This understanding is obtained by applying an unsupervised ML algorithm based on non-negative matrix factorization coupled with customized k-means clustering (NMFk). NMFk can automatically identify (1) hidden signatures characterizing analyzed datasets, (2) the optimal number of these signatures, (3) the dominant data attributes associated with each signature, and (4) the spatial distribution of the extracted signatures. Here, in this study, NMFk is applied to analyze 18 geological, geophysical, hydrogeological, and geothermal attributes at 44 locations in SWNM. Using NMFk, we find data patterns and identify the spatial associations of hydrothermal signatures within two physiographic provinces (Colorado Plateau and Basin and Range) and two sub-regions of these provinces (the Mogollon-Datil volcanic field and the Rio Grande rift) in SWNM. The ML algorithm extracted five hydrothermal signatures in the SWNM datasets that differentiate between low (<90°C) and medium (90-150°C)-temperature hydrothermal systems. The algorithm also suggests that the Rio Grande rift and northern Mogollon-Datil volcanic field are the most favorable regions for future geothermal resource discovery. NMFk also identified critical attributes to identify medium-temperature hydrothermal systems in the study area. The resulting NMFk model can be applied to predict geothermal conditions and their uncertainties at new SWNM locations based on limited data from unexplored regions. The code to execute the performed analyses as well as the corresponding data can be found at https://github.com/SmartTensors/GeoThermalCloud.jl.

15 GEOTHERMAL ENERGY↗
DOE OSTI2024

A real-time energy and cost efficient vehicle route assignment neural recommender system

Here, this paper presents a neural network recommender system algorithm for assigning vehicles to routes based on energy and cost criteria. In this work, we applied this new approach to efficiently identify the most cost-effective medium and heavy duty truck (MDHDT) powertrain technology, from a total cost of ownership (TCO) perspective, for given trips. We employ a machine learning based approach to efficiently estimate the energy consumption of various candidate vehicles over given routes, defined as sequences of links (road segments), with little information known about internal dynamics, i.e. using high level macroscopic route information. A complete recommendation logic is then developed to allow for real-time optimum assignment for each route, subject to the operational constraints of the fleet. We show how this framework can be used to (1) efficiently provide a single trip recommendation with a top-k vehicles star ranking system, and (2) engage in more general assignment problems where n vehicles need to be deployed over m (m ≤ n) trips. This new assignment system has been deployed and integrated into the POLARIS. Transportation System Simulation Tool for use in research conducted by the Department of Energy's Systems and Modeling for Accelerated Research in Transportation (SMART) Mobility Consortium (SMART, 2024).

Energy consumption↗
DOE OSTI2024

Generation and validation of comprehensive synthetic weather histories using auto-regressive moving-average models

As energy system design moves to more complex methods of optimization including machine learning there is a significant need for more weather data than is available. One method to solve this is using synthetic data models such as the auto-regressive moving-average (ARMA) model which has been frequently utilized to create such data. This paper looks at extending the ARMA algorithm to generate solar components through the use of clearsky detrending, maintaining vector relationships and by leveraging physical relationships. The method for the creation of entirely synthetic weather data files including key weather variables for energy system analysis is presented. Furthermore, a detailed comparison of energy system simulations utilizing both real and synthetic data is made using NREL’s System Advisor Model. Whilst good agreement is made for the solar variables, and other weather variables, ARMA methods often fail to capture the standard deviation and skew of annual weather distributions. Vector-ARMA is shown to maintain correlations between variables and thus generate data sets that perform similarly in energy system design. Here, it is finally shown that the ARMA method fails to preserve day-today correlations in weather variables and thus over-predicts optimal energy storage by 21% for a residential solar application.

42 ENGINEERING↗
DOE OSTI2025

Automated and highly parallelized Bayesian optimization scheme for direct drive fusion experiments on OMEGA

Finding the optimal implosion design on existing experimental facilities for inertial confinement fusion requires an exhaustive search of the vast design parameter space. This is infeasible both with experiments and with simulations. Consequently, a large fraction of the experimentally realizable design space remains unexplored, and new design schemes are challenging to optimize in a reasonable time frame. On the OMEGA laser facility, predictive machine learning models have been developed to accurately forecast the result of an experiment using only inexpensive simulations and the large dataset of prior experimental data. However, the full design space remains vast enough to be unassailable with simple optimization techniques. Here we develop an automated and optimally parallel Bayesian optimization algorithm that can entirely optimize the target and pulse shape of a direct-drive ICF implosion under a given design paradigm. We use this algorithm to find a markedly improved design for the performance implosions on OMEGA that is predicted to hydroequivalently scale to ignition at 2.15 MJ.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

A Machine Learning-based Reliability Evaluation Model for Integrated Power-Gas Systems

This article proposes a hybrid machine learning method for the reliability evaluation of integrated power-gas systems (IPGS) under the uncertain component failure probability distributions. The Random Forest (RF) method is designed to select important features to solve the insufficient quantity of data and the curse of dimensionality problems. The Extreme Gradient Boosting (XGBoost) regression algorithm is developed to quantify the relationship between the uncertain parameters and reliability metrics. Moreover, a ten-fold cross-validation method is employed to further improve the accuracy of the regression model. Simulation results on three test systems show that the proposed method can achieve high accuracy for the reliability evaluation.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

AI/ML Expo Boosting Job Performance with AI: Innovative Approaches and Success Stories

Our technology leverages artificial intelligence (AI) to enhance the user experience in High Performance Computing (HPC) environments. By analyzing user behavior and providing personalized recommendations, our AI system helps HPC users optimize their workflows and improve productivity. Additionally, we offer an advanced image similarity search feature, which utilizes AI algorithms to identify and retrieve visually similar images, saving users valuable time and effort in their research and analysis.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Data-driven learning of nonlocal physics from high-fidelity synthetic data

A key challenge to nonlocal models is the analytical complexity of deriving them from first principles, and frequently their use is justified a posteriori. Here, we extract nonlocal models from data, circumventing these challenges and providing data-driven justification for the resulting model form. Extracting data-driven surrogates is a major challenge for machine learning (ML) approaches, due to nonlinearities and lack of convexity — it is particularly challenging to extract surrogates which are provably well-posed and numerically stable. Our scheme not only yields a convex optimization problem, but also allows extraction of nonlocal models whose kernels may be partially negative while maintaining well-posedness even in small-data regimes. To achieve this, based on established nonlocal theory, we embed in our algorithm sufficient conditions on the non-positive part of the kernel that guarantee well-posedness of the learnt operator. These conditions are imposed as inequality constraints to meet the requisite conditions of the nonlocal theory. We demonstrate this workflow for a range of applications, including reproduction of manufactured nonlocal kernels; numerical homogenization of Darcy flow associated with a heterogeneous periodic microstructure; nonlocal approximation to high-order local transport phenomena; and approximation of globally supported fractional diffusion operators by truncated kernels.

42 ENGINEERING↗
DOE OSTI2021

Deep Learning Estimation of Daily Ground–Level NO 2 Concentrations from Remote Sensing Data

The limited number of nitrogen dioxide (NO 2 ) surface measurements calls for the development of highly accurate approaches to estimating surface NO 2 concentrations. In this study, we leverage a new satellite instrument, the TROPOspheric Monitoring Instrument (TROPOMI), along with other predictor variables, to estimate daily surface NO 2 concentrations over Texas in 2019. We use the deep convolutional neural network (Deep-CNN), an advanced deep learning algorithm, to obtain estimates and achieve a correlation coefficient (R) of 0.91, an index of agreement (IOA) of 0.95, and a mean absolute bias (MAB) of 1.75 ppb in surface NO 2 estimation. Additionally, we leverage a novel approach, SHapley Additive exPlanations (SHAP), to describe how Deep-CNN understands each predictor variable. The SHAP results show that the Deep-CNN model has an advanced understanding of the dataset, revealing that TROPOMI closely captures levels of NO 2 . In addition, we show the superiority of our Deep-CNN model at estimating surface NO 2 over other well-known machine learning and regression models in the field, including the support vector machines (SVM), random forest (RF), and multiple linear regression (MLR). Although SVM and RF show strong capabilities at estimating surface NO 2 concentrations, their accuracy is inferior to that of the Deep-CNN model, ranking second and third in model accuracy in this study. The MLR, however, shows a poor ability at NO 2 estimation and ranks last among all models. Furthermore, testing the impact of sample size on model performance, we also show that, compared to other models, Deep-CNN needs more samples to trigger its strength at surface NO 2 estimation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Quantum-inspired weight-constrained neural network: Reducing variable numbers by 100× compared to standard neural networks

Although quantum machine learning has shown great promise, the practical application of quantum computers remains constrained in the noisy intermediate-scale quantum era. To take advantage of quantum machine learning, we investigate the underlying mathematical principles of these quantum models and find that the quantum neural network with amplitude encoding is equivalent to a weight-constrained neural network. Motivated by this discovery, we develop a classical weight-constrained neural network. We find that this approach can reduce the number of variables in a classical neural network by a factor of 135 while preserving its accuracy. In addition, we develop a dropout method to enhance the robustness of quantum machine learning models, which are highly susceptible to adversarial attacks. This technique can also be applied to improve the adversarial robustness of the classical weight-constrained neural network, which is essential for industry applications, such as self-driving vehicles. Our work offers an approach to reduce the complexity of large classical neural networks, addressing a critical challenge in machine learning.

quantum algorithms & computation↗
DOE OSTI2022

Dynamic calibration of differential equations using machine learning, with application to turbulence models

We present a methodology for calibration of parametric ordinary and partial differential equation models, using off-the-shelf software for back-propagation in Neural Networks (NN). As a prototypical example, we consider calibration of a Reynolds-averaged Navier-Stokes (RANS) turbulence closure model, against ground truth data from direct numerical simulations (DNS) of two different turbulent flows. Numerical time integration is represented as a custom NN, where only the RANS model parameters are trainable. A loss function is defined to quantify the mismatch between the NN prediction and the ground truth over a predefined, finite time integration window. This loss function is then minimized using a gradient descent method utilizing the back-propagation algorithm. Furthermore, this dynamic approach to training is to be contrasted with a static approach, wherein a least square regression estimate for parameters is obtained in the limit of an infinitesimal time integration window. In a first test of static and dynamic approaches against ground truth data generated by the model, the former proves to be significantly faster and more accurate than the latter at recovering the parameters. When both calibration approaches are tested against DNS data, for which it is known that the model cannot achieve a perfect fit, the static approach yields a good prediction only for short times, while the dynamic approach results in physical and stable predictions over the entire integration window. After optimization of the dynamic approach for time step, spatial resolution, stability, and physics-based constraints, we obtain a 50% improvement of outcomes over those obtained from the existing, manually calibrated set of parameters, demonstrating the merits of this systematic and automated procedure.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Flexible machine-learning interatomic potential for simulating structural disordering behavior of Li 7 La 3 Zr 2 O 12 solid electrolytes

Batteries based on solid-state electrolytes, including Li 7 La 3 Zr 2 O 12 (LLZO), promise improved safety and increased energy density; however, atomic disorder at grain boundaries and phase boundaries can severely deteriorate their performance. Machine-learning (ML) interatomic potentials offer a uniquely compelling solution for simulating chemical processes, rare events, and phase transitions associated with these complex interfaces by mixing high scalability with quantum-level accuracy, provided that they can be trained to properly address atomic disorder. To this end, we report the construction and validation of an ML potential that is specifically designed to simulate crystalline, disordered, and amorphous LLZO systems across a wide range of conditions. The ML model is based on a neural network algorithm and is trained using ab initio data. Performance tests prove that the developed ML potential can predict accurate structural and vibrational characteristics, elastic properties, and Li diffusivity of LLZO comparable to ab initio simulations. As a demonstration of its applicability to larger systems, we show that the potential can correctly capture grain boundary effects on diffusivity, as well as the thermal transition behavior of LLZO. Here these examples show that the ML potential enables simulations of transitions between well-defined and disordered structures with quantum-level accuracy at speeds thousands of times faster than ab initio methods.

25 ENERGY STORAGE↗
DOE OSTI2025

Constraining nuclear mass models using 𝑟-process observables with multiobjective optimization

Modeling nuclear masses, particularly for nuclei far from stability, remains a key objective in nuclear physics. One contemporary approach is machine learning (ML), which trains on experimental data, but can suffer large errors when extrapolating toward neutron-rich species. In nature, such masses shape observables for the rapid neutron capture process (𝑟 process), which in principle could inform ML models. Here, we introduce a multiobjective optimization approach using the Pareto front algorithm. We show that this technique, capable of identifying models that generate 𝑟-process abundances aligning with both solar and stellar data, is a promising method to select ML models with reliable extrapolation power.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

VoroClust: Scalable Clustering for Remote Sensing

Although supervised machine learning provides a powerful framework for image classification and segmentation, it requires comprehensive consistent datasets, which are not available for many remote-sensing applications. Remote-sensing datasets are expensive to collect, and each is acquired under different environmental conditions or with significant variations in system operating parameters. Unsupervised clustering algorithms analyze the structure of each dataset independently, rather than drawing on similarities with existing “training” examples, and are thus well suited for practical remote-sensing applications. We introduce VoroClust, a fast density-based unsupervised clustering algorithm applicable to high-resolution and high-dimensional data. VoroClust runs as fast as distance-based clustering methods, while capturing complex regional geometries at least as well as current-density-based methods. It uses a data-centered sphere cover to reduce computational demands, while still capturing data topology. It then propagates clusters outward from local peaks in density. We show that VoroClust provides fast state-of-the-art clustering for both high-resolution polarimetric synthetic aperture radar and high-dimensional hyperspectral imaging datasets.

42 ENGINEERING↗
DOE OSTI2023

Is Knowledge about Running Applications Helping Improve Runtime Prediction of HPC Jobs?

High-performance computing systems rely upon scheduling algorithms to achieve high utilization. These schedulers rely upon user estimates of job resource requirements, such as runtime, to determine optimal scheduling of incoming jobs. These user estimates, however, are prone to error. To mitigate this error, significant research has been directed at providing better estimates of job runtime, usually employing machine learning techniques. These techniques are dependent upon the input features selected. Among the possible features is the primary application used by the job. In a survey of more than 20 papers directed at improving runtime prediction, only four included primary application as an input feature. We focus this investigation specifically on the value of adding primary application as an input feature, and find that it does improve model performance, especially for jobs with longer runtimes, though this improvement varies based on the application used. We recommend further research to determine the cause of this variability as well as an optimal strategy for employing a mixture of models both including and not including primary application as a feature.

MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Is Knowledge About Running Applications Helping Improve Runtime Prediction of HPC Jobs?

High-performance computing systems rely upon scheduling algorithms to achieve high utilization. These schedulers rely upon user estimates of job resource requirements, such as runtime, to determine optimal scheduling of incoming jobs. These user estimates, however, are prone to error. To mitigate this error, significant research has been directed at providing better estimates of job runtime, usually employing machine learning techniques. These techniques are dependent upon the input features selected. Among the possible features is the primary application used by the job. In a survey of more than 20 papers directed at improving runtime prediction, only four included primary application as an input feature. We focus this investigation specifically on the value of adding primary application as an input feature, and find that it does improve model performance, especially for jobs with longer runtimes, though this improvement varies based on the application used. We recommend further research to determine the cause of this variability as well as an optimal strategy for employing a mixture of models both including and not including primary application as a feature.

feature selection↗
DOE OSTI2024

Temperature-dependent mechanical properties and crystal plasticity parameters for additively manufactured Haynes-214 alloy: Experiments and numerical modeling

Our experimental mechanical testing data demonstrated that the additively manufactured (AM) laser powder bed fusion (L-PBF) Haynes-214 alloy exhibits non-linear mechanical properties as the temperature rises from ambient to 870 °C. Crystal plasticity (CP) simulations provide an effective approach to gaining deeper insights into microstructure-property linkages under thermomechanical loading. This method can reduce the need for costly high-temperature mechanical testing while accounting for the effects of crystallographic texture and grain morphology on the mechanical behavior of AM materials. However, calibrating a CP model is time-consuming because individual simulations are computationally expensive and hundreds (or more) of iterations over parameter sets may be required. To address this issue, we have designed a machine learning-differential evolution (ML-DE) CP framework that can accurately interpolate the tensile properties of AM L-PBF Haynes-214 alloy across a wide temperature range from ambient to 870 °C, with minimal reliance on experimental data. The framework uses electron backscatter diffraction (EBSD) measurements to generate statistically equivalent microstructural volume elements to serve as inputs to the CP modeling framework. Stress–strain curves were generated from 1000 CP simulations, which serve as the training data set for the three ML regression algorithms explored: linear, extra-trees, and multi-layer perceptron. These three regression models were independently evaluated to compare their efficiency and identify the most suitable algorithm for the given problem. Results revealed that the extra-trees ML regressor outperforms the other models in both qualitative and quantitative aspects with an R 2 of 0.98. Subsequently, the differential evolution optimization approach is employed to calibrate the ML-based CP material parameters with experimental results obtained at various temperatures. Finally, temperature-dependent CP material parameters are formulated. The effectiveness and efficiency of the designed framework are validated through comparison with experimental results, demonstrating a high degree of agreement. These calibrated parametric constitutive equations enable further use of the CP model to study the deformation behavior of this alloy under a wide range of thermo-mechanical loading conditions.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Machine learning for the identification of phase transitions in interacting agent-based systems: A Desai-Zwanzig example

Deriving closed-form analytical expressions for reduced-order models, and judiciously choosing the closures leading to them, has long been the strategy of choice for studying phase- and noise-induced transitions for agent-based models (ABMs). In this paper, we propose a data-driven framework that pinpoints phase transitions for an ABM—the Desai-Zwanzig model—in its mean-field limit, using a smaller number of variables than traditional closed-form models. To this end, we use the manifold learning algorithm Diffusion Maps to identify a parsimonious set of data-driven latent variables, and we show that they are in one-to-one correspondence with the expected theoretical order parameter of the ABM. We then utilize a deep learning framework to obtain a conformal reparametrization of the data-driven coordinates that facilitates, in our example, the identification of a single parameter-dependent ordinary differential equation (ODE) in these coordinates. Additionally, we identify this ODE through a residual neural network inspired by a numerical integration scheme (forward Euler). We then use the identified ODE—enabled through an odd symmetry transformation—to construct the bifurcation diagram exhibiting the phase transition.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Challenges and Opportunities in Deep Reinforcement Learning With Graph Neural Networks: A Comprehensive Review of Algorithms and Applications

Deep reinforcement learning (DRL) has empowered a variety of artificial intelligence fields, including pattern recognition, robotics, recommendation-systems, and gaming. Similarly, graph neural networks (GNN) have also demonstrated their superior performance in supervised learning for graph-structured data. In recent times, the fusion of GNN with DRL for graph-structured environments has attracted a lot of attention. Here, this paper provides a comprehensive review of these hybrid works. These works can be classified into two categories: (1) algorithmic enhancement, where DRL and GNN complement each other for better utility; (2) application-specific enhancement, where DRL and GNN support each other. This fusion effectively addresses various complex problems in engineering and life sciences. Based on the review, we further analyze the applicability and benefits of fusing these two domains, especially in terms of increasing generalizability and reducing computational complexity. Finally, the key challenges in integrating DRL and GNN, and potential future research directions are highlighted, which will be of interest to the broader machine learning community.

97 MATHEMATICS AND COMPUTING↗