Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “machine learning optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,495 records · Page 83
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Photometric redshift estimation of BASS DR3 quasars by machine learning

ABSTRACT Correlating Beijing–Arizona Sky Survey (BASS) data release 3 (DR3) catalogue with the ALLWISE data base, the data from optical and infrared information are obtained. The quasars from Sloan Digital Sky Survey are taken as training and test samples while those from LAMOST are considered as external test sample. We propose two schemes to construct the redshift estimation models with XGBoost, CatBoost, and Random Forest. One scheme (namely one-step model) is to predict photometric redshifts directly based on the optimal models created by these three algorithms; the other scheme (namely two-step model) is to first classify the data into low- and high-redshift data sets, and then predict photometric redshifts of these two data sets separately. For one-step model, the performance of these three algorithms on photometric redshift estimation is compared with different training samples, and CatBoost is superior to XGBoost and Random Forest. For two-step model, the performances of these three algorithms on the classification of low and high redshift subsamples are compared, and CatBoost still shows the best performance. Therefore, CatBoost is regarded as the core algorithm of classification and regression in two-step model. In contrast to one-step model, two-step model is optimal when predicting photometric redshift of quasars, especially for high-redshift quasars. Finally, the two models are applied to predict photometric redshifts of all quasar candidates of BASS DR3. The number of high-redshift quasar candidates is 3938 (redshift ≥3.5) and 121 (redshift ≥4.5) by two-step model. The predicted result will be helpful for quasar research and follow-up observation of high-redshift quasars.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

PAIReD jet: A multi-pronged resonance tagging strategy across all Lorentz boosts

We propose a new approach of jet-based event reconstruction that aims to optimally exploit correlations between the products of a hadronic multi-pronged decay across all Lorentz boost regimes. The new approach utilizes clustered small-radius jets as seeds to define unconventional jets, referred to as PAIReD jets. The constituents of these jets are subsequently used as inputs to machine learning-based algorithms to identify the flavor content of the jet. We demonstrate that this approach achieves higher efficiencies in the reconstruction of signal events containing heavy-flavor jets compared to other event reconstruction strategies at all Lorentz boost regimes. Classifiers trained on PAIReD jets also have significantly better background rejections compared to those based on traditional event reconstruction approaches using small-radius jets at low Lorentz boost regimes. The combined effect of a higher signal reconstruction efficiency and better classification performance results in a two to four times stronger rejection of light-flavor jets compared to conventional strategies at low Lorentz-boosts, and rejection rates similar to classifiers based on large-radius multi-pronged jets at high Lorentz-boost regimes.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Assessing decision boundaries under uncertainty

In order to make design decisions, engineers may seek to identify regions of the design domain that are acceptable in a computationally efficient manner. A design is typically considered acceptable if its reliability with respect to parametric uncertainty exceeds the designer’s desired level of confidence. Despite major advancements in reliability estimation and in design classification via decision boundary estimation, the current literature still lacks a design classification strategy that incorporates parametric uncertainty and desired design confidence. To address this gap, this paper offers a novel interpretation of the acceptance region by defining the decision boundary as the hypersurface which isolates the designs that exceed a user-defined level of confidence given parametric uncertainty. This work addresses the construction of this novel decision boundary using computationally efficient algorithms that were developed for reliability analysis and decision boundary estimation. The approach proposed in this paper is verified on two physical examples from structural and thermal analysis using Support Vector Machines and Efficient Global Optimization-based contour estimation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Error-mitigated data-driven circuit learning on noisy quantum hardware

Application-level benchmarks measure how well a quantum device performs meaningful calculations. In the case of parameterized circuit training, the computational task is the preparation of a target quantum state via optimization over a loss landscape. This is complicated by various sources of noise, fixed hardware connectivity, and generative modeling, the choice of target distribution. Gradient-based training has become a useful benchmarking task for noisy intermediate-scale quantum computers because of the additional requirement that the optimization step uses the quantum device to estimate the loss function gradient. In this work, we use gradient-based data-driven circuit learning to qualitatively evaluate the performance of several superconducting platform devices and present results that show how error mitigation can improve the training of quantum circuit Born machines with 28 tunable parameters.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Manifold learning for coarse-graining atomistic simulations: Application to amorphous solids

In this work, we introduce a generalized machine learning framework to probabilistically parameterize upper-scale models in the form of nonlinear PDEs consistent with a continuum theory, based on coarse-grained atomistic simulation data of mechanical deformation and flow processes. The proposed framework utilizes a hypothesized coarse-graining methodology with manifold learning and surrogate-based optimization techniques. Coarse-grained high-dimensional data describing quantities of interest of the multiscale models are projected onto a nonlinear manifold whose geometric and topological structure is exploited for measuring behavioral discrepancies in the form of manifold distances. A surrogate model is constructed using Gaussian process regression to identify a mapping between stochastic parameters and distances. Derivative-free optimization is employed to adaptively identify a unique set of parameters of the upper-scale model capable of rapidly reproducing the system's behavior while maintaining consistency with coarse-grained atomic-level simulations. The proposed method is applied to learn the parameters of the shear transformation zone (STZ) theory of plasticity that describes plastic deformation in amorphous solids as well as coarse-graining parameters needed to translate between atomistic and continuum representations. We show that the methodology is able to successfully link coarse-grained microscale simulations to macroscale observables and achieve a high-level of parity between the models across scales.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Multiscale Flow for robust and optimal cosmological analysis

We propose Multiscale Flow, a generative Normalizing Flow that creates samples and models the field-level likelihood of two-dimensional cosmological data such as weak lensing. Multiscale Flow uses hierarchical decomposition of cosmological fields via a wavelet basis and then models different wavelet components separately as Normalizing Flows. The log-likelihood of the original cosmological field can be recovered by summing over the log-likelihood of each wavelet term. This decomposition allows us to separate the information from different scales and identify distribution shifts in the data such as unknown scale-dependent systematics. The resulting likelihood analysis can not only identify these types of systematics, but can also be made optimal, in the sense that the Multiscale Flow can learn the full likelihood at the field without any dimensionality reduction. We apply Multiscale Flow to weak lensing mock datasets for cosmological inference and show that it significantly outperforms traditional summary statistics such as power spectrum and peak counts, as well as machine learning–based summary statistics such as scattering transform and convolutional neural networks. We further show that Multiscale Flow is able to identify distribution shifts not in the training data such as baryonic effects. Finally, we demonstrate that Multiscale Flow can be used to generate realistic samples of weak lensing data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Criticality analysis of nuclear binding energy neural networks

Machine learning methods, in particular deep learning methods such as artificial neural networks (ANNs) with many layers, have become widespread and useful tools in nuclear physics. However, these ANNs are typically treated as ‘black boxes’, with their architecture (width, depth, and weight/bias initialization) and the training algorithm and parameters chosen empirically by optimizing learning based on limited exploration. We test a non-empirical approach to understanding and optimizing nuclear physics ANNs by adapting a criticality analysis based on renormalization group flows in terms of the hyperparameters for weight/bias initialization, training rates, and the ratio of depth to width. This treatment utilizes the statistical properties of neural network initialization to find a generating functional for network outputs at any layer, allowing for a path integral formulation of the ANN outputs as a Euclidean statistical field theory. We use a prototypical example to test the applicability of this approach: a simple ANN for nuclear binding energies. We find that with training using a stochastic gradient descent optimizer, the predicted criticality behavior is realized, and optimal performance is found with critical tuning. However, the use of an adaptive learning algorithm leads to somewhat superior results without concern for tuning and thus obscures the analysis. Nevertheless, the criticality analysis offers a way to look within the black box of ANNs, which is a first step towards potential improvements in network performance beyond using adaptive optimizers.

artificial neural network↗
DOE OSTI2026

Data driven drift correction for complex optical systems

To exploit the thousand-fold increase in spectral brightness of modern light sources, increasingly intricate experiments are being conducted that demand extremely precise beam trajectory. Maintaining the optimal trajectory over several hours of an experiment with the needed precision necessitates active drift control. Here, we outline time varying Bayesian optimization (TVBO) as a data driven approach for robust drift correction, and illustrate its application for a split and delay optical system composed of six crystals and twelve input dimensions. Using numerical simulations, we exhibit the application of TVBO for linear drift, non-smooth temporal drift as well as constrained TVBO for multi-objective control settings, representing real-life operating conditions. This approach can be easily adapted to other X-ray beam conditioning and guidance systems, including multi-crystal monochromators and grazing-incidence mirrors, to maintain sub-micrometer and nanoradian beam stability over the course of an experiment spanning several hours.

Bayesian optimization↗
DOE OSTI2021

SARS-CoV2 Docking Dataset

Description: Small-molecule conformations and docking scores for 1.4 billion molecules docked against 6 protein targets from SARS-CoV2: MPro 5R84, MPro 6WQF, NSP15 6WLC, PLPro 7JIR, Spike 6M0J, and a hand-optimized model of the RNA-dependent RNA polymerase. Docking was carried out using the Autodock-GPU program performing 20 independent structure minimizations per dock - saving 3 results per molecule. Scores reported include the Autodock free energy estimate as well as RF3 and VS-DUD-E v2 machine-learned rescoring models. Protein structure files and maps in the format input to Autodock-GPU are included. Literature Ref: Supercomputer-Based Ensemble Docking Drug Discovery Pipeline with Application to Covid-19, J. Chem. Inf. Model. 2020, 60(12): 5832–5852.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Artificial Intelligence/Machine Learning Technologies for Advanced Reactors (Workshop Summary Report)

A workshop on artificial intelligence and machine learning (AI/ML) for advanced reactors (AR) was held October 5-6, 2021. The workshop was to be attended in-person at ANL but COVID restrictions forced the workshop to go virtual. The objectives of the workshop were to identify the most promising AI/ML opportunities for improving advanced reactor design, optimizing plant performance, and enhancing economic competitiveness and to develop an understanding of the scientific, engineering and licensing challenges facing their application. The workshop planning committee included GAIN, EPRI and NEI and members of three national laboratories (ANL, INL, and ORNL). The workshop was attended by more than 200 individuals representing academic and scientific institutions and the nuclear power industry. The definition put forth for an AI/ML system was one that perceives its environment and takes actions that maximize its chance of achieving its goals. In this report AI/ML refers to next generation algorithms that include deep learning, statistical analysis and data analytics and associated scientific computing and their potential application to the design, licensing, operation and maintenance of ARs. These methods typically incorporate models built from process data and may also include data generated by simulations that represent the behavior of a system. The workshop was organized in response to the growing interest in application of AI/ML for improving the economic competitiveness of nuclear energy. Increasingly more resources are being allocated to investigating the benefits of AI/ML methods. The DOE created the Artificial Intelligence & Technology Office to promote their development. And within the Office of Nuclear Energy, resources have been allocated to explore and understand the potential benefits of AI/ML. Additionally, the national laboratories are strategically positioned with DOE computing facilities such as Summit, Perlmutter, Aurora and Frontier that support large-scale simulations, hybrid HPC models with AI surrogates, and the exploration of new types of generative models emerging from multi-model data streams and sources. The workshop was organized with members of the AR community to understand the effort and to identify the level of interest and progress in this emerging technology. The workshop discussions focused on identifying opportunities for AI/ML across diverse areas of the nuclear industry and identifying current scientific and engineering challenges for advanced reactors that might be addressed through transformational uses of AI/ML. Discussion panels focused on four high-interest technical domains for advanced reactors: design, maintenance and operations, energy storage, and materials. The results of those discussions are summarized in this report. This includes opportunities that were identified for exploiting AI techniques and methods to improve the efficacy and efficiency of reactor analysis and to improve the operation and optimization of advanced reactors. Advanced reactor developers expressed an interest in learning more about AI/ML methods and their application. This included understanding whether ML methods can provide an advantage over existing nonlinear data regression methods for collapsing high-fidelity simulation results into faster running models. A consensus emerged that AR advances planned for the next decade will benefit from the use of AI/ML tools. The need exists to understand and model complex systems across length scales and modalities. AI/ML is a tool for discovery that can yield a set of engineering principles for use by nuclear engineers, licensing bodies, and operators to solve problems in plant design, safety analyses, autonomous operation, and predictive maintenance. While AI/ML represents a new set of tools, an awareness by the nuclear community of the full potential is still in the early stages so there is a need to increase awareness. It appears that the wide-spread adoption of AI/ML tools for ARs would be facilitated by future educational workshops that describe foundational methods and capabilities and describe successful applications.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2021

Multi-fidelity modeling to predict the rheological properties of fiber suspensions

Unveiling the rheological properties of fiber suspensions is of paramount interest to many industrial applications like biofuel production. The 3D numerical simulations of the suspension of fibers are often computationally expensive and time-consuming. Machine learning methods such as neural networks can simplify the prediction of rheological behavior; however, they require a relatively large training data set. Multi -fidelity models, which combine high-fidelity data from numerical simulations and less expensive lower fidelity data from resources such as simplified physical equations, can lead to optimized predictions. Here, we focus on a neural network with two levels of fidelity, i.e., high and low fidelity networks. To produce high-fidelity data, we perform direct numerical simulations to model the fibers as one-dimensional inextensible slender bodies that obey the Euler- Bernoulli beam equation. The Navier-Stokes equations govern the suspended fluid, and an immersed boundary method is used to couple the fluid and solid motion. The low-fidelity data is produced by using constitutive equations. Noticeable improvements have been observed in the accuracy of predicting the rheological behavior when a multi-fidelity network is used compared to the single-fidelity network.

Boodaghidizaji, Miad↗
DOE OSTI2022

Machine learning-enabled model-data integration for predicting subsurface water storage

Subsurface water storage (SWS) is a key variable of the climate system and a storage component for precipitation and radiation anomalies, inducing persistence in the climate system. It plays a critical role in climate-change projections and can mitigate the impacts of climate change on ecosystems. However, because of the difficult accessibility of the underground, hydrologic properties and dynamics of SWS are poorly known. Direct observations of SWS are limited, and accurate incorporation of SWS dynamics into Earth system land models remains challenging. We propose a machine learning-enabled model-data integration framework to improve the SWS prediction at local to conus scales in a changing climate by leveraging all the available observation and simulation resources, as well as to inform the model development and guide the observation collection. The accurate prediction will enable an optimal decision of water management and land use and improve the ecosystem's resilience to the climate change.

Lu, Dan↗
DOE OSTI2024

Engagement: Hyperparameter Optimization of Generative Adversarial Network Models for High-Energy Physics Simulations

We present our SciDAC FASTMath-HEP partnership results for tuning generative adversarial models (GANs) for high energy physics applications. The GANs are used in hybrid simulations to accelerate otherwise time-consuming computations. We optimize for both, prediction accuracy and variability with the goal to find GAN architectures that are reliable and robust.

high energy physics↗
DOE OSTI2020

Chapter 7: Predictive Analytics in Future Power Systems: A Panorama and State-of-the-Art of Deep Learning Applications

The challenges surrounding the optimal operation of power systems are growing in various dimensions, due in part to increasingly distributed energy resources and a progression towards large-scale transportation electrification. Currently, the increasing uncertainties associated with both renewable energy generation and demand are largely being managed by increasing operational reserves - potentially at the cost of suboptimal economic conditions - in order to maintain the reliability of the system. This chapter looks at the big picture role of forecasting in power systems from generation to consumption and provides a comprehensive review of traditional approaches for forecasting generation and load in various contexts. This chapter then takes a deep dive into the state-of-the-art machine learning and deep learning approaches for power systems forecasting. Furthermore, a case study of multi-time-horizon solar irradiance forecasting using deep learning is discussed in detail. Smart grids form the backbone of the future interdependent networks. For addressing the challenges associated with the operations of smart grid, development and wide adoption of machine learning and deep learning algorithms capable of producing better forecasting accuracies is urgently needed. Along with exploring the implementation and benefits of these approaches, this chapter also considers the strengths and limitations of deep learning algorithms for power systems forecasting applications. This chapter, thus, provides a panoramic view of state-of-the-art of predictive analytics in power systems in the context of future smart grid operations.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Illuminating the Material World: Autonomous Microscopy to Understand Order, Disorder, and Everything In Between

Artificial intelligence (AI) holds immense promise for revolutionizing microscopy, yet its widespread adoption has been hindered by challenges ranging from user inexperience to limited model transferability and difficulties in operationalizing machine learning. This presentation showcases our approach to developing practical autonomy for materials discovery, aiming to accelerate the integration of AI into everyday microscopy workflows. As shown in Fig. 1, I will focus on three key areas: understanding order-disorder transitions, quantifying point defects, and achieving truly device-scale microscopy. First, I will demonstrate the power of multi-modal knowledge graphs for integrating diverse microscopy data. By combining imaging, spectroscopy, and diffraction data, these graphs provide a holistic view of material behavior, capturing the intricate relationships between different modalities [1,2]. I will present a case study on how these models illuminate the structural and chemical changes associated with irradiation in oxide thin films, revealing critical insights for designing materials for extreme environments like spaceflight and nuclear energy. Specifically, I will show how multi-modal analysis clarifies the evolution of order-disorder transitions under irradiation, a key factor influencing material performance in these applications. Next, I will address the challenge of quantifying point defects in 2D materials. We demonstrate the application of computer vision and transfer learning to accurately identify and classify various defect types, such as vacancies and substitutional atoms, and to quantify their concentrations. This information is crucial for understanding and tailoring the properties of 2D materials for applications in electronics, optoelectronics, and catalysis. For example, I will show how our models can characterize the topological distribution of point defects in MXene transition metal carbides, providing valuable insights for optimizing their performance in energy storage and separation science. Finally, I will discuss our progress toward autonomous device-scale microscopy [3,4]. We are fundamentally redesigning electron microscopes around the principles of machine reasoning, enabling automation beyond basic tasks like sample navigation and data acquisition to include sophisticated experimental design. This approach paves the way for truly reproducible and massively scaled analysis campaigns. I will emphasize the importance of autonomous microscopy platforms for high-throughput materials discovery and characterization, facilitating the rapid screening of materials for a broad range of applications and accelerating the development of next-generation technologies.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Machine learning from RANS and LES to inform coarse grid simulations

Nuclear system thermal hydraulic analysis has historically relied on computationally inexpensive 1D codes. However, such tools are unable to capture multiscale multidimensional effects in large nuclear reactor enclosures. On the other hand, simulations with higher fidelity can be too expensive for such purposes. One of the ways to reduce computational cost is to perform simulations on a coarse grid, which, unfortunately, introduces large discretization errors. In this paper, two high-to-low data-driven approaches are investigated: (1) a coarse grid turbulence model to predict eddy viscosity and (2) correction of errors in coarse grid velocity fields. The approaches aim to reduce grid- and turbulence model-induced errors in coarse grid Reynolds-averaged Navier–Stokes (RANS) simulations. Two sources of high-fidelity data, RANS and large eddy simulations (LES), are explored. To extract the eddy viscosity from the LES data, an inverse optimization problem is solved. However, the LES eddy viscosity is shown to be comparable to the RANS eddy viscosity in terms of error reduction. Therefore, the directly available RANS eddy viscosity was used to develop a coarse grid data-driven turbulence model. Additionally, error correction in velocity is used to reduce the remaining uncertainties and bring the results closer to reality. In conclusion, the performance of the frameworks is demonstrated for a scaled upper plenum of a gas-cooled reactor facility.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2026

Robust Design Under Uncertainty in Quantum Error Mitigation

Error mitigation techniques are crucial to achieving near-term quantum advantage. Classical postprocessing of quantum computation outcomes is a popular approach for error mitigation, which includes methods, such as zero noise extrapolation, virtual distillation, and learning-based error mitigation. However, these techniques have limitations due to the propagation of uncertainty resulting from the finite shot number of a quantum measurement. In this work, we introduce general and unbiased methods for quantifying the uncertainty and error of error-mitigated observables based on the strategic sampling of error mitigation outcomes. We then extend our approach to demonstrate the optimization of performance and robustness of error mitigation under uncertainty. To illustrate our methods, we apply them to zero noise extrapolation and Clifford date regression in the ground state of the XY model simulated using depolarizing and International Business Machines Corporation (IBM) Toronto noise models, respectively. In particular, we optimize the choice of noise levels and the allocation of shots for zero noise extrapolation and the distribution of the training circuits for Clifford data regression. While our methods are readily applicable to any postprocessing-based error mitigation approach, in practice they must not be prohibitively expensive—even though they perform optimizations of the error mitigation hyperparameters requiring sampling of a statistical distribution of error mitigation outcomes. By leveraging surrogate-based optimization, we show that our methods can efficiently perform optimal design for a zero noise extrapolation implementation. We then further demonstrate the transferability of learned zero noise extrapolation hyperparameters to other similar circuits.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Understanding and control of Zener pinning via phase field and ensemble learning

Zener pinning refers to the dispersion of fine particles which influences grain size distribution via movement of grain boundaries in a polycrystalline material. Grain size distribution in polycrystals has a significant impact on their properties including physical, chemical, mechanical, and optical to name a few. We explore the use of Phase-field modeling and machine-learning techniques to understand and improve the control of grain size distribution via Zener pinning in polycrystalline materials. We develop a machine learning model that determines the relative importance of various parameters to exercise microstructure control via Zener pinning. Our workflow combines high-throughput phase-field simulations and machine learning to address the computational bottlenecks associated with large-scale simulations as well as identify features necessary for microstructure control in polycrystals. A random forest (RF) regression model was developed to predict grain sizes based on five Phase-field model parameters, achieving an average prediction error of 0.72 nm for the training data and 1.44 nm for the test data. The importance of the input parameters is analyzed using the SHapley Additive exPlanations (SHAP) approach which reveals that diffusivity, volume fraction, and particle diameter are the most important parameters in determining the final grain size. These findings will allow us to select the best second-phase particles, optimize grain size distributions and thus design microstructures with the desired properties. The developed method is a highly versatile and generalizable approach that can be used to assess the combined effects of individual features in the presence of multiple variables.

36 MATERIALS SCIENCE↗