Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “statistical learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Accelerating the rate of discovery: toward high-repetition-rate HED science

As high-intensity short-pulse lasers that can operate at high-repetition-rate (HRR) (>10 H z ) come online around the world, the high energy density (HED) science they enable will experience a radical paradigm shift. The >10 3 increase in shot rate over today's shot-per-hour drivers translates into dramatically faster data acquisition, more experiments, and the ability to exploit machine learning, and thus the potential to significantly accelerate the advancement of HED science. A wide range of HED experiments, from opacity investigations to secondary source generation to plasma nuclear physics, will benefit from the increased statistics, precision, and exploration of phase space. Besides increasing the rate at which scientific experiments can be performed, HRR also allows for the rapid delivery of optimal experiments supported by simulations and modeling augmented by close coupling to empirical data. To fully realize such an HRR framework, numerous subsystems must be developed and brought together, including feedback laser control loops, high-throughput targetry and diagnostics, cognitive simulation, enhanced HED codes, and advanced data analytics. This paper describes the vision for an integrated HRR laser experimental HED system and outlines some of the major considerations and challenges for realizing it.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Inferring dark matter substructure with astrometric lensing beyond the power spectrum

Abstract Astrometry—the precise measurement of positions and motions of celestial objects—has emerged as a promising avenue for characterizing the dark matter population in our Galaxy. By leveraging recent advances in simulation-based inference and neural network architectures, we introduce a novel method to search for global dark matter-induced gravitational lensing signatures in astrometric datasets. Our method based on neural likelihood-ratio estimation shows significantly enhanced sensitivity to a cold dark matter population and more favorable scaling with measurement noise compared to existing approaches based on two-point correlation statistics. We demonstrate the real-world viability of our method by showing it to be robust to non-trivial modeled as well as unmodeled noise features expected in astrometric measurements. This establishes machine learning as a powerful tool for characterizing dark matter using astrometric data.

convolutional neural networks (1938)↗
DOE OSTI2024

UFNet: Joint U-Net and Fully Connected Neural Network to Bias Correct Precipitation Predictions from

Paper information. Shuang Yu, Indrasis Chakraborty, Gemma J. Anderson, Donald D. Lucas, Yannic Lops, and Daniel Galea. UFNet: Joint U-Net and fully connected neural network to bias correct precipitation predictions from climate models. Artificial Intelligence for the Earth Systems, 2024. Overview. This work develops the UFNet methodology to correct E3SM historical precipitation projection bias. The UFNet deep learning framework consists of a two-part architecture: a U-Net convolutional network to capture the spatiotemporal distribution of precipitation and a fully connected network to capture the distribution of higher-order statistics. The joint network, termed UFNet, can simultaneously improve the spatial structure of the modeled precipitation and capture the distribution of extreme precipitation values. Below we provide guidance for applying UFNet to correct the Energy Exascale Earth System Model (E3SM; Golaz et al. 2019) daily precipitation projection over the contiguous United States (CONUS). Getting started 1. Obtain the historical climate simulation and observation data. The E3SM historical simulation data are available through https://aims2.llnl.gov/search/cmip6/. The CPC unified gauge-based analysis of daily precipitation can be found through https://psl.noaa.gov/data/gridded/data.cpc.globalprecip.html. The ECMWF atmospheric reanalysis of the 20th century (ERA-20C) data are available through https://www.ecmwf.int/en/forecasts/datasets/reanalysis-datasets/era-20c. The spatial resolution of E3SM and observed datasets are both regridded to a common 1° resolution grid using conservative interpolation. The regridded E3SM, CPC and ERA-20C with 1° resolution can be found throught ./data/. 2. Train the fully connected network (DNN) Python train_dnn.py 3. Train the UFNet Python train_ufnet.py 4. Evaluation and compared with the baseline Python evaluation.py

Lucas, DonaldD↗
DOE OSTI2024

Divide and conquer: Learning chaotic dynamical systems with multistep penalty neural ordinary differential equations

Forecasting high-dimensional dynamical systems is a fundamental challenge in various fields, such as geosciences and engineering. Neural Ordinary Differential Equations (NODEs), which combine the power of neural networks and numerical solvers, have emerged as a promising algorithm for forecasting complex nonlinear dynamical systems. However, classical techniques used for NODE training are ineffective for learning chaotic dynamical systems. In this work, we propose a novel NODE-training approach that allows for robust learning of chaotic dynamical systems. Here, our method addresses the challenges of non-convexity and exploding gradients associated with underlying chaotic dynamics. Training data trajectories from such systems are split into multiple, non-overlapping time windows. In addition to the deviation from the training data, the optimization loss term further penalizes the discontinuities of the predicted trajectory between the time windows. The window size is selected based on the fastest Lyapunov time scale of the system. Multi-step penalty(MP) method is first demonstrated on Lorenz equation, to illustrate how it improves the loss landscape and thereby accelerates the optimization convergence. MP method can optimize chaotic systems in a manner similar to least-squares shadowing with significantly lower computational costs. Our proposed algorithm, denoted the Multistep Penalty NODE, is applied to chaotic systems such as the Kuramoto-Sivashinsky equation, the two-dimensional Kolmogorov flow, and ERA5 reanalysis data for the atmosphere. It is observed that MP-NODE provide viable performance for such chaotic systems, not only for short-term trajectory predictions but also for invariant statistics that are hallmarks of the chaotic nature of these dynamics.

Chaotic dynamical systems↗
DOE OSTI2023

Explicit solvent machine-learned coarse-grained model of sodium polystyrene sulfonate to capture polymer structure and dynamics

In this article, strongly charged polyelectrolytes (PEs) demonstrate complex solution behavior as a function of chain length, concentrations, and ionic strength. The viscosity behavior is important to understand and is a core quantity for many applications, but aspects remain a challenge. Molecular dynamics simulations using implicit solvent coarse-grained (CG) models successfully reproduce structure, but are often inappropriate for calculating viscosities. To address the need for CG models which reproduce viscoelastic properties of one of the most studied PEs, sodium polystyrene sulfonate (NaPSS), we report our recent efforts in using Bayesian optimization to develop CG models of NaPSS which capture both polymer structure and dynamics in aqueous solutions with explicit solvent. We demonstrate that our explicit solvent CG NaPSS model with the ML-BOP water model [Chan et al. Nat Commun 10, 379 (2019)] quantitatively reproduces NaPSS chain statistics and solution structure. The new explicit solvent CG model is benchmarked against diffusivities from atomistic simulations and experimental specific viscosities for short chains. We also show that our Bayesian-optimized CG model is transferable to larger chain lengths across a range of concentrations. Overall, this work provides a machine-learned model to probe the structural, dynamic, and rheological properties of polyelectrolytes such as NaPSS and aids in the design of novel, strongly charged polymers with tunable structural and viscoelastic properties

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2024

Coincident learning for unsupervised anomaly detection of scientific instruments

Abstract Anomaly detection is an important task for complex scientific experiments and other complex systems (e.g. industrial facilities, manufacturing), where failures in a sub-system can lead to lost data, poor performance, or even damage to components. While scientific facilities generate a wealth of data, labeled anomalies may be rare (or even nonexistent), and expensive to acquire. Unsupervised approaches are therefore common and typically search for anomalies either by distance or density of examples in the input feature space (or some associated low-dimensional representation). This paper presents a novel approach called coincident learning for anomaly detection (CoAD), which is specifically designed for multi-modal tasks and identifies anomalies based on coincident behavior across two different slices of the feature space. We define an unsupervised metric, F ^ β , out of analogy to the supervised classification F β statistic. CoAD uses F ^ β to train an anomaly detection algorithm on unlabeled data , based on the expectation that anomalous behavior in one feature slice is coincident with anomalous behavior in the other. The method is illustrated using a synthetic outlier data set and a MNIST-based image data set, and is compared to prior state-of-the-art on two real-world tasks: a metal milling data set and our motivating task of identifying RF station anomalies in a particle accelerator.

43 PARTICLE ACCELERATORS↗
DOE OSTI2021

Noise reduction in X-ray photon correlation spectroscopy with convolutional neural networks encoder–decoder models

Abstract Like other experimental techniques, X-ray photon correlation spectroscopy is subject to various kinds of noise. Random and correlated fluctuations and heterogeneities can be present in a two-time correlation function and obscure the information about the intrinsic dynamics of a sample. Simultaneously addressing the disparate origins of noise in the experimental data is challenging. We propose a computational approach for improving the signal-to-noise ratio in two-time correlation functions that is based on convolutional neural network encoder–decoder (CNN-ED) models. Such models extract features from an image via convolutional layers, project them to a low dimensional space and then reconstruct a clean image from this reduced representation via transposed convolutional layers. Not only are ED models a general tool for random noise removal, but their application to low signal-to-noise data can enhance the data’s quantitative usage since they are able to learn the functional form of the signal. We demonstrate that the CNN-ED models trained on real-world experimental data help to effectively extract equilibrium dynamics’ parameters from two-time correlation functions, containing statistical noise and dynamic heterogeneities. Strategies for optimizing the models’ performance and their applicability limits are discussed.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Turbulent Processes that Influence Boundary-Layer Cloud Structure (Final Technical Report)

Many models of the atmosphere simulate vertical profiles of observable fields, such as temperature, moisture, and vertical velocity. To assess the realism of such model output, it is important to evaluate the modeled profiles against observations. However, many extant evaluations consist of comparing time averaged profiles from the model and the observations. Such time averages can smooth out unrealistic features that appear in instantaneous profiles. In this way, the time averages may obscure certain model errors from detection. For instance, many models misrepresent turbulent processes, leading to profiles that are too smoothed out or too noisy. E.g., if a model has coarse vertical resolution and overly strong diffusion, then the model may be unable to produce observed sharp peaks. On the other hand, a model may have artificial approximations that produce spurious jumps in the vertical. Much can be learned by browsing through instantaneous snapshots of model output. But the volume of output produced by a global model is far too great to rely purely on visual inspection. In addition, we need a way to statistically characterize the profiles, so that anomalies can be detected more conveniently.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Constraining the Multiscale Structure of Geophysical Fields in Machine Learning: The Case of Precipitation

The use of deep-learning algorithms for estimating the value of geophysical variables from remotely sensed information is rapidly expanding. The typical objective function minimized in such algorithms is the mean square error (MSE), which is known to lead to smooth estimates with compressed dynamical range as compared to the true distribution of the variable of interest. Here, we introduce and evaluate alternative objective functions, focusing on the retrieval of precipitation rates from satellite passive microwave radiometric measurements using a deep convolutional neural network. For this testbed application, the results show that explicitly imposing the preservation of the statistical distribution and spatial wavelet power spectrum of the target variable allows to accurately reproduce extreme values and sharp gradients across multiple scales in this study.

58 GEOSCIENCES↗
DOE OSTI2022

Size-Dependent Nucleation in Crystal Phase Transition from Machine Learning Metadynamics

In this Letter, we present a framework that combines machine learning potential (MLP) and metadynamics to investigate solid-solid phase transition. Here, based on the spectral descriptors and neural networks regression, we develop a scalable MLP model to warrant an accurate interpolation of the energy surface where two phases coexist. Applying it to the simulation of B4–B1 phase transition of GaN under 50 GPa with different model sizes, we observe sequential change of the phase transition mechanism from collective modes to nucleation and growths. When the size is at or below 128 000 atoms, the nucleation and growth appear to follow a preferred direction. At larger sizes, the nuclei occur at multiple sites simultaneously and grow to microstructures by passing the critical size. The observed change of the atomistic mechanism manifests the importance of statistical sampling with large system size in phase transition modeling.

36 MATERIALS SCIENCE↗
DOE OSTI2024

A novel conditional generative model for efficient ensemble forecasts of state variables in large-scale geological carbon storage

Integrating monitoring data to efficiently update reservoir pressure and CO 2 plume distribution forecasts presents a significant challenge in geological carbon storage (GCS) applications. Inverse modeling techniques are commonly used to fuse observational data and refine reservoir model parameters, thereby improving state variable forecasts. However, these techniques often rely on linear or Gaussian assumptions, which can limit their effectiveness in accurately predicting state variables. Moreover, simulating large-scale three-dimensional (3D) GCS problems is computationally expensive, making iterative runs in inverse problems prohibitive. To address these challenges, we propose a conditional generative model utilizing the score-based diffusion method for real-time 3D pressure and saturation field distribution predictions. Our approach involves solving the score function with a mini-batch-based Monte Carlo estimator to generate labeled data. This data is subsequently employed to train a fully connected neural network, enabling it to learn the conditional sample generator within a supervised learning framework. This method enables the rapid generation of a large ensemble of predictions, facilitating comprehensive uncertainty quantification of state variables. Here we applied our method to forecast the dynamic 3D distributions of pressure and saturation fields over a 30-year injection period. The statistical assessment with low root mean square error (RMSE) values demonstrates that our method can accurately predict the spatiotemporal distributions of both pressure and saturation fields. Moreover, the developed conditional generative model shows high computational efficiency by generating 100 ensemble forecasts of 3D state variables in less than 10 min. The consistency between ensemble averages and ground truth values further illustrates the model’s capability to capture state variable dynamics during the CO 2 plume injection process. Notably, the ground truth values fall within the ensemble forecasts, indicating that our uncertainty quantification effectively captures variability and potential noise in the observations. Thus, the developed conditional generative model proves to be a more efficient, accurate, and practical tool for GCS applications, facilitating timely risk analysis and informed decision-making.

58 GEOSCIENCES↗
DOE OSTI2020

Quantifying uncertainties and correlations in the nuclear-matter equation of state

We perform statistically rigorous uncertainty quantification (UQ) for chiral effective field theory (χ EFT) applied to infinite nuclear matter up to twice nuclear saturation density. The equation of state (EOS) is based on high-order many-body perturbation theory calculations with nucleon-nucleon and three-nucleon interactions up to fourth order in the χ EFT expansion. From these calculations our newly developed Bayesian machine-learning approach extracts the size and smoothness properties of the correlated EFT truncation error. Furthermore, we then propose a novel extension that uses multitask machine learning to reveal correlations between the EOS at different proton fractions. The inferred in-medium χ EFT breakdown scale in pure neutron matter and symmetric nuclear matter is consistent with that from free-space nucleon-nucleon scattering. These significant advances allow us to provide posterior distributions for the nuclear saturation point and propagate theoretical uncertainties to derived quantities: the pressure and incompressibility of symmetric nuclear matter, the nuclear symmetry energy, and its derivative. Our results, which are validated by statistical diagnostics, demonstrate that an understanding of truncation-error correlations between different densities and different observables is crucial for reliable UQ. The methods developed here are publicly available as annotated Jupyter notebooks.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Large-scale scenarios of electric vehicle charging with a data-driven model of control

Transportation electrification is forecast to bring millions of new electric vehicles to roads worldwide this decade. Planning to support those vehicles depends on detailed scenarios of their electricity demand in both uncontrolled and controlled or smart charging scenarios. In this work, we present a novel modeling approach to enable rapid generation of demand estimates that represent the impact of controlled charging for large-scale scenarios with millions of individual drivers. To model the effect of load modulation control on aggregate charging profiles, we propose a novel machine learning approach that replaces traditional optimization approaches. We demonstrate its performance modeling workplace charging control under a range of electricity rate schedules, achieving small errors (2.5%–4.5%) while accelerating computations by more than 4000 times. To generate the uncontrolled charging demand for scenarios with residential, workplace, and public charging we use statistical representations of a large data set of real charging sessions. We demonstrate the methodology by generating diverse sets of scenarios for California's charging demand in 2030 which consider multiple charging segments and controls, each run locally in under 50 s. We further demonstrate support for rate design by modeling the large-scale impact of a new, custom rate schedule for workplace charging.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2020

Progress Toward Interpretable Machine Learning–Based Disruption Predictors Across Tokamaks

Here in this paper we lay the groundwork for a robust cross-device comparison of data-driven disruption prediction algorithms on DIII-D and JET tokamaks. In order to consistently carry on a comparative analysis, we define physics-based indicators of disruption precursors based on temperature, density, and radiation profiles that are currently not used in many other machine learning predictors for DIII-D data. These profile-based indicators are shown to well-describe impurity accumulation events in both DIII-D and JET discharges that eventually disrupt. The univariate analysis of the features used as input signals in the data-driven algorithms applied on the data of both tokamaks statistically highlights the differences in the dominant disruption precursors. JET with its ITER-like wall is more prone to impurity accumulation events, while DIII-D is more subject to edge-cooling mechanisms that destabilize dangerous magnetohydrodynamic modes. Even though the analyzed data sets are characterized by such intrinsic differences, we show through a few examples that the inclusion of physics-based disruption markers in data-driven algorithms is a promising path toward the realization of a uniform framework to predict and interpret disruptive scenarios across different tokamaks. As long as the destabilizing precursors are diagnosed in a device-independent way, the knowledge that data-driven algorithms learn on one device can be re-used to explain a disruptive behavior on another device.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Search for new particles in events with energetic jets and large missing transverse momentum in proton-proton collisions at $ \sqrt{s} $ = 13 TeV

A search is presented for new particles produced at the LHC in proton-proton collisions at s√ = 13 TeV, using events with energetic jets and large missing transverse momentum. The analysis is based on a data sample corresponding to an integrated luminosity of 101 fb -1 , collected in 2017–2018 with the CMS detector. Machine learning techniques are used to define separate categories for events with narrow jets from initial-state radiation and events with large-radius jets consistent with a hadronic decay of a W or Z boson. A statistical combination is made with an earlier search based on a data sample of 36 fb -1 , collected in 2016. No significant excess of events is observed with respect to the standard model background expectation determined from control samples in data. The results are interpreted in terms of limits on the branching fraction of an invisible decay of the Higgs boson, as well as constraints on simplified models of dark matter, on first-generation scalar leptoquarks decaying to quarks and neutrinos, and on models with large extra dimensions. Several of the new limits, specifically for spin-1 dark matter mediators, pseudoscalar mediators, colored mediators, and leptoquarks, are the most restrictive to date.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Deciphering the source of primary biological aerosol particles: a pollen case study

Primary biological aerosol particles (PBAPs) are microscopic solids suspended in the atmosphere emitted by biological systems and play critical roles in the atmosphere and at the atmosphere-biosphere interface, impacting human health, climate, and the ecosystem function. Understanding the sources of PBAPs is necessary to decipher the mechanistic interactions between aerosols, climate, and distinct ecosystem components. However, the detection of specific PBAPs in complex ambient aerosol samples is challenging. We performed metabolomics analyses of pollen from three pollinating tree species and ambient samples collected during the peak pollination period of each species. Random Forest and sPLS-DA machine learning methods were employed to evaluate whether metabolic signatures of ambient samples can reveal the source of the main pollen particles present in the atmosphere. Our results suggest that atmospheric eco-metabolomics techniques combined with sophisticated statistical methods can decipher the origin of abundant PBAPs from complex ambient samples. Developing complete libraries containing high-resolution metabolomic fingerprints of the major PBAPs present in the atmosphere would significantly advance future research to accurately understand the role of PBAPs in the atmosphere, ecosystems and human health.

Rivas-Ubach, Albert↗
DOE OSTI2023

Why is the winner the best?

International benchmarking competitions have become fundamental for the comparative performance assessment of image analysis methods. However, little attention has been given to investigating what can be learnt from these competitions. Do they really generate scientific progress? What are common and successful participation strategies? What makes a solution superior to a competing method? To address this gap in the literature, we performed a multi- center study with all 80 competitions that were conducted in the scope of IEEE ISBI 2021 and MICCAI 2021. Statistical analyses performed based on comprehensive descriptions of the submitted algorithms linked to their rank as well as the underlying participation strategies revealed common characteristics of winning solutions. These typically include the use of multi-task learning (63%) and/or multi-stage pipelines (61%), and a focus on augmentation (100%), im- age preprocessing (97%), data curation (79%), and post- processing (66%). The “typical” lead of a winning team is a computer scientist with a doctoral degree, five years of experience in biomedical image analysis, and four years of experience in deep learning. Two core general development strategies stood out for highly-ranked teams: the reflection of the metrics in the method design and the focus on analyzing and handling failure cases. According to the organizers, 43% of the winning algorithms exceeded the state of the art but only 11% completely solved the respective domain problem. The insights of our study could help researchers (1) improve algorithm development strategies when approaching new problems, and (2) focus on open research questions revealed by this work.

Eisenmann, Matthias↗
DOE OSTI2020

Advancing Fusion with Machine Learning Research Needs Workshop Report

Abstract Machine learning and artificial intelligence (ML/AI) methods have been used successfully in recent years to solve problems in many areas, including image recognition, unsupervised and supervised classification, game-playing, system identification and prediction, and autonomous vehicle control. Data-driven machine learning methods have also been applied to fusion energy research for over 2 decades, including significant advances in the areas of disruption prediction, surrogate model generation, and experimental planning. The advent of powerful and dedicated computers specialized for large-scale parallel computation, as well as advances in statistical inference algorithms, have greatly enhanced the capabilities of these computational approaches to extract scientific knowledge and bridge gaps between theoretical models and practical implementations. Large-scale commercial success of various ML/AI applications in recent years, including robotics, industrial processes, online image recognition, financial system prediction, and autonomous vehicles, have further demonstrated the potential for data-driven methods to produce dramatic transformations in many fields. These advances, along with the urgency of need to bridge key gaps in knowledge for design and operation of reactors such as ITER, have driven planned expansion of efforts in ML/AI within the US government and around the world. The Department of Energy (DOE) Office of Science programs in Fusion Energy Sciences (FES) and Advanced Scientific Computing Research (ASCR) have organized several activities to identify best strategies and approaches for applying ML/AI methods to fusion energy research. This paper describes the results of a joint FES/ASCR DOE-sponsored Research Needs Workshop on Advancing Fusion with Machine Learning, held April 30–May 2, 2019, in Gaithersburg, MD (full report available at https://science.osti.gov/-/media/fes/pdf/workshop-reports/FES_ASCR_Machine_Learning_Report.pdf ). The workshop drew on broad representation from both FES and ASCR scientific communities, and identified seven Priority Research Opportunities (PRO’s) with high potential for advancing fusion energy. In addition to the PRO topics themselves, the workshop identified research guidelines to maximize the effectiveness of ML/AI methods in fusion energy science, which include focusing on uncertainty quantification, methods for quantifying regions of validity of models and algorithms, and applying highly integrated teams of ML/AI mathematicians, computer scientists, and fusion energy scientists with domain expertise in the relevant areas.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗