SEARCH · Engineering Papers

Results for “training data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 973 records · Page 54

All sources NASA NTRS DOE OSTI

DOE OSTI2023

Constructing a Simulation Surrogate with Partially Observed Output

Gaussian process surrogates are a popular alternative to directly using computationally expensive simulation models. When the simulation output consists of many responses, dimension-reduction techniques are often employed to construct these surrogates. However, surrogate methods with dimension reduction generally rely on complete output training data. This article proposes a new Gaussian process surrogate method that permits the use of partially observed output while remaining computationally efficient. The new method involves the imputation of missing values and the adjustment of the covariance matrix used for Gaussian process inference. The resulting surrogate represents the available responses, disregards the missing responses, and provides meaningful uncertainty quantification. In conclusion, the proposed approach is shown to offer sharper inference than alternatives in a simulation study and a case study where an energy density functional model that frequently returns incomplete output is calibrated.

42 ENGINEERING↗

DOE OSTI2020

The ASHRAE Great Energy Predictor III competition: Overview and results

In late 2019, ASHRAE hosted the Great Energy Predictor III (GEPIII) machine learning competition on the Kaggle platform. This launch marked the third energy prediction competition from ASHRAE and the first since the mid-1990s. In this updated version, the competitors were provided with over 20 million points of training data from 2,380 energy meters collected for 1,448 buildings from 16 sources. This competition’s overall objective was to find the most accurate modeling solutions for the prediction of over 41 million private and public test data points. Furthermore, the competition had 4,370 participants, split across 3,614 teams from 94 countries who submitted 39,403 predictions. In addition to the top five winning workflows, the competitors publicly shared 415 reproducible online machine learning workflow examples (notebooks), including over 40 additional, full solutions. This paper gives a high-level overview of the competition preparation and dataset, competitors and their discussions, machine learning workflows and models generated, winners and their submissions, discussion of lessons learned, and competition outputs and next steps. The most popular and accurate machine learning workflows used large ensembles of mostly gradient boosting tree models, such as LightGBM. Similar to the first predictor competition, preprocessing of the data sets emerged as a key differentiator.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2014

THE MILKY WAY PROJECT: LEVERAGING CITIZEN SCIENCE AND MACHINE LEARNING TO DETECT INTERSTELLAR BUBBLES

We present Brut, an algorithm to identify bubbles in infrared images of the Galactic midplane. Brut is based on the Random Forest algorithm, and uses bubbles identified by >35,000 citizen scientists from the Milky Way Project to discover the identifying characteristics of bubbles in images from the Spitzer Space Telescope. We demonstrate that Brut's ability to identify bubbles is comparable to expert astronomers. We use Brut to re-assess the bubbles in the Milky Way Project catalog, and find that 10%-30% of the objects in this catalog are non-bubble interlopers. Relative to these interlopers, high-reliability bubbles are more confined to the mid-plane, and display a stronger excess of young stellar objects along and within bubble rims. Furthermore, Brut is able to discover bubbles missed by previous searches—particularly bubbles near bright sources which have low contrast relative to their surroundings. Brut demonstrates the synergies that exist between citizen scientists, professional scientists, and machine learning techniques. In cases where ''untrained' citizens can identify patterns that machines cannot detect without training, machine learning algorithms like Brut can use the output of citizen science projects as input training sets, offering tremendous opportunities to speed the pace of scientific discovery. A hybrid model of machine learning combined with crowdsourced training data from citizen scientists can not only classify large quantities of data, but also address the weakness of each approach if deployed alone.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

Optimization and supervised machine learning methods for fitting numerical physics models without derivatives

Here, we address the calibration of a computationally expensive nuclear physics model for which derivative information with respect to the fit parameters is not readily available. Of particular interest is the performance of optimization-based training algorithms when dozens, rather than millions or more, of training data are available and when the expense of the model places limitations on the number of concurrent model evaluations that can be performed. As a case study, we consider the Fayans energy density functional model, which has characteristics similar to many model fitting and calibration problems in nuclear physics. We analyze hyperparameter tuning considerations and variability associated with stochastic optimization algorithms and illustrate considerations for tuning in different computational settings.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Machine learning assisted hybrid models can improve streamflow simulation in diverse catchments across the conterminous US

Incomplete representations of physical processes often lead to structural errors in process-based (PB) hydrologic models. Machine learning (ML) algorithms can reduce streamflow modeling errors but do not enforce physical consistency. As a result, ML algorithms may be unreliable if used to provide future hydroclimate projections where climates and land use patterns are outside the range of training data. Here we test hybrid models built by integrating PB model outputs with a ML algorithm known as Long Short-Term Memory (LSTM) network on their ability to simulate streamflow in 531 catchments representing diverse conditions across the Conterminous United States. Model performance of hybrid models as measured by Nash-Sutcliffe efficiency (NSE) improved relative to standalone PB and LSTM models. More importantly, hybrid models provide highest improvement in catchments where PB models fail completely (i.e., NSE < 0). However, all models performed poorly in catchments with extended low flow periods, suggesting need for additional research.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

Cropland abandonment between 1986 and 2018 across the United States: spatiotemporal patterns and current land uses

Knowing where and when croplands have been abandoned or otherwise removed from cultivation is fundamental to evaluating future uses of these areas, e.g. as sites for ecological restoration, recultivation, bioenergy production, or other uses. However, large uncertainties remain about the location and time of cropland abandonment and how this process and the availability of associated lands vary spatially and temporally across the United States. Here, we present a nationwide, 30 m resolution map of croplands abandoned throughout the period of 1986–2018 for the conterminous United States (CONUS). We mapped the location and time of abandonment from annual cropland layers we created in Google Earth Engine from 30 m resolution Landsat imagery using an automated classification method and training data from the U.S. Department of Agriculture Cropland Data Layer. Our abandonment map has overall accuracies of 0.91 and 0.65 for the location and time of abandonment, respectively. From 1986 to 2018, 12.3 (±2.87) million hectares (Mha) of croplands were abandoned across CONUS, with areas of greatest change over the Ogallala Aquifer, the southern Mississippi Alluvial Plain, the Atlantic Coast, North Dakota, northern Montana, and eastern Washington state. The average annual nationwide abandoned area across our study period was 0.51 Mha per year. Annual abandonment peaked between 1997 and 1999 at a rate of 0.63 Mha year –1 , followed by a continuous decrease to 0.41 Mha year –1 in 2009–2011. Among the abandoned croplands, 53% (6.5 Mha) changed to grassland and pasture, 18.6% (2.28 Mha) to shrubland and forest, 8.4% (1.03 Mha) to wetlands, and 4.6% (0.56 Mha) to non-vegetated lands. Of the areas that we mapped as abandoned, 19.6% (2.41 Mha) were enrolled in the Conservation Reserve Program as of 2020. Our new map highlights the long-term dynamic nature of agricultural land use and its relation to various competitive pressures and land use policies in the United States.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

Hubbard-corrected oxide formation enthalpies without adjustable parameters

A density functional theory (DFT) approach to computing transition metal oxide heat of formation without adjustable parameters is presented. Different degrees of d-electron localization in oxides are treated within the DFT+U approach with site-dependent, first-principles Hubbard U-parameters obtained from linear response theory, and delocalized states in the metallic phases are treated without Hubbard corrections. Comparison of relative stabilities of these differently treated phases is enabled by a local d-electron density matrix-dependent model, which was found by genetic programming against experimental reference formation enthalpies. This mathematically simple model does not explicitly depend on the Hubbard-corrected ionic species and is shown to reproduce the heats of formation of the Mott insulators Ca 2 RuO 4 and Y 2 Ru 2 O 7 within ~3% of experimental results, where the experimental training data did not contain Ru oxides. This newly developed method thus absolves from the need for element-specific corrections fitted to experiments in existing Hubbard-corrected approaches to the prediction of reaction energies of transition metal oxides and metals. The absence of fitting parameters opens up here the possibility to predict relative thermodynamic stabilities and reaction energies involving d-states of varying degree of localization at transition metal oxide interfaces and defects, where site-dependent U-parameters will be particularly important and devising a fitting scheme against experimental data with predictive power would be exceedingly difficult.

transition metal oxides↗

DOE OSTI2025

Metal oxide candidates for thermochemical water splitting obtained with a generative diffusion model

Generative diffusion models (DMs) for inorganic crystalline materials are being actively investigated for their potential to expand the chemical and structural design spaces for known functional materials. Generative candidates are particularly useful for applications where few functional, let alone commercially viable, materials currently exist, such as metal oxides for thermochemical water-splitting, which have strict requirements for defect thermodynamics and host stability. Here, we critically examine generated metal oxides from the M ATTER G EN DM conditioned on select chemical systems for thermochemical water splitting applications. Perhaps most notably, we find that M ATTER G EN predicts a novel, thermodynamically stable, quinary metal oxide, Ba 2 SrInFeO 6 , although this compound represents an ordered and layered substitution within the same A 3 B 2 O 6 structural prototype as its two ternary end members. Detailed density functional theory calculations and spin configuration sampling for this material and its possible decomposition products—beyond what existed in M ATTER G EN training data—are required to quantitatively validate hull energy predictions and conclusions of stability. Furthermore, the material exhibits oxygen defect formation energies appropriate for thermochemical water splitting, warranting targeted investigation in an experimental validation campaign, along with other future M ATTER G EN candidates in this application space.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Recipes for when physics fails: recovering robust learning of physics informed neural networks

Abstract Physics-informed neural networks (PINNs) have been shown to be effective in solving partial differential equations by capturing the physics induced constraints as a part of the training loss function. This paper shows that a PINN can be sensitive to errors in training data and overfit itself in dynamically propagating these errors over the domain of the solution of the PDE. It also shows how physical regularizations based on continuity criteria and conservation laws fail to address this issue and rather introduce problems of their own causing the deep network to converge to a physics-obeying local minimum instead of the global minimum. We introduce Gaussian process (GP) based smoothing that recovers the performance of a PINN and promises a robust architecture against noise/errors in measurements. Additionally, we illustrate an inexpensive method of quantifying the evolution of uncertainty based on the variance estimation of GPs on boundary data. Robust PINN performance is also shown to be achievable by choice of sparse sets of inducing points based on sparsely induced GPs. We demonstrate the performance of our proposed methods and compare the results from existing benchmark models in literature for time-dependent Schrödinger and Burgers’ equations.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Artificial neural network potentials for mechanics and fracture dynamics of two-dimensional crystals **

Understanding the mechanics and failure of materials at the nanoscale is critical for their engineering and applications. The accurate atomistic modeling of brittle failure with crack propagation in covalent crystals requires a quantum mechanics-based description of individual bond-breaking events. Artificial neural network potentials (NNPs) have emerged to overcome the traditional, physics-based modeling tradeoff between accuracy and accessible time and length scales. Previous studies have shown successful applications of NNPs for describing the structure and dynamics of molecular systems and amorphous or liquid phases of materials. However, their application to deformation and failure processes in materials is still uncommon. In this study, we discuss the apparent limitations of NNPs for the description of deformation and fracture under loadings and propose a way to generate and select training data for their employment in simulations of deformation and fracture simulations of crystals. We applied the proposed approach to 2D crystalline graphene, utilizing the density-functional tight-binding method for more efficient and extensive data generation in place of density functional theory. Then, we explored how the data selection affects the accuracy of the developed artificial NNPs. It revealed that NNP’s reliability should not only be measured based on the total energy and atomic force comparisons for reference structures but also utilize comparisons for physical properties, e.g. stress–strain curves and geometric deformation. In sharp contrast to popular reactive bond order potentials, our optimized NNP predicts straight crack propagation in graphene along both armchair and zigzag (ZZ) lattice directions, as well as higher fracture toughness of ZZ edge direction. Our study provides significant insight into crack propagation mechanisms on atomic scales and highlights strategies for NNP developments of broader materials.

2D materials↗

DOE OSTI2021

Blinking statistics and molecular counting in direct stochastic reconstruction microscopy (dSTORM)

Abstract Motivation Many recent advancements in single-molecule localization microscopy exploit the stochastic photoswitching of fluorophores to reveal complex cellular structures beyond the classical diffraction limit. However, this same stochasticity makes counting the number of molecules to high precision extremely challenging, preventing key insight into the cellular structures and processes under observation. Results Modelling the photoswitching behaviour of a fluorophore as an unobserved continuous time Markov process transitioning between a single fluorescent and multiple dark states, and fully mitigating for missed blinks and false positives, we present a method for computing the exact probability distribution for the number of observed localizations from a single photoswitching fluorophore. This is then extended to provide the probability distribution for the number of localizations in a direct stochastic optical reconstruction microscopy experiment involving an arbitrary number of molecules. We demonstrate that when training data are available to estimate photoswitching rates, the unknown number of molecules can be accurately recovered from the posterior mode of the number of molecules given the number of localizations. Finally, we demonstrate the method on experimental data by quantifying the number of adapter protein linker for activation of T cells on the cell surface of the T-cell immunological synapse. Availability and implementation Software and data available at https://github.com/lp1611/mol_count_dstorm. Supplementary information Supplementary data are available at Bioinformatics online.

Patel, Lekha↗

DOE OSTI2021

A machine learning approach to galaxy properties: joint redshift–stellar mass probability distributions with Random Forest

We demonstrate that highly accurate joint redshift–stellar mass probability distribution functions (PDFs) can be obtained using the Random Forest (RF) machine learning (ML) algorithm, even with few photometric bands available. As an example, we use the Dark Energy Survey (DES), combined with the COSMOS2015 catalogue for redshifts and stellar masses. We build two ML models: one containing deep photometry in the griz bands, and the second reflecting the photometric scatter present in the main DES survey, with carefully constructed representative training data in each case. We validate our joint PDFs for 10 699 test galaxies by utilizing the copula probability integral transform and the Kendall distribution function, and their univariate counterparts to validate the marginals. Benchmarked against a basic set-up of the template-fitting code bagpipes, our ML-based method outperforms template fitting on all of our predefined performance metrics. In addition to accuracy, the RF is extremely fast, able to compute joint PDFs for a million galaxies in just under 6 min with consumer computer hardware. Such speed enables PDFs to be derived in real time within analysis codes, solving potential storage issues. As part of this work we have developed galpro 1, a highly intuitive and efficient python package to rapidly generate multivariate PDFs on-the-fly. galpro is documented and available for researchers to use in their cosmology and galaxy evolution studies.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2023

DeepNeo: a webserver for predicting immunogenic neoantigens

Abstract Non-self epitopes, whether originated from foreign substances or somatic mutations, trigger immune responses when presented by major histocompatibility complex (MHC) molecules and recognized by T cells. Identification of immunogenically active neoepitopes has significant implications in cancer and virus medicine. However, current methods are mostly limited to predicting physical binding of mutant peptides and MHCs. We previously developed a deep-learning based model, DeepNeo, to identify immunogenic neoepitopes by capturing the structural properties of peptide-MHC pairs with T cell reactivity. Here, we upgraded our DeepNeo model with up-to-date training data. The upgraded model (DeepNeo-v2) was improved in evaluation metrics and showed prediction score distribution that better fits known neoantigen behavior. The immunogenic neoantigen prediction can be conducted at https://deepneo.net.

Kim, Jeong Yeon↗

DOE OSTI2017

Self-learning Monte Carlo method

Monte Carlo simulation is an unbiased numerical tool for studying classical and quantum many-body systems. One of its bottlenecks is the lack of a general and efficient update algorithm for large size systems close to the phase transition, for which local updates perform badly. In this Rapid Communication, we propose a general-purpose Monte Carlo method, dubbed self-learning Monte Carlo (SLMC), in which an efficient update algorithm is first learned from the training data generated in trial simulations and then used to speed up the actual simulation. Lastly, we demonstrate the efficiency of SLMC in a spin model at the phase transition point, achieving a 10–20 times speedup.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2021

Deep learning simulations of the microwave sky

Here we present 500 high-resolution, full-sky millimeter-wave deep learning (DL) simulations that include lensed CMB maps and correlated foreground components. We find that these MillimeterDL simulations can reproduce a wide range of non-Gaussian summary statistics matching the input training simulations, while only being optimized to match the power spectra. The procedure we develop in this work enables the capability to mass produce independent full-sky realizations from a single expensive full-sky simulation, when ordinarily the latter would not provide enough training data. We also circumvent a common limitation of high-resolution DL simulations that they be confined to small sky areas, often due to memory or GPU issues; we do this by developing a “stitching” procedure that can recover the large-scale, high-order statistics and avoid discontinuities or repeated features in the maps. In addition, since our network takes as input a full-sky lensing convergence map, it can in principle take a full-sky lensing convergence map from any large-scale structure (LSS) simulation and generate the corresponding lensed CMB and correlated foreground components at millimeter wavelengths; this is especially useful in the current era of combining results from both CMB and LSS surveys, which require a common set of simulations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2025

Resimulation-based self-supervised learning for pretraining physics foundation models

Self-supervised learning (SSL) is at the core of training modern large machine learning models, providing a scheme for learning powerful representations that can be used in a variety of downstream tasks. However, SSL strategies must be adapted to the type of training data and downstream tasks required. We propose resimulation-based self-supervised representation learning (RS3L), a novel simulation-based SSL strategy that employs a method of resimulation to drive data augmentation for contrastive learning in the physical sciences, particularly, in fields that rely on stochastic simulators. By intervening in the middle of the simulation process and rerunning simulation components downstream of the intervention, we generate multiple realizations of an event, thus producing a set of augmentations covering all physics-driven variations available in the simulator. Using experiments from high-energy physics, we explore how this strategy may enable the development of a foundation model; we show how RS3L pretraining enables powerful performance in downstream tasks such as discrimination of a variety of objects and uncertainty mitigation. In addition to our results, we make the RS3L dataset publicly available for further studies on how to improve SSL strategies.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Robust impurity detection and tracking for tokamaks

A robust impurity detection and tracking code, able to generate large sets of dust tracks from tokamak camera footage, is presented. This machine learning–based code is tested with cameras from the Joint European Torus, Doublet-III-D, and Magnum-PSI and is able to generate dust tracks with a 65 – 100 % classification accuracy. Moreover, the number dust particles detected from a single camera shot can be up to the order of 1000. Several areas of improvement for the code are highlighted, such as generating more significant training data sets and accounting for selection biases. Finally, although the code is tested with dust in single two-dimensional camera views, it could easily be applied to multiple-camera stereoscopic reconstruction or nondust impurities.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2021

Experimental quantum learning of a spectral decomposition

Currently available quantum hardware allows for small-scale implementations of quantum machine learning algorithms. Such experiments aid the search for applications of quantum computers by benchmarking the near-term feasibility of candidate algorithms. Here we demonstrate the quantum learning of a two-qubit unitary by a sequence of three parameterized quantum circuits containing a total of 21 variational parameters. Moreover, we variationally diagonalize the unitary to learn its spectral decomposition, i.e., its eigenvalues and eigenvectors. We illustrate how this can be used as a subroutine to compress the depth of dynamical quantum simulations. One can view our implementation as a demonstration of entanglement-enhanced machine learning, as only a single (entangled) training data pair is required to learn a 4 × 4 unitary matrix.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗