Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “ensemble methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

STSR-INR: Spatiotemporal super-resolution for multivariate time-varying volumetric data via implicit neural representation

Implicit neural representation (INR) has surfaced as a promising direction for solving different scientific visualization tasks due to its continuous representation and flexible input and output settings. We present STSR-INR, an INR solution for generating simultaneous spatiotemporal super-resolution for multivariate time-varying volumetric data. Inheriting the benefits of the INR-based approach, STSR-INR supports unsupervised learning and permits data upscaling with arbitrary spatial and temporal scale factors. Unlike existing GAN- or INR-based super-resolution methods, STSR-INR focuses on tackling variables or ensembles and enabling joint training across datasets of various spatiotemporal resolutions. Here we achieve this capability via a variable embedding scheme that learns latent vectors for different variables. In conjunction with a modulated structure in the network design, we employ a variational auto-decoder to optimize the learnable latent vectors to enable latent-space interpolation. To combat the slow training of INR, we leverage a multi-head strategy to improve training and inference speed with significant speedup. We demonstrate the effectiveness of STSR-INR with multiple scalar field datasets and compare it with conventional tricubic+linear interpolation and state-of-the-art deep-learning-based solutions (STNet and CoordNet).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Data Imbalance, Uncertainty Quantification, and Transfer Learning in Data‐Driven Parameterizations: Lessons From the Emulation of Gravity Wave Momentum Transport in WACCM

Abstract Neural networks (NNs) are increasingly used for data‐driven subgrid‐scale parameterizations in weather and climate models. While NNs are powerful tools for learning complex non‐linear relationships from data, there are several challenges in using them for parameterizations. Three of these challenges are (a) data imbalance related to learning rare, often large‐amplitude, samples; (b) uncertainty quantification (UQ) of the predictions to provide an accuracy indicator; and (c) generalization to other climates, for example, those with different radiative forcings. Here, we examine the performance of methods for addressing these challenges using NN‐based emulators of the Whole Atmosphere Community Climate Model (WACCM) physics‐based gravity wave (GW) parameterizations as a test case. WACCM has complex, state‐of‐the‐art parameterizations for orography‐, convection‐, and front‐driven GWs. Convection‐ and orography‐driven GWs have significant data imbalance due to the absence of convection or orography in most grid points. We address data imbalance using resampling and/or weighted loss functions, enabling the successful emulation of parameterizations for all three sources. We demonstrate that three UQ methods (Bayesian NNs, variational auto‐encoders, and dropouts) provide ensemble spreads that correspond to accuracy during testing, offering criteria for identifying when an NN gives inaccurate predictions. Finally, we show that the accuracy of these NNs decreases for a warmer climate (4 × CO 2 ). However, their performance is significantly improved by applying transfer learning, for example, re‐training only one layer using ∼1% new data from the warmer climate. The findings of this study offer insights for developing reliable and generalizable data‐driven parameterizations for various processes, including (but not limited to) GWs.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Grafting nanometer metal/oxide interface towards enhanced low-temperature acetylene semi-hydrogenation

Metal/oxide interface is of fundamental significance to heterogeneous catalysis because the seemingly “inert” oxide support can modulate the morphology, atomic and electronic structures of the metal catalyst through the interface. The interfacial effects are well studied over a bulk oxide support but remain elusive for nanometer-sized systems like clusters, arising from the challenges associated with chemical synthesis and structural elucidation of such hybrid clusters. We hereby demonstrate the essential catalytic roles of a nanometer metal/oxide interface constructed by a hybrid Pd/Bi 2 O 3 cluster ensemble, which is fabricated by a facile stepwise photochemical method. The Pd/Bi 2 O 3 cluster, of which the hybrid structure is elucidated by combined electron microscopy and microanalysis, features a small Pd-Pd coordination number and more importantly a Pd-Bi spatial correlation ascribed to the heterografting between Pd and Bi terminated Bi 2 O 3 clusters. The intra-cluster electron transfer towards Pd across the as-formed nanometer metal/oxide interface significantly weakens the ethylene adsorption without compromising the hydrogen activation. As a result, a 91% selectivity of ethylene and 90% conversion of acetylene can be achieved in a front-end hydrogenation process with a temperature as low as 44 °C.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Edge Reconstruction in a Quantum Spin Hall Insulator

We study interaction-driven edge reconstruction in a quantum spin Hall insulator described by the Bernevig-Hughes-Zhang model with Kanamori-Hubbard interactions using the real-space density matrix renormalization group method in both the grand-canonical and canonical ensembles. For a two-dimensional cylinder with a smooth edge, we identify discrete particle-number transitions that lead to a spin-polarized edge state stabilized by an emergent ferromagnetic exchange interaction. The reconstruction is orbital-selective, occurring predominantly in the 𝑠-orbital channel. Our results reveal a microscopic mechanism for emergent fluctuating moments at the edge that could compromise the topological protection of helical edge states by time-reversal symmetry.

Soni, Rahul [ORNL] (ORCID:0000000317714299)↗
DOE OSTI2025

MCCCS-MN

The MCCCS‒MN (Monte Carlo for Complex Chemical Systems‒Minnesota) software is developed by the Siepmann research group at the University of Minnesota. MCCCS‒MN allows for the simulation of multi-component molecular systems in the canonical, isobaric-isothermal (including constant stress for solids), grand-canonical, semi-grand, and Gibbs (NVT, NPT, and more than two simulation boxes) ensembles. It uses the configurational-bias Monte Carlo method to efficiently sample phase space for linear, branched and cyclic chain molecules, the adiabatic nuclear and electronic sampling Monte Carlo method to treat many-body polarization effects, and the aggregation-volume-bias Monte Carlo algorithm to efficiently sample the spatial distribution of associating molecules. MCCCS-MN employs a molecular representation of the system where force fields contain bonded and non-bonded terms. Funding for the development of MCCCS-MN through grants from the National Science Foundation (simulation of fluid phase equilibria and chromatography) and the Department of Energy (simulation of adsorption equilibria) is gratefully acknowledged.

Siepmann, J.Ilja [University of Minnesota - Twin C↗
DOE OSTI2020

Conservative Estimation of Tail Probabilities from Limited Sample Data

Several sparse-sample uncertainty quantification (UQ) methods are compared for conservative but not overly conservative estimation of small tail probabilities involving responses that lay beyond specified thresholds in the tails of probability distributions. Sixteen very differently shaped distributions (or probability density functions, PDFs) and tail probability magnitudes ranging from 10 -5 to 10 -1 are considered in order for the study to be relevant to a wide range of risk analysis and quantification of margins and uncertainty (QMU) problems. The emphasis of the study is on limited data regimes ranging from N = 2 to 20 samples, reflective of most experimental and some expensive computational situations. Relatively simple sparse-sample UQ methods tested for this regime involve statistical tolerance interval "Equivalent Normal and related "Ensemble of Normals" and "Superdistribution (SD) approaches. (The independently derived SD is effectively equivalent to the Bayesian posterior predictive distribution given the assumptions of the derivation.) The performance of the methods was generally improved for N ≥ 5 samples with a generalized Jackknife resampling technique, which determines a tail probability estimate by averaging estimates from smaller sub-samples. Several quantitative metrics for method conservatism and accuracy of tail probability estimation are used to assess and rank the methods' performance over many random trials for each test PDF and probability magnitude. A variant of Bootstrap resampling was also tried, but did not significantly improve tail probability estimates in most cases. Detailed results are presented from over 100-million tests over the above factors that provide useful granular information on which methods or combination of methods perform best in various areas of the factor space.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Data Science and Machine Learning for Genome Security

This report describes research conducted to use data science and machine learning methods to distinguish targeted genome editing versus natural mutation and sequencer machine noise. Genome editing capabilities have been around for more than 20 years, and the efficiencies of these techniques has improved dramatically in the last 5+ years, notably with the rise of CRISPR-Cas technology. Whether or not a specific genome has been the target of an edit is concern for U.S. national security. The research detailed in this report provides first steps to address this concern. A large amount of data is necessary in our research, thus we invested considerable time collecting and processing it. We use an ensemble of decision tree and deep neural network machine learning methods as well as anomaly detection to detect genome edits given either whole exome or genome DNA reads. The edit detection results we obtained with our algorithms tested against samples held out during training of our methods are significantly better than random guessing, achieving high F1 and recall scores as well as with precision overall.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Gradient-Based Novelty Detection Boosted by Self-Supervised Binary Classification

Novelty detection aims to automatically identify out-of-distribution (OOD) data, without any prior knowledge of them. It is a critical step in data monitoring, behavior analysis and other applications, helping enable continual learning in the field. Conventional methods of OOD detection perform multi-variate analysis on an ensemble of data or features, and usually resort to the supervision with OOD data to improve the accuracy. In reality, such supervision is impractical as one cannot anticipate the anomalous data. In this paper, we propose a novel, self-supervised approach that does not rely on any pre-defined OOD data: (1) The new method evaluates the Mahalanobis distance of the gradients between the in-distribution and OOD data. (2) It is assisted by a self-supervised binary classifier to guide the label selection to generate the gradients, and maximize the Mahalanobis distance. In the evaluation with multiple datasets, such as CIFAR-10, CIFAR-100, SVHN and TinyImageNet, the proposed approach consistently outperforms state-of-the-art supervised and unsupervised methods in the area under the receiver operating characteristic (AUROC) and area under the precision-recall curve (AUPR) metrics. We further demonstrate that this detector is able to accurately learn one OOD class in continual learning.

Sun, Jingbo↗
DOE OSTI2021

Analytical derivatives of the individual state energies in ensemble density functional theory. II. Implementation on graphical processing units (GPUs)

Conical intersections control excited state reactivity, and thus, elucidating and predicting their geometric and energetic characteristics are crucial for understanding photochemistry. Locating these intersections requires accurate and efficient electronic structure methods. Unfortunately, the most accurate methods (e.g., multireference perturbation theories such as XMS-CASPT2) are computationally challenging for large molecules. The state-interaction state-averaged restricted ensemble referenced Kohn–Sham (SI-SA-REKS) method is a computationally efficient alternative. The application of SI-SA-REKS to photochemistry was previously hampered by a lack of analytical nuclear gradients and nonadiabatic coupling matrix elements. We have recently derived analytical energy derivatives for the SI-SA-REKS method and implemented the method effectively on graphical processing units. We demonstrate that our implementation gives the correct conical intersection topography and energetics for several examples. Furthermore, our implementation of SI-SA-REKS is computationally efficient, with observed sub-quadratic scaling as a function of molecular size. This demonstrates the promise of SI-SA-REKS for excited state dynamics of large molecular systems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Global Carbon Budget 1800-2018 Loss of Additional Sink Capacity and Present Transient Difference

A dataset of two global carbon fluxes: the loss of additional sink capacity (LASC) and present versus transient difference (PTD). The loss of additional sink capacity is the loss of indirect anthropogenic sink capacity in land ecosystems caused by the loss of ecosystems that can increase carbon in response to changing environments, primarily forests. The present transient difference refers to the direct anthropogenic flux in land ecosystems (from land cover and land use change) and the difference in that flux when calculated assuming "present day" (actually more like 1980) ecosystem carbon stocks or when calculated assuming transient carbon stocks (i.e. caused by environmental change). In this dataset, the LASC and PTD fluxes were calculated using the TRENDY ensemble of Dynamic Global Vegetation Models (DGVMs; Sitch et al., 2024) following the methods described therein and in Obermeier et al. (2021). For further details see Obermeier et al. (2021), Friedlingstein et al. (2019), Sitch et al. (2024), and Walker et al. (2025). The dataset includes ensemble means and standard deviations for annual fluxes, fluxes cumulated annually over the whole time period, and cumulated since 1959. These data were originally collected to compare different approaches (DGVMs versus bookkeeping models) for estimating land use and land cover change (LULCC) emissions (Obermeier et al. 2021). More recently, and similarly, these data were used to harmonize the estimates of the direct and indirect anthropogenic fluxes in the global carbon cycle and to recalculate the global carbon budget (Walker et al., 2025).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

coh3

CoH3 (CoH ver.3) is an optical model, exciton pre-equilibrium, and Hauser-Feshbach statistical model code, which calculates nuclear reaction cross sections for medium to heavy targets in the keV to MeV energy region. This program is written in standard C++, divided into approximately 200 source and header files. CoH solves the Schroedinger equation for optical potentials defined in the code, and calculates differential elastic scattering, reaction, and total cross sections, for neutron, proton, deuteron, triton, 3He, and alpha-particle. Deformed optical potentials are solved with the coupled-channels method, in which the ground state rotational band members, or vibrational phonon states are coupled. The optical model gives particle transmission coefficients that are fed into the statistical model calculations. CoH includes the pre-equilibrium model (exciton model), the direct/semidirect capture model, and the multi-stage Hauser-Feshbach statistical decay with width fluctuation correction based on the Gaussian orthogonal ensemble. For weakly coupled levels, the DWBA (distorted wave Born approximation) method is used to calculate the direct inelastic scattering process to the excited states.

Kawano, Toshihiko↗
DOE OSTI2021

Scalable statistical inference of photometric redshift via data subsampling

Handling big data has largely been a major bottleneck in traditional statistical models. Consequently, when accurate point prediction is the primary target, machine learning models are often preferred over their statistical counterparts for bigger problems. But full probabilistic statistical models often outperform other models in quantifying uncertainties associated with model predictions. We develop a data-driven statistical modeling framework that combines the uncertainties from an ensemble of statistical models learned on smaller subsets of data carefully chosen to account for imbalances in the input space. We demonstrate this method on a photometric redshift estimation problem in cosmology, which seeks to infer a distribution of the redshift—the stretching effect in observing the light of far-away galaxies—given multivariate color information observed for an object in the sky. Our proposed method performs balanced partitioning, graph-based data subsampling across the partitions, and training of an ensemble of Gaussian process models.

data subsampling↗
DOE OSTI2022

Hybrid quantum-classical simulations of magic angle spinning dynamic nuclear polarization in very large spin systems

Solid-state nuclear magnetic resonance can be enhanced using unpaired electron spins with a method known as dynamic nuclear polarization (DNP). Fundamentally, DNP involves ensembles of thousands of spins, a scale that is difficult to match computationally. This scale prevents us from gaining a complete understanding of the spin dynamics and applying simulations to design sample formulations. We recently developed an ab initio model capable of calculating DNP enhancements in systems of up to ~1000 nuclei; however, this scale is insufficient to accurately simulate the dependence of DNP enhancements on radical concentration or magic angle spinning (MAS) frequency. We build on this work by using ab initio simulations to train a hybrid model that makes use of a rate matrix to treat nuclear spin diffusion. We show that this model can reproduce the MAS rate and concentration dependence of DNP enhancements and build-up time constants. Finally, we then apply it to predict the DNP enhancements in core–shell metal-organic-framework nanoparticles and reveal new insights into the composition of the particles’ shells.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Sequential Kalman tuning of the t -preconditioned Crank-Nicolson algorithm: efficient, adaptive and gradient-free inference for Bayesian inverse problems

Ensemble Kalman Inversion (EKI) has been proposed as an efficient method for the approximate solution of Bayesian inverse problems with expensive forward models. However, when applied to the Bayesian inverse problem EKI is only exact in the regime of Gaussian target measures and linear forward models. Here, in this work we propose embedding EKI and Flow Annealed Kalman Inversion, its normalizing flow (NF) preconditioned variant, within a Bayesian annealing scheme as part of an adaptive implementation of the t-preconditioned Crank-Nicolson (tpCN) sampler. The tpCN sampler differs from standard pCN in that its proposal is reversible with respect to the multivariate t-distribution. The more flexible tail behaviour allows for better adaptation to sampling from non-Gaussian targets. Within our Sequential Kalman Tuning (SKT) adaptation scheme, EKI is used to initialize and precondition the tpCN sampler for each annealed target. The subsequent tpCN iterations ensure particles are correctly distributed according to each annealed target, avoiding the accumulation of errors that would otherwise impact EKI. We demonstrate the performance of SKT for tpCN on three challenging numerical benchmarks, showing significant improvements in the rate of convergence compared to adaptation within standard SMC with importance weighted resampling at each temperature level, and compared to similar adaptive implementations of standard pCN. The SKT scheme applied to tpCN offers an efficient, practical solution for solving the Bayesian inverse problem when gradients of the forward model are not available. Code implementing the SKT schemes for tpCN is available at https://github.com/RichardGrumitt/KalmanMC.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Combining Emergent Constraints for Climate Sensitivity

A method is proposed for combining information from several emergent constraints into a probabilistic estimate for a climate sensitivity proxy Y such as equilibrium climate sensitivity (ECS). The method is based on fitting a multivariate Gaussian PDF for Y and the emergent constraints using an ensemble of global climate models (GCMs); it can be viewed as a form of multiple linear regression of Y on the constraints. The method accounts for uncertainties in sampling this multidimensional PDF with a small number of models, for observational uncertainties in the constraints, and for overconfidence about the correlation of the constraints with the climate sensitivity. Its general form (Method C) accounts for correlations between the constraints. Method C becomes less robust when some constraints are too strongly related to each other; this can be mitigated using regularization approaches such as ridge regression. An illuminating special case, Method U, neglects any correlations between constraints except through their mutual relationship to the climate proxy; it is more robust to small GCM sample size and is appealingly interpretable. These methods are applied to ECS and the climate feedback parameter using a previously published set of 11 possible emergent constraints derived from climate models in the Coupled Model Intercomparison Project (CMIP). The ±2σ posterior range of ECS for Method C with no overconfidence adjustment is 4.3 ± 0.7 K. For Method U with a large overconfidence adjustment, it is 4.0 ± 1.3 K. This study adds confidence to past findings that most constraints predict higher climate sensitivity than the CMIP mean.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Distilling Knowledge from Ensembles of Cluster-Constrained-Attention Multiple-Instance Learners for Whole Slide Image Classification

The peculiar nature of whole slide imaging (WSI), digitizing conventional glass slides to obtain multiple high resolution images which capture microscopic details of a patient’s histopathological features, has garnered increased interest from the computer vision research community over the last two decades. Given the unique computational space and time complexity inherent to gigapixel-size whole slide image data, researchers have proposed novel machine learning algorithms to aid in the performance of diagnostic tasks in clinical pathology. One effective algorithm represents a Whole slide image as a bag of smaller image patches, which can be represented as low-dimension image patch embeddings. Weakly supervised deep-learning methods, such as cluster-constrained-attention multiple instance learning (CLAM), have shown promising results when combined with image patch embeddings. While traditional ensemble classifiers yield improved task performance, such methods come with a steep cost in model complexity. Through knowledge distillation, it is possible to retain some performance improvements from an ensemble, while minimizing costs to model complexity. In this work, we implement a weakly supervised ensemble using clustering-constrained-attention multiple-instance learners (CLAM), which uses attention and instance-level clustering to identify task salient regions and feature extraction in whole slides. By applying logit-based and attention-based knowledge distillation, we show it is possible to retain some performance improvements resulting from the ensemble at zero cost to model complexity.

Alamudun, Folami↗
DOE OSTI2021

Galaxy Zoo DECaLS: Detailed visual morphology measurements from volunteers and deep learning for 314,000 galaxies

We present Galaxy Zoo DECaLS: detailed visual morphological classifications for Dark Energy Camera Legacy Survey images of galaxies within the SDSS DR8 footprint. Deeper DECaLS images (r = 23.6 versus r = 22.2 from SDSS) reveal spiral arms, weak bars, and tidal features not previously visible in SDSS imaging. To best exploit the greater depth of DECaLS images, volunteers select from a new set of answers designed to improve our sensitivity to mergers and bars. Galaxy Zoo volunteers provide 7.5 million individual classifications over 314 000 galaxies. 140 000 galaxies receive at least 30 classifications, sufficient to accurately measure detailed morphology like bars, and the remainder receive approximately 5. All classifications are used to train an ensemble of Bayesian convolutional neural networks (a state-of-the-art deep learning method) to predict posteriors for the detailed morphology of all 314 000 galaxies. We use active learning to focus our volunteer effort on the galaxies which, if labelled, would be most informative for training our ensemble. When measured against confident volunteer classifications, the trained networks are approximately 99 per cent accurate on every question. Morphology is a fundamental feature of every galaxy; our human and machine classifications are an accurate and detailed resource for understanding how galaxies evolve.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Atomistic learning in the electronically grand-canonical ensemble

Abstract A strategy is presented for the machine-learning emulation of electronic structure calculations carried out in the electronically grand-canonical ensemble. The approach relies upon a dual-learning scheme, where both the system charge and the system energy are predicted for each image. The scheme is shown to be capable of emulating basic electrochemical reactions at a range of potentials, and coupling it with a bootstrap-ensemble approach gives reasonable estimates of the prediction uncertainty. The method is also demonstrated to accelerate saddle-point searches, and to extrapolate to systems with one to five water layers. We anticipate that this method will allow for larger length- and time-scale simulations necessary for electrochemical simulations.

36 MATERIALS SCIENCE↗