Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Probabilistic numerics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Enhancing Gaussian Process Surrogates for Optimization and Posterior Approximation via Random Exploration

This paper proposes novel noise-free Bayesian optimization strategies that rely on a random exploration step to enhance the accuracy of Gaussian process surrogate models. The new algorithms retain the ease of implementation of the classical GP-UCB algorithm, but the additional random exploration step accelerates their convergence, nearly achieving the optimal convergence rate. Furthermore, to facilitate Bayesian inference with intractable likelihoods, we propose to utilize optimization iterates for maximum a posteriori estimation to build a Gaussian process surrogate model for the unnormalized log-posterior density. We provide bounds for the Hellinger distance between the true and the approximate posterior distributions in terms of the number of design points. We demonstrate the effectiveness of our Bayesian optimization algorithms in nonconvex benchmark objective functions, in a machine learning hyperparameter tuning problem, and in a black-box engineering design problem. The effectiveness of our posterior approximation approach is demonstrated in two Bayesian inference problems for parameters of dynamical systems.

Bayesian inference↗
DOE OSTI2025

A comparison of probabilistic generative frameworks for molecular simulations

Generative artificial intelligence is now a widely used tool in molecular science. Despite the popularity of probabilistic generative models, numerical experiments benchmarking their performance on molecular data are lacking. Here, in this work, we introduce and explain several classes of generative models, broadly sorted into two categories: flow-based models and diffusion models. We select three representative models: neural spline flows, conditional flow matching, and denoising diffusion probabilistic models, and examine their accuracy, computational cost, and generation speed across datasets with tunable dimensionality, complexity, and modal asymmetry. Our findings are varied, with no one framework being the best for all purposes. In a nutshell, (i) neural spline flows do best at capturing mode asymmetry present in low-dimensional data, (ii) conditional flow matching outperforms other models for high-dimensional data with low complexity, and (iii) denoising diffusion probabilistic models appear the best for low-dimensional data with high complexity. Our datasets include a Gaussian mixture model and the dihedral torsion angle distribution of the Aib9 peptide, generated via a molecular dynamics simulation. We hope our taxonomy of probabilistic generative frameworks and numerical results may guide model selection for a wide range of molecular tasks.

Artificial intelligence↗
DOE OSTI2025

Probabilistic flux limiters

The stable numerical integration of shocks in compressible flow simulations relies on the reduction or elimination of Gibbs phenomena (unstable, spurious oscillations). A popular method to virtually eliminate Gibbs oscillations caused by numerical discretization in under-resolved simulations is to use a flux limiter. A wide range of flux limiters have been studied in the literature, with recent interest in their optimization via machine learning methods trained on high-resolution datasets. The common use of flux limiters in numerical codes as plug-and-play blackbox components makes them key targets for design improvement. Even for deterministic dynamical models, numerical uncertainty is introduced via coarse-graining required by insufficient computational power to solve all scales of motion. Conventional flux limiters are deterministic and lack the capacity to address uncertainties, both aleatoric (inherent randomness) and epistemic (modeling uncertainty due to limited knowledge), which arise in coarse-grained numerical simulations. Here, we introduce a conceptually distinct type of flux limiter that is designed to handle the effects of randomness in the model and uncertainty in model parameters. Unlike traditional single-function flux limiters, these new probabilistic flux limiters incorporate multiple flux limiting functions, each applied with a learned probability drawn from high-resolution data to mitigate the effects of uncertainty in numerical simulations. This approach departs from traditional single-function limiters by explicitly modeling and incorporating uncertainty into the shock capturing process. Using the example of Burgers' equation as a testbed, we show that a machine learned, probabilistic flux limiter may be used in a shock capturing code to more accurately capture shock profiles. In particular, we show that our probabilistic flux limiter outperforms standard limiters and can be successively improved upon (up to a point) by expanding the set of probabilistically chosen flux limiting functions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Stage-local partitioned two-step runge-kutta methods for large systems of ordinary differential equations

We introduce stage-local partitioned two-step Runge-Kutta methods are an extension of standard two-step Runge-Kutta methods, which are an alternative to the standard additive two-step Runge-Kutta methods currently existing in the literature. Furthermore, these new schemes are designed with an eye towards truly N-partitioned systems and leverage local stage approximations to make several computationally interesting approximations viable. Specifically, the focus on local stage approximations makes possible the construction of truly asynchronous schemes, in the parallel sense, possible. In addition, we show that an implicit-explicit approach to these schemes can lead to methods that require the inversion of only local nonlinear systems.

Applied Dynamical Systems↗
DOE OSTI2024

GP-BayesOpInf

SAND2025-01851O GP-BayesOpInf is a software tool that uses algorithms to combine Gaussian process regression, principal component analysis, and linear Bayesian inference to produce a probabilistic reduced-order model for time-dependent systems. Numerical examples include the compressible Euler equations for an ideal gas, a heat diffusion process with a nonlinear reaction term, and a set of ordinary differential equations describing a compartmental model in epidemiology. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

SciDAC↗
DOE OSTI2026

On-chip probabilistic inference for charged-particle tracking at the sensor edge

Modern scientific instruments operate under increasingly extreme constraints on bandwidth, latency, and power. Inference at the sensor edge determines experimental data collection efficiency by deciding which information to save for further analysis. Particle tracking detectors at the Large Hadron Collider exemplify this challenge: pixelated silicon sensors generate rich spatiotemporal ionization patterns, yet most of this information is discarded due to data-rate limitations. Concurrently, advancements in co-design tools provide rapid turn-around for incorporating machine learning into application-specific integrated circuits, motivating designs for particle detectors with new integrated technologies. We demonstrate that neural networks embedded in the front-end electronics can infer charged-particle kinematic parameters from a single silicon layer. We regress hit positions and incident angles with calibrated uncertainties, while satisfying stringent constraints on numerical precision, latency, and silicon area. Our results establish a path toward probabilistic inference directly at the edge, opening new opportunities for intelligent sensing in high-rate scientific instruments.

Das, Arghya Ranjan [Purdue U.] (ORCID:000000018451↗
DOE OSTI2025

Dominant balance-based adaptive mesh refinement for incompressible fluid flows

This work introduces a novel adaptive mesh refinement (AMR) method that utilizes dominant balance analysis (DBA) for efficient and accurate grid adaptation in computational fluid dynamics (CFD) simulations. The proposed method leverages a Gaussian mixture model (GMM) to classify grid cells into active and passive regions based on the dominant physical interactions within the equation space. By modeling truncation error probabilistically from discretized terms, the method identifies regions of high interaction where numerical accuracy is most sensitive to resolution. Unlike traditional AMR strategies, this approach does not rely on heuristic-based sensors or user-defined thresholds, providing a fully automated and problem-independent framework for AMR. Applied to the incompressible Navier-Stokes equations for steady and unsteady flow past a cylinder, the DBA-based AMR method achieves comparable accuracy to high-resolution grids while reducing computational costs by up to 70 %. The validation highlights the method’s effectiveness in capturing complex flow features while minimizing grid cells, directing computational resources toward regions with the most critical dynamics. This modular and scalable strategy is adaptable to a wide range of applications, presenting a promising tool for efficient high-fidelity simulations in CFD and other multiphysics domains.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2026

A Bayesian desmearing algorithm for Bonse–Hart USANS with anisotropic scattering

Ultra-small-angle neutron scattering (USANS) using Bonse–Hart optics provides micrometer-scale structural insights but suffers from severe slit-geometry smearing. While well-established for isotropic systems, quantitative desmearing of anisotropic data remains a challenge because conventional corrections break down for non-radial scattering. In this work, we address this by developing a resolution-aware Bayesian framework that explicitly incorporates anisotropy via an affine deformation to the scattering pattern, guided by the principle of parsimony. This results in orientation-resolved point-spread functions that enable a self-consistent determination of both the resolution and deformation parameters. Using Gaussian process regression with uncertainty quantification and a probabilistic correction for multiple scattering, we demonstrate the framework’s effectiveness through numerical benchmarks and experimental studies of a stretched polymer melt. Our approach enables the seamless integration of SANS and USANS data, facilitating quantitative structural analysis of deformed materials at nanometer to micrometer scales.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Idaho National Laboratory Integrated Multisite SSHAC Level 3: Probabilistic Volcanic Hazards Assessment

The Idaho National Laboratory (INL) resides on the eastern Snake River Plain (ESRP), part of the Snake River Plain (SRP) with a complex origin and geologic history of volcanism. Much of the Quaternary (last 2.58 million years) volcanism within 400 km of INL is genetically associated with a major thermal anomaly referred to as the Yellowstone hotspot, which is currently located more than 180 km northeast of INL. Near INL, local volcanic sources include silicic domes near its southern border, and numerous dike-fed basaltic vents of the ESRP, some of which are near or within the INL boundaries. Quantitative probabilistic assessments of screened-in volcanic hazardous phenomena for nine different facility complexes at the INL are presented for a Senior Seismic Hazard Analysis Committee (SSHAC) Level 3 (SL3) study. The comprehensive, integrated multisite SSHAC study consists of a single regional Probabilistic Volcanic Hazards Assessment (PVHA) that pertains to all nine INL facility complexes, with site-specific information developed at each respective area of interest (AOI), referred to as the "INL facility AOI" (Figure ES-1). Hazard products are generated for specified INL facility AOIs for use by multiple stakeholders from different agencies in risk-informed decision-making regarding site selection, operations, and design of nuclear facilities at INL, consistent with U.S. Department of Energy (DOE) and U.S. Nuclear Regulatory Commission (NRC) regulatory guidance. The study also serves as the basis for future periodic safety assessments required for existing DOE facilities, such as 10-year evaluations of natural-phenomena hazards. Elements of the INL PVHA including initial characterization, screening, quantitative assessments of eruption and hazard potential, and consideration of facility needs are conducted using the three phases of the SSHAC process: evaluation, integration, and documentation. As per regulatory guidance, the SSHAC framework provided the necessary processes and procedures for the PVHA Technical Integration (TI) team, at three workshops, five formal working meetings and many TI team remote meetings, to conduct initial characterization, screen the volcanic hazards, create PVHA model inputs, exercise those models in the PVHA, consider facility-specific volcanic hazard needs, and perform final hazard calculations. The Participatory Peer Review Panel (PPRP) provided independent oversight and performed process and technical reviews of the PVHA throughout its duration. The study included an extensive New Data Collection and Analyses (NDCA) program developed by the PVHA TI team to reduce uncertainties in hazard-significant elements in the PVHA model. NDCA activities generated 20 reports providing important contributory datasets and results to the project database for characterizing the ESRP. For example, a report compiling the dimensions of ESRP shield volcanoes and lava fields was used to construct volcanic footprints (areas of impact), was compared with data from INL subsurface cores, and was used to validate the results of lava-flow inundation modeling on the contemporary terrain. Another example is the acquisition of aeromagnetic data over INL and its surrounding area, with maps of buried magmatic features (e.g., subsurface volcanoes and swarms of feeder dikes) that informed the PVHA conceptual model of volcanism. The SSHAC evaluation process was used to conduct all elements of the PVHA including initial characterization and screening. Existing data and NDCA activities provided the foundation to develop the tectonomagmatic conceptual model of volcanism for the region of geographical interest in the SRP and Yellowstone hotspot volcanic system, and for considering volcanoes in the western US. Considering Quaternary volcanic sources active during this period, the screening approach identified and evaluated magma compositions, types of eruptive and intrusive phenomena, types of hazardous phenomena, and proximity of sources to INL. The PVHA TI team evaluated 18 types of magmatic sources in terms of 20 potentially hazardous phenomena, resulting in 360 screening decisions. The screening process resulted in 140 screened-in hazardous volcanic phenomena for Quaternary volcanic sources 1) proximal to INL facility complexes in the ESRP (64 basaltic and 54 silicic), 2) regional sources associated with the Yellowstone caldera system and Blackfoot Reservoir volcanic field (7), and 3) more distal sources from thirteen Cascade volcanoes and two volcanoes at Long Valley caldera (CA).

58 GEOSCIENCES↗
DOE OSTI2024

Short-Term Probabilistic Solar Forecasting via Reinforcement Learning over ECMWF

In this paper, we present an innovative reinforcement learning approach for short-term solar forecasting, leveraging data from the European Centre for Medium-Range Weather Forecasts (ECMWF). The methodology begins with the application of the System Advisor Model (SAM) to transform various ECMWF numerical weather prediction members into predictive photovoltaic power generation. To enhance the precision of deterministic forecasting, we introduce a dynamic model selection algorithm based on Q-learning. This algorithm dynamically identifies and utilizes the most accurate ensemble member for forecasting purposes. Furthermore, we employ a support vector regression surrogate model with a Gaussian distribution to generate probabilistic forecasts, providing a holistic view of solar energy generation uncertainty. To expedite the training process and make it more practical for real-world applications, we integrate a rolling update workflow. This innovative workflow reduces the training period from months to a mere 19 days, making our method highly efficient. Numerical results of the case study show that in comparison to benchmark models, the proposed method improves the deterministic and probabilistic solar forecasting accuracy by up to 40.84% and 48.42%, respectively.

ensemble forecasting↗
DOE OSTI2025

Surrogate-Based Autotuning for Randomized Sketching Algorithms in Regression Problems

Algorithms from Randomized Numerical Linear Algebra (RandNLA) are known to be effective in handling high-dimensional computational problems, providing high-quality empirical performance as well as strong probabilistic guarantees. However, their practical application is complicated by the fact that the user needs to set various algorithm-specific tuning parameters which are different from those used in traditional NLA. This paper demonstrates how a surrogate-based autotuning approach can be used to address fundamental problems of parameter selection in RandNLA algorithms. In particular, we provide a detailed investigation of surrogate-based autotuning for sketch-and-precondition (SAP)-based randomized least squares methods, which have been one of the great success stories in modern RandNLA. Empirical results show that our surrogate-based autotuning approach can achieve near-optimal performance with much less tuning cost than a random search (up to about 7.6x fewer trials of different parameter configurations). Moreover, while our experiments focus on least squares, our results demonstrate a general-purpose autotuning pipeline applicable to any kind of RandNLA algorithm.

Cho, Younghyun↗
DOE OSTI2024

Synthesizing realistic sand assemblies with denoising diffusion in latent space

Abstract The shapes and morphological features of grains in sand assemblies have far‐reaching implications in many engineering applications, such as geotechnical engineering, computer animations, petroleum engineering, and concentrated solar power. Yet, our understanding of the influence of grain geometries on macroscopic response is often only qualitative, due to the limited availability of high‐quality 3D grain geometry data. In this paper, we introduce a denoising diffusion algorithm that uses a set of point clouds collected from the surface of individual sand grains to generate grains in the latent space. By employing a point cloud autoencoder, the three‐dimensional point cloud structures of sand grains are first encoded into a lower‐dimensional latent space. A generative denoising diffusion probabilistic model is trained to produce synthetic sand that maximizes the log‐likelihood of the generated samples belonging to the original data distribution measured by a Kullback‐Leibler divergence. Numerical experiments suggest that the proposed method is capable of generating realistic grains with morphology, shapes and sizes consistent with the training data inferred from an F50 sand database. We then use a rigid contact dynamic simulator to pour the synthetic sand in a confined volume to form granular assemblies in a static equilibrium state with targeted distribution properties. To ensure third‐party validation, 50,000 synthetic sand grains and the 1542 real synchrotron microcomputed tomography (SMT) scans of the F50 sand, as well as the granular assemblies composed of synthetic sand grains are made available in an open‐source repository.

Vlassis, Nikolaos N.↗
DOE OSTI2025

Optimization problems governed by systems of PDEs with uncertainties

This paper reviews current theoretical and numerical approaches to optimization problems governed by partial differential equations (PDEs) that depend on random variables or random fields. Such problems arise in many engineering, science, economics and societal decision-making tasks. This paper focuses on problems in which the governing PDEs are parametrized by the random variables/fields, and the decisions are made at the beginning and are not revised once uncertainty is revealed. Examples of such problems are presented to motivate the topic of this paper, and to illustrate the impact of different ways to model uncertainty in the formulations of the optimization problem and their impact on the solution. A linear–quadratic elliptic optimal control problem is used to provide a detailed discussion of the set-up for the risk-neutral optimization problem formulation, study the existence and characterization of its solution, and survey numerical methods for computing it. Different ways to model uncertainty in the PDE-constrained optimization problem are surveyed in an abstract setting, including risk measures, distributionally robust optimization formulations, probabilistic functions and chance constraints, and stochastic orders. Furthermore, approximation-based optimization approaches and stochastic methods for the solution of the large-scale PDE-constrained optimization problems under uncertainty are described. Some possible future research directions are outlined.

Heinkenschloss, Matthias [Rice Univ., Houston, TX ↗
DOE OSTI2024

Machine Learning‐Assisted Microearthquake Location Workflow for Monitoring the Newberry Enhanced Geothermal System

Abstract Enhanced geothermal systems (EGS) offer a sustainable energy source but face challenges in accurately locating microearthquakes induced during reservoir stimulation. Locating these microearthquakes provides reliable feedback on the stimulation progress. Current deep learning methods for locating earthquakes require extensive data sets for training, which is problematic as detected microearthquakes are often limited. To address the scarcity of training data, we propose a practical workflow using probabilistic multilayer perceptron (PMLP) which predicts microearthquake locations from cross‐correlation time lags in waveforms. Utilizing a 3D velocity model of Newberry site derived from ambient noise interferometry, we generate numerous synthetic microearthquakes and 3D acoustic waveforms for PMLP training. Accurate synthetic tests prompt us to apply the trained network to the 2012 and 2014 stimulation field waveforms. To enhance the accuracy of source localization, we carefully handpick the P‐arrival times. Predictions on the 2012 stimulation data set show major microseismic activity at depths of 0.5–1.2 km, correlating with a known casing leakage scenario. In the 2014 data set, the majority of predictions concentrate at 2.0–2.9 km depths, consistent with results obtained from conventional physics‐based inversion, and align with the presence of natural fractures from 2.0 to 2.7 km. We validate our findings by comparing the synthetic and field picks, demonstrating a satisfactory match for the first arrivals. By combining the benefits of quick inference speeds and accurate location predictions, we demonstrate the feasibility of using realistic synthetic data set to locate microseismicity for EGS monitoring.

15 GEOTHERMAL ENERGY↗
DOE OSTI2026

Active learning using hybrid surrogate tool life modeling for machining process optimization

Here, this paper describes an active learning approach for part-to-part iterative machining process optimization using a hybrid surrogate tool life model. A probabilistic interpolating tool life model is developed by combining the empirical Taylor-type tool life equation and the model fit error. The probabilistic tool life model is then used to calculate the machining cost per part distribution. The optimal machining parameters are selected using an expected improvement in machining cost per part criterion. The method is validated numerically using experimental results; the results show a median convergence error of 2.2% after three tests over 400 simulations. The method is validated experimentally on two industrial applications for Ti-6Al-4V roughing resulting in a cost per part reduction greater than 23% after two tests. The described method is a robust solution for rapid convergence to optimal machining parameters in an industrial production environment.

Active learning↗
DOE OSTI2024

Yet Another Discriminant Analysis (YADA): A Probabilistic Model for Machine Learning Applications

This paper presents a probabilistic model for various machine learning (ML) applications. While deep learning (DL) has produced state-of-the-art results in many domains, DL models are complex and over-parameterized, which leads to high uncertainty about what the model has learned, as well as its decision process. Further, DL models are not probabilistic, making reasoning about their output challenging. In contrast, the proposed model, referred to as Yet Another Discriminate Analysis(YADA), is less complex than other methods, is based on a mathematically rigorous foundation, and can be utilized for a wide variety of ML tasks including classification, explainability, and uncertainty quantification. YADA is thus competitive in most cases with many state-of-the-art DL models. Ideally, a probabilistic model would represent the full joint probability distribution of its features, but doing so is often computationally expensive and intractable. Hence, many probabilistic models assume that the features are either normally distributed, mutually independent, or both, which can severely limit their performance. YADA is an intermediate model that (1) captures the marginal distributions of each variable and the pairwise correlations between variables and (2) explicitly maps features to the space of multivariate Gaussian variables. Numerous mathematical properties of the YADA model can be derived, thereby improving the theoretic underpinnings of ML. Validation of the model can be statistically verified on new or held-out data using native properties of YADA. However, there are some engineering and practical challenges that we enumerate to make YADA more useful.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Emulation With Uncertainty Quantification of Regional Sea‐Level Change Caused by the Antarctic Ice Sheet

Abstract Projecting regional sea‐level change under various climate‐change scenarios typically involves running forward simulations of the Earth's gravitational, rotational and deformational (GRD) response to ice‐mass change, which requires substantial computational cost if applied to probabilistic frameworks requiring thousands to millions of samples. Here we build emulators of regional sea‐level change at 27 coastal locations, due to the GRD effects associated with future Antarctic Ice Sheet mass change over the 21st century. The emulators are evaluated against a numerical sea‐level model applied to an ensemble of ice‐sheet model simulations of the Antarctic Ice Sheet through 2100. We build a physics‐based emulator using a recent sensitivity kernel approach and compare it to machine learning based emulators (neural network and conditional variational autoencoder methods). In order to quantify uncertainty, we derive well‐calibrated prediction intervals for regional sea‐level change via split‐conformal inference and linear regression, and show that Monte Carlo dropout does not yield well‐calibrated uncertainties in this instance. We also demonstrate substantial gains in computational efficiency using both the physics‐based emulator and neural networks in comparison to the numerical model for the complete regional sea‐level solution. Overall, we find the physics‐based emulator modestly outperforms the machine learning emulators for this problem.

58 GEOSCIENCES↗
DOE OSTI2024

Bayesian stability and force modeling for uncertain machining processes

Accurately simulating machining operations requires knowledge of the cutting force model and system frequency response. However, this data is collected using specialized instruments in an ex-situ manner. Bayesian statistical methods instead learn the system parameters using cutting test data, but to date, these approaches have only considered milling stability. This paper presents a physics-based Bayesian framework which incorporates both spindle power and milling stability. Initial probabilistic descriptions of the system parameters are propagated through a set of physics functions to form probabilistic predictions about the milling process. The system parameters are then updated using automatically selected cutting tests to reduce parameter uncertainty and identify more productive cutting conditions, where spindle power measurements are used to learn the cutting force model. The framework is demonstrated through both numerical and experimental case studies. Results show that the approach accurately identifies both the system natural frequency and cutting force model.

42 ENGINEERING↗