Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Probabilistic models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Yet Another Discriminant Analysis (YADA): A Probabilistic Model for Machine Learning Applications

This paper presents a probabilistic model for various machine learning (ML) applications. While deep learning (DL) has produced state-of-the-art results in many domains, DL models are complex and over-parameterized, which leads to high uncertainty about what the model has learned, as well as its decision process. Further, DL models are not probabilistic, making reasoning about their output challenging. In contrast, the proposed model, referred to as Yet Another Discriminate Analysis(YADA), is less complex than other methods, is based on a mathematically rigorous foundation, and can be utilized for a wide variety of ML tasks including classification, explainability, and uncertainty quantification. YADA is thus competitive in most cases with many state-of-the-art DL models. Ideally, a probabilistic model would represent the full joint probability distribution of its features, but doing so is often computationally expensive and intractable. Hence, many probabilistic models assume that the features are either normally distributed, mutually independent, or both, which can severely limit their performance. YADA is an intermediate model that (1) captures the marginal distributions of each variable and the pairwise correlations between variables and (2) explicitly maps features to the space of multivariate Gaussian variables. Numerous mathematical properties of the YADA model can be derived, thereby improving the theoretic underpinnings of ML. Validation of the model can be statistically verified on new or held-out data using native properties of YADA. However, there are some engineering and practical challenges that we enumerate to make YADA more useful.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Diffusion Probabilistic Modeling for Video Generation

Denoising diffusion probabilistic models are a promising new class of generative models that mark a milestone in high-quality image generation. This paper showcases their ability to sequentially generate video, surpassing prior methods in perceptual and probabilistic forecasting metrics. We propose an autoregressive, end-to-end optimized video diffusion model inspired by recent advances in neural video compression. The model successively generates future frames by correcting a deterministic next-frame prediction using a stochastic residual generated by an inverse diffusion process. We compare this approach against six baselines on four datasets involving natural and simulation-based videos. We find significant improvements in terms of perceptual quality and probabilistic frame forecasting ability for all datasets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Continuous-time probabilistic models for longitudinal electronic health records

Analysis of longitudinal Electronic Health Record (EHR) data is an important goal for precision medicine. Difficulty in applying Machine Learning (ML) methods, either predictive or unsupervised, stems in part from the heterogeneity and irregular sampling of EHR data. Here, we present an unsupervised probabilistic model that captures nonlinear relationships between variables over continuous-time. This method works with arbitrary sampling patterns and captures the joint probability distribution between variable measurements and the time intervals between them. Inference algorithms are derived that can be used to evaluate the likelihood of future using under a trained model. As an example, we consider data from the United States Veterans Health Administration (VHA) in the areas of diabetes and depression. Likelihood ratio maps are produced showing the likelihood of risk for moderate-severe vs minimal depression as measured by the Patient Health Questionnaire-9 (PHQ-9).

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

FY2022 Status Update: A Probabilistic Model for Stress Corrosion Cracking of SNF Dry Storage Canisters

Understanding the potential risk of stress corrosion cracking of spent nuclear fuel dry storage canisters has been identified as a knowledge gap for determining the safety of long-term interim storage of spent nuclear fuel. To address this, the DOE is funding a multi-lab DOE effort to understand the timing, occurrence, and consequences of potential canister SCC. Sandia National Laboratories has developed a probabilistic model for canister penetration by SCC. This model has been continuously updated at SNL since 2014. Model uncertainties are treated using a nested loop structure, where the outer loop accounts for uncertainties due to lack of data and the inner aleatoric loop accounts for uncertainties due to variation in nature. By separating uncertainties into these categories, it is possible to focus future work on reducing the most influential epistemic uncertainties. Several experimental studies have already been performed to improve the modeling approach through expanded process understanding and improved model parameterization. The resulting code is physics-based and intended to inform future work by identifying (1) important modeling assumptions, (2) experimental data needs, and (3) necessary model developments. In this document, several of the sub-models in the probabilistic SCC model have been exercised, and the intermediate results, as the model progresses from one sub-model to the next, are presented. Evaluating the sub-models in this manner provides a better understanding of sub-model outputs and has identified several unintended consequences of model assumptions or parameterizations, requiring updates to the modeling approach. The following updates have been made, and future updates have been identified.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2022

Unsupervised probabilistic models for sequential Electronic Health Records

We develop an unsupervised probabilistic model for heterogeneous Electronic Health Record (EHR) data. Utilizing a mixture model formulation, our approach directly models sequences of arbitrary length, such as medications and laboratory results. This allows for subgrouping and incorporation of the dynamics underlying heterogeneous data types. The model consists of a layered set of latent variables that encode underlying structure in the data. These variables represent subject subgroups at the top layer, and unobserved states for sequences in the second layer. We train this model on episodic data from subjects receiving medical care in the Kaiser Permanente Northern California integrated healthcare delivery system. The resulting properties of the trained model generate novel insight from these complex and multifaceted data. In addition, we show how the model can be used to analyze sequences that contribute to assessment of mortality likelihood.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Denoising diffusion probabilistic models for generative alloy design

Inverse material design is an extremely challenging optimization task made difficult by, in part, the highly nonlinear relationship linking performance with composition. Quantitative approaches have improved significantly owing to advances in high throughput experimentation and computational thermodynamics. However, existing physics-based tools are mostly forward models; input a chemistry and obtain a prediction. More recently the materials community has leveraged advances in the machine learning community to establish novel inverse design frameworks. Very recently denoising diffusion probabilistic models have been shown to be extremely powerful generators producing synthetic data of various modalities e.g. images, text, audio, tables, etc.. In this work a novel framework for alloy design and optimization is proposed leveraging these class of models. Five key generative tasks are demonstrated (1) unconditional generation (2) composition conditioned generation (3) property conditioned generation (4) multi-feedstock conditioned generation and (5) generative optimization. These methods were tested on three case studies: high entropy alloy design, superalloy binder jet additive manufacturing, and in-situ dual-feedstock wire-arc additive manufacturing. Results indicate that the established models are extremely flexible, expressive, and robust. The architecture’s flexibility and training procedure empower the model to learn complex intra-compositional and composition-property relationships. Furthermore, the probabilistic nature of these models makes them well suited for addressing solution non-uniqueness and tackling uncertainty quantification tasks. While the fidelity and quantity of the underlying training data is paramount, we envision that future alloy design frameworks will make extensive use of these kinds of machine learning models as “search” tools bolstering the utility of experimental and computational approaches.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Digital polycrystalline microstructure generation using diffusion probabilistic models

Accurate micromechanical simulation of polycrystalline materials requires a realistic digital representation of the grain scale microstructure. Here, this work demonstrates the use of a generative diffusion probabilistic model for synthesizing single phase polycrystalline realizations. The model performs well and is capable of producing realistic microstructures consisting of not just simple equiaxed structures but also structures exhibiting more complex spatial arrangements. Masked microstructure generation reveals that the model is context aware of morphological descriptors which may be encoded in the latent space. Training on more diverse data sets, with scaled up architectures, may enable development of future models capable of synthesizing even more complex microstructural features.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Towards universal unfolding of detector effects in high-energy physics using denoising diffusion probabilistic models

Correcting for detector effects in experimental data, particularly through unfolding, is critical for enabling precision measurements in high-energy physics. However, traditional unfolding methods face challenges in scalability, flexibility, and dependence on simulations. We introduce a novel approach to multidimensional object-wise unfolding using conditional Denoising Diffusion Probabilistic Models (cDDPM). Our method utilizes the cDDPM for a non-iterative, flexible posterior sampling approach, incorporating distribution moments as conditioning information, which exhibits a strong inductive bias that allows it to generalize to unseen physics processes without explicitly assuming the underlying distribution. Our results highlight the potential of this method as a step towards a "universal" unfolding tool that reduces dependence on truth-level assumptions, while enabling the unfolding of a wide range of measured distributions with improved adaptability and accuracy.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Tropical Cyclone Super Resolution using conditional diffusion denoising probabilistic model from mesoscale simulation to LES

Accurate modeling of tropical cyclone wind fields is essential for the design, risk assessment, and operational planning of offshore energy infrastructure. While mesoscale simulations are widely used thanks to their computational efficiency, they lack the necessary resolution to capture key features such as wind shear and veer profiles as well as the distribution turbulent kinetic energy (TKE). High-fidelity large-eddy simulation (LES) models on the other hand, can resolve turbulent structures and provide a more accurate representation of the complex wind field, albeit at a higher computational cost. To address this modeling gap, we introduce a two-part generative framework to enhance the resolution and physics-capturing ability of mesoscale simulations. First, a reduced-order model based on Karhunen–Loève (KL) decomposition is used to extract dominant spatial modes from one-dimensional mean wind profiles. A multilayer perceptron (MLP) is trained to map mesoscale mode weights to their LES counterparts, enabling accurate reconstruction of vertical velocity profiles. Second, a conditional Diffusion Denoising Probabilistic Model (DDPM) is developed to super-resolve coarse and low-fidelity mesoscale velocity fields, recovering fine-scale turbulence structures and stress distributions. The framework is evaluated across different tropical cyclone intensity categories defined by the Saffir–Simpson scale and demonstrates strong performance in both interpolation and extrapolation tasks. The generated fields accurately reproduce spatial coherence, stress distributions, and spectral energy characteristics observed in LES data. By bridging the fidelity gap between mesoscale and LES outputs, this approach offers a scalable, data-driven solution for enhancing the representation of tropical cyclone wind fields, enabling more robust offshore energy infrastructure systems design in tropical-cyclone-prone areas.

17 WIND ENERGY↗
DOE OSTI2022

Probabilistic Modeling of Climate Change Impacts on Renewable Energy and Storage Requirements for NM's Energy Transition Act (SAND Report)

This report provides a study of the potential impacts of climate change on intermittent renewable energy resources, battery storage, and resource adequacy in Public Service Company of New Mexico’s Integrated Resource Plan for 2020 – 2040. Climate change models and available data were first evaluated to determine uncertainty and potential changes in solar irradiance, temperature, and wind speed in NM in the coming decades. These changes were then implemented in solar and wind energy models to determine impacts on renewable energy resources in NM. Results for the extreme climate-change scenario show that the projected wind power may decrease by ~13% due to projected decreases in wind speed. Projected solar power may decrease by ~4% due to decreases in irradiance and increases in temperature in NM. Uncertainty in these climate-induced changes in wind and solar resources was accommodated in probabilistic models assuming uniform distributions in the annual reductions in solar and wind resources. Uncertainty in battery storage performance was also evaluated based on increased temperature, capacity fade, and degradation in round-trip efficiency. The hourly energy balance was determined throughout the year given uncertainties in the renewable energy resources and energy storage. The loss of load expectation (LOLE) was evaluated for the 2040 No New Combustion portfolio and found to increase from 0 days/year to a median value of ~2 days/year due to potential reductions in renewable energy resources and battery storage performance and capacity. A rank-regression analyses revealed that battery round-trip efficiency was the most significant parameter that impacted LOLE, followed by solar resource, wind resource, and battery fade. An increase in battery storage capacity to ~25,000 – 30,000 MWh from a baseline value of ~14,000 MWh was required to reduce the median value of LOLE to ~0.2 days/year with consideration of potential climate impacts and battery degradation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Stealthy Cyber Anomaly Detection On Large Noisy Multi-material 3D Printer Datasets Using Probabilistic Models

As Additive Layer Manufacturing (ALM) becomes pervasive in industry, its applications in safety critical component manufacturing are being explored and adopted. However, ALM's reliance on embedded computing renders it vulnerable to tampering through cyber-attacks. Sensor instrumentation of ALM devices allows for rigorous process and security monitoring, but also results in a massive volume of noisy data for each run. As such, in-situ, near-real-time anomaly detection is very challenging. The ideal algorithm for this context is simple, computationally efficient, minimizes false positives, and is accurate enough to resolve small deviations. In this paper, we present a probabilistic-model-based approach to address this challenge. To test our approach, we analyze current measurements from a polymer composite 3D printer during emulated tampering attacks. Our results show that our approach can consistently and efficiently locate small changes in the presence of substantial operational noise.

Yoginath, Srikanth↗
DOE OSTI2024

Effectiveness of denoising diffusion probabilistic models for fast and high-fidelity whole-event simulation in high-energy heavy-ion experiments

Artificial intelligence (AI) generative models, such as generative adversarial networks (GANs), variational autoencoders, and normalizing flows, have been widely used and studied as efficient alternatives for traditional scientific simulations. However, they have several drawbacks, including training instability and inability to cover the entire data distribution, especially for regions where data are rare. This is particularly challenging for whole-event, full-detector simulations in high-energy heavy-ion experiments, such as sPHENIX at the Relativistic Heavy Ion Collider and Large Hadron Collider experiments, where thousands of particles are produced per event and interact with the detector. This work investigates the effectiveness of denoising diffusion probabilistic models (DDPMs) as an AI-based generative surrogate model for the sPHENIX experiment that includes the heavy-ion event generation and response of the entire calorimeter stack. DDPM performance in sPHENIX simulation data is compared with a popular rival, GANs. Results show that both DDPMs and GANs can reproduce the data distribution where the examples are abundant (low-to-medium calorimeter energies). Nonetheless, DDPMs significantly outperform GANs, especially in high-energy regions where data are rare. Additionally, DDPMs exhibit superior stability compared to GANs. The results are consistent between both central and peripheral centrality heavy-ion collision events. Moreover, DDPMs offer a substantial speedup of approximately a factor of 100 compared to the traditional Geant4 simulation method.

42 ENGINEERING↗
DOE OSTI2024

Sequential Inference of Hospitalization Electronic Health Records Using Probabilistic Models

In the dynamic hospital setting, decision support can be a valuable tool for improving patient outcomes. Data-driven inference of future outcomes is challenging in this dynamic setting, where long sequences such as laboratory tests and medications are updated frequently. This is due in part to heterogeneity of data types and mixed-sequence types contained in variable length sequences. In this work we design a probabilistic unsupervised model for multiple arbitrary-length sequences contained in hospitalization Electronic Health Record (EHR) data. The model uses a latent variable structure and captures complex relationships between medications, diagnoses, laboratory tests, neurological assessments, and medications. It can be trained on original data, without requiring any lossy transformations or time binning. Inference algorithms are derived that use partial data to infer properties of the complete sequences, including their length and presence of specific values. We train this model on data from subjects receiving medical care in the Kaiser Permanente Northern California integrated healthcare delivery system. The results are evaluated against held-out data for predicting the length of sequences and presence of Intensive Care Unit (ICU) in hospitalization bed sequences. Our method outperforms a baseline approach, showing that in these experiments the trained model captures information in the sequences that is informative of their future values.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Probabilistic modeling approach for tracking each collision and breakup of computational parcels

A modeling approach for binary collision and particle breakup is presented for a stochastic particle model in which a computational parcel represents a number of similar physical particles. Presented in this report includes sampling from a distribution of discrete data, parcel number reduction, probabilistic calculation of collision rates, and various particle-particle interactions. Modeling of elastic scattering, coalescence and agglomeration, and breakup are discussed for solid-solid, solid-liquid, and liquid-liquid particle collisions. Numerical procedures are also summarized.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Bayesian-Motivated Probabilistic Model of Hurricane-Induced Multimechanism Flood Hazards

Multimechanism floods (MMFs) are caused by the simultaneous occurrence of more than one flood mechanism such as storm surge, precipitation, tides, and waves. MMFs can lead to more severe or differing impacts than single-mechanism floods. As a result, comprehensive risk assessments require the ability to assess the multivariate probabilistic behaviors of hazards from MMFs. Here this study introduces a novel Bayesian-motivated approach for the probabilistic assessment of hurricane-induced hazards from the combination of the surge, precipitation, tides, and river antecedent flow. A Bayesian network (BN) is developed to capture the physical (conditional) relationship between variables and facilitate the generation of a hazard curve for river discharge that captures the contributions from multiple flood drivers. A case study located along the Delaware River is used to illustrate the proposed approach. Five computationally efficient representative predictive models are developed to estimate the conditional distributions required for the BN as a means of demonstrating the overall framework. The predictive models used in this study act as placeholders and can be replaced with more sophisticated and high-fidelity models depending on the desired accuracy level. While the predictive models are intended to be representative and illustrative, the model performance is evaluated using three historical storms that affected the area. Overall, the proposed framework is shown to be transparent, effective, and adaptable.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Thermal Overloading Risk Mitigation With a Semi-Analytical Probabilistic Model on Branch Current

A semi-analytical formulation is presented in this paper for the probability computation of branch current in multiphase systems. The developed formula is derived based on the linear power flow model in rectangular coordinates. The system uncertainty injections can be renewable energy resources or loads and are modeled using a Gaussian mixture model (GMM). The developed formula can be used to compute the line current violation probability as well as integrate into optimal power flow problem as chance-constraint relaxation. Here, the proposed formula is first compared with the Matlab embedded numerical integration function to show its performance. Besides, the semi-analytical formula is validated and compared with the Monte Carlo simulation method using the IEEE 123-bus system, EPRI Ckt5, and Ckt7 systems.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Probabilistic modeling and prediction of out-of-plane unidirectional composite lamina properties

Computational simulation provides an efficient means to predict the behavior of customized hybrid material configurations using validated, physics-based models. One limitation to this approach is the quality and quantity of available data to characterize the many constituent input properties. Therefore, a systematic approach to identify the most influential parameters on the hybrid behavior and quantify the corresponding uncertainty in predictive capabilities is required. Here, an approach using Bayesian multimodel inference and imprecise global sensitivity analysis is presented to investigate the effects of sparse constituent data on the prediction of composite material properties. The methodology allows the identification, using quantified uncertainties, of the most influential constituent material parameters for specified homogenized properties. This sensitivity analysis further enables a dimension reduction when assessing the influence of uncertainties on material properties and can be used to inform testing programs of the constituent properties that require additional testing/data collection in order to minimize uncertainty in macro-scale composite properties. The methodology is specifically demonstrated on the prediction and sensitivity analysis of out-of-plane mechanical properties of a unidirectional lamina.

36 MATERIALS SCIENCE↗