SEARCH · Engineering Papers

Results for “Probabilistic Machine Learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

All sources NASA NTRS DOE OSTI

DOE OSTI2024

EFIT-Prime: Probabilistic and physics-constrained reduced-order neural network model for equilibrium reconstruction in DIII-D

We introduce EFIT-Prime, a novel machine learning surrogate model for EFIT (Equilibrium FIT) that integrates probabilistic and physics-informed methodologies to overcome typical limitations associated with deterministic and ad hoc neural network architectures. EFIT-Prime utilizes a neural architecture search-based deep ensemble for robust uncertainty quantification, providing scalable and efficient neural architectures that comprehensively quantify both data and model uncertainties. Physically informed by the Grad–Shafranov equation, EFIT-Prime applies a constraint on the current density J tor and a smoothness constraint on the first derivative of the poloidal flux, ensuring physically plausible solutions. Furthermore, the spatial location of the diagnostics is explicitly incorporated in the inputs to account for their spatial correlation. Extensive evaluations demonstrate EFIT-Prime's accuracy and robustness across diverse scenarios, most notably showing good generalization on negative-triangularity discharges that were excluded from training. Timing studies indicate an ensemble inference time of 15 ms for predicting a new equilibrium, offering the possibility of plasma control in real-time, if the model is optimized for speed.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2025

LandScan Mosaic

The LandScan program at Oak Ridge National Laboratory (ORNL), in collaboration with the National Geospatial-Intelligence Agency (NGA), continues to deliver the most accurate and up to date global, high resolution gridded population data. Additionally, the latest advancements in the LandScan HD methodology led to reduced latency in development of rapid updates for geopolitical events. With momentum towards reporting more up to date population estimates, feedback from the user community expressed interest in reporting population estimates in ranges - whether to express a level of uncertainty or confirm to leadership and stakeholders the modeled data are estimates. Building upon the need to understand uncertainty or confidence in the modeled data and report ranges at the global scale, LandScan Mosaic was developed. LandScan Mosaic represents the next generation of high-resolution population modeling, building upon the established success of previous LandScan HD iterations. While LandScan HD employed a deterministic big data fusion approach, LandScan Mosaic enhances this methodology by integrating advanced machine learning techniques to impute missing, yet crucial, population model parameters. This advancement allows for probabilistic modeling of building occupancy and population distribution, incorporating uncertainty quantification through Monte Carlo sampling methods. By combining big data fusion with machine learning-driven imputation and stochastic modeling, LandScan Mosaic provides a more comprehensive and robust representation of population dynamics. LandScan Mosaic will be following the in the footsteps of its longstanding counterpart LandScan Global and releasing a global gridded population raster, at the 3-arcsecond resolution. This technical report documents the current stage of development of LandScan Mosaic, detailing the methodologies and data sources behind the modeling. Stakeholders are encouraged to use this document as an authoritative reference for insight into Mosaic’s data development processes. However, readers should note that LandScan Mosaic remains in a late-stage research and development phase, and methodologies and data presented here are subject to refinements ahead of the anticipated global release in Summer 2025. Feedback and inquiries from users and stakeholders are welcomed as we continue to refine and enhance this important population resource.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Logical Activation Functions v.1.1

SAND2024-01501O Logical Activation Functions software is a PyTorch implementation from the paper, "Logical Activation Functions for Training Arbitrary Probabilistic Boolean Logic." The activation functions approximate logit-space marginalization of probabilistic truth tables from probabilistic interpretations of inputs. They also provide a general methodology to approximate logical relationships between abstract antecedents and consequents for machine learning architectures. They do not target any specific application or use-case. By training probabilistic truth tables, these activation functions can capture more expressive relationships in a neural network than typical elementwise activation functions. This code is only designed for a single compute node with a GPU and is limited to machine learning architectures than can fit within the memory of a single GPU. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Duersch, Jed↗

DOE OSTI2023

Systems and methods for improved aircraft safety

Methods and system for improved aircraft safety are described herein. A machine learning model may be trained using historical flight data for a number of previously flown flights where an automation function autonomously disengaged. The trained machine learning model may be used to generate probabilistic alert rules. The probabilistic alert rules may be used by a computing device onboard an aircraft to provide contextual information to flight crew relating to engagement status for one or more automation functions of the aircraft.

Sherry, Lance↗

DOE OSTI2023

Applying AI/ML Techniques to U.S. Nuclear Operating Experience Program

Idaho National Laboratory (INL) has provided technical assistance to the U.S. Nuclear Regulatory Commission (NRC) in reliability and risk analysis including the operating experience (OpE) program since the 1980s. The U.S. nuclear OpE program provides input parameters to the NRC Standardized Plant Analysis Risk models and the industry probabilistic risk assessment (PRA) models. While earlier PRA focuses were on at-power, internal event analysis, the risks from external hazards and during low power shutdown (LPSD) operation could be significant and the needs to develop LPSD PRA and external hazards PRA are on the rise. One issue in developing LPSD PRA is the reasonable estimation of shutdown initiative event (SDIE) frequencies. INL has developed and is maintaining an SDIE database for the NRC. However, this database is based on the reviewing of Licensee Event Reports (LERs), which is believed to be only a subset of “actual” shutdown initiating events occurred in the industry. This paper investigates a new approach to identify and characterize shutdown initiating events from the Institute of Nuclear Power Operations (INPO) industry database using machine learning techniques. The main process in this approach is to find out the relationship between key words in event descriptions and the SDIE categories as in the NRC SDIE database. The relationship can then be applied to the INPO database and search for SDIEs.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2022

Operational Probabilistic Tools for Solar Uncertainty (OPTSUN) (Final Project Report for DOE Solar Forecasting II Project)

Increasing levels of solar PV can challenge system operations and may require novel methods to operate the power system reliably and efficiently. Power system operating plans generally use deterministic forecasts, in which the variable energy resources are represented by the expected value for each interval of the decision horizon. Probabilistic forecasts are relatively new but have the potential to address the shortfalls of deterministic forecasts. However, understanding how best to use such forecasts is still a key gap in industry and was the focus of this project. The project had three workstreams. In a forecasting workstream, improvements were made to baseline probabilistic forecasts using a number of new approaches such as machine learning methods and improved input data. In a design workstream, advanced simulation tools used these forecasts to investigate newly proposed reserve determination methods. Lastly, in a demonstration workstream a scheduling management platform (SMP) was developed to leverage probabilistic forecasts in a modular and customizable manner. In order to study the benefits that could be accrued, the project team collaborated with three utility partners (Duke Energy, Southern Company and Hawaiian Electric) to deliver improved probabilistic forecasts for each region and to model each region in case studies using advanced production cost modeling tools. Different methods to determine operating reserve requirements from probabilistic forecasts were developed, simulated, and tested across each region. The benefits of using these newly proposed methods varied by utility, but, in general, using probabilistic forecasts as well as historical data to set the reserve requirements seems to improve reliability related results, with less risk of reserve or supply shortfalls. The cost implications were not always straightforward; in some cases the new methods could show a reduction in expected operating costs, but often the increase in reserves associated with better risk mitigation using probabilistic forecasts could result in an increase in operating costs in the simulations. The SMP tool was developed to process probabilistic forecasts from their initial receipt through to scheduling decisions. This open-source tool consists of several modules for scenario development, reserve requirements calculation, and visualization. The SMP tool was demonstrated to a wide range of operators and stakeholders at all three utilities and further improved based on their feedback. The tool will be available on www.epri.com/optsun. The proposed probabilistic information-based reserve determination approaches have the potential to be implemented by different regions to ensure an economic and reliable power system operation on power systems integrating increasing levels of variable renewable resources. The innovative yet practical methods developed in this project demonstrated tangible benefits from using probabilistic forecasts beyond just study-based assessments to include three unique balancing areas. The demonstrated benefits across the multiple utility environments, are expected to provide system operators in all regions the confidence required and a platform to adopt the new forecasting and operating methods.

14 SOLAR ENERGY↗

DOE OSTI2022

Predicting peak day and peak hour of electricity demand with ensemble machine learning

Battery energy storage systems can be used for peak demand reduction in power systems, leading to significant economic benefits. Two practical challenges are 1) accurately determining the peak load days and hours and 2) quantifying and reducing uncertainties associated with the forecast in probabilistic risk measures for dispatch decision-making. In this study, we develop a supervised machine learning approach to generate 1) the probability of the next operation day containing the peak hour of the month and 2) the probability of an hour to be the peak hour of the day. Guidance is provided on preparation and augmentation of data as well as selection of machine learning models and decision-making thresholds. The proposed approach is applied to the Duke Energy Progress system and successfully captures 69 peak days out of 72 testing months with a 3% exceedance probability threshold. On 90% of the peak days, the actual peak hour is among the 2 h with the highest probabilities.

25 ENERGY STORAGE↗

DOE OSTI2023

Risk-Informed Condition Evaluation of Solar-centered Energy Generation and Distribution Networks through Bayesian Learning and Inference

We develop a methodology based on Bayesian inference over Probabilistic Graphical Models (PGMs) to understand and quantify risk in solar-centered grids using targeted measurements and learned system behavior. Being non-prescriptive but, rather, able to infer system behavior and, ultimately, address risk queries from data, our machine learning-type paradigm is tailored for diverse topologies and threat scenarios often associated with distributed energy generation and photovoltaic distributed energy resources (PV-DERs) in particular. We describe algorithmic processes for: (i) learning the structure of PGMs that result from attack-prone PV-DER-proliferated distribution systems, (ii) quantifying cause-effect relationships, and (iii) evaluating risk queries based on diverse evidence. The contributions are illustrated on a residential grid subject to output impairment attacks on its PV-DER infrastructure.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2025

Accurate and Fast Anomaly Detection in Additive Composite-Based Manufacturing using Thermal Cameras

Today, large-scale additive manufacturing with plastics and composite materials requires continuous monitoring by experienced staff to prevent, detect and correct anomalous events affecting the performance of the printed part. We address the complexity of this demanding task by designing a camera-based anomaly detection system utilizing probabilistic principal component analysis (PPCA). This is a machine learning technique is trained with thermal images collected during normal operation of the large-scale printer (Cincinnati BAAM). This technique is advantageous for practical applications as there is no need to artificially introduce anomalous conditions into model training. During deployment, we challenge this model by introducing deliberate variations of the extruder speed. We reduce extrusion speed to a lower level, between 70 and 95% of the nominal value to collected test images. Our results show that images are easily identified as anomalous for extruder speeds at or below 85% of the nominal speed, meaning that an anomalous reduction of the material deposition rate can be detected within seconds of its onset. We show that our results are robust to (a) camera-to-camera variability and (b) print-to-print variability.

Pike, John [ORNL]↗

DOE OSTI2024

Denoising diffusion probabilistic models for generative alloy design

Inverse material design is an extremely challenging optimization task made difficult by, in part, the highly nonlinear relationship linking performance with composition. Quantitative approaches have improved significantly owing to advances in high throughput experimentation and computational thermodynamics. However, existing physics-based tools are mostly forward models; input a chemistry and obtain a prediction. More recently the materials community has leveraged advances in the machine learning community to establish novel inverse design frameworks. Very recently denoising diffusion probabilistic models have been shown to be extremely powerful generators producing synthetic data of various modalities e.g. images, text, audio, tables, etc.. In this work a novel framework for alloy design and optimization is proposed leveraging these class of models. Five key generative tasks are demonstrated (1) unconditional generation (2) composition conditioned generation (3) property conditioned generation (4) multi-feedstock conditioned generation and (5) generative optimization. These methods were tested on three case studies: high entropy alloy design, superalloy binder jet additive manufacturing, and in-situ dual-feedstock wire-arc additive manufacturing. Results indicate that the established models are extremely flexible, expressive, and robust. The architecture’s flexibility and training procedure empower the model to learn complex intra-compositional and composition-property relationships. Furthermore, the probabilistic nature of these models makes them well suited for addressing solution non-uniqueness and tackling uncertainty quantification tasks. While the fidelity and quantity of the underlying training data is paramount, we envision that future alloy design frameworks will make extensive use of these kinds of machine learning models as “search” tools bolstering the utility of experimental and computational approaches.

36 MATERIALS SCIENCE↗

DOE OSTI2021

A novel probabilistic regression model for electrical peak demand estimate of commercial and manufacturing buildings

Due to the high cost of electricity in commercial and industrial sectors, demand forecast models have gained increasing attention. However, there are two unresolved issues: (1) Models are not adaptable when exposed to previously unknown data (2) The value of regression methods vs. state-of-the-art machine learning models has not been made apparent before. This study’s goal is to develop probabilistic demand estimation models. Herein, we propose a probabilistic Bayesian regression framework that can not only estimate future demands with high accuracy but also be updated once new information is available. By applying the proposed algorithm to two real-world case studies (commercial and manufacturing), we show a 40.3% and 30.8% improvement in terms of mean absolute error for the two cases. Moreover, the proposed technique outperforms powerful machine learning approaches, including support vector machine by 10.39%, random forest by 6.17%, and multilayer perceptron by 9.14% in terms of mean absolute percentage error.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2023

Trust-Enhancing Probabilistic Transfer Learning for Sparse and Noisy Data Environments

There is an increasing aspiration to utilize machine learning (ML) for various tasks of relevance to national security. ML models have thus far been mostly applied to tasks and domains that, while impactful, have sufficient volume of data. For predictive tasks of national security relevance, ML models of great capacity (ability to approximate nonlinear trends in input-output maps) are often needed to capture the complex underlying physics. However, scientific problems of relevance to national security are often accompanied by various sources of sparse and/or incomplete data, including experiments and simulations, across different regimes of operation, of varying degrees of fidelity, and include noise with different characteristics and/or intensity. State-of-the-art ML models, despite exhibiting superior performance on the task and domain they were trained on, may suffer detrimental loss in performance in such sparse data environments. This report summarizes the results of the Laboratory Directed Research and Development project entitled Trust-Enhancing Probabilistic Transfer Learning for Sparse and Noisy Data Environments. The objective of the project was to develop a new transfer learning (TL) framework that aims to adaptively blend the data across different sources in tackling one task of interest, resulting in enhanced trustworthiness of ML models for mission- and safety-critical systems. The proposed framework determines when it is worth applying TL and how much knowledge is to be transferred, despite uncontrollable uncertainties. The framework accomplishes this by leveraging concepts and techniques from the fields of Bayesian inverse modeling and uncertainty quantification, relying on strong mathematical foundations of probability and measure theories to devise new uncertainty-aware TL workflows.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

A Bayesian Deep Learning Approach to Near-Term Climate Prediction

Since model bias and associated initialization shock are serious shortcomings that reduce prediction skills in state-of-the-art decadal climate prediction efforts, we pursue a complementary machine-learning-based approach to climate prediction. The example problem setting we consider consists of predicting natural variability of the North Atlantic sea surface temperature on the interannual timescale in the pre-industrial control simulation of the Community Earth System Model. While previous works have considered the use of recurrent networks such as convolutional LSTMs and reservoir computing networks in this and other similar problem settings, we currently focus on the use of feedforward convolutional networks. In particular, we find that a feedforward convolutional network with a Densenet architecture is able to outperform a convolutional LSTM in terms of predictive skill. Next, we go on to consider a probabilistic formulation of the same network based on Stein variational gradient descent and find that in addition to providing useful measures of predictive uncertainty, the probabilistic (Bayesian) version improves on its deterministic counterpart in terms of predictive skill. Finally, we characterize the reliability of the ensemble of machine learning models obtained in the probabilistic setting by using analysis tools developed in the context of ensemble numerical weather prediction.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Performance evaluation of automated data-driven feature extraction and selection methods for practical and scalable building energy consumption prediction models

Here, this study quantifies the impact of automated feature engineering methods (feature extraction and selection) on the quality and accuracy of machine learning models that predict building energy consumption. The case study compares model performance for three main scenarios: baseline (no feature extraction and selection), feature extraction only, and feature extraction combined with feature selection (filter and/or wrapper methods) for fully trained machine learning models for 200 metered/sub-metered energy measurements across 118 real buildings. For consistency, the same machine learning model architecture (a black box deep learning neural network with probabilistic forecast output) was used for all scenarios. Based on results, all feature engineering methods provided noticeable prediction accuracy improvements (e.g., 29%-68% median prediction improvement) compared to baseline scenarios. However, in this application, feature selection methods provide little practical value due to their limited performance gains and high computational cost. Smarter algorithm development supported by better computational environments will be needed before feature selection methods can reliably and efficiently improve predictive model performance.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Application of machine learning for the estimation of electron energy distribution from optical emission spectra

Abstract This paper discusses the use of probabilistic deep neural networks for the prediction of the electron energy probability function in low-temperature non-thermal plasmas. The neural networks are trained using optical emission spectroscopy and Langmuir probe measurements, with the goal of providing a reliable estimate of the electron energy probability function solely from optical emission data. The performance of both non-Bayesian and Bayesian networks is evaluated. It is found that Bayesian models are preferable as they assign a higher level of uncertainty to their prediction especially when the dataset used to train them is small. This work describes one of the many potential applications of machine learning in plasma science and technology.

Physics↗

DOE OSTI2023

Comparing quantile regression forest and mixture density long short-term memory models for probabilistic post-processing of satellite precipitation-driven streamflow simulations

Abstract. Deep learning (DL) and machine learning (ML) are widely used in hydrological modelling, which plays a critical role in improving the accuracy of hydrological predictions. However, the trade-off between model performance and computational cost has always been a challenge for hydrologists when selecting a suitable model, particularly for probabilistic post-processing with large ensemble members. This study aims to systematically compare the quantile regression forest (QRF) model and countable mixtures of asymmetric Laplacians long short-term memory (CMAL-LSTM) model as hydrological probabilistic post-processors. Specifically, we evaluate their ability in dealing with biased streamflow simulations driven by three satellite precipitation products across 522 nested sub-basins of the Yalong River basin in China. Model performance is comprehensively assessed using a series of scoring metrics from both probabilistic and deterministic perspectives. Our results show that the QRF model and the CMAL-LSTM model are comparable in terms of probabilistic prediction, and their performances are closely related to the flow accumulation area (FAA) of the sub-basin. The QRF model outperforms the CMAL-LSTM model in most sub-basins with smaller FAA, while the CMAL-LSTM model has an undebatable advantage in sub-basins with FAA larger than 60 000 km2 in the Yalong River basin. In terms of deterministic predictions, the CMAL-LSTM model is preferred, especially when the raw streamflow is poorly simulated and used as input. However, setting aside the differences in model performance, the QRF model with 100-member quantiles demonstrates a noteworthy advantage by exhibiting a 50 % reduction in computation time compared to the CMAL-LSTM model with the same ensemble members in all experiments. As a result, this study provides insights into model selection in hydrological post-processing and the trade-offs between model performance and computational efficiency. The findings highlight the importance of considering the specific application scenario, such as the catchment size and the required accuracy level, when selecting a suitable model for hydrological post-processing.

Geology↗

DOE OSTI2021

Probabilistic learning on manifolds constrained by nonlinear partial differential equations for small datasets

A novel extension of the Probabilistic Learning on Manifolds (PLoM) is presented. It makes it possible to synthesize solutions to a wide range of nonlinear stochastic boundary value problems described by partial differential equations (PDEs) for which a stochastic computational model (SCM) is available and which depend on a vector-valued random control parameter. The cost of a single numerical evaluation of this SCM is assumed to be such that only a limited number of points can be computed for constructing the training dataset (small data). Each point of the training dataset is made up of realizations from a vector-valued stochastic process (the stochastic solution) and the associated random control parameter on which it depends. The presented PLoM constrained by PDE allows for generating a large number of learned realizations of the stochastic process and its corresponding random control parameter. These learned realizations are generated so as to minimize the vector-valued random residual of the PDE in the mean-square sense. Appropriate novel methods are developed to solve this challenging problem. Three applications are presented. The first one is a simple uncertain nonlinear dynamical system with a nonstationary stochastic excitation. The second one concerns the 2D nonlinear unsteady Navier–Stokes equations for incompressible flows in which the Reynolds number is the random control parameter. Here, the last one deals with the nonlinear dynamics of a 3D elastic structure with uncertainties. The results obtained make it possible to validate the PLoM constrained by stochastic PDE but also provide further validation of the PLoM without constraint.

Machine learning↗

DOE OSTI2025

Power System Feature-Based Event Classification by Means of Multiple PMU Data

Abstract—Phasor Measurement Units (PMUs) provide time synchronized measurements across the power grid, enabling data driven event detection and classification for enhanced system monitoring and situational awareness. However, variations in event duration, spatial extent, and severity, along with coincident events, pose challenges for conventional classification models that require fixed-size inputs. This paper presents a feature-based framework that aggregates diverse attributes from all available PMUs for each event into a fixed-length vector, facilitating the application of standard machine learning classifiers, including Random Forest, XGBoost, and Multilayer Perceptron. A probabilistic post-processing scheme is further introduced to enable multi-label classification in the presence of overlapping events. Experiments using real-world PMU data demonstrate that the Random Forest model achieves 95% accuracy, while the proposed post-processing method yields an additional 3% improvement.

Nematirad, Reza↗