Bayesian Multitask with Structure Learning
Software including a Bayesian multitask learning model. The pipeline includes functionality to process data, train, evaluate models, and generate reports.
Soper, BradenC.↗
SEARCH · Engineering Papers
Results for “Bayesian learning”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Application of Machine Learning to Rotorcraft Health Monitoring
Machine learning is a powerful tool for data exploration and model building with large data sets. This project aimed to use machine learning techniques to explore the inherent structure of data from rotorcraft gear tests, relationships between features and damage states, and to build a system for predicting gear health for future rotorcraft transmission applications. Classical machine learning techniques are difficult, if not irresponsible to apply to time series data because many make the assumption of independence between samples. To overcome this, Hidden Markov Models were used to create a binary classifier for identifying scuffing transitions and Recurrent Neural Networks were used to leverage long distance relationships in predicting discrete damage states. When combined in a workflow, where the binary classifier acted as a filter for the fatigue monitor, the system was able to demonstrate accuracy in damage state prediction and scuffing identification. The time dependent nature of the data restricted data exploration to collecting and analyzing data from the model selection process. The limited amount of available data was unable to give useful information, and the division of training and testing sets tended to heavily influence the scores of the models across combinations of features and hyper-parameters. This work built a framework for tracking scuffing and fatigue on streaming data and demonstrates that machine learning has much to offer rotorcraft health monitoring by using Bayesian learning and deep learning methods to capture the time dependent nature of the data. Suggested future work is to implement the framework developed in this project using a larger variety of data sets to test the generalization capabilities of the models and allow for data exploration.
machine learning↗
Improved naive Bayesian probability classifier in predictions of nuclear mass
Recently, novel statistical methods such as neural networks and Bayesian learning methods are implemented to describe the nuclear masses. Based on previous studies, an improved naive Bayesian probability (iNBP) classifier is proposed to study the nuclear masses by refining the results of sophisticated nuclear models. In the iNBP method, the prediction for nuclear masses is treated as a classification problem. The residuals are classified into several groups to generate prior and conditional probabilities, and the posterior probabilities are further determined by the Bayesian formula. We choose the expectation with maximum probability as the final prediction. Reliability of the iNBP method is assessed by analyzing the global optimizations and the extrapolating capabilities. Here, the iNBP method exhibits impressive improvements on global descriptions for different mass models. Moreover, the method shows robust extrapolating capabilities. Results demonstrate the iNBP method can be applied to predict the nuclear masses of unknown regions. Considering the local mass relations, the iNBP method can offer considerable fine-tuning of the mass descriptions from nuclear models. The methodology proposed in this paper can also be applied to other model-based extrapolations of nuclear observables.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
Future applications of artificial intelligence to Mission Control Centers
Future applications of artificial intelligence to Mission Control Centers are presented in the form of the viewgraphs. The following subject areas are covered: basic objectives of the NASA-wide AI program; inhouse research program; constraint-based scheduling; learning and performance improvement for scheduling; GEMPLAN multi-agent planner; planning, scheduling, and control; Bayesian learning; efficient learning algorithms; ICARUS (an integrated architecture for learning); design knowledge acquisition and retention; computer-integrated documentation; and some speculation on future applications.
Friedland, Peter↗
TCAD-Machine Learning Enabled TID Compact Model Development for Commercial SiC MOSFET
We propose a TCAD (Technology Computer Aided Design)-machine learning coupled approach that combines a TCAD tool (Charon), optimization/uncertainty quantification tool (Dakota), surrogate models, and Bayesian learning capabilities. The coupling approach is used for accurate modeling and calibration of total ionizing dose (TID) induced threshold voltage (V th ) shifts in Commercial-Off-The-Shelf (COTS) semiconductor devices and to develop physics-informed TID compact models. This versatile approach is applied to model the TID effect in an exemplar COTS 3.3 kV SiC power MOSFET (Metal-Oxide-Semiconductor Field-Effect Transistor). With the Charon-Dakota coupling, we can determine key device geometry and doping values based on device physics, which are difficult to obtain or not available for COTS devices but important for TCAD simulation; additionally, we can efficiently generate thousands of simulation results in a large parameter space, which makes it possible to develop data-driven surrogate models and perform Bayesian calibration. Utilizing the full tool-coupling approach, we achieve calibrated TCAD simulation models that accurately capture the average TID-induced V th shifts behavior with total doses and V th shifts saturation at high doses as observed in experimental data. More importantly, the calibrated TCAD simulations are obtained with determined TID model parameters (e.g., hole trap density and capture cross section) values that contain well quantified uncertainties. Furthermore, we can isolate and quantify the noises that are not captured by the TCAD models but exist in the measured data due to measurements and devices variabilities. Lastly, the calibrated surrogate models are used to develop physics-informed TID compact models. The method is generalizable to other devices and/or radiation conditions with few modifications and can provide well-determined uncertainties.
COTS↗
A Multi-Timescale Data-Driven Approach to Enhance Distribution System Observability
Not provided.
Engineering↗
Comprehensive compartmental model and calibration algorithm for the study of clinical implications of the population-level spread of COVID-19: a study protocol
The complex dynamics of the coronavirus disease 2019 (COVID-19) pandemic has made obtaining reliable long-term forecasts of the disease progression difficult. Simple mechanistic models with deterministic parameters are useful for short-term predictions but have ultimately been unsuccessful in extrapolating the trajectory of the pandemic because of unmodelled dynamics and the unrealistic level of certainty that is assumed in the predictions. We propose a 22-compartment epidemiological model that includes compartments not previously considered concurrently, to account for the effects of vaccination, asymptomatic individuals, inadequate access to hospital care, post-acute COVID-19 and recovery with long-term health complications. Additionally, new connections between compartments introduce new dynamics to the system and provide a framework to study the sensitivity of model outputs to several concurrent effects, including temporary immunity, vaccination rate and vaccine effectiveness. Subject to data availability for a given region, we discuss a means by which population demographics (age, comorbidity, socioeconomic status, sex and geographical location) and clinically relevant information (different variants, different vaccines) can be incorporated within the 22-compartment framework. Considering a probabilistic interpretation of the parameters allows the model's predictions to reflect the current state of uncertainty about the model parameters and model states. We propose the use of a sparse Bayesian learning algorithm for parameter calibration and model selection. This methodology considers a combination of prescribed parameter prior distributions for parameters that are known to be essential to the modelled dynamics and automatic relevance determination priors for parameters whose relevance is questionable. This is useful as it helps prevent overfitting the available epidemiological data when calibrating the parameters of the proposed model. Population-level administrative health data will serve as partial observations of the model states.
59 BASIC BIOLOGICAL SCIENCES↗
SubTSBR to tackle high noise and outliers for data-driven discovery of differential equations
Data-driven discovery of differential equations has been an emerging research topic. We propose a novel algorithm subsampling-based threshold sparse Bayesian regression (SubTSBR) to tackle high noise and outliers. The subsampling technique is used for improving the accuracy of the Bayesian learning algorithm. It has two parameters: subsampling size and the number of subsamples. When the subsampling size increases with fixed total sample size, the accuracy of our algorithm goes up and then down. When the number of subsamples increases, the accuracy of our algorithm keeps going up. We demonstrate how to use our algorithm step by step and compare our algorithm with threshold sparse Bayesian regression (TSBR) for the discovery of differential equations. We show that our algorithm produces better results. We also discuss the merits of discovering differential equations from data and demonstrate how to discover models with random initial and boundary condition as well as models with bifurcations. The numerical examples are: (1) predator-prey model with noise, (2) shallow water equations with outliers, (3) heat diffusion with random initial and boundary condition, and (4) fish-harvesting problem with bifurcations.
97 MATHEMATICS AND COMPUTING↗
WE-H-BRC-06: A Unified Machine-Learning Based Probabilistic Model for Automated Anomaly Detection in the Treatment Plan Data
Purpose: The purpose of this work was to investigate the ability of a machine-learning based probabilistic approach to detect radiotherapy treatment plan anomalies given initial disease classes information. Methods In total we obtained 1112 unique treatment plans with five plan parameters and disease information from a Mosaiq treatment management system database for use in the study. The plan parameters include prescription dose, fractions, fields, modality and techniques. The disease information includes disease site, and T, M and N disease stages. A Bayesian network method was employed to model the probabilistic relationships between tumor disease information, plan parameters and an anomaly flag. A Bayesian learning method with Dirichlet prior was useed to learn the joint probabilities between dependent variables in error-free plan data and data with artificially induced anomalies. In the study, we randomly sampled data with anomaly in a specified anomaly space.We tested the approach with three groups of plan anomalies – improper concurrence of values of all five plan parameters and values of any two out of five parameters, and all single plan parameter value anomalies. Totally, 16 types of plan anomalies were covered by the study. For each type, we trained an individual Bayesian network. Results: We found that the true positive rate (recall) and positive predictive value (precision) to detect concurrence anomalies of five plan parameters in new patient cases were 94.45±0.26% and 93.76±0.39% respectively. To detect other 15 types of plan anomalies, the average recall and precision were 93.61±2.57% and 93.78±3.54% respectively. The computation time to detect the plan anomaly of each type in a new plan is ∼0.08 seconds. Conclusion: The proposed method for treatment plan anomaly detection was found effective in the initial tests. The results suggest that this type of models could be applied to develop plan anomaly detection tools to assist manual and automated plan checks. The senior author received research grants from ViewRay Inc. and Varian Medical System.
60 APPLIED LIFE SCIENCES↗
Solving Stochastic Inverse Problems for Property–Structure Linkages Using Data-Consistent Inversion and Machine Learning
Determining process–structure–property linkages is one of the key objectives in material science, and uncertainty quantification plays a critical role in understanding both process–structure and structure–property linkages. In this work, we seek to learn a distribution of microstructure parameters that are consistent in the sense that the forward propagation of this distribution through a crystal plasticity finite element model matches a target distribution on materials properties. This stochastic inversion formulation infers a distribution of acceptable/consistent microstructures, as opposed to a deterministic solution, which expands the range of feasible designs in a probabilistic manner. Furthermore, to solve this stochastic inverse problem, we employ a recently developed uncertainty quantification framework based on push-forward probability measures, which combines techniques from measure theory and Bayes’ rule to define a unique and numerically stable solution. This approach requires making an initial prediction using an initial guess for the distribution on model inputs and solving a stochastic forward problem. To reduce the computational burden in solving both stochastic forward and stochastic inverse problems, we combine this approach with a machine learning Bayesian regression model based on Gaussian processes and demonstrate the proposed methodology on two representative case studies in structure–property linkages.
36 MATERIALS SCIENCE↗
Machine learning-assisted identification of potential sources of bias in measurements of prompt-fission neutron spectra
Unrecognized sources of uncertainty (USU) can bias the reported mean and/or covariance of experimental nuclear data. These biases, in turn, can propagate through evaluated nuclear data to application simulations or may poorly inform nuclear theory that is fitted to the experimental data. Such unknown sources of bias must be tied to the inherent physical constituents of the measurements such as the characteristics of a detector response or a background reduction technique. Here, in this article, a sparse Bayesian learning model is used to support experts in their efforts to identify and characterize USU in experimental prompt fission neutron spectra (PFNS) for spontaneous fissioning of 252 Cf by linking observed biases to features of the measurement system. Three different bias components were found. The first acts as a verification case for the algorithm as it identifies a bias coming from a well-known source related to the use of 6 Li in the neutron detection system. The second two cases demonstrate how this method can benefit the evaluation of experimental nuclear data by identifying, quantifying, and relating unknown biases to potential causes.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
Fermilab Booster loss modelling and rebalancing using Bayesian methods
Fermilab Booster is being upgraded for the PIP-II project to support 20Hz ramp rate at higher intensities. Loss trip limits determine the achievable peak power. To meet PIP-II requirements, losses need to be halved as compared to current levels. Losses primarily occur at injection and transition crossing, with both gradually increasing and threshold-like intensity-dependent behaviors. The existing simulation models are not yet good enough for quantitative loss predictions. In practice, it will be necessary to tune up the Booster using iterative methods and operator intuition. In this paper we present an effort to systematically model Booster losses using active learning (Bayesian exploration) techniques, and subsequently to rebalance them for higher trip limit margins. We first created several sets of spatially and temporally isolated orbit and optics knobs, and trained Gaussian process models for each beam loss monitor as well as beam current. This is a complex task due to safety and timing requirements – we discuss mitigations such as uncertainty constraints and approximate fitting. Once models are stable, we perform large-scale single and multi-objective tuning using scalarized objectives made up of critical beam loss locations. Our results demonstrate significant rebalancing of losses, increasing trip margins, as well as an overall improvement in beam transmission efficiency. We are exploring how to combine existing simulations with experimental data and automate the collection procedure so that more advanced surrogate models can be created over time.
Kuklev, Nikita [Fermilab]↗
On-the-fly active learning of interpretable Bayesian force fields for atomistic rare events
Abstract Machine learned force fields typically require manual construction of training sets consisting of thousands of first principles calculations, which can result in low training efficiency and unpredictable errors when applied to structures not represented in the training set of the model. This severely limits the practical application of these models in systems with dynamics governed by important rare events, such as chemical reactions and diffusion. We present an adaptive Bayesian inference method for automating the training of interpretable, low-dimensional, and multi-element interatomic force fields using structures drawn on the fly from molecular dynamics simulations. Within an active learning framework, the internal uncertainty of a Gaussian process regression model is used to decide whether to accept the model prediction or to perform a first principles calculation to augment the training set of the model. The method is applied to a range of single- and multi-element systems and shown to achieve a favorable balance of accuracy and computational efficiency, while requiring a minimal amount of ab initio training data. We provide a fully open-source implementation of our method, as well as a procedure to map trained models to computationally efficient tabulated force fields.
36 MATERIALS SCIENCE↗
Proteomic Profiling of the Substantia Nigra to Identify Determinants of Lewy Body Pathology and Dopaminergic Neuronal Loss
Proteinaceous aggregates containing ?-synuclein protein called Lewy bodies in the substantia nigra is a hallmark of Parkinson’s disease. The molecular mechanisms of Lewy body formation and associated neuronal loss remain largely unknown. To gain insights on proteins and pathways associated with Lewy body pathology, we performed quantitative profiling of the proteome. We analyzed substantia nigra tissue from 51 subjects arranged into three groups: cases with Lewy body pathology, Lewy body-negative controls with matching neuronal loss and controls with no neuronal loss. Using label-free LC-MS/MS approach we quantified the 2,963 most abundant proteins. Statistical testing for differential protein abundance, followed by pathway enrichment and Bayesian learning of the causal network structure were performed to identify likely drivers of Lewy body formation and dopaminergic neuronal loss. The identified pathways include (1) Arp2/3 complex-mediated actin nucleation; (2) synaptic function; (3) poly(A) RNA binding; (4) basement membrane and endothelium; and (5) hydrogen peroxide metabolic process. According to the data, the endothelial/basement membrane pathway is tightly connected with both pathologies and likely to be one of the drivers of the neuronal loss. The poly(A) RNA-binding proteins, including the ones relevant to other neurodegenerative disorders (e.g. TDP-43 and FUS) have strong inverse correlation with Lewy bodies and may reflect alternative mechanism of nigral neurodegeneration.
Petyuk, Vladislav A.↗
Writing the Programs of Programmable Catalysis
It has long been known that non-steady state and periodic catalytic reactor operation in terms of temperature, pressure, and composition can lead to higher overall productivity and/or product selectivity than the best steady operation. Recently, the emergence of catalysts whose intrinsic properties can be made to oscillate with time, introduces advanced forcing capabilities that can be “programmed” into the catalysts to broaden the scope and applicability of periodic operation to surface chemistry. In this work, an algorithmic approach is implemented to significantly accelerate the discovery and optimization of periodic steady states of catalytic reactors. Decomposition of complex dynamics into fundamental mechanistic fast–slow steps is seen to improve conceptual understanding of the relationship between binding energy oscillation protocols and overall catalytic rates. Finding structured forcing protocols, optimally tailored to the multiple time scales of a given individual mechanism, requires an efficient search of high-dimensional parameter spaces. Here, this is enabled here through active learning (Bayesian optimization, enhanced by our proposed Bayesian continuation). Implementation of these methods is shown to accelerate the evaluation of catalyst programs by up to several orders of magnitude. Faster screening of programmable catalysts to discover periodic steady states enables the optimization of catalytic operating protocols and thus opens the possibility for catalyst engineering based on optimal forcing programs to control rate and product selectivity, even for complex multistep catalytic mechanisms.
catalysis↗
Gaussian processes meet NeuralODEs: a Bayesian framework for learning the dynamics of partially observed systems from scarce and noisy data
We present a machine learning framework (GP-NODE) for Bayesian model discovery from partial, noisy and irregular observations of nonlinear dynamical systems. The proposed method takes advantage of differentiable programming to propagate gradient information through ordinary differential equation solvers and perform Bayesian inference with respect to unknown model parameters using Hamiltonian Monte Carlo sampling and Gaussian Process priors over the observed system states. This allows us to exploit temporal correlations in the observed data, and efficiently infer posterior distributions over plausible models with quantified uncertainty. The use of the Finnish Horseshoe as a sparsity-promoting prior for free model parameters also enables the discovery of parsimonious representations for the latent dynamics. A series of numerical studies is presented to demonstrate the effectiveness of the proposed GP-NODE method including predator–prey systems, systems biology and a 50-dimensional human motion dynamical system. This article is part of the theme issue ‘Data-driven prediction in dynamical systems’.
Science & Technology - Other Topics↗
Multi-fidelity equations of state and transport coefficient datasets for pulsed-power applications
Reliably simulating experiments relevant to the National Nuclear Security Administration (NNSA) requires a detailed description of material properties across a wide range of conditions. Such properties include the equations of state, charged-particle transport coefficients, and optical properties like the opacity. Together, these properties make up the material models used in radiation-magnetohydrodynamic simulations of nuclear fusion experiments. Many of these models do not incorporate uncertainties in the data used to produce them. It is unknown whether these uncertainties significantly impact the interpretation of simulation results and diagnostics. The purpose of this work is to quantify how such uncertainties impact simulations of pulsed-power experiments. We accomplished this task by first assessing discrepancies between approaches used to generate the data. This included bringing together members of the high-energy-density community spanning the three NNSA laboratories and multiple universities. Then, using these data, we developed a general framework that systematically incorporates physical uncertainties within the material models suitable for uncertainty quantification analyses. The framework utilizes machine learning, Bayesian inference, and incorporates multi-fidelity datasets. We demonstrated the framework by quantifying the impact that material model uncertainties have on simulations of pulsed-power experiments underway on Z at Sandia National Laboratories. As a result of this work, we discovered that modest uncertainties in material models (roughly 20%) correspond to significant uncertainties in the outputs from simulations. Our framework has enabled rapid construction of material models through an automated procedure and allows for the generation of material models of interest to the NNSA.
36 MATERIALS SCIENCE↗
Technical Report for Bayesian Optimization and Reinforcement Learning for Beam Polarization Increase in the BNL Hadron Injectors
This project developed and evaluated physics-informed Bayesian learning and machine learning (ML)-based optimization methods for improving beam polarization preservation in the BNL hadron injector chain. The work focused on uncertainty-aware digital twin modeling, Bayesian calibration of accelerator simulations using beam measurements, and data-efficient optimization strategies including Bayesian optimization and reinforcement learning. These methods were applied to injector tuning and RF control problems in realistic accelerator settings to support improved operational robustness and readiness for RHIC operations and future Electron–Ion Collider facilities. No subject inventions were disclosed under this award.
43 PARTICLE ACCELERATORS↗