Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “statistical learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 793 records · Page 44
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Real-Time Health Monitoring for Gas Turbine Components Using Online Learning and High-Dimensional Data (Final Report)

Capital-intensive turbomachinery, such as gas turbines and combined cycle plants, are constantly being monitored for performance anomalies, faults, and physical degradation. Although these power-generating assets are equipped with hundreds of sensors, existing monitoring tools can only handle moderate-sized data. As a result, only a handful of aggregate metrics are used to monitor machine health. At the same time, developing advanced tools suitable for large datasets have been restricted by the lack of appropriate data. The objective of this proposal was to demonstrate a Big Data analytics framework for fault detection and diagnosis in gas turbine applications. We develop a predictive analytics framework methodology guided by these experimental data, industrial data from our collaborators, and physics-based models with engineering domain knowledge. Our analytics framework consists of four key components: (1) a data curation process that addresses data storage, data quality assessments, and integrity checks, (2) a feature engineering component that utilizes statistical methods and transformation algorithms guided by physics-based models to extract high-fidelity fault features that can be leveraged for fault detection and classifying fault severities, (3) a Machine Learning-based fault detection and diagnostics algorithms for detecting operational and hardware faults in the combustion and the turbines section. We utilize two industry-class gas turbine component test rigs to generate first of its kind data for critical gas turbine faults with varying severity levels. Advanced gas turbine test facilities will be interrogated using state-of-the-art instrumentation techniques to build fault signatures and data trends for key combustor and turbine faults. Data generated from a combustor test rig (Georgia Tech) and a turbine test rig (Penn State) during both normal operation and with seeded faults serve as the basis for the Big Data sets. The test conditions in the two test facilities include common, critical events that occur in the operation. Utilizing the combustor test rig, we examine two common combustor faults: lean blowout and centerbody degradation. For the turbine section we develop analytic models for monitoring cooling faults in the gas turbine.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2025

Criticality analysis of nuclear binding energy neural networks

Machine learning methods, in particular deep learning methods such as artificial neural networks (ANNs) with many layers, have become widespread and useful tools in nuclear physics. However, these ANNs are typically treated as ‘black boxes’, with their architecture (width, depth, and weight/bias initialization) and the training algorithm and parameters chosen empirically by optimizing learning based on limited exploration. We test a non-empirical approach to understanding and optimizing nuclear physics ANNs by adapting a criticality analysis based on renormalization group flows in terms of the hyperparameters for weight/bias initialization, training rates, and the ratio of depth to width. This treatment utilizes the statistical properties of neural network initialization to find a generating functional for network outputs at any layer, allowing for a path integral formulation of the ANN outputs as a Euclidean statistical field theory. We use a prototypical example to test the applicability of this approach: a simple ANN for nuclear binding energies. We find that with training using a stochastic gradient descent optimizer, the predicted criticality behavior is realized, and optimal performance is found with critical tuning. However, the use of an adaptive learning algorithm leads to somewhat superior results without concern for tuning and thus obscures the analysis. Nevertheless, the criticality analysis offers a way to look within the black box of ANNs, which is a first step towards potential improvements in network performance beyond using adaptive optimizers.

artificial neural network↗
DOE OSTI2023

Mathematical nuances of Gaussian process-driven autonomous experimentation

Abstract The fields of machine learning (ML) and artificial intelligence (AI) have transformed almost every aspect of science and engineering. The excitement for AI/ML methods is in large part due to their perceived novelty, as compared to traditional methods of statistics, computation, and applied mathematics. But clearly, all methods in ML have their foundations in mathematical theories, such as function approximation, uncertainty quantification, and function optimization. Autonomous experimentation is no exception; it is often formulated as a chain of off-the-shelf tools, organized in a closed loop, without emphasis on the intricacies of each algorithm involved. The uncomfortable truth is that the success of any ML endeavor, and this includes autonomous experimentation, strongly depends on the sophistication of the underlying mathematical methods and software that have to allow for enough flexibility to consider functions that are in agreement with particular physical theories. We have observed that standard off-the-shelf tools, used by many in the applied ML community, often hide the underlying complexities and therefore perform poorly. In this paper, we want to give a perspective on the intricate connections between mathematics and ML, with a focus on Gaussian process-driven autonomous experimentation. Although the Gaussian process is a powerful mathematical concept, it has to be implemented and customized correctly for optimal performance. We present several simple toy problems to explore these nuances and highlight the importance of mathematical and statistical rigor in autonomous experimentation and ML. One key takeaway is that ML is not, as many had hoped, a set of agnostic plug-and-play solvers for everyday scientific problems, but instead needs expertise and mastery to be applied successfully. Graphical abstract

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Shower Reconstruction for Cross-Section Measurements with Photon(s) in the Final State at SBND

The Short-Baseline Near Detector (SBND) is a 112-ton liquid argon time projection chamber operating in the Booster Neutrino Beam at Fermilab as part of the Short-Baseline Neutrino Program. Precise reconstruction of electromagnetic showers from photons and electrons is essential for searches for neutrino oscillations, cross-section measurements, and rare-process studies in liquid argon. In this poster, we present a unified set of SBND analyses enabled by the SPINE (Scalable Particle Imaging with Neural Embeddings) end-to-end machine-learning–based reconstruction framework, highlighting charged-current neutral pion (CC π⁰) production and neutral-current (NC) Δ radiative decay as complementary physics cases. Charged-current π⁰ interactions provide a high-statistics benchmark for validating photon reconstruction, constraining the electromagnetic energy scale, and modeling backgrounds to other cross-section measurements and BSM searches with one or more photons in the final state. We summarize progress toward the ongoing development of a CC π⁰ selection in SBND and, report on the current status of a search for NC resonant Δ production followed by radiative decay (Δ→Nγ) in SBND. This rare process, previously investigated in MicroBooNE, provides a stringent test of photon-only final-state reconstruction and is of interest in the context of the MiniBooNE low-energy excess. Together, these analyses demonstrate how modern machine-learning–based shower reconstruction enables a diverse and scalable physics program at SBND.

Chung, Seokju [Nevis Labs, Columbia U.]↗
NASA NTRSDate not supplied

Interpretable Machine Learning Models for Autonomous Characterization of Analogue Ocean World Seawater Chemistry and Biosignature Potential Using Isotope Ratio Data

Background: Future missions to ocean worlds, such as Enceladus and Europa, will attempt to characterize the subsurface seawater chemistry and assess the potential for life. Such missions will be equipped with capabilities to precisely measure volatile isotopes in plumes, atmospheres, and exospheres. Motivation: While large isotopic fractionations can indicate a biological source, there are signatures resulting from abiotic geochemical processes that mimic isotopic biosignatures. While machine learning (ML) has the potential to disentangle competing effects and biotic mimicry, high-dimensional isotope ratio mass spectrometry (IRMS) data is likely to contain noise/irrelevant features and involve complex statistical interactions that make human inference and interpretation difficult. Further, ML predictions with as far-reaching implications as an extraterrestrial biosignature on an ocean world requires the use of interpretable models (i.e., not “black box” models) with physically and mathematically meaningful feature spaces along with false positive diagnostics. Methods: We use volatile CO2 IRMS data of analogue ocean world seawaters to validate an ML approach to provide biogeochemical context for biosignature detection. We employ a feature selection method called nearest-neighbor projected distance regression (NPDR) that detects statistical interactions and helps elucidate the mechanisms of the Random Forest classification models. Results: We train and validate predictive ML models on volatile CO2 IRMS data of analogue ocean world seawaters to predict major salt components (e.g., MgSO4, NaHCO3), pH, ionic strength, and the presence of biosignatures. Features derived from IRMS measurements are augmented with extracted time-series features. Our results show high test accuracy and interpretability, which is increased by interaction network visualization, sample-wise variable importance scores, and single-sample class probability estimates. We demonstrate an ML mission software solution that triggers autonomous data transmission and biogeochemical sample prediction.

geochemistry↗
DOE OSTI2020

Machine learning and ligand binding predictions: A review of data, methods, and obstacles

We report that computational predictions of ligand binding is a difficult problem, with more accurate methods being extremely computationally expensive. The use of machine learning for drug binding predictions could possibly leverage the use of biomedical big data in exchange for time-intensive simulations. This paper reviews current trends in the use of machine learning for drug binding predictions, data sources to develop machine learning algorithms, and potential problems that may lead to overfitting and ungeneralizable models. A few popular datasets that can be used to develop virtual high-throughput screening models are characterized using spatial statistics to quantify potential biases. We can see from evaluating some common benchmarks that good performance correlates with models with high-predicted bias scores and models with low bias scores do not have much predictive power. A better understanding of the limits of available data sources and how to fix them will lead to more generalizable models that will lead to novel drug discovery.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Electron Neutrino Selection for Neutrino Oscillation Searches in the SBN Program

The Short- Baseline Neutrino (SBN) Pprogram at Fermilab employs a near detector, SBND, and a far detector, ICARUS, to search for baseline-dependent changes to the measure muon neutrino disappearance and electron neutrino appearance and disappearance from the muon neutrino-dominated Booster Neutrino Beam flux. The use of the same liquid argon time projection chamber (LArTPC) technology in both detectors and the same neutrino beam enables a systematically well-constrained oscillation measurement, by leveraging the high statistics of O(10 million) neutrino interactions expected at SBND through 2027 to constrain flux and cross-section systematics. In this poster, we present high-statistics, high-purity electron neutrino selections in SBND and ICARUS and related electron shower energy calibration measurements in SBND enabled by the SPINE deep learning-based particle reconstruction framework. Furthermore, we discuss how these samples can enable high sensitivity to '3+1' sterile neutrino oscillations in a combined fit accounting for systematic uncertainties and correlations.

Oza, Nupur [Nevis Labs, Columbia U.; Columbia U.]↗
NASA NTRS1992

Determining significant material properties: A discovery approach

The following is a laboratory experiment designed to further understanding of materials science. The experiment itself can be informative for persons of any age past elementary school, and even for some in elementary school. The preparation of the plastic samples is readily accomplished by persons with resonable dexterity in the cutting of paper designs. The completion of the statistical Design of Experiments, which uses Yates' Method, requires basic math (addition and subtraction). Interpretive work requires plotting of data and making observations. Knowledge of statistical methods would be helpful. The purpose of this experiment is to acquaint students with the seven classes of recyclable plastics, and provide hands-on learning about the response of these plastics to mechanical tensile loading.

Karplus, Alan K.↗
NASA NTRS2016

Thermodynamic Spectrum of Solar Flares Based on SDO/EVE Observations: Techniques and First Results

The Solar Dynamics Observatory (SDO)/EUV Variability Experiment (EVE) provides rich information on the thermodynamic processes of solar activities, particularly on solar flares. Here, we develop a method to construct thermodynamic spectrum (TDS) charts based on the EVE spectral lines. This tool could potentially be useful for extreme ultraviolet (EUV) astronomy to learn about the eruptive activities on distant astronomical objects. Through several cases, we illustrate what we can learn from the TDS charts. Furthermore, we apply the TDS method to 74 flares equal to or greater than the M5.0 class, and reach the following statistical results. First, EUV peaks are always behind the soft X-ray (SXR) peaks and stronger flares tend to have faster cooling rates. There is a power-law correlation between the peak delay times and the cooling rates, suggesting a coherent cooling process of flares from SXR to EUV emissions. Second, there are two distinct temperature drift patterns, called Type I and Type II. For Type I flares, the enhanced emission drifts from high to low temperature like a quadrilateral, whereas for Type II flares the drift pattern looks like a triangle. Statistical analysis suggests that Type II flares are more impulsive than Type I flares. Third, for late-phase flares, the peak intensity ratio of the late phase to the main phase is roughly correlated with the flare class, and the flares with a strong late phase are all confined. We believe that the re-deposition of the energy carried by a flux rope, which unsuccessfully erupts out, into thermal emissions is responsible for the strong late phase found in a confined flare. Furthermore, we show the signatures of the flare thermodynamic process in the chromosphere and transition region in the TDS charts. These results provide new clues to advance our understanding of the thermodynamic processes of solar flares and associated solar eruptions, e.g., coronal mass ejections.

Wang, Yuming↗
NASA NTRSDate not supplied

Use of Design of Experiments in Determining Neural Network Architectures for Loss of Control Detection

We describe empirical methods for selecting a neural network architecture to implement belief state inference on generic commercial transport aircraft. We highlight a case study on the planning, execution, and analysis of a set of experiments to determine the configurations of a conditional variational autoencoder (CVAE). Our main contribution is the application of a structured method that can be used for machine learning in many aerospace applications. This method optimizes the structure and training parameters of a neural network for belief state inference, using Design of Experiments (DOE) statistical methodologies. The motivation for this specific DOE analysis was to identify the appropriate hyperparameters for measuring the CVAE reconstruction probability and latent space, such that the measurements can be used to infer qualitative state changes for the aircraft. We demonstrate that this process yields information about a trained neural network’s utility for this specific application, along with a quantifiable range of certainty. We execute 84 experiments using loss-of-control flight maneuver data from the NASA T-2 aircraft, demonstrating that this empirical process allows us to construct cheap and simple models with specific attributes amenable to belief state inference in aerospace applications.

Loss of Control↗
NASA NTRSDate not supplied

Use of Design of Experiments in Determining Neural Network Architectures for Loss of Control Detection

We describe empirical methods for selecting a neural network architecture to implement belief state inference on generic commercial transport aircraft. We highlight a case study on the planning, execution, and analysis of a set of experiments to determine the configurations of a conditional variational autoencoder (CVAE). Our main contribution is the application of a structured method that can be used for machine learning in many aerospace applications. This method optimizes the structure and training parameters of a neural network for belief state inference, using Design of Experiments (DOE) statistical methodologies. The motivation for this specific DOE analysis was to identify the appropriate hyperparameters for measuring the CVAE reconstruction probability and latent space, such that the measurements can be used to infer qualitative state changes for the aircraft. We demonstrate that this process yields information about a trained neural network’s utility for this specific application, along with a quantifiable range of certainty. We execute 84 experiments using loss-of-control flight maneuver data from the NASA T 2 aircraft, demonstrating that this empirical process allows us to construct cheap and simple models with specific attributes amenable to belief state inference in aerospace applications.

Loss of Control↗
NASA NTRSDate not supplied

Use of Design of Experiments in Determining Neural Network Architectures for Loss of Control Detection

Abstract—We describe empirical methods for selecting a neural network architecture to implement belief state inference on generic commercial transport aircraft. We highlight a case study on the planning, execution, and analysis of a set of experiments to determine the configurations of a conditional variational autoencoder (CVAE). Our main contribution is the application of a structured method that can be used for machine learning in many aerospace applications. This method optimizes the structure and training parameters of a neural network for belief state inference, using Design of Experiments (DOE) statistical methodologies. The motivation for this specific DOE analysis was to identify the appropriate hyperparameters for measuring the CVAE reconstruction probability and latent space, such that the measurements can be used to infer qualitative state changes for the aircraft. We demonstrate that this process yields information about a trained neural network’s utility for this specific application, along with a quantifiable range of certainty. We execute 84 experiments using loss-of-control flight maneuver data from the NASA T-2 aircraft, demonstrating that this empirical process allows us to construct cheap and simple models with specific attributes amenable to belief state inference in aerospace applications.

neural networks↗
DOE OSTI2020

Retrofit-ability: A supplementary metric to inform energy efficiency policies and programs

Building Energy Use Intensity (EUI) has been commonly used to facilitate policy makers and utilities to design energy efficacy programs and help building managers prioritize investment in building upgrades. Buildings with higher EUIs are usually chosen as better candidates for retrofit. Our study of nearly one hundred buildings in Seattle’s Building Tune Up Accelerator program and thousands of buildings in the Asset Score database reveals that EUI (after weather/location/use type normalization) alone is not the best indicator of a building’s potential to save energy, especially for those buildings in the mid-range (30-70 percentile EUI). In this paper, we introduce the concept of “retrofit-ability” — a building’s real potential to reduce its energy use cost-effectively — as a supplementary metric to inform energy efficiency policy and investment. Currently, such potential analyses are predicted for a building stock using prototypical building models or statistical data. However, typical buildings or general historical data may not sufficiently represent building configurations in a portfolio and predict their saving potentials. We utilized machine learning to investigate how key building characteristics (such as envelope attributes, HVAC type, location, use type, etc.) affect a building’s improvement potentials and developed a low-cost method to quantify such potentials for a portfolio of buildings. The supplementary perspective provided by “retrofit-ability” highlights which building assets correlate most with a building’s energy savings potential across region, building use type and more.

Wang, Na↗
DOE OSTI2021

Multiscale Modeling Meets Machine Learning: What Can We Learn?

Machine learning is increasingly recognized as a promising technology in the biological, biomedical, and behavioral sciences. There can be no argument that this technique is incredibly successful in image recognition with immediate applications in diagnostics including electrophysiology, radiology, or pathology, where we have access to massive amounts of annotated data. However, machine learning often performs poorly in prognosis, especially when dealing with sparse data. This is a field where classical physics-based simulation seems to remain irreplaceable. In this review, we identify areas in the biomedical sciences where machine learning and multiscale modeling can mutually benefit from one another: Machine learning can integrate physics-based knowledge in the form of governing equations, boundary conditions, or constraints to manage ill-posted problems and robustly handle sparse and noisy data; multiscale modeling can integrate machine learn- ing to create surrogate models, identify system dynamics and parameters, analyze sensitivities, and quantify uncertainty to bridge the scales and understand the emergence of function. With a view towards applications in the life sciences, we discuss the state of the art of combining machine learning and multiscale modeling, identify applications and opportunities, raise open questions, and address potential challenges and limitations. We anticipate that it will stimulate discussion within the community of computational mechanics and reach out to other disciplines including mathematics, statistics, computer science, artificial intelligence, biomedicine, systems biology, and precision medicine to join forces towards creating robust and efficient models for biological systems.

machine learning, multiscale modeling, physics-bas↗
DOE OSTI2022

Benchmarking structural evolution methods for training of machine learned interatomic potentials

When creating training data for machine-learned interatomic potentials (MLIPs), it is common to create initial structures and evolve them using molecular dynamics (MD) to sample a larger configuration space. Here, we benchmark two other modalities of evolving structures, contour exploration (CE) and dimer-method (DM) searches against MD for their ability to produce diverse and robust density functional theory training data sets for MLIPs. We also discuss the generation of initial structures which are either from known structures or from random structures in detail to further formalize the structure-sourcing processes in the future. The polymorph-rich zirconium-oxygen composition space is used as a rigorous benchmark system for comparing the performance of MLIPs trained on structures generated from these structural evolution methods. Using Behler–Parrinello neural networks as our MLIP models, we find that CE and the DM searches are generally superior to MD in terms of spatial descriptor diversity and statistical accuracy.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2021

Learning the Low Frequency Earthquake Activity on the Central San Andreas Fault

Abstract Low frequency earthquakes (LFEs) originating below the central San Andreas Fault are associated with slow‐slip beneath the seismogenic zone within the more ductile portion of the crust. Monitoring efforts over 15 years detected >1 million LFEs. We train a gradient boosted tree model using statistical features describing the seismic waveforms to estimate the hourly LFE event count. The burst‐like LFE behavior is reproduced, while lower amplitudes are predicted during the most active periods. The hourly event counts are up to 18% greater than the catalog. The ability to continuously monitor LFE activity provides insight to when geodetic measurements of slow slip are possible, without the need for developing a computational‐intensive template‐matching catalog. Similar waveform statistical features are found between detecting LFEs and tremors, which provides additional evidence tremors are composed of LFEs. The approach extracts information contained in continuous seismic waveforms that might benefit detecting precursory signals.

58 GEOSCIENCES↗
DOE OSTI2022

Differentiable Preisach Modeling for Characterization and Optimization of Particle Accelerator Systems with Hysteresis

Future improvements in particle accelerator performance are predicated on increasingly accurate online modeling of accelerators. Hysteresis effects in magnetic, mechanical, and material components of accelerators are often neglected in online accelerator models used to inform control algorithms, even though reproducibility errors from systems exhibiting hysteresis are not negligible in high precision accelerators. Here, we combine the classical Preisach model of hysteresis with machine learning techniques to efficiently create nonparametric, high-fidelity models of arbitrary systems exhibiting hysteresis. We experimentally demonstrate how these methods can be used in situ, where a hysteresis model of an accelerator magnet is combined with a Bayesian statistical model of the beam response, allowing characterization of magnetic hysteresis solely from beam-based measurements. Finally, we explore how using these joint hysteresis-Bayesian statistical models allows us to overcome optimization performance limitations that arise when hysteresis effects are ignored.

43 PARTICLE ACCELERATORS↗
NASA NTRS2023

A Machine Learning-Based Approach to Time-Series Wave Identification in the Solar Wind

The Wind spacecraft has yielded several decades of high-resolution magnetic field data, a large fraction of which displays small-scale structures. In particular, the solar wind is full of wavelike fluctuations that appear in both the field magnitude and its components. The nature of these fluctuations can be tied to the properties of other structures in the solar wind, such as shocks, that have implications for the time evolution of the solar wind. As such, having a large collection of wave events would facilitate further study of the effects that these fluctuations have on solar wind evolution. Given the large volume of magnetic field data available, machine learning is the most practical approach to classifying the myriad small-scale structures observed. To this end, a subset of Wind data is labeled and used as a training set for a multi-branch 1D convolutional neural network aimed at classifying circularly polarized wave modes. Using this algorithm, a preliminary statistical study of one year of data is performed, yielding about 300,000 wave intervals out of about 5,000,000 solar wind intervals. The wave intervals come about more often in the fast solar wind and at higher temperatures, and the number of waves per day is highly periodic. This machine learning-based approach to wave detection has the potential to be a powerful, inexpensive way to catalog waves throughout decades of spacecraft data.

Samuel Fordin↗