Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “statistical learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
NASA NTRS2012

Interactive Web Graphs with Fewer Restrictions

There is growing popularity for interactive, statistical web graphs and programs to generate them. However, it seems that these programs tend to be somewhat restricted in which web browsers and statistical software are supported. For example, the software might use SVG (e.g., Protovis, gridSVG) or HTML canvas, both of which exclude most versions of Internet Explorer, or the software might be made specifically for R (gridSVG, CRanvas), thus excluding users of other stats software. There are more general tools (d3, Rapha lJS) which are compatible with most browsers, but using one of these to make statistical graphs requires more coding than is probably desired, and requires learning a new tool. This talk will present a method for making interactive web graphs, which, by design, attempts to support as many browsers and as many statistical programs as possible, while also aiming to be relatively easy to use and relatively easy to extend.

Fiedler, James↗
DOE OSTI2021

Constrained non-negative matrix factorization enabling real-time insights of in situ and high-throughput experiments

Non-negative matrix factorization (NMF) is an appealing class of methods for performing unsupervised learning on streaming spectral data, particularly in time-sensitive applications such as in situ characterization of materials. These methods seek to decompose a dataset into a small number of components and weights that can compactly represent the underlying signal while effectively reconstructing the observations with minimal error. However, canonical NMF methods have no underlying requirement that the reconstruction uses components or weights that are representative of the true physical processes. In this work, we demonstrate how constraining a subset of the NMF weights or components as rigid priors, provided as known or assumed values, can provide significant improvement in revealing true underlying phenomena. We present a PyTorch-based method for efficiently applying constrained NMF and demonstrate its application to several synthetic examples. Our implementation allows an expert researcher-in-the-loop to provide and dynamically adjust the constraints during a live experiment involving streaming spectral data. Such interactive priors allow researchers to specify known or identified independent components, as well as functional expectations about the mixing or transitions between the components. We further demonstrate the application of this method to measured synchrotron x-ray total scattering data from in situ beamline experiments. In such a context, constrained NMF can result in a more interpretive and scientifically relevant decomposition than canonical NMF or other decomposition techniques. As a result, the details of the method are provided, along with general guidance for employing constrained NMF in the extraction of critical information and insights during time-sensitive experimental applications.

36 MATERIALS SCIENCE↗
NASA NTRS2013

Metric Learning to Enhance Hyperspectral Image Segmentation

Unsupervised hyperspectral image segmentation can reveal spatial trends that show the physical structure of the scene to an analyst. They highlight borders and reveal areas of homogeneity and change. Segmentations are independently helpful for object recognition, and assist with automated production of symbolic maps. Additionally, a good segmentation can dramatically reduce the number of effective spectra in an image, enabling analyses that would otherwise be computationally prohibitive. Specifically, using an over-segmentation of the image instead of individual pixels can reduce noise and potentially improve the results of statistical post-analysis. In this innovation, a metric learning approach is presented to improve the performance of unsupervised hyperspectral image segmentation. The prototype demonstrations attempt a superpixel segmentation in which the image is conservatively over-segmented; that is, the single surface features may be split into multiple segments, but each individual segment, or superpixel, is ensured to have homogenous mineralogy.

Thompson, David R.↗
DOE OSTI2021

Point Adversarial Self-Mining: A Simple Method for Facial Expression Recognition

In this article, we propose a simple yet effective approach, called point adversarial self mining (PASM), to improve the recognition accuracy in facial expression recognition (FER). Unlike previous works focusing on designing specific architectures or loss functions to solve this problem, PASM boosts the network capability by simulating human learning processes: providing updated learning materials and guidance from more capable teachers. Specifically, to generate new learning materials, PASM leverages a point adversarial attack method and a trained teacher network to locate the most informative position related to the target task, generating harder learning samples to refine the network. The searched position is highly adaptive since it considers both the statistical information of each sample and the teacher network capability. Other than being provided new learning materials, the student network also receives guidance from the teacher network. After the student network finishes training, the student network changes its role and acts as a teacher, generating new learning materials and providing stronger guidance to train a better student network. Here, the adaptive learning materials generation and teacher/student update can be conducted more than one time, improving the network capability iteratively. Extensive experimental results validate the efficacy of our method over the existing state of the arts for FER.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A treatment-effect model to quantify human dimensions of disaster impacts: the case of Hurricane Maria in Puerto Rico

Here, we propose a supervised learning approach using publicly available panel data to statistically quantify the specific manifestations of human impacts of an extreme event, such as changes number of suicides, substance abuse, excess mortality, and unemployment. This allows us to conceptually focus our framework on human impacts and how by attributing them to disaster events along widely accepted psychological, economic, and social dimensions. Our modified treatment-effect model allows counterfactual baseline conditions to be posited for each manifestation from which an aggregated quantitative multi-faceted measure of human impacts can be determined. The developed statistical methodology could be beneficial to policymakers who must allocate scarce resources to those communities in greater need. We illustrate the applicability of our approach using annual and monthly panel data from 2012 to 2018 encompassing the 2017 Hurricane Maria event across various municipalities in Puerto Rico. Our statistical modeling methodology stands apart since (i) it explicitly and more realistically captures the effect of different human-oriented manifestations of an actual event and (ii) it is flexible enough to accommodate individual preferences of various stakeholders in how they assign importance to multiple manifestations of human impacts.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Crystal Orientation and Defect Mapping in Electron-Beam-Sensitive Zeolites with Near-Axis Transmission Kikuchi Diffraction

Porous materials are vital in catalysis, energy conversion, and environmental remediation. Understanding structural heterogeneity in zeolites is key to linking synthesis, framework intergrowths, and catalytic performance, yet current methods for phase identification and spatial mapping lack sufficient resolution or throughput. We present a high-throughput approach using near-axis transmission Kikuchi diffraction in a scanning electron microscope, achieving high phase and spatial resolution for electron-beam-sensitive zeolites, including ZSM-5 and, for the first time, Zeolite A. Here, this method enables direct visualization of intergrowth features that critically affect catalytic and adsorption behavior, bridging the gap between ensemble-averaged X-ray diffraction and high-resolution but low-throughput transmission electron microscopy. Combining nanoscale mapping with statistical sampling is highly suited for machine-learning pipelines guiding new structure function understanding and could be extended to other beam-sensitive porous materials such as metal–organic or covalent–organic frameworks.

crystallography↗
DOE OSTI2021

Wavelet Analysis of Properties of Marine Boundary Layer Mesoscale Cells Observed From AMSR–E

Marine boundary layer clouds tend to organize into closed or open mesoscale cellular convection (MCC). Here, two-dimensional wavelet analysis is applied for the first time to passive microwave retrievals of cloud water path (CWP), water vapor path (WVP), and rain rate (RR) from Advanced Microwave Scanning Radiometer for Earth Observing System in 2008 over the Northeast and Southeast Pacific, and the Southeast Atlantic subtropical stratocumulus to cumulus transition regions. The (co-)variability between CWP, WVP, and RR in 160 × 160 km 2 analysis boxes is partitioned between four mesoscale wavelength octaves (20, 40, 80, and 160 km). The cell scale is identified as the wavelength of the peak CWP variance. Together with a machine-learning classification of cell type, this allows the statistical characteristics of open and closed MCC of various scales, and its relation to WVP, RR, and potential environmental controlling factors to be analyzed across a very large set of cases. Here, the results show that the cell wavelength is most commonly 40–80 km. Cell-scale CWP perturbations are good predictors of the WVP and RR perturbations. For cells larger than 20 km, there is no obvious dependence of cell scale on the environmental controlling factors tested, suggesting that the cell scale may depend more on its historical evolution than the current environmental conditions.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

An exponential build-up in seismic energy suggests a months-long nucleation of slow slip in Cascadia

Slow slip events result from the spontaneous weakening of the subduction megathrust and bear strong resemblance to earthquakes, only slower. This resemblance allows us to study fundamental aspects of nucleation that remain elusive for classic, fast earthquakes. We rely on machine learning algorithms to infer slow slip timing from statistics of seismic waveforms. We find that patterns in seismic power follow the 14-month slow slip cycle in Cascadia, arguing in favor of the predictability of slow slip rupture. Here, we show that seismic power exponentially increases as the slowly slipping portion of the subduction zone approaches failure, a behavior that shares a striking similarity with the increase in acoustic power observed prior to laboratory slow slip events. Our results suggest that the nucleation phase of Cascadia slow slip events may last from several weeks up to several months.

58 GEOSCIENCES↗
DOE OSTI2021

Improved accuracy and transferability of molecular-orbital-based machine learning: Organics, transition-metal complexes, non-covalent interactions, and transition states

We report molecular-orbital-based machine learning (MOB-ML) provides a general framework for the prediction of accurate correlation energies at the cost of obtaining molecular orbitals. The application of Nesbet’s theorem makes it possible to recast a typical extrapolation task, training on correlation energies for small molecules and predicting correlation energies for large molecules, into an interpolation task based on the properties of orbital pairs. We demonstrate the importance of preserving physical constraints, including invariance conditions and size consistency, when generating the input for the machine learning model. Numerical improvements are demonstrated for different datasets covering total and relative energies for thermally accessible organic and transition-metal containing molecules, non-covalent interactions, and transition-state energies. MOB-ML requires training data from only 1% of the QM7b-T dataset (i.e., only 70 organic molecules with seven and fewer heavy atoms) to predict the total energy of the remaining 99% of this dataset with sub-kcal/mol accuracy. This MOB-ML model is significantly more accurate than other methods when transferred to a dataset comprising of 13 heavy atom molecules, exhibiting no loss of accuracy on a size intensive (i.e., per-electron) basis. It is shown that MOB-ML also works well for extrapolating to transition-state structures, predicting the barrier region for malonaldehyde intramolecular proton-transfer to within 0.35 kcal/mol when only trained on reactant/product-like structures. Finally, the use of the Gaussian process variance enables an active learning strategy for extending the MOB-ML model to new regions of chemical space with minimal effort. We demonstrate this active learning strategy by extending a QM7b-T model to describe non-covalent interactions in the protein backbone–backbone interaction dataset to an accuracy of 0.28 kcal/mol.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

2023 Roadmap on molecular modelling of electrochemical energy materials

New materials for electrochemical energy storage and conversion are the key to the electrification and sustainable development of our modern societies. Molecular modelling based on the principles of quantum mechanics and statistical mechanics as well as empowered by machine learning techniques can help us to understand, control and design electrochemical energy materials at atomistic precision. Therefore, this roadmap, which is a collection of authoritative opinions, serves as a gateway for both the experts and the beginners to have a quick overview of the current status and corresponding challenges in molecular modelling of electrochemical energy materials for batteries, supercapacitors, CO 2 reduction reaction, and fuel cell applications.

25 ENERGY STORAGE↗
DOE OSTI2020

Deeply uncertain: comparing methods of uncertainty quantification in deep learning algorithms

We present a comparison of methods for uncertainty quantification (UQ) in deep learning algorithms in the context of a simple physical system. Three of the most common uncertainty quantification methods - Bayesian Neural Networks (BNN), Concrete Dropout (CD), and Deep Ensembles (DE) - are compared to the standard analytic error propagation. We discuss this comparison in terms endemic to both machine learning ("epistemic" and "aleatoric") and the physical sciences ("statistical" and "systematic"). The comparisons are presented in terms of simulated experimental measurements of a single pendulum - a prototypical physical system for studying measurement and analysis techniques. Our results highlight some pitfalls that may occur when using these UQ methods. For example, when the variation of noise in the training set is small, all methods predicted the same relative uncertainty independently of the inputs. This issue is particularly hard to avoid in BNN. On the other hand, when the test set contains samples far from the training distribution, we found that no methods sufficiently increased the uncertainties associated to their predictions. This problem was particularly clear for CD. In light of these results, we make some recommendations for usage and interpretation of UQ methods.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Improved naive Bayesian probability classifier in predictions of nuclear mass

Recently, novel statistical methods such as neural networks and Bayesian learning methods are implemented to describe the nuclear masses. Based on previous studies, an improved naive Bayesian probability (iNBP) classifier is proposed to study the nuclear masses by refining the results of sophisticated nuclear models. In the iNBP method, the prediction for nuclear masses is treated as a classification problem. The residuals are classified into several groups to generate prior and conditional probabilities, and the posterior probabilities are further determined by the Bayesian formula. We choose the expectation with maximum probability as the final prediction. Reliability of the iNBP method is assessed by analyzing the global optimizations and the extrapolating capabilities. Here, the iNBP method exhibits impressive improvements on global descriptions for different mass models. Moreover, the method shows robust extrapolating capabilities. Results demonstrate the iNBP method can be applied to predict the nuclear masses of unknown regions. Considering the local mass relations, the iNBP method can offer considerable fine-tuning of the mass descriptions from nuclear models. The methodology proposed in this paper can also be applied to other model-based extrapolations of nuclear observables.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2022

Neural simulation-based inference approach for characterizing the Galactic Center γ-ray excess

The nature of the Fermi γ-ray Galactic Center Excess (GCE) has remained a persistent mystery for over a decade. Although the excess is broadly compatible with emission expected due to dark matter annihilation, an explanation in terms of a population of unresolved astrophysical point sources e.g., millisecond pulsars, remains viable. The effort to uncover the origin of the GCE is hampered in particular by an incomplete understanding of diffuse emission of Galactic origin. This can lead to spurious features that make it difficult to robustly differentiate smooth emission, as expected for a dark matter origin, from more “clumpy” emission expected from a population of relatively bright, unresolved point sources. Here, we use recent advancements in the field of simulation-based inference, in particular density estimation techniques using normalizing flows, in order to characterize the contribution of modeled components, including unresolved point source populations, to the GCE. Compared to traditional techniques based on the statistical distribution of photon counts, our machine learning-based method is able to utilize more of the information contained in a given model of the Galactic Center emission, and in particular can perform posterior parameter estimation while accounting for pixel-to-pixel spatial correlations in the γ-ray map. This makes the method demonstrably more resilient to certain forms of model misspecification. On application to Fermi data, the method generically attributes a smaller fraction of the GCE flux to unresolved point sources when compared to traditional approaches. We nevertheless infer such a contribution to make up a non-negligible fraction of the GCE across all analysis variations considered, with at least $38^{+9}_{-19}$% of the excess attributed to unresolved point sources in our baseline analysis.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Neural units with time-dependent functionality

We show that the time-resolved dynamics of an underdamped harmonic oscillator can be used to do multifunctional computation, performing distinct computations at distinct times within a single dynamical trajectory. We consider the amplitude of an oscillator whose inputs influence its frequency. The activity of the oscillator at fixed times is a nonmonotonic function of its inputs, so it can solve problems such as XOR that are not linearly separable. The activity of the oscillator at fixed input is a nonmonotonic function of time, so it is multifunctional in a temporal sense, and able to carry out distinct nonlinear computations at distinct times within the same dynamical trajectory. We show that a single oscillator, observed at different times, can act as all of the elementary logic gates and perform binary addition, the latter usually implemented in hardware using five logic gates. We show that a set of n oscillators, observed at different times, can perform an arbitrary number of analog-to-n-bit digital conversions. We also show that oscillators can be trained by gradient descent to perform distinct classification tasks at distinct times. Computing with time-dependent functionality can be done in or out of equilibrium, and suggests a way of reducing the number of parameters or devices required to do nonlinear computations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A hybrid data–model approach to map soil thickness in mountain hillslopes

Abstract. Soil thickness plays a central role in the interactions between vegetation, soils, and topography, where it controls the retention and release of water, carbon, nitrogen, and metals. However, mapping soil thickness, here defined as the mobile regolith layer, at high spatial resolution remains challenging. Here, we develop a hybrid model that combines a process-based model and empirical relationships to estimate the spatial heterogeneity of soil thickness with fine spatial resolution (0.5 m). We apply this model to two aspects of hillslopes (southwest- and northeast-facing, respectively) in the East River watershed in Colorado. Two independent measurement methods – auger and cone penetrometer – are used to sample soil thickness at 78 locations to calibrate the local value of unconstrained parameters within the hybrid model. Sensitivity analysis using the hybrid model reveals that the diffusion coefficient used in hillslope diffusion modeling has the largest sensitivity among all input parameters. In addition, our results from both sampling and modeling show that, in general, the northeast-facing hillslope has a deeper soil layer than the southwest-facing hillslope. By comparing the soil thickness estimated between a machine-learning approach and this hybrid model, the hybrid model provides higher accuracy and requires less sampling data. Modeling results further reveal that the southwest-facing hillslope has a slightly faster surface soil erosion rate and soil production rate than the northeast-facing hillslope, which suggests that the relatively less dense vegetation cover and drier surface soils on the southwest-facing slopes influence soil properties. With seven parameters in total for calibration, this hybrid model can provide a realistic soil thickness map with a relatively small amount of sampling dataset comparing to machine-learning approach. Integrating process-based modeling and statistical analysis not only provides a thorough understanding of the fundamental mechanisms for soil thickness prediction but also integrates the strengths of both statistical approaches and process-based modeling approaches.

58 GEOSCIENCES↗
NASA NTRS2016

Analyzing and Predicting Effort Associated with Finding and Fixing Software Faults

Context: Software developers spend a significant amount of time fixing faults. However, not many papers have addressed the actual effort needed to fix software faults. Objective: The objective of this paper is twofold: (1) analysis of the effort needed to fix software faults and how it was affected by several factors and (2) prediction of the level of fix implementation effort based on the information provided in software change requests. Method: The work is based on data related to 1200 failures, extracted from the change tracking system of a large NASA mission. The analysis includes descriptive and inferential statistics. Predictions are made using three supervised machine learning algorithms and three sampling techniques aimed at addressing the imbalanced data problem. Results: Our results show that (1) 83% of the total fix implementation effort was associated with only 20% of failures. (2) Both safety critical failures and post-release failures required three times more effort to fix compared to non-critical and pre-release counterparts, respectively. (3) Failures with fixes spread across multiple components or across multiple types of software artifacts required more effort. The spread across artifacts was more costly than spread across components. (4) Surprisingly, some types of faults associated with later life-cycle activities did not require significant effort. (5) The level of fix implementation effort was predicted with 73% overall accuracy using the original, imbalanced data. Using oversampling techniques improved the overall accuracy up to 77%. More importantly, oversampling significantly improved the prediction of the high level effort, from 31% to around 85%. Conclusions: This paper shows the importance of tying software failures to changes made to fix all associated faults, in one or more software components and/or in one or more software artifacts, and the benefit of studying how the spread of faults and other factors affect the fix implementation effort.

software fix implementation effort↗
DOE OSTI2026

Enabling probabilistic learning on manifolds through double diffusion maps

Here, we present a generative learning framework for probabilistic sampling that extends Probabilistic Learning on Manifolds (PLoM), which is designed to generate statistically consistent realizations of a random vector in a finite-dimensional Euclidean space, informed by a (representative) set of observations. In its original form, PLoM constructs a reduced-order probabilistic model by combining three main components: (a) kernel density estimation to approximate the underlying probability measure, (b) Diffusion Maps to characterize the manifold of the data, and (c) a reduced-order Itô Stochastic Differential Equation (ISDE) to sample from the learned distribution. However, its sampling dynamics are posed in the ambient space and the retained number of reduced coordinates is chosen by projection-reconstruction error. In practice, this often (i) requires more coordinates than the data’s intrinsic dimension to achieve stable sampling and (ii) lacks a smooth, basis-independent lifting back to the data domain; moreover, standard Diffusion Maps emphasize harmonic eigenfunctions and can miss non-harmonic latent structure. We address these limitations by decoupling geometry learning from sampling: a first Diffusion Maps pass identifies non-harmonic coordinates on which we formulate a full-order ISDE directly in the latent space, while Double Diffusion Maps captures multiscale geometric features and Geometric Harmonics (GH) learns a smooth lifting map to the ambient variables that is independent of the particular diffusion basis. This hybrid design preserves the system’s dynamical richness with a compact geometric representation and enables principled out-of-sample inference. The effectiveness and robustness of the proposed method are illustrated through two numerical studies: one based on data generated from two-dimensional Hermite polynomial functions and another based on high-fidelity simulations of a detonation wave in a reactive flow.

Double diffusion maps↗
DOE OSTI2021

Stopping criteria for ending autonomous, single detector radiological source searches

While the localization of radiological sources has traditionally been handled with statistical algorithms, such a task can be augmented with advanced machine learning methodologies. The combination of deep and reinforcement learning has provided learning-based navigation to autonomous, single-detector, mobile systems. However, these approaches lacked the capacity to terminate a surveying/search task without outside influence of an operator or perfect knowledge of source location (defeating the purpose of such a system). Two stopping criteria are investigated in this work for a machine learning navigated system: one based upon Bayesian and maximum likelihood estimation (MLE) strategies commonly used in source localization, and a second providing the navigational machine learning network with a “stop search” action. A convolutional neural network was trained via reinforcement learning in a 10 m × 10 m simulated environment to navigate a randomly placed detector-agent to a randomly placed source of varied strength (stopping with perfect knowledge during training). The network agent could move in one of four directions (up, down, left, right) after taking a 1 s count measurement at the current location. During testing, the stopping criteria for this navigational algorithm was based upon a Bayesian likelihood estimation technique of source presence, updating this likelihood after each step, and terminating once the confidence of the source being in a single location exceeded 0.9. A second network was trained and tested with similar architecture as the previous but which contained a fifth action: for self-stopping. The accuracy and speed of localization with set detector and source initializations were compared over 50 trials of MLE-Bayesian approach and 1000 trials of the CNN with self-stopping. The statistical stopping condition yielded a median localization error of ~1.41 m and median localization speed of 12 steps. The machine learning stopping condition yielded a median localization error of 0 m and median localization speed of 17 steps. This work demonstrated two stopping criteria available to a machine learning guided, source localization system.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗