Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “statistical learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,117 records · Page 62
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

A hybrid data-driven and model-based approach for computationally efficient stochastic unit commitment and economic dispatch under wind and solar uncertainty

Stochastic unit commitment (UC) and economic dispatch (ED) are imperative in dealing with uncertainty in renewable forecast for power system operation and planning such that the overall expected production cost is minimized over the planning horizon. However, accurate calculation of the expected production cost requires assessment of a very large number of different scenarios of uncertain renewable resources, such as solar and wind, which is practically infeasible to simulate in real time. This article proposes a hybrid datadriven and physics-based model-predictive paradigm to efficiently solve for stochastic unit commitment and economic dispatch considering uncertainty in wind and solar power forecasts. Here, the novelty of the approach lies in decoupling the production cost estimation from the unit commitment and economic dispatch optimization problems under uncertainty without compromising on the fidelity of the solutions. A data-driven machine learning model is first developed to predict the mean optimal production cost. A physics-based inverse problem is then solved to get the stochastic UC and ED profiles from the expected cost. The presented approach considers, for the first time, solar uncertainty in UC/ED determination and enables efficient and accurate propagation of wind and solar uncertainty to estimate the statistics of the production cost. The effectiveness of the developed approach is demonstrated systematically on a stylized RTS-GMLC single-node system. The overall framework predicts the expected cost 62.5% more accurately than the existing state-of-the-art, on unforeseen days during the entire year, and yields, for the first time, the associated physically consistent UC and ED profiles. The solutions are also shown to be flexible in providing adequate daily reserves to address any statistical deviations from probabilistic power forecasts. The computational time associated with the presented method is only about 10 s compared to over 24 h needed for a conventional stochastic UC/ED determination under uncertainty on an Intel Core i9 processor with 32 GB of RAM.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2023

Kernel-based global sensitivity analysis obtained from a single data set

Results from global sensitivity analysis (GSA) often guide the understanding of complicated input–output systems. Kernel-based GSA methods have recently been proposed for their capability of treating a broad scope of complex systems. In this paper, we develop a new set of kernel GSA tools when only a single set of input–output data is available. Three key advances are made: (1) A new numerical estimator is proposed that demonstrates an empirical improvement over previous procedures. (2) A computational method for generating inner statistical functions from a single data set is presented. (3) A theoretical extension is made to define conditional sensitivity indices, which reveal the degree that the inputs carry shared information about the output when inherent input–input correlations are present. Utilizing these conditional sensitivity indices, a decomposition is derived for the output uncertainty based on what is called the optimal learning sequence of the input variables, which remains consistent when correlations exist between the input variables. Further, while these advances cover a range of GSA subjects, a common single data set numerical solution is provided by a technique known as the conditional mean embedding of distributions. The new methodology is implemented on benchmark systems to demonstrate the provided insights.

42 ENGINEERING↗
DOE OSTI2023

Infinite neural network quantum states: entanglement and training dynamics

We study infinite limits of neural network quantum states (∞-NNQS), which exhibit representation power through ensemble statistics, and also tractable gradient descent dynamics. Ensemble averages of entanglement entropies are expressed in terms of neural network correlators, and architectures that exhibit volume-law entanglement are presented. The analytic calculations of entanglement entropy bound are tractable because the ensemble statistics are simplified in the Gaussian process limit. A general framework is developed for studying the gradient descent dynamics of neural network quantum states (NNQS), using a quantum state neural tangent kernel (QS-NTK). For ∞-NNQS the training dynamics is simplified, since the QS-NTK becomes deterministic and constant. An analytic solution is derived for quantum state supervised learning, which allows an ∞-NNQS to recover any target wavefunction. Numerical experiments on finite and infinite NNQS in the transverse field Ising model and Fermi Hubbard model demonstrate excellent agreement with theory. ∞-NNQS opens up new opportunities for studying entanglement and training dynamics in other physics applications, such as in finding ground states.

quantum state supervised learning↗
DOE OSTI2025

Generalizable, fast, and accurate DeepQSPR with fastprop

Abstract Quantitative Structure–Property Relationship studies (QSPR), often referred to interchangeably as QSAR, seek to establish a mapping between molecular structure and an arbitrary target property. Historically this was done on a target-by-target basis with new descriptors being devised to specifically map to a given target. Today software packages exist that calculate thousands of these descriptors, enabling general modeling typically with classical and machine learning methods. Also present today are learned representation methods in which deep learning models generate a target-specific representation during training. The former requires less training data and offers improved speed and interpretability while the latter offers excellent generality, while the intersection of the two remains under-explored. This paper introduces , a software package and general Deep-QSPR framework that combines a cogent set of molecular descriptors with deep learning to achieve state-of-the-art performance on datasets ranging from tens to tens of thousands of molecules. provides both a user-friendly Command Line Interface and highly interoperable set of Python modules for the training and deployment of feedforward neural networks for property prediction. This approach yields improvements in speed and interpretability over existing methods while statistically equaling or exceeding their performance across most of the tested benchmarks. is designed with Research Software Engineering best practices and is free and open source, hosted at github.com/jacksonburns/fastprop.

Burns, Jackson W. (ORCID:0000000206579426)↗
NASA NTRS2015

Advanced Bayesian Method for Planetary Surface Navigation

Autonomous Exploration, Inc., has developed an advanced Bayesian statistical inference method that leverages current computing technology to produce a highly accurate surface navigation system. The method combines dense stereo vision and high-speed optical flow to implement visual odometry (VO) to track faster rover movements. The Bayesian VO technique improves performance by using all image information rather than corner features only. The method determines what can be learned from each image pixel and weighs the information accordingly. This capability improves performance in shadowed areas that yield only low-contrast images. The error characteristics of the visual processing are complementary to those of a low-cost inertial measurement unit (IMU), so the combination of the two capabilities provides highly accurate navigation. The method increases NASA mission productivity by enabling faster rover speed and accuracy. On Earth, the technology will permit operation of robots and autonomous vehicles in areas where the Global Positioning System (GPS) is degraded or unavailable.

Center, Julian↗
NASA NTRSDate not supplied

Application of Support Vector Regression to Derive Crater Depth/Diameter From Satellite Images

Through the study of impact crater shapes, one can draw important conclusions about the nature and evolution of planetary surfaces [e.g., 1-4].In particular, studying the depth (d) to diameter (D)ratio (d/D) of a population of impact craters, in combination with crater count statistics, can yield valuable insights regarding rates of erosion and burial[5]. Motivated by the great abundance of available planetary surface image data, the goal of this project is to develop an efficient way to estimate d/D from satellite images of impact craters for which stereo information is not available [6]. We set out to develop and train a machine learning algorithm to extract d/D from a dataset of synthetic impact crater images for which model d/D is known. The applications of machine learning to planetary science are numerous and diverse [7], including automatic planetary surface mapping [8] and the detection of impact craters [9]. Our algorithm makes use of Support Vector Regression (SVR), which is a type of Support Vector Machine (SVM) [10, 11].SVMs are a branch of supervised machine learning valued for their straightforward implementation and versatility in solving both classification and regression problems. In regression analysis, an SVR algorithm produces a hyperplane function to fit the training data points, as well as an ε-tube that surrounds the hyperplane. Tunable hyperparameters include the width of the ε-tube (ε) and the amount an algorithm is penalized for points which fall outside the ε-tube.

L R Chin↗
DOE OSTI2023

Machine-learning-assisted analysis of transition metal dichalcogenide thin-film growth

In situ reflective high-energy electron diffraction (RHEED) is widely used to monitor the surface crystalline state during thin-film growth by molecular beam epitaxy (MBE) and pulsed laser deposition. With the recent development of machine learning (ML), ML-assisted analysis of RHEED videos aids in interpreting the complete RHEED data of oxide thin films. The quantitative analysis of RHEED data allows us to characterize and categorize the growth modes step by step, and extract hidden knowledge of the epitaxial film growth process. In this study, we employed the ML-assisted RHEED analysis method to investigate the growth of 2D thin films of transition metal dichalcogenides (ReSe2) on graphene substrates by MBE. Principal component analysis (PCA) and K-means clustering were used to separate statistically important patterns and visualize the trend of pattern evolution without any notable loss of information. Using the modified PCA, we could monitor the diffraction intensity of solely the ReSe2 layers by filtering out the substrate contribution. These findings demonstrate that ML analysis can be successfully employed to examine and understand the film-growth dynamics of 2D materials. Further, the ML-based method can pave the way for the development of advanced real-time monitoring and autonomous material synthesis techniques.

36 MATERIALS SCIENCE↗
DOE OSTI2023

En echelon faults reactivated by wastewater disposal near Musreau Lake, Alberta

We use machine-learning and cross-correlation techniques to enhance earthquake detectability by two magnitude units for the earthquake sequence near Musreau Lake, Alberta, which is induced by wastewater disposal. This deep catalogue reveals a series of en echelon ~N–S oriented strike-slip faults that are favourably oriented for reactivation. These faults require only ~0.6 MPa overpressure for triggering to occur. Earthquake activity occurs in bursts, or episodes; episodes restricted to the largest fault tend to have earthquakes starting near the southern end (distant from injectors) and progressing northwards (towards the injectors). While most events are concentrated along these ~N–S oriented faults, we also delineate smaller faults. Together, these findings suggest pore pressure as the triggering mechanism, where a time-dependent increase in pore pressure likely caused these faults to progressively reawaken. Analysis of the ‘next record-breaking event’, a statistical model that forecasts the sequencing of earthquake magnitudes, suggests that the next largest event would be M L ~4.3. The seismically illuminated length of the largest fault indicates potential magnitudes as large as M w 5.3.

58 GEOSCIENCES↗
DOE OSTI2020

Genome-Wide Analysis of RNA Decay in the Cyanobacterium Synechococcus sp. Strain PCC 7002

ABSTRACT RNA degradation is an important process that influences the ultimate concentration of individual proteins inside cells. While the main enzymes that facilitate this process have been identified, global maps of RNA turnover are available for only a few species. Even in these cases, there are few sequence elements that are known to enhance or destabilize a native transcript; even fewer confer the same effect when added to a heterologous transcript. To address this knowledge gap, we assayed genome-wide RNA degradation in the cyanobacterium Synechococcus sp. strain PCC 7002 by collecting total RNA samples after stopping nascent transcription with rifampin. We quantified the abundance of each position in the transcriptome as a function of time using RNA-sequencing data and later analyzed the global mRNA decay map using machine learning principles. Half-lives, calculated on a per-ORF (open reading frame) basis, were extremely short, with a median half-life of only 0.97 min. Despite extremely rapid turnover of most mRNA, transcripts encoding proteins involved in photosynthesis were both highly expressed and highly stable. Upon inspection of these stable transcripts, we identified an enriched motif in the 3′ untranslated region (UTR) that had similarity to Rho-independent terminators. We built statistical models for half-life prediction and used them to systematically identify sequence motifs in both 5′ and 3′ UTRs that correlate with stabilized transcripts. We found that transcripts linked to a terminator containing a poly(U) tract had a longer half-life than both those without a poly(U) tract and those without a terminator. IMPORTANCE RNA degradation is an important process that affects the final concentration of individual mRNAs, affecting protein expression and cellular physiology. Studies of how RNA is degraded increase our knowledge of this fundamental process as well as enable the creation of genetic tools to manipulate RNA stability. By studying global transcript turnover, we searched for sequence elements that correlated with transcript (in)stability and used these sequences to guide tool design. This study probes global RNA turnover in a cyanobacterium, Synechococcus sp. strain PCC 7002, that both has a unique array of RNases that facilitate RNA degradation and is an industrially relevant strain that could be used to convert CO 2 and sunlight into useful products.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

A Multiple Filter Based Neural Network Approach to the Extrapolation of Adsorption Energies on Metal Surfaces for Catalysis Applications

Computational catalyst discovery involves the development of microkinetic reactor models based on estimated parameters determined from density functional theory (DFT). For complex surface chemistries, the number of reaction intermediates can be very large, and the cost of calculating the adsorption energies by DFT for all surface intermediates even for one active site model can become prohibitive. In this paper, we have identified appropriate descriptors and machine learning models that can be used to predict a significant part of these adsorption energies given data on the rest of them. Furthermore, our investigations also included the case when the species data used to train the predictive model are of different size relative to the species the model tries to predict—this is an extrapolation in the data space which is typically difficult with regular machine learning models. Due to the relative size of the available data sets, we have attempted to extrapolate from the larger species to the smaller ones in the current work. Here, we have developed a neural network based predictive model that combines an established additive atomic contribution based model with the concepts of a convolutional neural network that, when extrapolating, achieves a statistically significant improvement over the previous models.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Efficient Super-Resolution of Near-Surface Climate Modeling Using the Fourier Neural Operator

Downscaling methods are critical in efficiently generating high-resolution atmospheric data. However, state-of-the-art statistical or dynamical downscaling techniques either suffer from the high computational cost of running a physical model or require high-resolution data to develop a downscaling tool. Here, we demonstrate a recently proposed zero-shot super-resolution method, the Fourier neural operator (FNO), to efficiently perform downscaling without the need for high-resolution data. Because the FNO learns dynamics in Fourier space, FNO is a resolution-invariant emulator; it can be trained at a coarse resolution and produces emulation at any high resolution. We applied FNO to downscale a 4-km resolution Weather Research and Forecasting (WRF) Model simulation of near-surface heat-related variables over the Great Lakes region. The FNO is driven by the atmospheric forcings, and topographic features used in the WRF model at the same resolution. We incorporated a physics-constrained loss in FNO by using the Clausius-Clapeyron relation to better constrain the relations among the emulated states. Trained on merely 600 WRF snapshots at 4-km resolution, the FNO shows comparable performance with a widely used convolutional network, U-Net, achieving averaged modified Kling-Gupta Efficiency of 0.88 and 0.94 on the test dataset for temperature and pressure, respectively. We then employed the FNO to produce 1-km emulations to reproduce the fine climate features. Further, by taking the WRF simulation as ground truth, we show consistent performances at the two resolutions, suggesting the reliability of FNO in producing high-resolution dynamics. Our study demonstrates the potential of using FNO for zero-shot super-resolution in generating first-order estimation on atmospheric modeling.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Machine Learning for Daily Forecasts of Arctic Sea Ice Motion: An Attribution Assessment of Model Predictive Skill

Physics-based simulations of Arctic sea ice are highly complex, involving transport between different phases, length scales, and time scales. Resultantly, numerical simulations of sea ice dynamics have a high computational cost and model uncertainty. We employ data-driven machine learning (ML) to make predictions of sea ice motion. The ML models are built to predict present-day sea ice velocity given present-day wind velocity and previous-day sea ice concentration and velocity. Models are trained using reanalysis winds and satellite-derived sea ice properties. We compare the predictions of three different models: persistence (PS), linear regression (LR), and a convolutional neural network (CNN). We quantify the spatiotemporal variability of the correlation between observations and the statistical model predictions. Additionally, we analyze model performance in comparison to variability in properties related to ice motion (wind velocity, ice velocity, ice concentration, distance from coast, bathymetric depth) to understand the processes related to decreases in model performance. Results indicate that a CNN makes skillful predictions of daily sea ice velocity with a correlation up to 0.81 between predicted and observed sea ice velocity, while the LR and PS implementations exhibit correlations of 0.78 and 0.69, respectively. The correlation varies spatially and seasonally: lower values occur in shallow coastal regions and during times of minimum sea ice extent. LR parameter analysis indicates that wind velocity plays the largest role in predicting sea ice velocity on 1-day time scales, particularly in the central Arctic. Regions where wind velocity has the largest LR parameter are regions where the CNN has higher predictive skill than the LR.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Deep learning of parameterized equations with applications to uncertainty quantification

We propose a learning algorithm for discovering unknown parameterized dynamical systems by using observational data of the state variables. Our method is built upon and extends the recent work of discovering unknown dynamical systems, in particular those using deep neural network (DNN). We propose a DNN structure, largely based upon the residual network (ResNet), to not only learn the unknown form of the governing equation but also take into account the random effect embedded in the system, which is generated by the random parameters. Once the DNN model is successfully constructed, it is able to produce system prediction over longer term and for arbitrary parameter values. For uncertainty quantification, it allows us to conduct uncertainty analysis by evaluating solution statistics over the parameter space.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

The Evaluation of Machine Learning Techniques for Isotope Identification Contextualized by Training and Testing Spectral Similarity

Precise gamma-ray spectral analysis is crucial in high-stakes applications, such as nuclear security. Research efforts toward implementing machine learning (ML) approaches for accurate analysis are limited by the resemblance of the training data to the testing scenarios. The underlying spectral shape of synthetic data may not perfectly reflect measured configurations, and measurement campaigns may be limited by resource constraints. Consequently, ML algorithms for isotope identification must maintain accurate classification performance under domain shifts between the training and testing data. To this end, four different classifiers (Ridge, Random Forest, Extreme Gradient Boosting, and Multilayer Perceptron) were trained on the same dataset and evaluated on twelve other datasets with varying standoff distances, shielding, and background configurations. A tailored statistical approach was introduced to quantify the similarity between the training and testing configurations, which was then related to the predictive performance. Wilcoxon signed-rank tests revealed that the OVR-wrapped XGB significantly outperformed the other algorithms, with confidence levels of 99.0% or above for the 133Ba, 60Co, 137Cs, and 152Eu sources. The findings from this work are significant as they outline techniques to promote the development of robust ML-based approaches for isotope identification.

domain adaptation↗
DOE OSTI2026

Generation of random geological models using multi-randomization for machine learning

Generating high-fidelity geological models is essential for advancing machine learning (ML) methods in automated seismic interpretation. For instance, seismic images paired with corresponding fault labels are foundational for ML-based fault detection from seismic migration sections. While several open-access datasets of random geological models exist, open-source tools specifically designed to produce large volumes of such models for ML applications remain scarce. To address this gap, we present RGM (Random Geological Model), an open-source software package for efficiently generating 2D and 3D synthetic geological models tailored for ML workflows. RGM supports the creation of diverse model components, including medium property distributions (P-/S-wave velocities and density), seismic reflectivity images (i.e., synthetic migration sections), relative geological time, and discrete fault attributes such as probability, dip, strike, rake, and displacement. It also accommodates the creation of complex geological features such as salt bodies and unconformities. The model generation algorithm employs a multi-randomization strategy, yielding an effectively infinite-dimensional model space that encompasses a wide range of geological scenarios and associated seismic features. Furthermore, RGM incorporates a method to generate synthetic elastic migration images using analytical elastic reflection coefficients combined with frequency-dependent scaling. This functionality enables the creation of training datasets for ML models that leverage elastic seismic images. RGM is implemented in modern object-oriented Fortran, allowing users to flexibly control statistical parameters governing model variability. We demonstrate the capability, performance, and geological realism of the package through comprehensive 2D and 3D examples.

58 GEOSCIENCES↗
DOE OSTI2022

Monitoring the propagation of mechanical discontinuity using data-driven causal discovery and supervised learning

Mechanical wave transmission through a material is influenced by the mechanical discontinuity in the material. The propagation of embedded discontinuities can be monitored by analyzing the wave-transmission measurements recorded by a multipoint sensor system placed on the surface of the material. The proposed workflow monitors the propagation of mechanical discontinuity through three stages, namely initial, intermediate, and final stages, by using supervised learning followed by data-driven causal discovery. To the end, the workflow processes the multipoint waveform measurements resulting from a single impulse source, while considering the effects of wave attenuation, dispersion and multiple wave-propagation modes due to the discontinuity and material boundaries. Among various feature reduction techniques ranging from decomposition methods to manifold approximation methods, the features derived based on statistical parameterizations of the measured waveforms lead to reliable monitoring that is robust to changes in precision, resolution, and signal-to-noise ratio of the multipoint sensor measurements. The numbers of zero-crossing, negative-turning, and positive turning in the waveforms are the strongest causal signatures of the propagation of mechanical discontinuity. Higher order moments of the waveforms, such as variance, skewness and kurtosis, are also strong causal signatures of the propagation. Finally, the newly discovered causal signatures confirm that the statistical correlations and conventional feature rankings are not always statistically significant indicators of causality.

42 ENGINEERING↗
DOE OSTI2021

Sparse expansions of multicomponent oxide configuration energy using coherency and redundancy

We report that compressed sensing has become a widely accepted paradigm to construct high dimensional cluster expansion models used for statistical mechanical studies of atomic configuration in complex multicomponent crystalline materials. However, strict sampling requirements necessary to obtain minimal coherence measurements for compressed sensing to guarantee accurate estimation of model parameters are difficult and in some cases impossible to satisfy due to the inability of physical systems to access certain configurations. Nevertheless, the dependence of energy on atomic configuration can still be adequately learned without these strict requirements by using compressed sensing by way of coherent measurements using redundant function sets known as frames. We develop a particular frame constructed from the union of all occupancy-based cluster expansion basis sets. We illustrate how using this highly redundant frame yields sparse expansions of the configuration energy of complex oxide materials that are competitive and often surpass the prediction accuracy and sparsity of models obtained from standard cluster expansions.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗