Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “training data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Comparative Analysis of Machine Learning Models for Day-Ahead Photovoltaic Power Production Forecasting

A main challenge for integrating the intermittent photovoltaic (PV) power generation remains the accuracy of day-ahead forecasts and the establishment of robust performing methods. The purpose of this work is to address these technological challenges by evaluating the day-ahead PV production forecasting performance of different machine learning models under different supervised learning regimes and minimal input features. Specifically, the day-ahead forecasting capability of Bayesian neural network (BNN), support vector regression (SVR), and regression tree (RT) models was investigated by employing the same dataset for training and performance verification, thus enabling a valid comparison. The training regime analysis demonstrated that the performance of the investigated models was strongly dependent on the timeframe of the train set, training data sequence, and application of irradiance condition filters. Furthermore, accurate results were obtained utilizing only the measured power output and other calculated parameters for training. Consequently, useful information is provided for establishing a robust day-ahead forecasting methodology that utilizes calculated input parameters and an optimal supervised learning approach. Finally, the obtained results demonstrated that the optimally constructed BNN outperformed all other machine learning models achieving forecasting accuracies lower than 5%.

14 SOLAR ENERGY↗
DOE OSTI2021

Automated Parameterized CFD Simulations of Phase-Change Material Embedded Heat Exchangers

Thermal storage can be implemented using Phase-Change Materials (PCM), which absorb significant latent heat with a relatively small temperature change. PCM phase-change processes are transient and are driven by thermal diffusion and natural convection – the latter, especially for melting process. Modeling and simulation of PCM heat exchangers is typically computationally intensive due to the relative complex time-dependent underlying physics. This paper presents a study investigating a single straight tube with circular transverse fins in the vertical orientation, using PCM’s with 37oC nominal melting temperature, leveraging automated parametric CFD simulations. The computational domain is reduced to 2D axisymmetric for a segment of the HX comprising a region in between two fins with constant wall temperature as boundary condition. This approach allows for faster evaluations of the transient thermal characteristics for a given design space of interest. Correlations and reduced order models (ROM’s) are developed. Results indicate that the ROM’s reproduce the CFD simulations with high accuracy for designs from the training data set, however, designs from cross-validation and test sets do not always have good agreement. The limited number of designs in the training set is a possible cause for the lack of robustness of the ROM’s and thus additional training designs will be added in future work. The ROM’s solve the simulations on average 500,000 times faster than CFD.

25 ENERGY STORAGE↗
NASA NTRSDate not supplied

Algorithmic Detection of Elemental Biosignatures

Machine learning models that classify a sample as indicative or non-indicative of life could play an important role in life-detection missions. Their predictions result from agnostic algorithms and thereby add redundancy to judgements resulting from human expertise. Additionally, their important features can reveal the most informative measurements within the operational constraints of a life-detection mission. The Ladder of Life Detection (Neveu 2018) identifies the need for an understanding of how combinations of multiple biosignatures affect overall confidence. The present work provides a starting point to answer this need, and future work will expand the data types to obtain even more predictive combinations of features. Elemental abundance was chosen as a starting set of features due to its availability in diverse sample types, which are needed to train a generalizable model. A standardized dataset was collected, including 35 non-indicative, e.g., lunar rock, basalt; 19 indicative mixed, e.g., seawater, agricultural soil; 46 indicative non-alive, e.g., coal, chalk; and 10 indicative alive, e.g., biofilm, bacteria. This dataset could be valuable for complementary biosignature research. The samples were standardized to the same limit of detection of a simulated mission scenario. Four classification models were used: k-nearest neighbors (KNN), logistic regression (LR), linear support vector machines (SVM), and Gaussian naïve Bayes (GNB). To obtain feature importances, KNN was run on three principal components of the training data and LR and SVM were run with L1 and L2 regularization. The performances and feature importances of the six model variants on 40:60 train to validation ratios were assessed with Monte Carlo simulations. ROC AUC and mean accuracy scores ranged between 82% - 94%, with sensitivity greater than specificity. For indicative of life predictors, all models had C and Ca as strong and Cl as medium; a majority of models had N, K, and P as medium. For non-indicative of life predictors, all models had Si as strong, and a majority of models had Mg, Al, and Ti as medium. Varied elements were Fe (slightly non-indicative), H (slightly indicative), O (widely varied), Na, Mn, and S. These results serve as a proof of concept and suggest important elemental signals beyond merely the CHNOPS of Earth-based life.

Algorithmic↗
NASA NTRS2007

Multi-Sensor Data from A-Train Instruments Brought Together for Atmospheric Research

The A-Train is comprised of a series of instruments, developed independently, that measure highly related atmospheric components along the same flight path. In order to intercompare data from this multitude of sensors, researchers must access, subset, visualize, analyze and correlate distributed atmosphere measurements from the various A-Train instruments. The A-Train Data Depot (ATDD) has been operational for over a year, successfully performing the aforementioned functions on behalf of researchers, thus providing co-registered data from the Cloudsat, CALIOP, AIRS, and MODIS instruments for further intercomparisons. Of late, significant data from OM1 and POLDER are now included in the 'depot'. By specifying the desired spatial and temporal range, the researcher can subset, visualize, co-register, and access multi-sensor A-Train data related to: Cloud, aerosol, atmospheric temperature, and water vapor parameters (vertical profile visualizations); Cloud Pressure, cloud top temperature, water vapor, cloud optical thickness, and aerosol products (horizontal strips subsetted +/- 100km from the profile visualizations), and; Cloud pressure parameters (2-D line plots overlayed on the vertical profiles). All data is plotted using the GIOVANNI data exploration tool. A new feature of GIOVANNI is its ability to have collocated and subsetted data sets as well as PNG image files downloaded to the researcher's computing facility. By providing a convenient way to visualize and acquire multi-sensor data, ATDD affords users more time and effort to further their research.

Smith, Peter M.↗
DOE OSTI2021

Component-wise reduced order model lattice-type structure design

Lattice-type structures can provide a combination of stiffness with light weight that is desirable in a variety of applications. Design optimization of these structures must rely on approximations of the governing physics to render solution of a mathematical model feasible. In this paper, we propose a topology optimization (TO) formulation that approximates the governing physics using component-wise reduced order modeling as introduced in Huynh et al. (2013); Eftang and Patera (2013), which can reduce solution time by multiple orders of magnitude over a full-order finite element model while providing a relative error in the solution of 1%. In addition, the offline training data set from such component-wise models is reusable, allowing its application to many design problems for only the cost of a single offline training phase, and the component-wise method is nearly embarrassingly parallel. We also show how the parameterization chosen in our optimization allows a simplification of the component-wise reduced order model (CWROM) not noted in previous literature, for further speedup of the optimization process. Furthermore, the sensitivity of the compliance with respect to the particular parameterization is derived solely at the component level. In numerical examples, we demonstrate a 1000x speedup over a full-order FEM model with relative error of 1% and show minimum compliance designs for two different cantilever beam examples, one smaller and one larger. Finally, error bounds for the displacement field, compliance, and compliance sensitivity of the CWROM are derived.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Deep-learning-based workflow for boundary and small target segmentation in digital rock images using UNet++ and IK-EBM

We report three-dimensional (3D) X-ray micro-computed tomography (μCT) has been widely used in petroleum engineering because it can provide detailed pore structural information for a reservoir rock, which can be imported into a pore-scale numerical model to simulate the transport and distribution of multiple fluids in the pore space. The partial volume blurring (PVB) problem is a major challenge in segmenting raw μCT images of rock samples, which impacts boundaries and small targets near the resolution limit. We developed a deep-learning (DL)-based workflow for accurate and fast partial volume segmentation. The DL model's performance depends primarily on the training data quality and model architecture. This study employed the entropy-based-masking indicator kriging (IK-EBM) to segment 3D Berea sandstone images as training datasets. The comparison between IK-EBM and manual segmentation using a 3D synthetic sphere pack, which had a known ground truth, showed that IK-EBM had higher accuracy on partial volume segmentation. We then trained and tested the UNet++ model, a state-of-the-art supervised encoder-decoder model, for binary (i.e., void and solid) and four-class segmentation. We compared the UNet++ with the commonly used U-Net and wide U-Net models and showed that the UNet++ had the best performance in terms of pixel-wise and physics-based evaluation metrics. Specifically, boundary-scaled accuracy demonstrated that the UNet++ architecture outperformed the regular U-Net architecture in the segmentation of pixels near boundaries and small targets, which were subjected to the PVB effect. Feature map visualization illustrated that the UNet++ bridged the semantic gaps between the feature maps extracted at different depths of the network, thereby enabling faster convergence and more accurate extraction of fine-scale features. The developed workflow significantly enhances the performance of supervised encoder-decoder models in partial volume segmentation, which has extensive applications in fundamental studies of subsurface energy, water, and environmental systems.

02 PETROLEUM↗
DOE OSTI2023

Lorentz group equivariant autoencoders

Abstract There has been significant work recently in developing machine learning (ML) models in high energy physics (HEP) for tasks such as classification, simulation, and anomaly detection. Often these models are adapted from those designed for datasets in computer vision or natural language processing, which lack inductive biases suited to HEP data, such as equivariance to its inherent symmetries. Such biases have been shown to make models more performant and interpretable, and reduce the amount of training data needed. To that end, we develop the Lorentz group autoencoder (LGAE), an autoencoder model equivariant with respect to the proper, orthochronous Lorentz group $$\textrm{SO}^+(3,1)$$ SO + ( 3 , 1 ) , with a latent space living in the representations of the group. We present our architecture and several experimental results on jets at the LHC and find it outperforms graph and convolutional neural network baseline models on several compression, reconstruction, and anomaly detection metrics. We also demonstrate the advantage of such an equivariant model in analyzing the latent space of the autoencoder, which can improve the explainability of potential anomalies discovered by such ML models.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS1973

A Kalman filter approach to adaptive estimation of multispectral signatures

The signatures of remote sensing data from agricultural crops exhibit significant non-stationarity, so that the performance of fixed parameter classifiers degenerates with time and distance from the initial training data. A class of adaptive decision-directed classifiers are being developed, based on Kalman filter theory. Limited results to date on two data sets indicate approximately a 25 to 40% reduction in rates of misclassification.

Crane, R. B.↗
DOE OSTI2026

Multidimensional Distributional Neural Network Output Demonstrated in Super‐Resolution of Surface Wind Speed

Accurate quantification of uncertainty in neural network predictions remains a central challenge for scientific applications involving high-dimensional, correlated data. While existing methods capture either aleatoric or epistemic uncertainty, few offer closed-form, multidimensional distributions that preserve spatial correlation while remaining computationally tractable. In this work, we present a framework for training neural networks with a multidimensional Gaussian loss, generating a closed-form predictive distribution over outputs informed by non-identically distributed training data. Our approach captures aleatoric uncertainty by iteratively estimating the means and covariance matrices, and is demonstrated on a super-resolution example out-of-training-sample. We leverage a Fourier representation of the covariance matrix to stabilize network training and preserve spatial correlation. We introduce a novel regularization strategy—referred to as information sharing—that interpolates between image-specific and global covariance estimates, enabling convergence of the super-resolution downscaling network trained on image-specific distributional loss functions. This framework allows for efficient sampling, explicit correlation modeling, and extensions to more complex distribution families all without disrupting prediction performance. We demonstrate the method on a surface wind speed downscaling task and discuss its broader applicability to uncertainty-aware prediction in scientific models.

17 WIND ENERGY↗
DOE OSTI2018

Data Driven Temperature Control of Heat Exchangers

The following poster presents a couple of approaches for temperature control at heat exchanger’s exit. One of the approaches is based on using n-step ahead neural network while the other approach utilizes a combination of control + learning module for temperature control. Designing robust controllers for typical heat exchangers is very challenging primarily because of complexities associated with the synthesis of dynamics of industrial heat exchangers. Even if a dynamics model is synthesized, the accuracy of it would be called into question as there can be lot of unmodeled system dynamics and random noise in input parameters. In many situations, the degree of uncertainty in the model of system being controlled limits the utility of optimal control design. Controller can be manually tuned on the field but it is very difficult to determine manual adjustments that result in overall improvement. These practical challenges call for learning based approaches for temperature control that would result in optimal heat exchanger performance. We utilize the existing mathematical model for air-water system synthesized by Underwood and Crawford for testing our algorithms. Underwood and Crawford developed a model of heating coil by fitting a second-order non-linear equations to measurements of air and water temperature and flow rates obtained from the actual coil. The uncertainties in the model include air and water inlet temperatures and air-flow rate. These uncertainties are modeled as random variables that are sampled from a uniform distribution. The control variable is the water flow rate which is controlled for regulating the air-outlet temperature. Neural networks are frequently used in various literatures for learning time-series. We propose a modified version of neural networks known as n-step ahead neural network that take the state variables and output at time t as input to successfully predict output at time t+n. The variable n is a hyper-parameter which can be optimized based on performance constraints. The n-step ahead neural network was then trained on training data generated from experimental model to high training accuracies. This trained model performed appreciably in its objective to regulate outlet temperature of a heat exchanger. The second approach uses a combination of control and learning module for temperature regulation. The control module used is a proportional controller while the learning module used is steady state neural network. The steady state neural network is used to learn the controller output as a function of random variables and temperature set-point. The proportional controller is then used to compensate for transient behavior of linearized system around various operating points. The algorithm works effectively for temperature regulation as neural network captures the non-linear input-output behavior of closed loop system leaving proportional controller to compensate for steady state errors of a linearized plant. The proposed algorithm’s performances were evaluated against the performance of a PI controller. Further, the robustness of the algorithms were also studied in great detail under small perturbations around tuned parameters of the proposed algorithms.

Sundar, Sreenath↗
NASA NTRS2021

Decoding the effects of synonymous variants

Synonymous single nucleotide variants (sSNVs) are common in the human genome but are often overlooked. However, sSNVs can have significant biological impact and may lead to disease. Existing computational methods for evaluating the effect of sSNVs suffer from the lack of gold-standard training/evaluation data and exhibit over-reliance on sequence conservation signals. We developed synVep (synonymous Variant effect predictor), a machine learning-based method that overcomes both of these limitations. Our training data was a combination of variants reported by gnomAD (observed) and those unreported, but possible in the human genome (generated). We used positive-unlabeled learning to purify the generated variant set of any likely unobservable variants. We then trained two sequential extreme gradient boosting models to identify subsets of the remaining variants putatively enriched and depleted in effect. Our method attained 90% precision/recall on a previously unseen set of variants. Furthermore, although synVep does not explicitly use conservation, its scores correlated with evolutionary distances between orthologs in cross-species variation analysis. synVep was also able to differentiate pathogenic vs. benign variants, as well as splice-site disrupting variants (SDV) vs. non-SDVs. Thus, synVep provides an important improvement in annotation of sSNVs, allowing users to focus on variants that most likely harbor effects.

Zishuo Zeng↗
DOE OSTI2024

Learning Nonlinear Reduced Models from Data with Operator Inference

This review discusses Operator Inference, a nonintrusive reduced modeling approach that incorporates physical governing equations by defining a structured polynomial form for the reduced model, and then learns the corresponding reduced operators from simulated training data. The polynomial model form of Operator Inference is sufficiently expressive to cover a wide range of nonlinear dynamics found in fluid mechanics and other fields of science and engineering, while still providing efficient reduced model computations. The learning steps of Operator Inference are rooted in classical projection-based model reduction; thus, some of the rich theory of model reduction can be applied to models learned with Operator Inference. This connection to projection-based model reduction theory offers a pathway toward deriving error estimates and gaining insights to improve predictions. Furthermore, through formulations of Operator Inference that preserve Hamiltonian and other structures, important physical properties such as energy conservation can be guaranteed in the predictions of the reduced model beyond the training horizon. This review illustrates key computational steps of Operator Inference through a large-scale combustion example.

Mechanics↗
DOE OSTI2020

A priori analysis on deep learning of subgrid-scale parameterizations for Kraichnan turbulence

In the present study, we investigate different data-driven parameterizations for large eddy simulation of two-dimensional turbulence in the a priori settings. These models utilize resolved flow field variables on the coarser grid to estimate the subgrid-scale stresses. We use data-driven closure models based on localized learning that employs a multilayer feedforward artificial neural network with point-to-point mapping and neighboring stencil data mapping, and convolutional neural network fed by data snapshots of the whole domain. The performance of these data-driven closure models is measured through a probability density function and is compared with the dynamic Smagorinksy model (DSM). The quantitative performance is evaluated using the cross-correlation coefficient between the true and predicted stresses. We analyze different frameworks in terms of the amount of training data, selection of input and output features, their characteristics in modeling with accuracy, and training and deployment computational time. We also demonstrate computational gain that can be achieved using the intelligent eddy viscosity model that learns eddy viscosity computed by the DSM instead of subgrid-scale stresses. Here, we detail the hyperparameters optimization of these models using the grid search algorithm.

42 ENGINEERING↗
DOE OSTI2023

Accelerated alloy discovery using synthetic data generation and data mining

The search for new alloys with improved properties is never ending with infinite combinations and amounts of alloying elements in the alloy. Advancements in machine learning have made navigating this enormous search space feasible. However, training the machine learning models and tuning their hyper-parameters to make accurate predictions can be time-consuming and often require high-performance computing resources. Furthermore, the quality of the predictions depend on the availability of sufficient training data. Here, we present a generic approach to accelerate alloy discovery by coupling high throughput CALPHAD calculations, synthetic data generation, and data mining. Finally, as a demonstration of the approach, we design super bainitic steels that form bainite at 200 C in lower transformation times.

36 MATERIALS SCIENCE↗
DOE OSTI2024

On the Training and Generalization of Deep Operator Networks

Here, we present a novel training method for deep operator networks (DeepONets), one of the most popular neural network models for operators. DeepONets are constructed by two subnetworks, namely the branch and trunk networks. Typically, the two subnetworks are trained simultaneously, which amounts to solving a complex optimization problem in a high dimensional space. In addition, the nonconvex and nonlinear nature makes training very challenging. To tackle such a challenge, we propose a two-step training method that trains the trunk network first and then sequentially trains the branch network. The core mechanism is motivated by the divide-and-conquer paradigm and is the decomposition of the entire complex training task into two subtasks with reduced complexity. Therein the Gram–Schmidt orthonormalization process is introduced which significantly improves stability and generalization ability. On the theoretical side, we establish a generalization error estimate in terms of the number of training data, the width of DeepONets, and the number of input and output sensors. Numerical examples are presented to demonstrate the effectiveness of the two-step training method, including Darcy flow in heterogeneous porous media.

deep operator networks↗
DOE OSTI2021

Mapping Stochastic Devices to Probabilistic Algorithms

Probabilistic and Bayesian neural networks have long been proposed as a method to incorporate uncertainty about the world (both in training data and operation) into artificial intelligence applications. One approach to making a neural network probabilistic is to leverage a Monte Carlo sampling approach that samples a trained network while incorporating noise. Such sampling approaches for neural networks have not been extensively studied due to the prohibitive requirement of many computationally expensive samples. While the development of future microelectronics platforms that make this sampling more efficient is an attractive option, it has not been immediately clear how to sample a neural network and what the quality of random number generation should be. This research aimed to start addressing these two fundamental questions by examining basic “off the shelf” neural networks can be sampled through a few different mechanisms (including synapse “dropout” and neuron “dropout”) and examine how these sampling approaches can be evaluated both in terms of evaluating algorithm effectiveness and the required quality of random numbers.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Theory-Guided Machine Learning in Materials Science

Materials scientists are increasingly adopting the use of machine learning tools to discover hidden trends in data and make predictions. Applying concepts from data science without foreknowledge of their limitations and the unique qualities of materials data, however, could lead to errant conclusions. The differences that exist between various kinds of experimental and calculated data require careful choices of data processing and machine learning methods. Here, we outline potential pitfalls involved in using machine learning without robust protocols. We address some problems of overfitting to training data using decision trees as an example, rational descriptor selection in the field of perovskites, and preserving physical interpretability in the application of dimensionality reducing techniques such as principal component analysis. We show how proceeding without the guidance of domain knowledge can lead to both quantitatively and qualitatively incorrect predictive models.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Geometrical-Based Generative Adversarial Network to Enhance Digital Rock Image Quality

X-ray microcomputed tomography (micro-CT) is a common tool for the study of porous media structures and properties. High-quality micro-CT data are required to accurately capture pore structures. Acquiring high-quality micro-CT data, however, is not always possible, owing to application limitations and experimental constraints. Therefore, we propose a geometrical-based generative adversarial network (GAN) to rapidly restore noisy micro-CT images to their clean counterparts. The training data and related ground-truth (GT) data are scanned for 7 min and 9.5 h, respectively. To evaluate the performance of the geometrical-based GAN, a 6003 voxel image that has never been used for training is reconstructed and compared with the corresponding GT image. Histogram matching and linear normalization are implemented to adjust the histogram of the reconstructed image to that of the GT image. A watershed-based segmentation method is then applied to delineate pore and solid phases. Lastly, we measure the Minkowski functionals and petrophysical properties, including absolute permeability, pore size distribution, drainage capillary pressure-saturation curve, and imbibition relative permeability, to estimate the physical accuracy of the denoised image. The results show that the proposed geometrical-based GAN can accurately restore noisy micro-CT data. By reducing the scanning time from 9.5 h to 7 min, the expenditure of collecting micro-CT can be decreased significantly. This is particularly important for applications where time-lapse images of a dynamic process are required, high-throughput imaging is necessary for real-time data analysis, or where the quantification of large sample volumes is required.

58 GEOSCIENCES↗