Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “convolutional neural network model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Enhancing stability, magnetic anisotropy, and coercivity of manganese aluminum: Machine learning, ab initio , and micromagnetic modeling

The binary manganese aluminum (MnAl) alloy with L⁢1 0 crystal structure is a promising rare earth (RE) element-free permanent magnetic material because of its exceptional magnetic properties. However, experimentally synthesizing it in a stable bulk form is extremely challenging. Here, in this study, an alternative method of stabilizing the material, a pathway for experimental synthesis and validation, is proposed and theoretically verified. This is done by partially substituting Mn and Al sites with Fe and Ni and identifying its enhanced phase stability, saturation magnetization density, magnetic anisotropy, and coercivity from density functional theory (DFT), machine learning (ML) crystal graph convolution neural network (CGCNN), and micro-magnetic modeling. When considering a fixed 50% Ni, the magnetic anisotropy increases with the increasing Fe content but decreases the formation energy. The calculated formation energies, elastic constants, and phonon frequencies demonstrate that the binary and quaternary compositions are stable. Most importantly, in 50% Fe and Ni-substituted-equiatomic phase, magnetic anisotropy constants and saturation magnetization density increase by 56% and 23% as compared to the MnAl. Further, the coercivity of the equiatomic phase predicted with micro-magnetic modeling is higher by 17% than the parent compound.

Bhandari, Churna [Ames National Laboratory, and Io↗
DOE OSTI2021

Hierarchical Inference with Bayesian Neural Networks: An Application to Strong Gravitational Lensing

In the past few years, approximate Bayesian Neural Networks (BNNs) have demonstrated the ability to produce statistically consistent posteriors on a wide range of inference problems at unprecedented speed and scale. However, any disconnect between training sets and the distribution of real-world objects can introduce bias when BNNs are applied to data. This is a common challenge in astrophysics and cosmology, where the unknown distribution of objects in our universe is often the science goal. In this work, we incorporate BNNs with flexible posterior parameterizations into a hierarchical inference framework that allows for the reconstruction of population hyperparameters and removes the bias introduced by the training distribution. We focus on the challenge of producing posterior PDFs for strong gravitational lens mass model parameters given Hubble Space Telescope–quality single-filter, lens-subtracted, synthetic imaging data. We show that the posterior PDFs are sufficiently accurate (statistically consistent with the truth) across a wide variety of power-law elliptical lens mass distributions. We then apply our approach to test data sets whose lens parameters are drawn from distributions that are drastically different from the training set. We show that our hierarchical inference framework mitigates the bias introduced by an unrepresentative training set's interim prior. Simultaneously, we can precisely reconstruct the population hyperparameters governing our test distributions. Our full pipeline, from training to hierarchical inference on thousands of lenses, can be run in a day. The framework presented here will allow us to efficiently exploit the full constraining power of future ground- and space-based surveys (https://github.com/swagnercarena/ovejero).

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2024

Evaluation of Machine Learning and Deep Learning Algorithms for Fire Prediction in Southeast Asia

Vegetation fires are prevalent in South/Southeast Asian countries, making fire prediction crucial due to their potential environmental, economic, and social impacts. Accurate predictions of fires facilitate timely interventions, helping to mitigate uncontrolled fires that can lead to biodiversity loss and air quality issues. In this study, we utilize VIIRS satellite-derived fire data alongside six machine learning and deep learning models—Simple Persistence, Multi-Layer Perceptron (MLP), Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM), CNN-LSTM, and ConvLSTM—to determine the most effective fire prediction model, using Root Mean Square Error (RMSE) as the metric. Our results indicate that the CNN model is the most reliable in regions with spatial dependencies, such as Brunei, Indonesia, Malaysia, the Philippines, Timor-Leste, and Thailand. Conversely, the ConvLSTM model excels in countries with complex spatiotemporal dynamics like Laos, Myanmar, and Vietnam. The CNN-LSTM hybrid model also performed well in Cambodia, suggesting a need for a balanced approach in areas requiring both spatial and temporal feature extraction. Furthermore, simpler models like Persistence and MLP showed limitations in capturing dynamic patterns and temporal dependencies. Our findings highlight the importance of evaluating models before implementing any decision support systems (DSS) in fire management. By tailoring models to specific regional fire data, we can enhance prediction accuracy and responsiveness, ultimately improving fire risk management in Southeast Asia and beyond.

Deep learning↗
DOE OSTI2024

Neural Networks to Find the Optimal Forcing for Offsetting the Anthropogenic Climate Change Effects

Abstract Of great relevance to climate engineering is the systematic relationship between the radiative forcing to the climate system and the response of the system, a relationship often represented by the linear response function (LRF) of the system. However, estimating the LRF often becomes an ill-posed inverse problem due to high-dimensionality and nonunique relationships between the forcing and response. Recent advances in machine learning make it possible to address the ill-posed inverse problem through regularization and sparse system fitting. Here, we develop a convolutional neural network (CNN) for regularized inversion. The CNN is trained using the surface temperature responses from a set of Green’s function perturbation experiments as imagery input data together with data sample densification. The resulting CNN model can infer the forcing pattern responsible for the temperature response from out-of-sample forcing scenarios. This promising proof of concept suggests a possible strategy for estimating the optimal forcing to negate certain undesirable effects of climate change. The limited success of this effort underscores the challenges of solving an inverse problem for a climate system with inherent nonlinearity. Significance Statement Predicting the climate response for a given climate forcing is a direct problem, while inferring the forcing for a given desired climate response is often an inverse, ill-posed, problem, posing a new challenge to the climate community. This study makes the first attempt to infer the radiative forcing for a given target pattern of global surface temperature response using a deep learning approach. The resulting deeply trained convolutional neural network inversion model shows promise in capturing the forcing pattern corresponding to a given surface temperature response, with a significant implication on the design of an optimal solar radiation management strategy for curbing global warming. This study also highlights the technical challenges that future research should prioritize in seeking feasible solutions to the inverse climate problem.

Ren, Huiying↗
DOE OSTI2023

Optical information transfer through random unknown diffusers using electronic encoding and diffractive decoding

Free-space optical information transfer through diffusive media is critical in many applications, such as biomedical devices and optical communication, but remains challenging due to random, unknown perturbations in the optical path. We demonstrate an optical diffractive decoder with electronic encoding to accurately transfer the optical information of interest, corresponding to, e.g., any arbitrary input object or message, through unknown random phase diffusers along the optical path. This hybrid electronic-optical model, trained using supervised learning, comprises a convolutional neural network-based electronic encoder and successive passive diffractive layers that are jointly optimized. After their joint training using deep learning, our hybrid model can transfer optical information through unknown phase diffusers, demonstrating generalization to new random diffusers never seen before. The resulting electronic-encoder and optical-decoder model was experimentally validated using a 3D-printed diffractive network that axially spans <70λ, where λ = 0.75 mm is the illumination wavelength in the terahertz spectrum, carrying the desired optical information through random unknown diffusers. The presented framework can be physically scaled to operate at different parts of the electromagnetic spectrum, without retraining its components, and would offer low-power and compact solutions for optical information transfer in free space through unknown random diffusive media.

36 MATERIALS SCIENCE↗
NASA NTRS2017

Very High Resolution Tree Cover Mapping for Continental United States using Deep Convolutional Neural Networks

Uncertainties in input land cover estimates contribute to a significant bias in modeled above ground biomass (AGB) and carbon estimates from satellite-derived data. The resolution of most currently used passive remote sensing products is not sufficient to capture tree canopy cover of less than ca. 10-20 percent, limiting their utility to estimate canopy cover and AGB for trees outside of forest land. In our study, we created a first of its kind Continental United States (CONUS) tree cover map at a spatial resolution of 1-m for the 2010-2012 epoch using the USDA NAIP imagery to address the present uncertainties in AGB estimates. The process involves different tasks including data acquisition ingestion to pre-processing and running a state-of-art encoder-decoder based deep convolutional neural network (CNN) algorithm for automatically generating a tree non-tree map for almost a quarter million scenes. The entire processing chain including generation of the largest open source existing aerial satellite image training database was performed at the NEX supercomputing and storage facility. We believe the resulting forest cover product will substantially contribute to filling the gaps in ongoing carbon and ecological monitoring research and help quantifying the errors and uncertainties in derived products.

High Resolution↗
DOE OSTI2021

Scalable balanced training of conditional generative adversarial neural networks on image data

Here, we propose a distributed approach to train deep convolutional generative adversarial neural network (DC-CGANs) models. Our method reduces the imbalance between generator and discriminator by partitioning the training data according to data labels, and enhances scalability by performing a parallel training where multiple generators are concurrently trained, each one of them focusing on a single data label. Performance is assessed in terms of inception score, Fréchet inception distance, and image quality on MNIST, CIFAR10, CIFAR100, and ImageNet1k datasets, showing a significant improvement in comparison to state-of-the-art techniques to training DC-CGANs. Weak scaling is attained on all the four datasets using up to 1000 processes and 2000 NVIDIA V100 GPUs on the OLCF supercomputer Summit.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Uncertainty Aware Deep Learning for Fault Prediction Using Multivariate Time Series Signals

The superconducting radio-frequency cavities are a crucial component of the Continuous Electron Beam Accelerator Facility (CEBAF) at Jefferson Lab. When a cavity faults, beam delivery to experimental end users is disrupted. Prediction of cavity faults prior to onset is essential to reduce operation and maintenance costs. In this work, a parallel long short-term memory (LSTM)-convolution neural network (CNN)-based deep learning (DL) model is proposed to predict impending faults using pre-fault signals. Further, we introduce an uncertainty quantification approach using Monte Carlo dropout with the LSTM-CNN model to ascertain confidence in the prediction. The model was tested using multivariate time series signals from stable cavity operations and before faults. Initial results show that on the test dataset, the model can identify impending faults before their onset with an average 10-fold cross validation accuracy of 97.39% and a standard deviation of 0.12% using a 100-ms time window. It is also observed that the model performs better as the prediction time moves closer to the fault onset. For additional context, we compare the performance of the model with three machine-learning-based (ML) fault prediction models. Our proposed parallel LSTM-CNN-based DL method shows better performance than the ML-based methods.

Rahman, Md Monibor↗
DOE OSTI2023

GRAPH CONVOLUTIONAL NEURAL NETWORK (GCNN)FOR POLYMER PROPERTY PREDICTION

SF-22-121 Graph convolutional neural networks (GCNN) are powerful machine learning models that apply a message-passing algorithm on adjacent nodes in a graph. Due to the natural representation of molecules via discrete graphs (where nodes encode atoms and edges encode bonds), GCNNs are an intuitive machine learning model chemical space informatics, including the prediction of polymer properties. Here, molecules are represented in feature and adjacency matrices where they are directly used as inputs to a GCNN.

FEINSTEIN, JEREMY↗
NASA NTRS2018

Advances in Hyperspectral Image Classification Methods for Vegetation and Agricultural Cropland Studies

Hyperspectral data are becoming more widely available via sensors on airborne and unmanned aerial vehicle (UAV) platforms, as well as proximal platforms. While space-based hyperspectral data continue to be limited in availability, multiple spaceborne Earth-observing missions on traditional platforms are scheduled for launch, and companies are experimenting with small satellites for constellations to observe the Earth, as well as for planetary missions. Land cover mapping via classification is one of the most important applications of hyperspectral remote sensing and will increase in significance as time series of imagery are more readily available. However, while the narrow bands of hyperspectral data provide new opportunities for chemistry-based modeling and mapping, challenges remain. Hyperspectral data are high dimensional, and many bands are highly correlated or irrelevant for a given classification problem. For supervised classification methods, the quantity of training data is typically limited relative to the dimension of the input space. The resulting Hughes phenomenon, often referred to as the curse of dimensionality, increases potential for unstable parameter estimates, overfitting, and poor generalization of classifiers. This is particularly problematic for parametric approaches such as Gaussian maximum likelihood–based classifiers that have been the backbone of pixel-based multispectral classification methods. This issue has motivated investigation of alternatives, including regularization of the class covariance matrices, ensembles of weak classifiers, development of feature selection and extraction methods, adoption of nonparametric classifiers, and exploration of methods to exploit unlabeled samples via semi-supervised and active learning. Data sets are also quite large, motivating computationally efficient algorithms and implementations. This chapter provides an overview of the recent advances in classification methods for mapping vegetation using hyperspectral data. Three data sets that are used in the hyperspectral classification literature (e.g., Botswana Hyperion satellite data and AVIRIS airborne data over both Kennedy Space Center and Indian Pines) are described in Section 3.2 and used to illustrate methods described in the chapter. An additional high-resolution hyperspectral data set acquired by a SpecTIR sensor on an airborne platform over the Indian Pines area is included to exemplify the use of new deep learning approaches, and a multiplatform example of airborne hyperspectral data is provided to demonstrate transfer learning in hyperspectral image classification. Classical approaches for supervised and unsupervised feature selection and extraction are reviewed in Section 3.3. In particular, nonlinearities exhibited in hyperspectral imagery have motivated development of nonlinear feature extraction methods in manifold learning, which are outlined in Section 3.3.1.4. Spatial context is also important in classification of both natural vegetation with complex textural patterns and large agricultural fields with significant local variability within fields. Approaches to exploit spatial features at both the pixel level (e.g., co-occurrence–based texture and extended morphological attribute profiles [EMAPs]) and integration of segmentation approaches (e.g., HSeg) are discussed in this context in Section 3.3.2. Recently, classification methods that leverage nonparametric methods originating in the machine learning community have grown in popularity. An overview of both widely used and newly emerging approaches, including support vector machines (SVMs), Gaussian mixture models, and deep learning based on convolutional neural networks is provided in Section 3.4. Strategies to exploit unlabeled samples, including active learning and metric learning, which combine feature extraction and augmentation of the pool of training samples in an active learning framework, are outlined in Section 3.5. Integration of image segmentation with classification to accommodate spatial coherence typically observed in vegetation is also explored, including as an integrated active learning system. Exploitation of multisensor strategies for augmenting the pool of training samples is investigated via a transfer learning framework in Section 3.5.1.2. Finally, we look to the future, considering opportunities soon to be provided by new paradigms, as hyperspectral sensing is becoming common at multiple scales from ground-based and airborne autonomous vehicles to manned aircraft and space-based platforms.

Pasolli, Edoardo↗
DOE OSTI2025

Leveraging Natural Language Processing and Generative Models in Molecular Chemistry: Property Prediction and Novel Compound Generation

The accurate prediction of molecular properties is important for the rational design and the advancement of green chemistry and sustainable materials research. However, the predictive power of traditional computational chemistry methods is limited due to computational restrictions. Here, in this study, we examine an alternative approach to the accurate prediction of properties of organic compounds: natural language processing (NLP)-based molecular embedding. Using viscosity, partition coefficient (log P), and enthalpy of vaporization as test properties through a survey of comprehensive datasets comprising 5695 data points for viscosity, 25 870 data points for log P, and 2296 data points for enthalpy of vaporization. These are important properties for the design of greener, safer, and sustainable chemical processes. Models were trained using NLP methods such as Mol2vec and fine-tuned ChemBERTa, and results were compared with traditional input featurization techniques such as Morgan fingerprints and quantum chemistry derived sigma profiles and DFT features. Among the various machine learning models, Mol2vec demonstrated superior predictive capabilities, achieving the highest correlation coefficient (R 2 = 0.945) and lowest RMSE (0.106 mPa s) for viscosity, as well as high accuracy for log P and enthalpy of vaporization predictions. These findings establish the Mol2vec featurization technique, graph-convolutional neural networks (GCNN), and fine-tuned ChemBERTa model as powerful tools for predictive modeling of organic compounds properties, offering a significant improvement over previously used featurization techniques and opening up strategies for very-high-throughput computational screening. Finally, we integrated ML models with hybrid language-model-based generative adversarial networks (LM-GAN) to generate novel molecular sequences with desirable properties for different research applications. The ability to computationally design solvents with lower viscosity, lower log P, and lower enthalpy of vaporization offers a data-driven route to accelerating the discovery of sustainable alternatives to traditionally toxic solvents.

ChemBERTa↗
DOE OSTI2024

Hyperparameter Studies for Vision Transformers Trained on High-Fidelity Simulations

This library is a collection of python modules that define, train, and analyze vision-transformer (ViT) machine learning models. The code implements, with mild modifications, ViT models that have been made publicly available through publication and GitHub code. The training data for these models is hydrodynamic simulation output in the form of numpy arrays. This library contains code to train these ViT models on the hydrodynamic simulation output with a variety of hyperparameters, and to compare the results of such models. Furthermore, the library contains definitions of simple convolutional neural network (CNN) machine learning architectures which can be trained on the same hydrodynamic simulation output. These are included as a reference point to compare the ViT models to. Additionally, the library includes trained ViT and CNN models and example input data for demonstration purposes. The code is based on the PyTorch python library.

Callis, Skylar↗
DOE OSTI2020

Using Machine Learning to Predict Future Temperature Outputs in Geothermal Systems

Optimizing the power output, and economic value, of geothermal power plants over decades of operation is a major challenge in renewable energy. Optimizing the output requires the ability to predict the mass flow rates and the output temperatures of production wells based on the inputs of injection wells, as well as the time history of the system. Machine Learning (ML) that incorporates the known physics of geothermal systems is one possible solution to this challenge. In this work, we explore the ability of ML algorithms to predict future temperature outputs based on historical data. Considering the challenges with obtaining an empirical dataset from field data that is large enough to enable reliable ML, we propose an alternate approach: developing a high-fidelity reservoir model and using computational resources to build a dataset that enables ML. As a first step towards achieving this goal, we present preliminary results from applying ML to predict the temperature timeseries of simple modeled geothermal systems. We describe the application of relevant state-of-the-art ML approaches, such as the Long Short-Term Memory (LSTM) networks and Convolutional Neural Networks (CNN), to extract temporal structures in the model data. We assess the accuracy of the forecasts we obtain, compare the selected approaches, and share the lessons learned that would inform the process of training and utilizing ML algorithms for larger and more complex geothermal systems.

GEOTHERMAL ENERGY↗
DOE OSTI2023

Noise signal identification in time projection chamber data using deep learning model

Deep learning has been employed in various scientific fields and has provided promising results. Here, in this study, a deep learning classifier was implemented to improve the quality of data obtained from a time projection chamber. Digital waveforms of the detected signals were classified into the following three categories: particles, noises, and particles piled up with noises. A simple 1-dimensional convolutional neural network was developed for the classification. The model demonstrated an excellent performance on the test dataset. Its practical performance was also examined using track images and particle identification plots by comparing the original and clean data without the noise signals. The comparison clearly showed that the deep learning model improved the quality of data. The current study presents an effective application of the deep learning model for the time projection chamber data.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Machine Learning Benchmarks for the Classification of Equivalent Circuit Models from Electrochemical Impedance Spectra

Analysis of Electrochemical Impedance Spectroscopy (EIS) data for electrochemical systems often consists of defining an Equivalent Circuit Model (ECM) using expert knowledge and then optimizing the model parameters to deconvolute various resistance, capacitive, inductive, or diffusion responses. For small data sets, this procedure can be conducted manually; however, it is not feasible to manually define a proper ECM for extensive data sets with a wide range of EIS responses. Automatic identification of an ECM would substantially accelerate the analysis of large sets of EIS data. We showcase machine learning methods to classify the ECMs of 9,300 impedance spectra provided by QuantumScape for the BatteryDEV hackathon. The best-performing approach is a gradient-boosted tree model utilizing a library to automatically generate features, followed by a random forest model using the raw spectral data. A convolutional neural network using boolean images of Nyquist representations is presented as an alternative, although it achieves a lower accuracy. We publish the data and open source the associated code. The approaches described in this article can serve as benchmarks for further studies. A key remaining challenge is the identifiability of the labels, underlined by the model performances and the comparison of misclassified spectra.

25 ENERGY STORAGE↗
DOE OSTI2021

A Benchmark to Test Generalization Capabilities of Deep Learning Methods to Classify Severe Convective Storms in a Changing Climate

Abstract This is a test case study assessing the ability of deep learning methods to generalize to a future climate (end of 21st century) when trained to classify thunderstorms in model output representative of the present‐day climate. A convolutional neural network (CNN) was trained to classify strongly rotating thunderstorms from a current climate created using the Weather Research and Forecasting model at high‐resolution, then evaluated against thunderstorms from a future climate and found to perform with skill and comparatively in both climates. Despite training with labels derived from a threshold value of a severe thunderstorm diagnostic (updraft helicity), which was not used as an input attribute, the CNN learned physical characteristics of organized convection and environments that are not captured by the diagnostic heuristic. Physical features were not prescribed but rather learned from the data, such as the importance of dry air at mid‐levels for intense thunderstorm development when low‐level moisture is present (i.e., convective available potential energy). Explanation techniques also revealed that thunderstorms classified as strongly rotating are associated with learned rotation signatures. Results show that the creation of synthetic data with ground truth is a viable alternative to human‐labeled data and that a CNN is able to generalize a target using learned features that would be difficult to encode due to spatial complexity. Most importantly, results from this study show that deep learning is capable of generalizing to future climate extremes and can exhibit out‐of‐sample robustness with hyperparameter tuning in certain applications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Physics-informed machine learning with differentiable programming for heterogeneous underground reservoir pressure management

Abstract Avoiding over-pressurization in subsurface reservoirs is critical for applications like CO $$_2$$ 2 sequestration and wastewater injection. Managing the pressures by controlling injection/extraction are challenging because of complex heterogeneity in the subsurface. The heterogeneity typically requires high-fidelity physics-based models to make predictions on CO $$_2$$ 2 fate. Furthermore, characterizing the heterogeneity accurately is fraught with parametric uncertainty. Accounting for both, heterogeneity and uncertainty, makes this a computationally-intensive problem challenging for current reservoir simulators. To tackle this, we use differentiable programming with a full-physics model and machine learning to determine the fluid extraction rates that prevent over-pressurization at critical reservoir locations. We use DPFEHM framework, which has trustworthy physics based on the standard two-point flux finite volume discretization and is also automatically differentiable like machine learning models. Our physics-informed machine learning framework uses convolutional neural networks to learn an appropriate extraction rate based on the permeability field. We also perform a hyperparameter search to improve the model’s accuracy. Training and testing scenarios are executed to evaluate the feasibility of using physics-informed machine learning to manage reservoir pressures. We constructed and tested a sufficiently accurate simulator that is 400 000 times faster than the underlying physics-based simulator, allowing for near real-time analysis and robust uncertainty quantification.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Reconstructing High Resolution ESM Data Through a Novel Fast Super Resolution Convolutional Neural Network (FSRCNN)

In this work, we present the first application of a fast super resolution convolutional neural network (FSRCNN) based approach for downscaling earth system model (ESM) simulations. Unlike other SR approaches, FSRCNN uses the same input feature dimensions as the low resolution input. This allows it to have smaller convolution layers, avoiding over-smoothing, and reducing computational costs. We adapt the FSRCNN to improve reconstruction on ESM data, we term the FSRCNN-ESM. We use high-resolution (~0.25°) monthly averaged model output of five surface variables over North America from the US Department of Energy's Energy Exascale Earth System Model's control simulation. These high-resolution and corresponding coarsened low-resolution (~1°) pairs of images are used to train the FSRCNN-ESM and evaluate its use as a downscaling approach. We find that FSRCNN-ESM outperforms FSRCNN and other super-resolution methods in reconstructing high resolution images producing finer spatial scale features with better accuracy for surface temperature, surface radiative fluxes, and precipitation.

58 GEOSCIENCES↗