Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “NEURAL NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,621 records · Page 90
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Viscosity in water from first-principles and deep-neural-network simulations

Abstract We report on an extensive study of the viscosity of liquid water at near-ambient conditions, performed within the Green-Kubo theory of linear response and equilibrium ab initio molecular dynamics (AIMD), based on density-functional theory (DFT). In order to cope with the long simulation times necessary to achieve an acceptable statistical accuracy, our ab initio approach is enhanced with deep-neural-network potentials (NNP). This approach is first validated against AIMD results, obtained by using the Perdew–Burke–Ernzerhof (PBE) exchange-correlation functional and paying careful attention to crucial, yet often overlooked, aspects of the statistical data analysis. Then, we train a second NNP to a dataset generated from the Strongly Constrained and Appropriately Normed (SCAN) functional. Once the error resulting from the imperfect prediction of the melting line is offset by referring the simulated temperature to the theoretical melting one, our SCAN predictions of the shear viscosity of water are in very good agreement with experiments.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Physics informed deep neural network embedded in a chemical transport model for the Amazon rainforest

Secondary organic aerosols (SOA) are fine particles in the atmosphere, which interact with clouds, radiation and affect the Earth’s energy budget. SOA formation involves chemistry in gas phase, aqueous aerosols, and clouds. Simulating these chemical processes involve solving a stiff set of differential equations, which are computationally expensive steps for three-dimensional chemical transport models. Deep neural networks (DNNs) are universal function approximators that could be used to represent the complex nonlinear changes in aerosol physical and chemical processes; however, key challenges such as generalizability to extended time periods, preservation of mass balance, simulating sparse model outputs, and maintaining physical constraints have limited their use in atmospheric chemistry. Here, we develop an approach of using a physics-informed DNN that overcomes previous such challenges and demonstrates its applicability for the chemical formation processes of isoprene epoxydiol SOA (IEPOX-SOA) over the Amazon rainforest. The DNN is trained with data generated by simulating IEPOX-SOA over the entire atmospheric column, using the Weather Research and Forecasting Model coupled with Chemistry (WRF-Chem). The trained DNN is then embedded within WRF-Chem to replace the computationally expensive default solver of IEPOX-SOA formation. The trained DNN predictions generalizes well with the default model simulation of the IEPOX-SOA mass concentrations and its size distribution (20 size bins) over several days of simulations in both dry and wet seasons. The embedded DNN reduces the computational expense of WRF-Chem by a factor of 2. Our approach shows promise in terms of application to other computationally expensive chemistry solvers in climate models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Bioplastic design using multitask deep neural networks

Abstract Non-degradable plastic waste jeopardizes our environment, yet our modern lifestyle and current technologies are impossible to sustain without plastics. Bio-synthesized and biodegradable alternatives such as polyhydroxyalkanoates (PHAs) have the potential to replace large portions of the world’s plastic supply with cradle-to-cradle materials, but their chemical complexity and diversity limit traditional resource-intensive experimentation. Here, we develop multitask deep neural network property predictors using available experimental data for a diverse set of nearly 23,000 homo- and copolymer chemistries. Using the predictors, we identify 14 PHA-based bioplastics from a search space of almost 1.4 million candidates which could serve as potential replacements for seven petroleum-based commodity plastics that account for 75% of the world’s yearly plastic production. We also discuss possible synthesis routes for the identified promising materials.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Heat transport in liquid water from first-principles and deep neural network simulations

In this work, we compute the thermal conductivity of water within linear response theory from equilibrium molecular dynamics simulations, by adopting two different approaches. In one, the potential energy surface (PES) is derived on the fly from the electronic ground state of density functional theory (DFT) and the corresponding analytical expression is used for the energy flux. In the other, the PES is represented by a deep neural network (DNN) trained on DFT data, whereby the PES has an explicit local decomposition and the energy flux takes a particularly simple expression. By virtue of a gauge invariance principle, established by Marcolongo, Umari, and Baroni, the two approaches should be equivalent if the PES were reproduced accurately by the DNN model. We test this hypothesis by calculating the thermal conductivity, at the GGA (PBE) level of theory, using the direct formulation and its DNN proxy, finding that both approaches yield the same conductivity, in excess of the experimental value by approximately 60%. Besides being numerically much more efficient than its direct DFT counterpart, the DNN scheme has the advantage of being easily applicable to more sophisticated DFT approximations, such as meta-GGA and hybrid functionals, for which it would be hard to derive analytically the expression of the energy flux. We find in this way that a DNN model, trained on meta-GGA (SCAN) data, reduces the deviation from experiment of the predicted thermal conductivity by about 50%, leaving the question open as to whether the residual error is due to deficiencies of the functional, to a neglect of nuclear quantum effects in the atomic dynamics, or, likely, to a combination of the two.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

TULIP: An RNA-seq-based Primary Tumor Type Prediction Tool Using Convolutional Neural Networks

Background: With cancer as one of the leading causes of death worldwide, accurate primary tumor type prediction is critical in identifying genetic factors that can inhibit or slow tumor progression. There have been efforts to categorize primary tumor types with gene expression data using machine learning, and more recently with deep learning, in the last several years. Methods In this paper, we developed four 1-dimensional (1D) Convolutional Neural Network (CNN) models to classify RNA-seq count data as one of 17 highly represented primary tumor types or 32 primary tumor types regardless of imbalanced representation. Additionally, we adapted the models to take as input either all Ensembl genes (60,483) or protein coding genes only (19,758). Unlike previous work, we avoided selection bias by not filtering genes based on expression values. RNA-seq count data expressed as FPKM-UQ of 9,025 and 10,940 samples from The Cancer Genome Atlas (TCGA) were downloaded from the Genomic Data Commons (GDC) corresponding to 17 and 32 primary tumor types respectively for training and validating the models. Results: All 4 1D-CNN models had an overall accuracy of 94.7% to 97.6% on the test dataset. Further evaluation indicates that the models with protein coding genes only as features performed with better accuracy compared to the models with all Ensembl genes for both 17 and 32 primary tumor types. For all models, the accuracy by primary tumor type was above 80% for most primary tumor types. Conclusions: We packaged all 4 models as a Python-based deep learning classification tool called TULIP (TUmor CLassIfication Predictor) for performing quality control on primary tumor samples and characterizing cancer samples of unknown tumor type. Further optimization of the models is needed to improve the accuracy of certain primary tumor types.

Jones, Sara↗
NASA NTRS1992

Application of neural networks in the acousto-ultrasonic evaluation of metal-matrix composite specimens

Acousto-ultrasonics (AU) is a nondestructive evaluation (NDE) technique that was devised for the testing of various types of composite materials. A study has been done to determine how effectively the AU technique may be applied to metal-matrix composites (MMCs). The authors use the results and data obtained from that study and apply neural networks to them, particularly in the assessment of mechanical property variations of a specimen from AU measurements. It is assumed that there is no information concerning the important features of the AU signal which relate to the mechanical properties of the specimen. Minimally processed AU measurements are used while relying on the network's ability to extract the significant features of the signal.

Cios, Krzysztof J.↗
DOE OSTI2024

Addressing GPU memory limitations for Graph Neural Networks in High-Energy Physics applications

Introduction Reconstructing low-level particle tracks in neutrino physics can address some of the most fundamental questions about the universe. However, processing petabytes of raw data using deep learning techniques poses a challenging problem in the field of High Energy Physics (HEP). In the Exa.TrkX Project, an illustrative HEP application, preprocessed simulation data is fed into a state-of-art Graph Neural Network (GNN) model, accelerated by GPUs. However, limited GPU memory often leads to Out-of-Memory (OOM) exceptions during training, due to the large size of models and datasets. This problem is exacerbated when deploying models on High-Performance Computing (HPC) systems designed for large-scale applications. Methods We observe a high workload imbalance issue during GNN model training caused by the irregular sizes of input graph samples in HEP datasets, contributing to OOM exceptions. We aim to scale GNNs on HPC systems, by prioritizing workload balance in graph inputs while maintaining model accuracy. Our paper introduces diverse balancing strategies aimed at decreasing the maximum GPU memory footprint and avoiding the OOM exception, across various datasets. Results Our experiments showcase memory reduction of up to 32.14% compared to the baseline. We also demonstrate the proposed strategies can avoid OOM in application. Additionally, we create a distributed multi-GPU implementation using these samplers to demonstrate the scalability of these techniques on the HEP dataset. Discussion By assessing the performance of these strategies as data loading samplers across multiple datasets, we can gauge their effectiveness in both single-GPU and distributed environments. Our experiments, conducted on datasets of varying sizes and across multiple GPUs, broaden the applicability of our work to various GNN applications that handle input datasets with irregular graph sizes.

Lee, Claire Songhyun↗
NASA NTRS1993

Classification of the LCVF AVIRIS test site with a Kohonen artificial neural network

We present a classification of an AVIRIS spectral image of the Lunar Crater Volcanic Field (LCVF). Geologic mapping from such data is made possible by distinctive mineral signatures: absorption features and the shape of the spectral continuum. The subtle spectral shape differences between some of the geological units in this scene along with the high dimensionality of the spectral presents a challenging pattern recognition task. We found an artificial neural network powerful in separating 13 geological units based on the full spectral resolution. The LCVF, in northern Nye County, Nevada, was the primary focus of the NASA-sponsored Geologic Remote Sensing Field Experiment in the summer of 1989. It consists of over 100 square miles of Quaternary basaltic pyroclastic and flow deposits. These deposits lie atop ignimbrites and silicic lava flows of Tertiary age and in turn are overlain by Quaternary alluvial and playa deposits. This AVIRIS image was collected on September 29, 1989 at 11:44 at 11:44 PDT. The 256-by-256 pixel subsection in this study contains oxidized basaltic cinder deposits, the southern half of the Lunar Lake playa, and outcrops of the Rhyollite of Big Sand Spring Valley. Vegetation in LCVF is sparse, but locally abundant within washes and near springs.

Merenyi, Erzsebet↗
DOE OSTI2021

Explore Spatio‐Temporal Learning of Large Sample Hydrology Using Graph Neural Networks

Abstract Streamflow forecasting over gauged and ungauged basins play a vital role in water resources planning, especially under the changing climate. Increased availability of large sample hydrology data sets, together with recent advances in deep learning techniques, has presented new opportunities to explore temporal and spatial patterns in hydrological signatures for improving streamflow forecasting. The purpose of this study is to adapt and benchmark several state‐of‐the‐art graph neural network (GNN) architectures, including ChebNet, Graph Convolutional Network (GCN), and GraphWaveNet, for end‐to‐end graph learning. We explicitly represent river basins as nodes in a graph, learn the spatiotemporal nodal dependencies, and then use the learned relations to predict streamflow simultaneously across all nodes in the graph. The efficacy of the developed GNN models is investigated using the Catchment Attributes and MEteorology for Large‐sample Studies (CAMELS) data set under two settings, fixed graph topology (transductive learning), and variable graph topology (inductive learning), with the latter applicable to prediction in ungauged basins (PUB). Results indicate that GNNs are generally robust and computationally efficient, achieving similar or better performance than a baseline model trained using the long short‐term memory (LSTM) network. Further analyses are conducted to interpret the graph learning process at the edge and node levels and to investigate the effect of different model configurations. We conclude that graph learning constitutes a viable machine learning‐based method for aggregating spatiotemporal information from a multitude of sources for streamflow forecasting

Sun, Alexander Y.↗
DOE OSTI2022

Real-time X-ray phase-contrast imaging using SPINNet—a speckle-based phase-contrast imaging neural network

X-ray phase-contrast imaging has become indispensable for visualizing samples with low absorption contrast. In this regard, speckle-based techniques have shown significant advantages in spatial resolution, phase sensitivity, and implementation flexibility compared with traditional methods. However, the computational cost associated with data inversion has hindered their wider adoption. By exploiting the power of deep learning, we developed a speckle-based phase-contrast imaging neural network (SPINNet) that significantly improves the imaging quality and boosts the phase retrieval speed by at least 2 orders of magnitude compared to existing methods. To achieve this performance, we combined SPINNet with a coded-mask-based technique, an enhanced version of the speckle-based method. Using this scheme, we demonstrate the simultaneous reconstruction of absorption and phase images on the order of 100 ms, where a traditional correlation-based analysis would take several minutes even with a cluster. In addition to significant improvement in speed, our experimental results show that the imaging and phase retrieval quality of SPINNet outperform existing single-shot speckle-based methods. Furthermore, we successfully demonstrate SPINNet application in x-ray optics metrology and 3D x-ray phase-contrast tomography. Our result shows that SPINNet could enable many applications requiring high-resolution and fast data acquisition and processing, such as in situ and in operando 2D and 3D phase-contrast imaging and real-time at-wavelength metrology and wavefront sensing.

36 MATERIALS SCIENCE↗
NASA NTRS2000

A neural network to retrieve the mesoscale instantaneous latent heat flux over oceans from SSM/I observations

The turbulent latent and sensible heat fluxes are necessary to study heat budget of the upper ocean or initialize ocean general circulation models. In order to retrieve the latent heat flux from satellite observations authors mostly use a bulk approximation of the flux whose parameters are derived from different instrument. In this paper, an approach based on artificial neural networks is proposed and compared to the bulk method on a global data set and 3 local data sets.

SSM/I latent heat flux air-sea interactions↗
DOE OSTI2024

Neuromorphic overparameterisation and few-shot learning in multilayer physical neural networks

Abstract Physical neuromorphic computing, exploiting the complex dynamics of physical systems, has seen rapid advancements in sophistication and performance. Physical reservoir computing, a subset of neuromorphic computing, faces limitations due to its reliance on single systems. This constrains output dimensionality and dynamic range, limiting performance to a narrow range of tasks. Here, we engineer a suite of nanomagnetic array physical reservoirs and interconnect them in parallel and series to create a multilayer neural network architecture. The output of one reservoir is recorded, scaled and virtually fed as input to the next reservoir. This networked approach increases output dimensionality, internal dynamics and computational performance. We demonstrate that a physical neuromorphic system can achieve an overparameterised state, facilitating meta-learning on small training sets and yielding strong performance across a wide range of tasks. Our approach’s efficacy is further demonstrated through few-shot learning, where the system rapidly adapts to new tasks.

Science & Technology - Other Topics↗
DOE OSTI2021

Model Evaluation and Intercomparison of Marine Warm Low Cloud Fractions With Neural Network Ensembles

Abstract Low cloud fractions (LCFs) and meteorological factors (MFs) over an oceanic region containing multiple cloud regimes are examined for three data sets: one Energy Exascale Earth System Model (E3SM) simulation with the default 72‐layer vertical grid (E3SM72), another one with 8‐times vertical resolution via the Framework for Improvement by Vertical Enhancement (E3SM 8), and one with MFs from ERA5 reanalysis and LCFs from the CERES SSF product (ERA5‐SSF). Neural networks (NNs) are trained to capture the relationship between MFs and LCF and to select the best‐performing MF subsets for predicting LCF. NN ensembles are used to (a) confirm the performance of selected MF subsets, (b) to serve as proxy models for each data set to predict LCFs for MFs from all data sets, and (c) to classify MFs into those in shared and uniquely occupied MF subspaces. Overall, E3SM72 and E3SM 8 have large fractions of MFs in shared MF subspace, but less so near the Californian and Peruvian stratocumulus decks. E3SM 8 and ERA5 have small fractions of MFs in shared MF subspace but greater than E3SM72 and ERA5, especially in the Southeast Pacific. The differences in LCFs between three pairs of data sets are decomposed into those associated with the differences in the LCF‐MF relationship and those involving different MFs. Given the same MFs, LCFs produced by E3SM 8 are greater than those produced by E3SM72 but are still different from those in ERA5‐SSF. In general, the shift in MFs dominates the difference in the LCFs.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Galaxy Morphology Classification Using Bayesian Neural Networks for LSST

Within the decade, many new ground and space-based observatories will become operational, generating massive amounts of data on short timescales. New surveys like Rubin Observatory's Legacy Survey of Space and Time (LSST) will be capable of observing objects with greater resolution than ever before, but processing and analyzing these datasets optimally will pose a significant challenge. In an effort to prepare for this, we explore how incorporating Deep Neural Networks can better support future data-intensive Astrophysics tasks such as galaxy morphology classification.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Towards computational polar-topotronics: Multiscale neural-network quantum molecular dynamics simulations of polar vortex states in SrTiO3/PbTiO3 nanowires

Recent discoveries of polar topological structures ( e.g ., skyrmions and merons) in ferroelectric/paraelectric heterostructures have opened a new field of polar topotronics. However, how complex interplay of photoexcitation, electric field and mechanical strain controls these topological structures remains elusive. To address this challenge, we have developed a computational approach at the nexus of machine learning and first-principles simulations. Our multiscale neural-network quantum molecular dynamics molecular mechanics approach achieves orders-of-magnitude faster computation, while maintaining quantum-mechanical accuracy for atoms within the region of interest. This approach has enabled us to investigate the dynamics of vortex states formed in PbTiO 3 nanowires embedded in SrTiO 3 . We find topological switching of these vortex states to topologically trivial, uniformly polarized states using electric field and trivial domain-wall states using shear strain. These results, along with our earlier results on optical control of polar topology, suggest an exciting new avenue toward opto-electro-mechanical control of ultrafast, ultralow-power polar topotronic devices.

Linker, Thomas↗
NASA NTRS1990

Solving inversion problems with neural networks

A class of inverse problems in remote sensing can be characterized by Q = F(x), where F is a nonlinear and noninvertible (or hard to invert) operator, and the objective is to infer the unknowns, x, from the observed quantities, Q. Since the number of observations is usually greater than the number of unknowns, these problems are formulated as optimization problems, which can be solved by a variety of techniques. The feasibility of neural networks for solving such problems is presently investigated. As an example, the problem of finding the atmospheric ozone profile from measured ultraviolet radiances is studied.

Kamgar-Parsi, Behzad↗
NASA NTRS2006

Using Neural Networks to Explore Air Traffic Controller Workload

When a new system, concept, or tool is proposed in the aviation domain, one concern is the impact that this will have on operator workload. As an experience, workload is difficult to measure in a way that will allow comparison of proposed systems with those already in existence. Chatterji and Sridhar (2001) suggested a method by which airspace parameters can be translated into workload ratings, using a neural network. This approach was employed, and modified to accept input from a non-real time airspace simulation model. The following sections describe the preparations and testing work that will enable comparison of a future airspace concept with a current day baseline in terms of workload levels.

Martin, Lynne↗
NASA NTRS2019

AladynPi – Adaptive Neural Network Molecular Dynamics Simulation Code with Physically Informed Potential: Computational Materials Mini-Application

This report provides an overview and description of commands used in the Computational Materials mini-application, AladynPi. AladynPi is an extension of a previously released mini-application, Aladyn (https://github.com/nasa/aladyn; Yamakov, V.I., and Glaessgen, E.H., NASA/TM-2018-220104). Aladyn and AladynPi are basic molecular dynamics codes written in FORTRAN 2003, which are designed to demonstrate the use of adaptive neural networks (ANNs) in atomistic simulations. The role of ANNs is to efficiently reproduce the very complex energy landscape resulting from the atomic interactions in materials with the accuracy of the more expensive quantum mechanics-based calculations. The ANN is trained on a large set of atomic structures calculated using the density functional theory method. An input for the ANN is a set of structure coefficients, characterizing the local atomic environment of each atom, for which the atomic energy is obtained in the ANN inference process. In Aladyn, the ANN gives directly the energy of interatomic interactions. In AladynPi, the ANN gives optimized parameters for a predefined empirical function, known as bond-order-potential (BOP). The parameterized BOP function is then used to calculate the energy. AladynPi code is being released to serve as a training testbed for students and professors in academia to explore possible optimization algorithms for parallel computing on multicore central processing unit (CPU) computers or computers utilizing manycore architectures based on graphic processing units (GPUs). The effort is supported by the High Performance Computing incubator (HPCi) project at NASA Langley Research Center.

Yamakov, Vesselin I.↗