SEARCH · Engineering Papers

Results for “NEURAL NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,657 records · Page 92

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Photometric redshift estimation with convolutional neural networks and galaxy images: Case study of resolving biases in data-driven methods

Deep-learning models have been increasingly exploited in astrophysical studies, but these data-driven algorithms are prone to producing biased outputs that are detrimental for subsequent analyses. In this work, we investigate two main forms of biases: class-dependent residuals, and mode collapse. We do this in a case study, in which we estimate photometric redshift as a classification problem using convolutional neural networks (CNNs) trained with galaxy images and associated spectroscopic redshifts. We focus on point estimates and propose a set of consecutive steps for resolving the two biases based on CNN models, involving representation learning with multichannel outputs, balancing the training data, and leveraging soft labels. The residuals can be viewed as a function of spectroscopic redshift or photometric redshift, and the biases with respect to these two definitions are incompatible and should be treated individually. We suggest that a prerequisite for resolving biases in photometric space is resolving biases in spectroscopic space. Experiments show that our methods can better control biases than benchmark methods, and they are robust in various implementing and training conditions with high-quality data. Our methods hold promises for future cosmological surveys that require a good constraint of biases, and they may be applied to regression problems and other studies that make use of data-driven models. Nonetheless, the bias-variance tradeoff and the requirement of sufficient statistics suggest that we need better methods and optimized data usage strategies.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

Training Spiking Neural Networks with Synaptic Plasticity under Integer Representation

Neuromorphic computing is emerging as a promising Beyond Moore computing paradigm that employs event-triggered computation and non-von Neumann hardware. Spike Timing Dependent Plasticity (STDP) is a well-known bio-inspired learning rule that relies on activities of locally connected neurons to adjust the weights of their respective synapses. In this work, we analyze a basic STDP rule and its sensitivity on the different hyperparameters for training spiking neural networks (SNNs) with supervision, customized for a neuromorphic hardware implementation with integer weights. We compare the classification performance on four UCI datasets (iris, wine, breast cancer and digits) that depict varying levels of complexity. We perform a search for optimal set of hyperparameters using both grid search and Bayesian optimization. Through the use of Bayesian optimization, we show the general trends in hyperparameter sensitivity in SNN classification problem. With the best sets of hyperparameters, we achieve accuracies comparable to some of the best performing SNNs on these four datasets. With a highly optimized supervised STDP rule we show that these accuracies can be achieved with just 20 epochs of training.

Kulkarni, Shruti↗

DOE OSTI2021

Short Term Plasticity for Artificial Neural Networks

Achieving efficient learning for AI systems was identified as a major challenge in the DOE's recently released, AI for Science , report. The human brain is capable of efficient and low-powered learning. It is likely that implementing brain-like principles will lead to more efficient AI systems. In this LDRD, I aim to contribute to this goal by creating a foundation for implementing and studying a brain phenomenon termed short term plasticity (STP) in spiking artificial neural networks within Sandia. First, data collected by the Allen Institute for Brain Science (AIBS) was analyzed to see if STP could be classified into types using the data collected. Although the data was inadequate at the time, AIBS has updated their database and created models that could be utilized in the future. Second, I began creating a software package to assess the ability of a Boltzmann machine utilizing STP to sample from national security data.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2023

The Feasibility of Incorporating a 3D Velocity Model Into Earthquake Location Around Salt Lake City, UT Using a Physics Informed Neural Network

Earthquake location algorithms typically require travel time calculation. Doing this calculation in 3D, despite advances in algorithm efficiency and computational power, can still be prohibitively expensive in terms of resources and storage. Implementation of high-resolution 3D models in routine earthquake location would be a significant step forward in most of the world. Machine learning algorithms have potential to act as substitutes for travel time calculation algorithms or stored travel time tables. We investigate EikoNet - a physics informed neural network machine learning model that estimates travel times very quickly and comes with negligible memory-overhead. Specifically, we apply EikoNet to the Wasatch Fault Community Velocity Model (WFCVM), a highly detailed and complex 3D velocity model of the Salt Lake City, UT region. While routine locations in the area and studies of the 2020 Magna, UT earthquake sequence used a 1D velocity model, a 3D model may help better our understanding the structure of the major fault in the region. Our primary goal was to test the speed, memory requirements, and accuracy of EikoNet compared to a reference eikonal solver. We find that while the EikoNet is exceedingly fast and requires little memory overhead, achieving acceptable accuracy in estimated travel times is difficult and requires extensive computational resources.

58 GEOSCIENCES↗

NASA NTRS2018

A Global Spatially Contiguous Solar-Induced Fluorescence (CSIF) Dataset Using Neural Networks

Satellite-retrieved solar-induced chlorophyll fluorescence (SIF) has shown great potential to monitor the photosynthetic activity of terrestrial ecosystems. However, several issues, including low spatial and temporal resolution of the gridded datasets and high uncertainty of the individual retrievals, limit the applications of SIF. In addition, inconsistency in measurement footprints also hinders the direct comparison between gross primary production (GPP) from eddy covariance (EC) flux towers and satellite-retrieved SIF. In this study, by training a neural network (NN) with surface reflectance from the MODerate-resolution Imaging Spectroradiometer (MODIS) and SIF from Orbiting Carbon Observatory-2 (OCO-2), we generated two global spatially contiguous SIF (CSIF) datasets at moderate spatiotemporal (0.05° 4-day) resolutions during the MODIS era, one for clear-sky conditions (2000–2017) and the other one in all-sky conditions (2000–2016). The clear-sky instantaneous CSIF (CSIF(sub clear-inst)) shows high accuracy against the clear-sky OCO-2 SIF and little bias across biome types. The all-sky daily average CSIF (CSIF(sub all-daily)) dataset exhibits strong spatial, seasonal and interannual dynamics that are consistent with daily SIF from OCO-2 and the Global Ozone Monitoring Experiment-2 (GOME-2). An increasing trend (0.39 %) of annual average CSIFall-daily is also found, confirming the greening of Earth in most regions. Since the difference between satellite-observed SIF and CSIF is mostly caused by the environmental down-regulation on SIF(sub yield), the ratio between OCO-2 SIF and CSIF(sub clear-inst) can be an effective indicator of drought stress that is more sensitive than the normalized difference vegetation index and enhanced vegetation index. By comparing CSIF(sub all-daily) with GPP estimates from 40 EC flux towers across the globe, we find a large cross-site variation (c.v. = 0.36) of the GPP–SIF relationship with the highest regression slopes for evergreen needleleaf forest. However, the cross-biome variation is relatively limited (c.v. = 0.15). These two contiguous SIF datasets and the derived GPP–SIF relationship enable a better understanding of the spatial and temporal variations of the GPP across biomes and climate.

Zhang, Yao↗

NASA NTRS2019

Sea Surface Salinity Retrievals from Aquarius Using Neural Networks

Even though the Sea Surface Salinity (SSS) retrieved from Aquarius are generally very close to in-situ measurements, the level of similarity varies with the region and with the circumstances of the observations (wind speed, sea surface temperature, etc.). SSS is currently retrieved from the brightness temperatures measured by Aquarius and applying the current theoretical model for the propagation and emission of the natural thermal radiation. In this contribution we consider an alternative retrieval approach based on a Neural Network (NN) with the goal of improving the subsets of Aquarius SSS data that are in poorer agreement within-situ measurements. The subset considered here are the SSS retrieved at latitudes higher than 30 ̊. The output of the NN approach are compared against in-situ measurements using four statistical metrics (correlation coefficient, bias, RMSD and 5% trimmed range). The output of the NN and the nominal Aquarius SSS are compared against SSS values from in-situ measurements and from ocean models. From these comparisons it appears that the output of the NN matches the in-situ measurements better than the nominal Aquarius SSS.

Soldo, Yan↗

NASA NTRS2018

Advantages of Measuring the Q Stokes Parameter in Addition to the Total Radiance I in the Detection of Absorbing Aerosols: a Proof of Concept Using Vector Radiative Transfer, Neural Networks, and Optimal Estimation

A simple but novel study was conducted to investigate whether an imager-type spectroradiometer instrument like MODIS, currently flying on board the Aqua and Terra satellites, or MERIS, which flew on board Envisat, could detect absorbing aerosols if they could measure the Q Stokes parameter in addition to the total radiance I, that is if they could also measure the linear polarization of the light. Accurate radiative transfer calculations were used to train a fast neural network forward model, which together with a simple statistical optimal estimation scheme was used to retrieve three aerosol parameters: aerosol optical depth, fraction of absorbing aerosols, and aerosol location. The aerosols were assumed to be located either between 0 and 2 km or between 2 and 4 km in the Earth’s atmosphere. From simulated data it was found that by itself the total radiance I was generally insufficient to retrieve all three aerosol parameters, but that together with the Q Stokes component it was possible to reliably retrieve values of aerosol optical depth, fraction, and location.

Stamnes, Snorre↗

DOE OSTI2020

Direct detection of dark matter substructure in strong lens images with convolutional neural networks

Strong gravitational lensing is a potential way of uncovering the nature of dark matter, by finding perturbations to images that cannot be accounted for well by modeling the lens galaxy without additional structure, be it subhalos (smaller halos within the smooth lens) or line-of-sight (LOS) halos. Here, we introduce results attempting to infer the presence of substructure from images without requiring an intermediate step in which a smooth model has to be subtracted, using a simple convolutional neural network (CNN). We find that the network is only able to infer the presence of subhalos with greater than 75% accuracy when they have masses of greater than or equal to 5 × 10 9 M ⊙ if they lie within the main lens galaxy. Since less massive foreground LOS halos can have the same effect as higher-mass subhalos, the CNN can probe lower masses in the halo mass function. The accuracy does not improve significantly if we add a population of less massive subhalos. With the expectation of experiments such as Hubble Space Telescope and Euclid yielding thousands of high-quality strong lensing images in the next years, having a way of analyzing images quickly to identify candidates that merit further analysis to determine individual subhalo properties while preventing extensive resources being used for images that would yield null detections could be very useful. By understanding the sensitivity as a function of substructure mass, nondetections could be combined with the information from images with substructure to constrain the cold dark matter scenario, in particular if the sensitivity can be pushed to lower masses.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2025

Accelerating iterative ptychography with an integrated neural network

Electron ptychography is a powerful and versatile tool for high-resolution and dose-efficient imaging. Iterative reconstruction algorithms are powerful but also computationally expensive due to their relative complexity and the many hyperparameters that must be optimised. Gradient descent-based iterative ptychography is a popular method, but it may converge slowly when reconstructing low spatial frequencies. Here, in this work, we present a method for accelerating a gradient descent-based iterative reconstruction algorithm by training a neural network (NN) that is applied in the reconstruction loop. The NN works in Fourier space and selectively boosts low spatial frequencies, thus enabling faster convergence in a manner similar to accelerated gradient descent algorithms. We discuss the difficulties that arise when incorporating a NN into an iterative reconstruction algorithm and show how they can be overcome with iterative training. We apply our method to simulated and experimental data of gold nanoparticles on amorphous carbon and show that we can significantly speed up ptychographic reconstruction of the nanoparticles.

4DSTEM↗

DOE OSTI2024

NuGraph2: A Graph Neural Network for Neutrino Event Reconstruction

Neutrino experiments are set to probe some of the most important open questions in physics, from CP violation and the nature of dark matter. The technology of choice for many of these experiments is the liquid argon time projection chamber (LArTPC). In current LArTPC experiments, reconstruction performance often represents a limiting factor for the sensitivity. New developments are therefore needed to unlock the full potential of LArTPC experiments. NuGraph2 is a state of the art Graph Neural Network for reconstruction of data in LArTPC experiments. NuGraph2 utilizes a heterogeneous graph structure, with separate subgraphs of 2D nodes (hits in each plane) connected across planes via 3D nodes (space points). The model provides a consistent description of the neutrino interaction across all planes. NuGraph2 is a multi-purpose network, with a common message-passing attention engine connected to multiple decoders with different classification or regression tasks. These include the classification of detector hits according to the particle type that produced them (semantic segmentation) and the separation of hits from the neutrino interaction from hits due to noise or cosmic-ray background. Additional decoders are being developed, performing tasks such as the regression of the neutrino interaction vertex position. Performance results will be presented based on publicly available samples from MicroBooNE. These include both physics performance metrics, achieving 95% accuracy for semantic segmentation and 98% classification of neutrino hits, as well as computational metrics for training and for inference on CPU or GPU. The status of the NuGraph integration in the LArSoft software framework will be presented, as well as initial studies about model interpretability and injection of domain knowledge.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2022

Invertible Neural Networks for Airfoil Design

We report the airfoil design problem, in which an engineer seeks a shape with desired performance characteristics, is fundamental to aerodynamics. Design workflows traditionally rely on iterative optimization methods using low-fidelity integral boundary-layer methods as higher-fidelity adjoint-based computational fluid dynamics methods are computationally expensive. Surrogate-based approaches can accelerate the design process but still rely on some iterative inverse design procedure. In this work, we leverage emerging invertible neural network (INN) tools to enable the rapid inverse design of airfoil shapes for wind turbines. INNs are specialized deep-learning models with well-defined inverse mappings. When trained appropriately, INN surrogate models are capable of forward prediction of aerodynamic and structural quantities for a given airfoil shape as well as inverse recovery of airfoil shapes with specified aerodynamic and structural characteristics. The INN approach offers a roughly 100 times speed-up compared to adjoint-based methods for inverse design. We demonstrate the INN tool for inverse design on three test cases of 100 airfoils each that satisfy the performance characteristics close to those of airfoils used in wind-turbine blades. All generated shapes satisfy the desired aerodynamic characteristics, demonstrating the success of the INN approach for inverse design of airfoils.

17 WIND ENERGY↗

NASA NTRS2018

New Neural Network Cloud Mask Algorithm Based on Radiative Transfer Simulations

Cloud detection and screening constitute critically important first steps required to derive many satellite data products. Traditional threshold-based cloud mask algorithms require a complicated design process and fine tuning for each sensor, and they have difficulties over areas partially covered with snow/ice. Exploiting advances in machine learning techniques and radiative transfer modeling of coupled environmental systems, we have developed a new, threshold-free cloud mask algorithm based on a neural network classifier driven by extensive radiative transfer simulations. Statistical validation results obtained by using collocated CALIOP and MODIS data show that its performance is consistent over different ecosystems and significantly better than the MODIS Cloud Mask (MOD35 C6) during the winter seasons over snow-covered areas in the mid-latitudes. Simulations using a reduced number of satellite channels also show satisfactory results, indicating its flexibility to be configured for different sensors. Comparedto threshold-based methods and previous machine-learning approaches, this new cloud mask (i) does not rely on thresholds, (ii) needs fewer satellite channels, (iii) has superior performance during winter seasons in mid-latitude areas, and (iv) can easily be applied to different sensors.

cloud mask algorithms↗

DOE OSTI2024

NuGraph2: A Graph Neural Network for Neutrino Event Reconstruction

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2024

NuGraph2: A Graph Neural Network for Neutrino Event Reconstruction

Neutrino experiments are set to probe some of the most important open questions in physics, from CP violation and the nature of dark matter. The technology of choice for many of these experiments is the liquid argon time projection chamber (LArTPC). In current LArTPC experiments, reconstruction performance often represents a limiting factor for the sensitivity. New developments are therefore needed to unlock the full potential of LArTPC experiments. NuGraph2 is a state of the art Graph Neural Network for reconstruction of data in LArTPC experiments [https://arxiv.org/abs/2403.11872]. NuGraph2 utilizes a heterogeneous graph structure, with separate subgraphs of 2D nodes (hits in each plane) connected across planes via 3D nodes (space points). The model provides a consistent description of the neutrino interaction across all planes. NuGraph2 is a multi-purpose network, with a common message-passing attention engine connected to multiple decoders with different classification or regression tasks. These include the classification of detector hits according to the particle type that produced them (semantic segmentation) and the separation of hits from the neutrino interaction from hits due to noise or cosmic-ray background. Additional decoders are being developed, performing tasks such as the regression of the neutrino interaction vertex position. Performance results will be presented based on publicly available samples from MicroBooNE. These include both physics performance metrics, achieving 95% accuracy for semantic segmentation and 98% classification of neutrino hits, as well as computational metrics for training and for inference on CPU or GPU. The status of the NuGraph integration in the LArSoft software framework will be presented, as well as initial studies about model interpretability and injection of domain knowledge.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2025

Neural Network Analysis of Nuclear Magnetic Resonance and Infrared Spectra

Nuclear magnetic resonance (NMR) spectroscopy and infrared (IR) spectroscopy are powerful chemical characterization techniques with broad general usage. However, the manual evaluation of the resulting spectra is time-consuming and requires significant expertise, preventing insights from being used in real-time applications. With recent advances in computation and artificial intelligence (AI), new tools are available for automating spectral interpretation. In this work, machine learning (ML) algorithms using 1-dimensional convolutional neural networks (CNNs) were applied to identify common functional groups from spectral information. Raw spectra were collected virtually from the Human Metabolome Database (HMDB) and National Institute of Standards and Technology (NIST) Chemistry WebBook and processed into a suitable standard. Algorithm design was tailored to best fit the nature of the problem, with built-in flexibility to accommodate relevant parameters beyond the raw spectral input, specifically solvent identity and magnetic frequency for NMR. The predictive capability of the algorithm in identifying functional groups is displayed in several examples. This methodology has been compiled into a code repository and could easily be modified to adapt alternative data sources, including other spectrum types. To mitigate overfitting, a common problem in mathematical modeling where overfamiliarity with training data produces trends that are not representative of the general data, a novel metric was developed, referred to as Accufit. Accufit includes a parameter that penalizes substantial differences in the training accuracy and the accuracy of an independent validation set. Examples are presented showing the effectiveness of Accufit in maintaining the model’s predictive capability while controlling the overfitting when used as a custom metric for hyperparameter tuning.

Sturgill, James↗

DOE OSTI2024

Phase Diagrams of Alloys and Their Hydrides via On-Lattice Graph Neural Networks and Limited Training Data

Efficient prediction of sampling-intensive thermodynamic properties is needed to evaluate material performance and permit high-throughput materials modeling for a diverse array of technology applications. To alleviate the prohibitive computational expense of high-throughput configurational sampling with density functional theory (DFT), surrogate modeling strategies like cluster expansion are many orders of magnitude more efficient but can be difficult to construct in systems with high compositional complexity. We therefore employ minimal-complexity graph neural network models that accurately predict and can even extrapolate to out-of-train distribution formation energies of DFT-relaxed structures from an ideal (unrelaxed) crystallographic representation. This enables the large-scale sampling necessary for various thermodynamic property predictions that may otherwise be intractable and can be achieved with small training data sets. Two exemplars, optimizing the thermodynamic stability of low-density high-entropy alloys and modulating the plateau pressure of hydrogen in metal alloys, demonstrate the power of this approach, which can be extended to a variety of materials discovery and modeling problems.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2021

Convolutional Neural Networks Based Remote Sensing Scene Classification under Clear and Cloudy Environments

Remote sensing (RS) scene classification has wide applications in the environmental monitoring and geological survey. In the real-world applications, the RS scene images taken by the satellite might have two scenarios: clear and cloudy environments. However, most of existing methods did not consider these two environments simultaneously. In this paper, we assume that the global and local features are iscriminative in either clear or cloudy environments. Many existing Convolution Neural Networks (CNN) based models have made excellent achievements in the image classification, however they somewhat ignored the global and local features in their network structure. In this paper, we propose a new CNN based network (named GLNet) with the Global Encoder and Local Encoder to extract the discriminative global and local features for the RS scene classification, where the constraints for inter-class dispersion and intra-class compactness are embedded in the GLNet training. The experimental results on two publicized RS scene classification datasets show that the proposed GLNet could achieve better performance based on many existing CNN backbones under both clear and cloudy environments.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Predicting Elastic Properties of Materials from Electronic Charge Density Using 3D Deep Convolutional Neural Networks

Materials representation plays a key role in machine learning-based prediction of materials properties and new materials discovery. Currently both graph and three-dimensional (3D) voxel representation methods are based on the heterogeneous elements of the crystal structures. Here, we propose to use electronic charge density (ECD) as a generic unified 3D descriptor for materials property prediction with the advantage of possessing close relation with the physical and chemical properties of materials. We developed an ECD-based 3D convolutional neural networks (CNNs) for predicting the elastic properties of materials, in which CNNs can learn effective hierarchical features with multiple convolving and pooling operations. Extensive benchmark experiments over 2170 $Fm\bar3m$ face-centered-cubic materials show that our ECD-based CNNs can achieve good performance for elasticity prediction. Especially, our CNN models based on the fusion of elemental Materials-Agnostic Platform for Informatics and Exploration features and ECD descriptors achieved the best fivefold cross-validation performance. More importantly, we showed that our ECD-based CNN models can achieve significantly better extrapolation performance when evaluated over nonredundant data sets, where there are few neighbor-training samples around test samples. As an additional validation, we evaluated the predictive performance of our models on 329 materials of space group $Fm\bar3m$ by comparing to density functional theory calculated values, which shows a better prediction power of our model for bulk modulus than shear modulus. Because of the unified representation power of ECD, it is expected that our ECD-based CNN approach can also be applied to predict other physical and chemical properties of crystalline materials.

36 MATERIALS SCIENCE↗