Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Deep Neural Networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Automatically parallelizing batch inference on deep neural networks using Fiats and Fortran 2023 `do concurrent`

This paper introduces novel programming strategies that leverage features of the Fortran 2023 standard of the International Standards Organization (ISO) to automatically parallelize computations on deep neural networks. The paper focuses on the interplay of object-oriented, parallel, and functional programming paradigms in the Fiats deep learning library. We demonstrate how several infrequently used language features play a role in enabling efficient, parallel execution. Specifically, the ability to explicitly declare that a procedure is pure facilitates inference in the context of the language’s loop-parallelism construct `do concurrent`. Also, explicitly prohibiting the overriding of a parent type’s type-bound procedures eliminates the need for dynamic dispatch in performance-critical code. Finally, this paper uses batch inference calculations on a neural network surrogate for atmospheric aerosol dynamics to demonstrate that LLVM Flang compiler’s automatic parallelization of `do concurrent` achieves roughly the same performance and scalability as achieved by OpenMP compiler directives. We also demonstrate that double-precision inference costs 37–72% longer runtime than default-real precision with most values in the range 57-60%.

Rouson, Damian↗
DOE OSTI2024

Achieving precise multiparameter measurements with distributed optical fiber sensor using wavelength diversity and deep neural networks

The development of advanced distributed optical fiber sensing systems that are capable of performing accurate and spatially resolved multiparameter measurements is of great interest to a wide range of scientific and industrial applications. Here, in this paper, we propose and experimentally demonstrate a wavelength diversity based advanced distributed optical fiber sensor system to accomplish multiparameter sensing while greatly enhancing measurement accuracy. A suite of deep neural network (DNN) algorithms are developed and verified for data denoising, rapid Brillouin frequency shift estimation, and vibration data event classification. As a proof-of-concept, we demonstrate the effectiveness of the proposed advanced wavelength diversity distributed fiber sensor system assisted by DNN for simultaneous, independent measurements of static strain, temperature, and acoustic vibrations over a 25 km long sensing fiber at 3 m spatial resolution. These results suggest the potential for an intelligent multiparameter monitoring system with enhanced performance in advanced structural health monitoring applications.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Modern deep neural networks for Direct Normal Irradiance forecasting: A classification approach

The escalating energy demand and the adverse environmental impacts of fossil-fuel use necessitate a shift towards cleaner and renewable alternatives. Concentrated Solar Power (CSP) technology emerges as a promising solution, offering a carbon-free alternative for power generation. The efficiency and profitability of CSP depend on the Direct Normal Irradiance (DNI) component of solar radiation; hence, accurate DNI forecasting can help optimize CSP plants’ operations and performance. The unpredictable nature of weather phenomena, particularly cloud cover, introduces uncertainty into DNI projections. Existing DNI forecasting models use meteorological factors, which are both challenging to estimate numerically over short prediction windows and expensive to model through data at a sufficiently high spatial and temporal resolution. This research addresses the challenge by presenting a novel approach that formulates DNI prediction as a multi-class classification problem, departing from conventional regression-based methods. The primary objective of this classification framework is to identify optimal periods aligning with specific operational thresholds for CSP plants, contributing to enhanced dispatch optimization strategies. We model the DNI classification problem using four advanced deep neural networks – rectified linear unit (ReLU) networks, 1D residual networks (ResNets), bidirectional long short-term memory (BiLSTM) networks, and transformers – achieving accuracies up to 93.5% without requiring meteorological parameters.

14 SOLAR ENERGY↗
DOE OSTI2025

Wide‐Field Bond Quality Evaluation Using Frequency Domain Thermoreflectance with Deep Neural Network Feature Reconstruction

Heterogeneous integration of microelectronic components provides a pathway to improve circuit/component performance; however, this comes with assembly challenges, in particular due to complex interfaces via subsurface bump bonds. The ability of these bonds to transmit electrical signals and conduct heat to the carrier substrate limits component performance. In this work, hyperspectral frequency‐domain thermoreflectance (FDTR) imaging is demonstrated as a robust technique for evaluating the quality of subsurface indium bump bonds in a surrogate microelectronic sample. By performing microscale FDTR imaging with coarse motion image stitching, thermal phase maps that cover a 4 mm by 4 mm field‐of‐view with subsurface feature sensitivity at depths greater than 50 µm are obtained. The resulting FDTR hyperspectral data contains more than three million pixels and reveal the quality of subsurface microbump arrays. Wide‐field analysis of bonded versus gap regions is enabled by deep neural network feature reconstruction, that after training, rapidly provides an interpretable representation of bond quality. Utility of noisy higher frequency FDTR phase maps, i.e., near the computationally predicted sensing depth limit, results in an average prediction error of 11%. Taken together, FDTR with neural network‐based analysis demonstrates subsurface bond monitoring at length scales relevant for heterogeneously integrated microelectronics.

FDTR↗
DOE OSTI2025

Deep Neural Network Assisted Distributed Strain and Temperature Fiber Sensor System for Natural Gas Pipeline Monitoring

Natural gas pipeline integrity monitoring is crucial to detect potential leaks, find structural issues, and prevent environmental damage. This article presents a system of natural gas pipeline monitoring that uses a specialized double Brillouin peak sensing fiber along with the Brillouin optical time domain analysis (BOTDAs) technique. The calibrated sensing fiber coefficients for strain and temperature are 41.8 kHz/ με and 0.9 MHz/°C for peak 1; and 47.2 kHz/ με , and 1.11 MHz/°C for peak 2, respectively. Initially, lab tests were performed by installing a short section of double Brillouin peak fiber (DBPF) on a 1-in steel pipe under pressure up to 1000 per square inch (psi) at elevated temperatures. Simultaneous distributed measurements of temperature and pressure-induced hoop strain were successfully measured. Considering the long processing speed to extract Brillouin frequency shift (BFS), we employ a novel probabilistic deep neural network (PDNN) framework for rapid BFS prediction. Additionally, using the Finite Element Method, the effects of the pipeline pressure on hoop strain were modeled and compared to the experimental hoop strain under the same set of pipeline conditions. Finally, an actual 4-in outer diameter steel natural gas pipeline was used for pilot-scale tests, where hoop strain was measured at various pressure levels. Leaks were simulated to demonstrate accurate pipeline integrity monitoring. At an internal pipe pressure of 1000 psi, hoop strain of approximately 300 με was observed, and the sensitivity was calculated as 0.28 με /psi. The results of this pilot-scale study demonstrated that the system is capable of performing distributed monitoring sufficient to detect pipeline pressure and the presence of leaks to ensure the safe operation of gas pipelines in the field.

03 NATURAL GAS↗
DOE OSTI2024

Simultaneous energy and mass calibration of large-radius jets with the ATLAS detector using a deep neural network

The energy and mass measurements of jets are crucial tasks for the Large Hadron Collider experiments. This paper presents a new calibration method to simultaneously calibrate these quantities for large-radius jets measured with the ATLAS detector using a deep neural network (DNN). To address the specificities of the calibration problem, special loss functions and training procedures are employed, and a complex network architecture, which includes feature annotation and residual connection layers, is used. The DNN-based calibration is compared to the standard numerical approach in an extensive series of tests. The DNN approach is found to perform significantly better in almost all of the tests and over most of the relevant kinematic phase space. In particular, it consistently improves the energy and mass resolutions, with a 30% better energy resolution obtained for transverse momenta $p$ T > $500$ GeV.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Benchmarking Operators in Deep Neural Networks for Improving Performance Portability of SYCL

SYCL is a portable programming model for heterogeneous computing, so it is important to obtain reasonable performance portability of SYCL. Towards the goal of better understanding and improving performance portability of SYCL for machine learning workloads, we have been developing benchmarks for basic operators in deep neural networks (DNNs). These operators could be offloaded to heterogeneous computing devices such as graphics processing units (GPUs) to speed up computation. In this paper, we introduce the benchmarks, evaluate the performance of the operators on GPU-based systems, and describe the causes of the performance gap between the SYCL and Compute Unified Device Architecture (CUDA) kernels. We find that the causes are related to the utilization of the texture cache for read-only data, optimization of the memory accesses with strength reduction, use of local memory, and register usage per thread. We hope that the efforts of developing benchmarks for studying performance portability will stimulate discussion and interactions within the community.

Jin, Zheming [ORNL] (ORCID:000000027197780X)↗
DOE OSTI2024

Evaluating Operators in Deep Neural Networks for Improving Performance Portability of SYCL

SYCL is a portable programming model for heterogeneous computing, so it is important to obtain reasonable performance portability of SYCL. Towards the goal of better understanding and improving performance portability of SYCL for machine learning workloads, we have been developing benchmarks for basic operators in deep neural networks (DNNs). These operators could be offloaded to heterogeneous computing devices such as graphics processing units (GPUs) to speed up computation. In this work, we introduce the benchmarks, evaluate the performance of the operators on GPU-based systems, and describe the causes of the performance gap between the SYCL and Compute Unified Device Architecture (CUDA) kernels. We find that the causes are related to the utilization of the texture cache for read-only data, optimization of the memory accesses with strength reduction, shared local memory accesses, and register usage per thread. We hope that the efforts of developing benchmarks for studying performance portability will stimulate discussion and interactions within the community.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2018

Deep Neural Network for Precision Multi-band Infrared Image Segmentation

Image segmentation is one of the fundamental steps in computer vision. Separating targets from background clutter with high precision is a challenging operation for both humans and computers. Currently, segmenting objects from IR images is done by tedious manual work. The implementation of a Deep Neural Network (DNN) to perform precision segmentation of multi-band IR video images is presented. A customized pix2pix DNN with multiple layers of generative encoder/decoder and discriminator architecture is used in the IR image segmentation process. Real and synthetic images and ground truths are employed to train the DNN. Iterative training is performed to achieve optimum accuracy of segmentation using a minimal number of training data. Special training images are created to enhance the missing features and to increase the segmentation accuracy of the objects. Retraining strategies are developed to minimize the DNN training time. Single pixel accuracy has been achieved in IR target boundary segmentation using DNNs. The segmentation accuracy between the customized pix2pix DNN and simple thresholding, GraphCut, simple neural network and ResNet models are compared.

Torres, Gil↗
NASA NTRS2018

Automated Pneumothorax Diagnosis using Deep Neural Networks

Thoracic ultrasound can provide information leading to rapid diagnosis of pneumothorax with improved accuracy over the standard physical examination and with higher sensitivity than anteroposterior chest radiography. However, the clinical We have Furthermore, remote environments, such as the battlefield or deep-space exploration, may lack expertise for diagnosing developed an automated image interpretation pipeline for the analysis of thoracic ultrasound data and the classification of pneumothorax events to provide decision support in such situations. Our pipeline consists of image preprocessing, data augmentation, and deep learning architectures for medical diagnosis. In this work, we demonstrate that robust, accurate interpretation of chest images and video can be achieved using deep neural networks. A number of novel image processing techniques were employed to achieve this result. Affine transformations were applied for data augmentation. Hyperparameters were optimized for learning rate, dropout regularization, batch size, and epoch iteration by a sequential model-based Bayesian approach. In addition, we utilized pretrained architecturesinterpretation of a patient medical image is highly operator dependent. certain pathologies., applying transfer learning and fine-tuning techniques to fully connected layers. Our pipeline yielded binary classification validation accuracies of 98.3% for M-mode images and 99.8% with B-mode video frames.

US Army collaboration↗
DOE OSTI2026

Improving missing transverse momentum estimation with a deep neural network

At hadron colliders, the net transverse momentum of particles that do not interact with the detector (missing transverse momentum, $^→_𝑝$$^{miss}_{T}$) is a crucial observable in many analyses. In the standard model, $^→_𝑝$$^{miss}_{T}$ originates from neutrinos. Many beyond-the-standard-model particles, such as dark matter candidates, are also expected to leave the experimental apparatus undetected. This paper presents a novel deep neural network based $^→_𝑝$$^{miss}_{T}$ estimator, DeepMET, developed by the CMS Collaboration at the LHC. The DeepMET algorithm produces a weight for each reconstructed particle based on its properties. The estimator is based on the negative vector sum of the weighted transverse momenta of all reconstructed particles in an event. Compared with other estimators currently employed by CMS, DeepMET improves the $^→_𝑝$$^{miss}_{T}$ resolution by 10%–30%, shows improvement for a wide range of final states, is easier to train, and is more resilient against the effects of additional proton-proton interactions accompanying the collision of interest.

artificial neural networks↗
NASA NTRSDate not supplied

Estimating Bidirectional Reflectance and Monitoring Stability of SNPP-VIIRS Reflective Solar Bands Using A Deep Neural Network

The NASA Clouds and the Earth's Radiant Energy System project provides the scientific community with observed top-of-atmosphere shortwave and longwave fluxes for climate monitoring and climate model validation. To provide consistent VIIRS cloud retrievals, the CERES Imager and Geostationary Calibration Group (IGCG) must understand and quantify the stability of the VIIRS instruments. To achieve this, the IGCG utilizes tropical deep convective clouds (DCCs) as invariant targets. Proper seasonal characterization of the DCC bidirectional reflectance distribution function (BRDF) is key to the success of DCC-based calibration methods, particularly for shortwave infrared (SWIR) bands. This article proposes the use of a deep neural network (DNN) to characterize VIIRS solar reflective band BRDF reflectance, with which individual channel trends are isolated by manipulating the DNN time input. Initial results show that the DNN method can extract statistically significant SNPP-VIIRS band trends, using only SNPP-VIIRS inputs, that are correlative to and match the magnitude of significant trends determined using methods that rely on an external angular distribution model. It may be possible to apply this approach to actively monitor the stability of new instruments without the need for predetermined seasonal BRDF corrections.

Benjamin Scarino↗
NASA NTRSDate not supplied

A Deep Neural Network for Achieving Spectrally Consistent and Seamless Infrared Radiance Measurements Across Geostationary Satellite Domains

The NASA Clouds and the Earth's Radiant Energy System (CERES) project provides the scientific community with observed top-of-atmosphere (TOA) shortwave and longwave fluxes for climate monitoring and climate model validation. To achieve this goal, CERES relies on TOA broadband fluxes derived from geostationary satellite (GEO) imagery to account for the diurnal flux variations between the CERES observation intervals. Consistent global flux derivation depends on accurate and consistent cloud retrievals. Scene-dependent spectral measurement inconsistency of the instruments that make up the contiguous ring of GEO observations (GEO-Ring), as well as limb darkening effects, can cause discontinuities in derived cloud properties and radiative fluxes at the boundaries of adjacent imager domains. Although the algorithms utilize radiative transfer models to account for instrument-band-dependent atmospheric correction and viewing zenith angle (VZA) dependency, small discontinuities may persist due to uncertainties inherent to the multiple imager-specific algorithms. Furthermore, while hyperspectral-instrument-based spectral band adjustment factors may effectively account for spectrally induced bias, they are less effective at reducing variance owed to the specific composition of the viewed scene, which is challenging to robustly characterize. As such, this article highlights the use of a deep neural network (DNN) to resolve spectral-and VZA-induced biases between GEO-Ring imagers. The DNN uses available infrared (IR) channels from the GEO instruments, along with viewing and solar illumination geometry, to estimate homogenized, VIIRS-like IR radiances for use in the GEO cloud algorithm. This approach is effective at mitigating scene-dependent spectral variance and VZA dependency, resulting in consistent radiance measurements across the GEO-Ring, thereby leading toward a more seamless global cloud assessment.

deep learning↗
NASA NTRS2024

Theoretical Prediction of Thermal Expansion Anisotropy for Y 2 Si 2 O 7 Environmental Barrier Coatings Using a Deep Neural Network Potential and Comparison to Experiment

Environmental barrier coatings (EBCs) are an enabling technology for silicon carbide (SiC)-based ceramic matrix composites (CMCs) in extreme environments such as gas turbine engines. However, development of new coating systems is hindered by the large design space and difficulty in predicting properties for these materials. Density Functional Theory (DFT) has successfully been used to model and predict some thermodynamic and thermo-mechanical properties of high-temperature ceramics for EBCs, although these calculations are challenging due to their high computational costs. In this work, we use machine learning to train a deep neural network potential (DNP) for Y 2 Si 2 O 7 , which is then applied to calculate thermodynamic and thermo-mechanical properties at near-DFT accuracy much faster and using less computational resources than DFT. We use this DNP to predict phonon-based thermodynamic properties of Y 2 Si 2 O 7 with good agreement to DFT and experiments. We also utilize the DNP to calculate the anisotropic, lattice direction-dependent coefficients of thermal expansion (CTEs) for Y 2 Si 2 O 7 . Molecular dynamics trajectories using the DNP correctly demonstrate accurate prediction of the anisotropy of the CTE in good agreement with diffraction experiments. In the future, this DNP could be applied to accelerate additional property calculations for Y 2 Si 2 O 7 compared to DFT or experiments.

rare earth silicates↗
NASA NTRS2023

Detail Enhancement of AIRS/AMSU Temperature and Moisture Profiles Using a 3D Deep Neural Network

In recent decades, spaceborne microwave and hyperspectral infrared sounding instruments have significantly benefited weather forecasting and climate science. However, existing retrievals of lower troposphere temperature and humidity profiles have limitations in vertical resolution, and often cannot accurately represent key features such as the mixed layer thermodynamic structure and the inversion at the planetary boundary layer (PBL) top. Because of the existing limitations in PBL remote sensing from space, there is a compelling need to improve routine, global observations of the PBL and enable advances in scientific understanding and weather and climate prediction. To address this, we have developed a new 3D deep neural network (DNN) which enhances detail and reduces noise in Level 2 granules of temperature and humidity profiles from the Atmospheric Infrared Sounder (AIRS)/Advanced Microwave Sounding Unit (AMSU) sounder instruments aboard NASA’s Aqua spacecraft. We show that the enhancement improves accuracy and detail including key features such as capping inversions at the top of the PBL over land, resulting in improved accuracy in estimations of PBL height.

Adam B Milstein↗
NASA NTRS2017

H31G-1596: DeepSAT's CloudCNN: A Deep Neural Network for Rapid Cloud Detection from Geostationary Satellites

Cloud and cloud shadow detection has important applications in weather and climate studies. It is even more crucial when we introduce geostationary satellites into the field of terrestrial remote sensing. With the challenges associated with data acquired in very high frequency (10-15 mins per scan), the ability to derive an accurate cloud shadow mask from geostationary satellite data is critical. The key to the success for most of the existing algorithms depends on spatially and temporally varying thresholds,which better capture local atmospheric and surface effects.However, the selection of proper threshold is difficult and may lead to erroneous results. In this work, we propose a deep neural network based approach called CloudCNN to classify cloudshadow from Himawari-8 AHI and GOES-16 ABI multispectral data. DeepSAT's CloudCNN consists of an encoderdecoder based architecture for binary-class pixel wise segmentation. We train CloudCNN on multi-GPU Nvidia Devbox cluster, and deploy the prediction pipeline on NASA Earth Exchange (NEX) Pleiades supercomputer. We achieved an overall accuracy of 93.29% on test samples. Since, the predictions take only a few seconds to segment a full multispectral GOES-16 or Himawari-8 Full Disk image, the developed framework can be used for real-time cloud detection, cyclone detection, or extreme weather event predictions.

GOES-1↗
DOE OSTI2024

Distributed-Memory Sparse Deep Neural Network Inference Using Global Arrays

Partitioned Global Address Space (PGAS) models exhibit tremendous promise in developing efficient and productive distributed-memory parallel applications. They have been used extensively in scientific computations due to conveniently offering a ``shared-memory''-like model and convenient interfaces that separate communication with synchronization. Traditionally, PGAS communication models have been applied to dense/contiguously distributed data, but most modern applications depict varied levels of sparsity. Existing PGAS models require certain adaptations to support distributed sparse computations, since associated computations often require matrix arithmetic, in addition to data movement. The Global Arrays toolkit from Pacific Northwest National Laboratory (PNNL) is one of the earliest PGAS models to combine one-sided data communication and distributed matrix operations and is still used in the popular NWChem quantum chemistry suite. Recently, we have expanded the Global Arrays toolkit to support common sparse operations, like sparse matrix-dense matrix multiplies (SpMM), sparse matrix-sparse matrix multiplication (SpGEMM) and Sampled Dense-Dense Matrix Multiplication (SDDMM). As it turns out, these operations are the bedrock of sparse Deep Learning (DL); sparse deep neural networks and Graph Neural Networks (GNNs) have gained increasing attention recently in achieving speedups on training and inference with reduced memory footprints. Unlike scientific applications in High Performance Computing (HPC), modern (distributed-memory capable) DL toolkits often rely on non-standardized and closed-source vendor software optimizations, creating challenges in software-hardware co-design at scale. Our goal is to support a variety of distributed-memory sparse matrix operations and helper functions in the newly created Sparse Global Arrays (SGA), such that it is possible to build portable and productive Machine Learning scenarios for algorithm/software and hardware codesign purposes. Contemporary data-parallel schemes for training/inference are undergoing a major overhaul since model replication limits scalability and causes resource inefficiencies. As such, we have adopted tensor parallelism in decomposing the model and inputs, to mitigate memory issues. Current implementation is built on top of MPI and uses CPUs to maximize the portability across the platforms.

Distributed computing, machine learning↗
NASA NTRSDate not supplied

CERES FluxByCldTyp NB2BB Fluxes Improvement based on Deep Neural Network

The NASA Clouds and the Earth's Radiant Energy System (CERES) product provides over 20 years of accurately observed top-of-the-atmosphere (TOA) and surface flux data record for climate monitoring and diagnostic studies. The interaction between clouds and radiation interaction is a key factor that dominate climate feedbacks but is not well understood. To further advance our understanding of the cloud-radiation interaction, a new CERES FluxByCldTyp (FBCT) product has been developed that contains radiative fluxes by cloud-type, which can provide more stringent constraints when validating models and reveal more insight into the interactions between clouds and climate. For CERES partly cloudy and multiple cloud-type footprints, the FBCT product utilizes Moderate Resolution Imaging Spectroradiometer (MODIS) narrow-band (NB) imager channel radiances partitioned by cloud-type within a CERES footprint to estimate the cloud-type broadband fluxes. The MODIS multi-channel derived broadband fluxes were compared with the CERES observed footprint fluxes and were found to be within 1% and 2.5% for LW and SW, respectively, as well as being mostly free of cloud property dependencies. The FBCT all-sky and clear-sky monthly averaged fluxes were found to be consistent with the CERES SSF1deg product. This study takes advantage of recent progress in machine learning (ML) field by applying deep neural network algorithm to improve fluxes based on MODIS NB radiances. The preliminary study shows ML produce are an improvement over the current FBCT Edition 4 NB2BB algorithm. Furthermore, unlike Ed4 NB2BB, the new ML method convert NB radiances directly to broadband fluxes. For future Ed5, new NB radiances are proposed and used by ML to improve fluxes calculation. Prelimary results show significant LW improvement.

Moguo Sun↗