Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Deep learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,495 records · Page 83
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Spatio–Temporal Machine Learning for Regional to Continental Scale Terrestrial Hydrology

Integrated hydrologic models can simulate coupled surface and subsurface processes but are computationally expensive to run at high resolutions over large domains. Here we develop a novel deep learning model to emulate subsurface flows simulated by the integrated ParFlow–CLM model across the contiguous US. We compare convolutional neural networks like ResNet and UNet run autoregressively against our novel architecture called the Forced SpatioTemporal RNN (FSTR). The FSTR model incorporates separate encoding of initial conditions, static parameters, and meteorological forcings, which are fused in a recurrent loop to produce spatiotemporal predictions of groundwater. We evaluate the model architectures on their ability to reproduce 4D pressure heads, water table depths, and surface soil moisture over the contiguous US at 1 km resolution and daily time steps over the course of a full water year. The FSTR model shows superior performance to the baseline models, producing stable simulations that capture both seasonal and event–scale dynamics across a wide array of hydroclimatic regimes. The emulators provide over 1,000× speedup compared to the original physical model, which will enable new capabilities like uncertainty quantification and data assimilation for integrated hydrologic modeling that were not previously possible. Our results demonstrate the promise of using specialized deep learning architectures like FSTR for emulating complex process–based models without sacrificing fidelity.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRSDate not supplied

Joint Spectrum Access and Power Control in Air-Air Communications - A Deep Reinforcement Learning Based Approach

This paper considers the dynamic spectrum access and power control problem in a single-hop point-to-point Air-Air Communication Network (AACN). Due to spectrum scarcity, we assume the number of Aircraft-to-Aircraft (A2A) communication links is greater than that of the available channels, such that some communication links need to share the same channel, causing co-channel interference. We formulate the joint channel selection and power control optimization problem to maximize the Weighted Sum Spectral Efficiency (WSSE). A distributed and dynamic deep Q learning-based algorithm is proposed to find the optimal solution. Specifically, we design two different policies that are trained by conducting a trial-and-error scheme. Each communication link can achieve the optimal policy by exploiting the local information from its neighbors, and this distributive approach make it scalable to large networks. Finally, our experimental results demonstrate the effectiveness of the proposed solution in various AACN scenarios.

Zhe Wang↗
NASA NTRSDate not supplied

Joint Spectrum Access and Power Control in Air-Air Communications - A Deep Reinforcement Learning Based Approach

This paper considers the dynamic spectrum access and power control problem in a single-hop point-to-point Air-Air Communication Network (AACN). Due to spectrum scarcity, we assume the number of Aircraft-to-Aircraft (A2A) communication links is greater than that of the available channels, such that some communication links need to share the same channel, causing co-channel interference. We formulate the joint channel selection and power control optimization problem to maximize the Weighted Sum Spectral Efficiency (WSSE). A distributed and dynamic deep Q learning-based algorithm is proposed to find the optimal solution. Specifically, we design two different policies that are trained by conducting a trial-and-error scheme. Each communication link can achieve the optimal policy by exploiting the local information from its neighbors, and this distributive approach make it scalable to large networks. Finally, our experimental results demonstrate the effectiveness of the proposed solution in various AACN scenarios.

Zhe Wang↗
DOE OSTI2024

Accurate and Data‐Efficient Micro X‐ray Diffraction Phase Identification Using Multitask Learning: Application to Hydrothermal Fluids

Traditional analysis of highly distorted micro X‐ray diffraction (μ‐XRD) patterns from hydrothermal fluid environments is a time‐consuming process, often requiring substantial data preprocessing and labeled experimental data. Herein, the potential of deep learning with a multitask learning (MTL) architecture to overcome these limitations is demonstrated. MTL models are trained to identify phase information in μ‐XRD patterns, minimizing the need for labeled experimental data and masking preprocessing steps. Notably, MTL models show superior accuracy compared to binary classification convolutional neural networks. Additionally, introducing a tailored cross‐entropy loss function improves MTL model performance. Most significantly, MTL models tuned to analyze raw and unmasked XRD patterns achieve close performance to models analyzing preprocessed data, with minimal accuracy differences. This work indicates that advanced deep learning architectures like MTL can automate arduous data handling tasks, streamline the analysis of distorted XRD patterns, and reduce the reliance on labor‐intensive experimental datasets.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2016

A Quantum-Assisted Algorithm for Sampling Applications in Machine Learning

An increase in the efficiency of sampling from Boltzmann distributions would have a significant impact in deep learning and other machine learning applications. Recently, quantum annealers have been proposed as a potential candidate to speed up this task, but several limitations still bar these state-of-the-art technologies from being used effectively. One of the main limitations is that, while the device may indeed sample from a Boltzmann-like distribution, quantum dynamical arguments suggests it will do so with an instance-dependent effective temperature, different from the physical temperature of the device. Unless this unknown temperature can be unveiled, it might not be possible to effectively use a quantum annealer for Boltzmann sampling. In this talk, we present a strategy to overcome this challenge with a simple effective-temperature estimation algorithm. We provide a systematic study assessing the impact of the effective temperatures in the learning of a kind of restricted Boltzmann machine embedded on quantum hardware, which can serve as a building block for deep learning architectures. We also provide a comparison to k-step contrastive divergence (CD-k) with k up to 100. Although assuming a suitable fixed effective temperature also allows to outperform one step contrastive divergence (CD-1), only when using an instance-dependent effective temperature we find a performance close to that of CD-100 for the case studied here. We discuss generalizations of the algorithm to other more expressive generative models, beyond restricted Boltzmann machines.

Perdomo-Ortiz, Alejandro↗
DOE OSTI2021

Machine Learning Predictions of Transition Probabilities in Atomic Spectra

Forward modeling of optical spectra with absolute radiometric intensities requires knowledge of the individual transition probabilities for every transition in the spectrum. In many cases, these transition probabilities, or Einstein A-coefficients, quickly become practically impossible to obtain through either theoretical or experimental methods. Complicated electronic orbitals with higher order effects will reduce the accuracy of theoretical models. Experimental measurements can be prohibitively expensive and are rarely comprehensive due to physical constraints and sheer volume of required measurements. Due to these limitations, spectral predictions for many element transitions are not attainable. In this work, we investigate the efficacy of using machine learning models, specifically fully connected neural networks (FCNN), to predict Einstein A-coefficients using data from the NIST Atomic Spectra Database. For simple elements where closed form quantum calculations are possible, the data-driven modeling workflow performs well but can still have lower precision than theoretical calculations. For more complicated nuclei, deep learning emerged more comparable to theoretical predictions, such as Hartree–Fock. Unlike experiment or theory, the deep learning approach scales favorably with the number of transitions in a spectrum, especially if the transition probabilities are distributed across a wide range of values. It is also capable of being trained on both theoretical and experimental values simultaneously. In addition, the model performance improves when training on multiple elements prior to testing. The scalability of the machine learning approach makes it a potentially promising technique for estimating transition probabilities in previously inaccessible regions of the spectral and thermal domains on a significantly reduced timeline.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2023

Data-driven prediction of α IIb β 3 integrin activation paths using manifold learning and deep generative modeling

The integrin heterodimer is a transmembrane protein critical for driving cellular process and is a therapeutic target in the treatment of multiple diseases linked to its malfunction. Activation of integrin involves conformational transitions between bent and extended states. Some of the conformations that are intermediate between bent and extended states of the heterodimer have been experimentally characterized, but the full activation pathways remain unresolved both experimentally due to their transient nature and computationally due to the challenges in simulating rare barrier crossing events in these large molecular systems. An understanding of the activation pathways can provide new fundamental understanding of the biophysical processes associated with the dynamic interconversions between bent and extended states and unveil new putative therapeutic targets. In this work, we apply nonlinear manifold learning to coarse-grained molecular dynamics simulations of bent, extended, and two intermediate states of αI I b β3 integrin to learn a low-dimensional embedding of the configurational phase space. We then train deep generative models to learn an inverse mapping between the low-dimensional embedding and high-dimensional molecular space and use these models to interpolate the molecular configurations constituting the activation pathways between the experimentally characterized states. Furthermore, this work furnishes plausible predictions of integrin activation pathways and reports a generic and transferable multi-scale technique to predict transition pathways for biomolecular systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Data-Driven Day-Ahead PV Estimation Using Autoencoder-LSTM and Persistence Model

Inherent variability in photovoltaic (PV) and associated impacts on power systems is a challenging problem for both the PV owners and the grid operators. Existing statistical and machine learning algorithms typically work well for weather conditions similar to historical data. Furthermore, uncertain weather conditions pose a great challenge to the estimation accuracy of the estimation models. With the enhanced integration of intelligent electronic devices and the realization of associated automation in the power grid, renewable energy data is becoming more accessible, which can be utilized by deep learning models and improve the PV power generation estimation accuracy. In this paper, a hybrid deep learning model driven by external weather data is proposed to do day-ahead PV output forecasting at 15-minute-interval. The proposed model is motivated by the recent advancement of Long-Short-Term-Memory (LSTM) networks and AutoEncoder (AE), which estimates uncertainties in sequence while making the prediction for complex weather conditions. Meanwhile, the persistence model (PM) is used to predict continuous sunny weather conditions. The forecasting result is validated with data from multiple locations

42 ENGINEERING↗
DOE OSTI2022

NaroNet: Discovery of tumor microenvironment elements from highly multiplexed images

Understanding the spatial interactions between the elements of the tumor microenvironment -i.e. tumor cells. fibroblasts, immune cells- and how these interactions relate to the diagnosis or prognosis of a tumor is one of the goals of computational pathology. We present NaroNet, a deep learning framework that models the multi-scale tumor microenvironment from multiplex-stained cancer tissue images and provides patient-level interpretable predictions using a seamless end-to-end learning pipeline. Trained only with multiplex-stained tissue images and their corresponding patient-level clinical labels, NaroNet unsupervisedly learns which cell phenotypes, cell neighborhoods, and neighborhood interactions have the highest influence to predict the correct label. To this end, NaroNet incorporates several novel and state-of-the-art deep learning techniques, such as patch-level contrastive learning, multi-level graph embeddings, a novel max-sum pooling operation, or a metric that quantifies the relevance that each microenvironment element has in the individual predictions. We validate NaroNet using synthetic data simulating multiplex-immunostained images where a patient label is artificially associated to the -adjustable- probabilistic incidence of different microenvironment elements. We then apply our model to two sets of images of human cancer tissues: 336 seven-color multiplex-immunostained images from 12 high-grade endometrial cancer patients; and 382 35-plex mass cytometry images from 215 breast cancer patients. In both synthetic and real datasets, NaroNet provides outstanding predictions of relevant clinical information while associating those predictions to the presence of specific microenvironment elements.

60 APPLIED LIFE SCIENCES↗
NASA NTRS2022

Hybrid Model Based Approaches for Systems Health Management and Prognostics

This is a previously approved and published presentation. To accurately predict the future state of any system, it is required to possess knowledge of its current health state and future operational conditions. Present achievements of data-driven algorithms in regression of complex nonlinear functions and classification tasks have generated a growing interest in artificial intelligence for industrial applications. Complex multi-physics models as well as digital twins, once purely built on physics and corresponding simplified lumped parameter iterations, can now benefit from machine learning algorithms to mitigate the lack of understanding of some complex behavior. Given models of the current and future system behavior, a general approach of model-based prognostics can solve the prediction problem and further decision-making. In principle, data-driven approaches can replace expensive experimental test-setups as well as reduce the number of simulations needed to explore, e.g., the parametric space of a multi-parameter model. Nonetheless, the limitations of pure data-driven methods came to light rather quickly, at least for some industries. In many industrial applications, data acquisition is costly, and the volume of data that can be collected does not satisfy the requirements for effective model training and cross-validation. Therefore, some recent works in the area of machine learning is focusing on blending physics with data-driven algorithms, thus mitigating the drawbacks of the two approaches and emphasizing respective advantages. Partial physical knowledge of the problem can aid the learning process by “guiding” the algorithm towards efficient solutions that satisfy the physics driving the system behavior. The result is a hybrid modeling approach combining physical knowledge as well data-driven methods to develop a unified hybrid approach. A hybrid framework for fusing information from physics-based performance models along with deep learning algorithms for prognostics of complex safety-critical systems is presented. In this framework, physics-based performance models infer unobservable model parameters related to the system's components health solving a calibration problem in the deep learning approach.

Prognostics↗
DOE OSTI2024

Distributed-Memory Sparse Deep Neural Network Inference Using Global Arrays

Partitioned Global Address Space (PGAS) models exhibit tremendous promise in developing efficient and productive distributed-memory parallel applications. They have been used extensively in scientific computations due to conveniently offering a ``shared-memory''-like model and convenient interfaces that separate communication with synchronization. Traditionally, PGAS communication models have been applied to dense/contiguously distributed data, but most modern applications depict varied levels of sparsity. Existing PGAS models require certain adaptations to support distributed sparse computations, since associated computations often require matrix arithmetic, in addition to data movement. The Global Arrays toolkit from Pacific Northwest National Laboratory (PNNL) is one of the earliest PGAS models to combine one-sided data communication and distributed matrix operations and is still used in the popular NWChem quantum chemistry suite. Recently, we have expanded the Global Arrays toolkit to support common sparse operations, like sparse matrix-dense matrix multiplies (SpMM), sparse matrix-sparse matrix multiplication (SpGEMM) and Sampled Dense-Dense Matrix Multiplication (SDDMM). As it turns out, these operations are the bedrock of sparse Deep Learning (DL); sparse deep neural networks and Graph Neural Networks (GNNs) have gained increasing attention recently in achieving speedups on training and inference with reduced memory footprints. Unlike scientific applications in High Performance Computing (HPC), modern (distributed-memory capable) DL toolkits often rely on non-standardized and closed-source vendor software optimizations, creating challenges in software-hardware co-design at scale. Our goal is to support a variety of distributed-memory sparse matrix operations and helper functions in the newly created Sparse Global Arrays (SGA), such that it is possible to build portable and productive Machine Learning scenarios for algorithm/software and hardware codesign purposes. Contemporary data-parallel schemes for training/inference are undergoing a major overhaul since model replication limits scalability and causes resource inefficiencies. As such, we have adopted tensor parallelism in decomposing the model and inputs, to mitigate memory issues. Current implementation is built on top of MPI and uses CPUs to maximize the portability across the platforms.

Distributed computing, machine learning↗
DOE OSTI2026

Federated learning for 2D synchrotron x-ray diffractometry: a cross-institutional approach for phase quantification of Ti–6Al–4V alloy

High-energy Two dimensional (2D) synchrotron x-ray diffractometry provides important insights into the atomistic structure and phase evolution of materials, yet traditional analysis methods remain complex, knowledge-intensive, and computationally demanding. Deep-learning models offer a powerful alternative for automating their analysis. Institutions that hold these datasets may be unwilling to share their data due to privacy and security policies, as well as the challenges associated with large-scale data transfer. As a result, models trained on local datasets often perform well only on their own data but exhibit bias and poor generalization across different instruments or facilities. To overcome these limitations, we explore federated learning (FL) for 2D synchrotron diffractograms, enabling collaborative model training without exchanging raw data. In this study, 2D synchrotron diffractograms of Ti–6Al–4V alloy collected from two independent facilities are used to train convolutional neural networks for predicting the β-phase volume fraction. Experimental results show that federated global models significantly outperform locally trained models in terms of generalization and achieve accuracy comparable to centralized trained models. These findings demonstrate the potential of FL to enable secure, cross-institutional collaboration and enhance the scalability of deep-learning-based materials characterization.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Understanding and Leveraging the I/O Patterns of Emerging Machine Learning Analytics

The scientific community is currently experiencing unprecedented amounts of data generated by cutting-edge science facilities. Soon facilities will be producing up to 1 PB/s which will force scientist to use more autonomous techniques to learn from the data. The adoption of machine learning methods, like deep learning techniques, in large-scale workflows comes with a shift in the workflow’s computational and I/O patterns. These changes often include iterative processes and model architecture searches, in which datasets are analyzed multiple times in different formats with different model configurations in order to find accurate, reliable and efficient learning models. This shift in behavior brings changes in I/O patterns at the application level as well at the system level. These changes also bring new challenges for the HPC I/O teams, since these patterns contain more complex I/O workloads. In this paper we discuss the I/O patterns experienced by emerging analytical codes that rely on machine learning algorithms and highlight the challenges in designing efficient I/O transfers for such workflows. We comment on how to leverage the data access patterns in order to fetch in a more efficient way the required input data in the format and order given by the needs of the application and how to optimize the data path between collaborative processes. We will motivate our work and show performance gains with a study case of medical applications.

Gainaru, Ana↗
DOE OSTI2024

Physics informed neural network can retrieve rate and state friction parameters from acoustic monitoring of laboratory stick-slip experiments

Various machine learning (ML) and deep learning (DL) techniques have been recently applied to the forecasting of laboratory earthquakes from friction experiments. The magnitude and timing of shear failures in stick-slip cycles are predicted using features extracted from the recorded ultrasonic or acoustic emission (AE) signals. In addition, the Rate and State Friction (RSF) constitutive laws are extensively used to model the frictional behavior of faults. In this work, we use data from shear experiments coupled with passive acoustic (variance, kurtosis, and AE rate) interleaved with active source ultrasonic monitoring (transmitted wave amplitude) to develop physics-informed neural network (PINN) models incorporating the RSF law and AE rate generation equation with wave amplitude serving as a proxy for friction state variable. This PINN framework allows learning RSF parameters from stick-slip experiments rather than measuring them through a series of velocity step experiments. We observe that when the stick-slip cycles are irregular, the PINN models outperform the data-driven DL models. Transfer learning (TL) PINN models are also developed by pre-training on data collected at one normal stress level followed by forecasting shear failures and retrieving RSF parameters at other stress levels (i.e., with different recurrence intervals) after retraining on a limited amount of new data. Our findings suggest that TL models perform better compared to standalone models. Both standalone and TL PINN-estimated RSF parameters and their ground truth values show excellent agreements thus demonstrating that RSF parameters can be retrieved from laboratory stick-slip experiments using the corresponding acoustic data and that the transmitted wave amplitude provides a good representation of the evolving frictional state during stick-slips.

58 GEOSCIENCES↗
DOE OSTI2021

Physics guided machine learning using simplified theories

Recent applications of machine learning, in particular deep learning, motivate the need to address the generalizability of the statistical inference approaches in physical sciences. In this Letter, we introduce a modular physics guided machine learning framework to improve the accuracy of such data-driven predictive engines. The chief idea in our approach is to augment the knowledge of the simplified theories with the underlying learning process. To emphasize their physical importance, our architecture consists of adding certain features at intermediate layers rather than in the input layer. To demonstrate our approach, we select a canonical airfoil aerodynamic problem with the enhancement of the potential flow theory. We include the features obtained by a panel method that can be computed efficiently for an unseen configuration in our training procedure. By addressing the generalizability concerns, our results suggest that the proposed feature enhancement approach can be effectively used in many scientific machine learning applications, especially for the systems where we can use a theoretical, empirical, or simplified model to guide the learning module.

42 ENGINEERING↗