SEARCH · Engineering Papers

Results for “Deep neural operator”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

All sources NASA NTRS DOE OSTI

DOE OSTI2020

nPINNs: Nonlocal physics-informed neural networks for a parametrized nonlocal universal Laplacian operator. Algorithms and applications

Physics-informed neural networks (PINNs) are effective in solving inverse problems based on differential and integro-differential equations with sparse, noisy, unstructured, and multifidelity data. PINNs incorporate all available information, including governing equations (reflecting physical laws), initial-boundary conditions, and observations of quantities of interest, into a loss function to be minimized, thus recasting the original problem into an optimization problem. In this paper, we extend PINNs to parameter and function inference for integral equations such as nonlocal Poisson and nonlocal turbulence models, and we refer to them as nonlocal PINNs (nPINNs). The contribution of the paper is three-fold. First, we propose a unified nonlocal Laplace operator, which converges to the classical Laplacian as one of the operator parameters, the nonlocal interaction radius $\delta$ goes to zero, and to the fractional Laplacian as $\delta$ goes to infinity. This universal operator forms a super-set of classical Laplacian and fractional Laplacian operators and, thus, has the potential to fit a broad spectrum of data sets. We also provide theoretical convergence rates with respect to $\delta$ and verify them via numerical experiments. Second, we use nPINNs to estimate the two parameters, $\delta$ and $\alpha$, characterizing the kernel of the unified operator. The strong non-convexity of the loss function yielding multiple (good) local minima reveals the occurrence of the operator mimicking phenomenon, that is, different pairs of estimated parameters could produce multiple solutions of comparable accuracy. Third, we propose another nonlocal operator with spatially variable order $\alpha(y)$, which is more suitable for modeling turbulent Couette flow. Our results show that nPINNs can jointly infer this function as well as $\delta$. More importantly, these parameters exhibit a universal behavior with respect to the Reynolds number, a finding that contributes to our understanding of nonlocal interactions in wall-bounded turbulence.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Learning diffractive optical communication around arbitrary opaque occlusions

Abstract Free-space optical communication becomes challenging when an occlusion blocks the light path. Here, we demonstrate a direct communication scheme, passing optical information around a fully opaque, arbitrarily shaped occlusion that partially or entirely occludes the transmitter’s field-of-view. In this scheme, an electronic neural network encoder and a passive, all-optical diffractive network-based decoder are jointly trained using deep learning to transfer the optical information of interest around the opaque occlusion of an arbitrary shape. Following its training, the encoder-decoder pair can communicate any arbitrary optical information around opaque occlusions, where the information decoding occurs at the speed of light propagation through passive light-matter interactions, with resilience against various unknown changes in the occlusion shape and size. We also validate this framework experimentally in the terahertz spectrum using a 3D-printed diffractive decoder. Scalable for operation in any wavelength regime, this scheme could be particularly useful in emerging high data-rate free-space communication systems.

36 MATERIALS SCIENCE↗

DOE OSTI2020

Deep Learning for Automated Identification of Eels in Sonar Data

Freshwater eels, such as the American eel (Anguilla rostrata) present numerous challenges related to safe downstream fish passage at hydroelectric facilities. One of those challenges is effective monitoring of their abundance, movements, and behavior to facilitate design and operation of eel protection and passage facilities. A previous EPRI study documented the ability of human analysts to reliably identify American eels in data obtained with a 1100/1800 kHz, multibeam sonar. This report describes a project to develop deep learning (a subset of artificial intelligence) tools to automate the time-consuming, subjective process of eel identification in multibeam sonar data. The project exploited new data collected in the laboratory and the existing data from the prior EPRI field study to develop and test deep learning and other data analytic tools, including wavelet filtering, differencing for static object removal, and convolutional neural network analysis. The analysis of the laboratory data demonstrated feasibility of the approach, revealed object characteristics observed with the sonar that distinguish eels from similarly sized and shaped acoustic targets, and provided additional data for algorithm selection and training. Deep learning algorithms trained and tested on the laboratory data alone achieved accuracy rates of greater than 98% when classifying acoustic images of eels and similar-sized neutrally buoyant sticks. The algorithm trained and tested on the pre-existing field data alone, and yielded classification accuracy of 9.3% false positives and 13.3% false negatives when distinguishing between eels and sticks/PVC pipes based on video clips (i.e., multiple, consecutive images). This performance is comparable to the classification accuracy achieved by human analysts in the prior study. The deep learning algorithm trained on a combination of video clips obtained in the laboratory and the field and tested on video clips from the field, was able to distinguish eels from sticks and PVC pipes (a river debris analog) of similar size with 100% accuracy. Outreach to the hardware, software, and end-user communities early in the project helped to identify needs and specify the application space. Outreach to those communities at the end of the project communicated project results and opportunities for further development. The project achieved proof of concept for automated identification of eel in multibeam sonar data. Future work should focus on acquisition of additional data for more robust algorithm training and testing; modification of the software tools to accommodate multiple acoustic targets in the acoustic field at a given time; identification of additional object classes; incorporation of motion in the object identification and classification algorithms; operationalizing the software tools, including integration with other existing sonar data analysis tools; and partnering with hardware and software providers for distribution of the software tools with their commercial products.

13 HYDRO ENERGY↗

DOE OSTI2023

Machine learning-based real-time kinetic profile reconstruction in DIII-D

Abstract Kinetic equilibrium reconstruction plays a vital role in the physical analysis of plasma stability and control in fusion tokamaks. However, the traditional approach is subjective and prone to human biases. To address this, the consistent automatic kinetic equilibrium reconstruction (CAKE) method was introduced, providing objective results. Nonetheless, its offline nature limits its application in real-time plasma control systems (PCSs). To address this limitation, we present RTCAKENN, a machine learning model that approximates 7 CAKE-level output profiles, namely pressure, inverse q , toroidal current density, electron temperature and density, carbon ion impurity temperature and rotation profiles, using real-time available inputs. The deep neural network consists of an encoder layer, where the scalars and interdependent inputs such as plasma boundary coordinates and motional Stark effect data are encoded using multi-layer perceptrons (MLPs), while profile inputs are encoded by 1D convolutional layers. The encoded data is passed through a MLP for latent feature extraction, before being decoded in the decoding layers, which consist of upsampling and convolutional layers. RTCAKENN has been implemented in the DIII-D PCS and our model achieves accuracy comparable to CAKE and surpasses existing real-time alternatives. Through clever dropout training, RTCAKENN exhibits robustness and can operate even in the absence of Thomson scattering data or charge exchange recombination data. It executes in under 8 ms in the real-time environment, enabling future application in real-time control and analysis.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2025

Improving the Transportability of a Deep Learning Denoising Model Using Transfer Learning Techniques

The adoption of machine learning techniques in the seismology community has led to great performance improvements in several areas, including signal processing. Specifically, the development of deep learning–based seismic waveform denoising models has the potential to yield improvements in signal detection capabilities for networks operating in particularly noisy environments. Recent advancements in the design of these deep learning denoising models have included the incorporation of continuous and discrete wavelet transform functions into the network architecture to improve the learning capabilities and efficiency of said models. These wavelet transform–based seismic denoising models have shown improved denoising capabilities in regions where there is good agreement between the data features present in the training and evaluation datasets. However, questions remain about the overall transportability of these models to other monitoring regions. Here, in this study, we will determine the baseline transportability of a newly developed multilevel wavelet‐transform convolutional neural network (MWCNN) seismic denoising model. We accomplish this by taking a version of the MWCNN denoising model trained on data collected from the Utah region and evaluating its denoising performance on datasets collected from the neighboring Nevada region, which differ with regard to monitoring sensor types and event histories. We find that there is a notable variability in denoising performance related to the degree of similarity between the initial and new target datasets. The most notable difference in denoising performance is the ability of the denoising model to preserve accurate amplitude information associated with the signal energy present in the waveform data. Finally, we evaluate the ability of transfer learning techniques to improve the transportability of the MWCNN denoising model. We find that although there is still a performance gap present in the denoising results of the MWCNN model, transfer learning did yield improved results.

Quinones, Louis [Sandia National Laboratories (SNL↗

DOE OSTI2020

Artificial Intelligence Applications in Renewable Energy

Addressing new methodologies in deep learning (DL), machine learning (ML) and artificial intelligence (AI), the webinar speakers will provide an overview of the literature spanning these three overlapping fields as applied to energy systems research. The audience will learn how developments in these areas have added new capabilities for pattern recognition and predictive modeling that are complementary to more traditional modeling approaches used in energy systems research. The speakers will illustrate several use cases of these new approaches in the energy space, such as physics-guided neural networks to improve ML regressions for solar data and how AI agents can be used to explore power plant operations.

40 EE - Geothermal Technologies Office (EE-4G)↗

DOE OSTI2024

Efficacy of using a dynamic length representation vs. a fixed-length for neuroarchitecture search

Deep learning neuroarchitecture and hyperparameter search are important in finding the best configuration that maximizes learned model accuracy. However, the number of types of layers, their associated hyperparameters, and the myriad of ways to connect layers poses a significant computational challenge in discovering ideal model configurations. Here, we assess two different approaches for neuroarchitecture search for a LeNet style neural network, one that uses a fixed-length approach where there is a preset number of possible layers that can be toggled on or off via mutation, and a variable-length approach where layers can be freely added or removed via special mutation operators. We found that the variable-length implementation trained better models while discovering unusual layer configurations worth further exploration.

Coletti, Mark↗

DOE OSTI2022

DeepZipper: A Novel Deep-learning Architecture for Lensed Supernovae Identification

Large-scale astronomical surveys have the potential to capture data on large numbers of strongly gravitationally lensed supernovae (LSNe). To facilitate timely analysis and spectroscopic follow-up before the supernova fades, an LSN needs to be identified soon after it begins. To quickly identify LSNe in optical survey data sets, we designed ZipperNet, a multibranch deep neural network that combines convolutional layers (traditionally used for images) with long short-term memory layers (traditionally used for time series). We tested ZipperNet on the task of classifying objects from four categories—no lens, galaxy-galaxy lens, lensed Type-Ia supernova, lensed core-collapse supernova—within high-fidelity simulations of three cosmic survey data sets: the Dark Energy Survey, Rubin Observatory’s Legacy Survey of Space and Time (LSST), and a Dark Energy Spectroscopic Instrument (DESI) imaging survey. Among our results, we find that for the LSST-like data set, ZipperNet classifies LSNe with a receiver operating characteristic area under the curve of 0.97, predicts the spectroscopic type of the lensed supernovae with 79% accuracy, and demonstrates similarly high performance for LSNe 1–2 epochs after first detection. We anticipate that a model like ZipperNet, which simultaneously incorporates spatial and temporal information, can play a significant role in the rapid identification of lensed transient systems in cosmic survey experiments.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2023

Informed Feature Selection for Data Clustering of CSP Plant Production

To make concentrating solar power (CSP) more cost competitive, rigourous optimizations must be run to improve plant design and operations. However, these optimizaitons rely on time consuming annual simulations that solve an electricity dispatch scheduling problem to maximize plant revenue. To reduce the runtime of annual dispatch simulations of CSP plants, a data clustering approach is utilized. This approach assumes that like days of revenue and electricity generation can be identified using weather and price data. Although weather and price are important factors for electricity production, this work investigates how thermal energy storage (TES) inventory at the beginning of a day, denoted as Si, can be used as a supplemental feature to group like days. A framework for creating and training a deep neural network to predict Si is proposed. This model is validated and assessed using eleven sets of testing data that were not used during training. Then, the data clustering approach is performed three seperate times with features of weather and price along with either Si from the neural network, Si from the full annual simulation, or no Si. Ultimately, the results suggest that using Si as an additional clustering feature improves the data clustering simulation accuracy by 1.4%.

Tuman, Matthew J. (ORCID:000900038772051X)↗

DOE OSTI2022

Metaplastic and energy-efficient biocompatible graphene artificial synaptic transistors for enhanced accuracy neuromorphic computing

CMOS-based computing systems that employ the von Neumann architecture are relatively limited when it comes to parallel data storage and processing. In contrast, the human brain is a living computational signal processing unit that operates with extreme parallelism and energy efficiency. Although numerous neuromorphic electronic devices have emerged in the last decade, most of them are rigid or contain materials that are toxic to biological systems. In this work, we report on biocompatible bilayer graphene-based artificial synaptic transistors (BLAST) capable of mimicking synaptic behavior. The BLAST devices leverage a dry ion-selective membrane, enabling long-term potentiation, with ~50 aJ/µm 2 switching energy efficiency, at least an order of magnitude lower than previous reports on two-dimensional material-based artificial synapses. The devices show unique metaplasticity, a useful feature for generalizable deep neural networks, and we demonstrate that metaplastic BLASTs outperform ideal linear synapses in classic image classification tasks. With switching energy well below the 1 fJ energy estimated per biological synapse, the proposed devices are powerful candidates for bio-interfaced online learning, bridging the gap between artificial and biological neural networks.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

Machine Learning-Enabled Wearable Piezoelectric Acoustic Sensor for Real-Time Breast Abnormality Detection

In contemporary society, breast health has become a significant public health concern, particularly among women. According to statistics from the World Health Organization, both the incidence and mortality rates of breast tumors have steadily increased in recent years. Therefore, effective early-stage screening and postoperative monitoring are essential for maintaining breast health. However, conventional clinical diagnostic modalities are typically bulky, operationally complex, and unsuitable for continuous real-time monitoring, which limits their use in portable and everyday health management applications. To address these limitations, this study proposes a machine learning-integrated wearable piezoelectric sensing platform as an auxiliary tool for breast health assessment. The device consists of a PDMS matching layer embedded with flexible silver nanowires, a P(VDF-TrFE) piezoelectric layer, and a multi-channel low-noise signal acquisition circuit. It is capable of acquiring acoustic echo signals from tissue-mimicking environments and automatically evaluating signal validity using a convolutional neural network (CNN). By integrating piezoelectric sensing with deep learning-based signal analysis, the proposed system achieves a signal-to-noise ratio exceeding 70 dB and a real-time classification accuracy above 96% under controlled conditions. These results demonstrate that the platform provides a compact, portable, and intelligent approach for wearable sensing of mechanical heterogeneity and highlight its potential for future development in continuous biomedical monitoring technologies.

He, Shuaitong↗

DOE OSTI2020

nPINNs: nonlocal Physics-Informed Neural Networks for a parametrized nonlocal universal Laplacian operator. Algorithms and Applications

Physics-informed neural networks (PINNs) are effective in solving inverse problems based on differential and integro-differential equations with sparse, noisy, unstructured, and multifidelity data. PINNs incorporate all available information, including governing equations (reflecting physical laws), initial-boundary conditions, and observations of quantities of interest, into a loss function to be minimized, thus recasting the original problem into an optimization problem. In this paper, we extend PINNs to parameter and function inference for integral equations such as nonlocal Poisson and nonlocal turbulence models, and we refer to them as nonlocal PINNs (nPINNs). The contribution of the paper is three-fold. First, we propose a unified nonlocal Laplace operator, which converges to the classical Laplacian as one of the operator parameters, the nonlocal interaction radius δ goes to zero, and to the fractional Laplacian as δ goes to infinity. This universal operator forms a super-set of classical Laplacian and fractional Laplacian operators and, thus, has the potential to fit a broad spectrum of data sets. We provide theoretical convergence rates with respect to δ and verify them via numerical experiments. Second, we use nPINNs to estimate the two parameters, δ and α, characterizing the kernel of the unified operator. The strong non-convexity of the loss function yielding multiple (good) local minima reveals the occurrence of the operator mimicking phenomenon, that is, different pairs of estimated parameters could produce multiple solutions of comparable accuracy. Third, we propose another nonlocal operator with spatially variable order α(γ), which is more suitable for modeling turbulent Couette flow. Our results show that nPINNs can jointly infer this function as well as δ. More importantly, these parameters exhibit a universal behavior with respect to the Reynolds number, a finding that contributes to our understanding of nonlocal interactions in wall-bounded turbulence.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2023

Digital holography for real-time non-invasive monitoring of larval fish at power plant intakes

Effective evaluation of technological and operational approaches to reduce entrainment of marine organisms at cooling water intake structures (CWIS) requires accurate organism-sensing systems. Current detection methods lead to large temporal data gaps, require tedious manual analysis, and are fatal to organisms. In this report we describe integrating deep learning with a non-lethal, non-intrusive imaging method—digital holography—to rapidly detect fish larvae. Laboratory experiments demonstrated that the instrument could successfully image fish larvae at flow rates exceeding ranges seen in CWIS. Holograms of two fish larvae species, in the presence of bubbles and detritus, were collected to build a large database for training a lightweight convolutional neural network. The model achieves 97% extraction accuracy in quantifying larvae, and distinguishing them from other particles, including detritus and bubbles, when applied to a dataset of manually classified images, exceeding previous metrics for non-lethal, accurate, and real-time detection. These results demonstrate the potential of in situ holographic imaging for monitoring endangered larval fish species at power plant intake structures, and for high-fidelity, real-time applications in monitoring aquatic ichthyoplankton.

endangered species↗

DOE OSTI2024

Deep Learning-Based Failure Prognostic Model for PV Inverter Using Field Measurements

Here, this study presents a novel approach for the precise monitoring and prognosis of photovoltaic (PV) inverter status, which is crucial for the proactive maintenance of PV systems. It addresses the gaps in traditional model-based methods, which tend to neglect the overall reliability of inverters, and the limitations of data-driven approaches that largely depend on simulated data. This research presents a robust solution applicable to real-world scenarios. The proposed data-driven model for PV inverter failure prognosis employs actual inverter measurements, integrating various operational and weather-related factors based on domain knowledge. This approach effectively represents inverter stressors and operational status. Utilizing an Enhanced Siamese Convolutional Neural Network (ESCNN), the model merges operational data with domain knowledge features, redefining the prognosis challenge as a classification task. Furthermore, the paper discusses an ESCNN-based real-time inverter failure monitoring method developed on the well-trained model. The proposed models are rigorously trained and tested with real inverter data and a novel filtering method is included to address accidental failures in practical scenarios. The results validate the model's efficacy, and the directions for future research are also outlined.

42 ENGINEERING↗

DOE OSTI2021

Surrogate Model of Flexible Research Platform EnergyPlus Models to Enable Sensitivity Analysis

This letter report describes the surrogate models developed from the EnergyPlus model of Oak Ridge National Laboratory’s Flexible Research Platform. Two data-driven black-box models were developed, and the outputs of the surrogate models were compared with the EnergyPlus model. The two models developed are a multilayer perceptron deep learning model, and a long short-term memory (LSTM) neural network model. The three factors for selecting the black-box models are scalability, computation time, and accuracy. A total of 107 input variables were the dominant variables in determining the outputs of building energy consumptions and thermal comfort. A total of 54 output variables were identified as the prediction targets, including the system- and zone-level outputs. The large set of the simulation cases were generated by integrating sensor errors into an emulator based on EnergyPlus and Python EMS, which includes advanced control sequences from ASHRAE Guideline 36-2018: High-Performance Sequences of Operation. The surrogate models were developed based on a set of large-scale simulation runs (i.e., 4,000 runs) on a cloud platform. The comparison analysis shows that the two black-box models had good accuracy for predicting new outputs for sensitivity analysis using the root mean square error metric. As a next step, the developed surrogate models will be used to perform sensitivity analysis for different sensor impacts (e.g., sensor types, sensor locations).

42 ENGINEERING↗

NASA NTRSDate not supplied

AEGIS: Autonomous Entity Global Intelligence System for Urban Air Mobility

This paper presents a global intelligence system that synthesizes aerial vehicles’ real-time physical data, planned actions, and historical behavior into engineered data frames representing the collective state of the airspace and suitable for efficient machine learning consumption. These data frames are then learnt by a deep neural net to build a prediction model that estimates the expected evolution path of the current state, thereby identifying potential future conflicts. This approach lends itself to an automated early warning system that the aerial vehicles can implement onboard with a suitable edge computing module more efficiently and effectively than non-AI methods, and eventually take preventive or corrective measures towards self/collaborative resolution of the issues. Contrary to a centralized early warning system where all vehicles’ task-space eventually converges to a global optimum state, the presented distributed global intelligence system brings in a balance between local utility functions of each vehicle and the global operating framework. This contributes to effectively handle the potential massive scaling in urban air mobility in the near future.

Artificial Intelligence↗

DOE OSTI2022

A deep learning-based direct forecasting of CO 2 plume migration

Accurate and timely forecasts of CO 2 plume evolution in geological reservoirs are crucial for CO 2 migration detection, leakage risk assessment, and operation decision support. Conventional forecasting usually adopts a two-step strategy, first calibrating reservoir model parameters against observations using iterative inverse modeling (or history matching) and then applying the calibrated model for predictions. This method impedes real-time forecasts due to the heavy computational demand in inverse modeling and may suffer from poor prediction accuracy because of the limited observation data. In this work, we propose a deep learning-based latent space mapping framework to forecast CO 2 plume migration directly by avoiding the inverse modeling. We first use the convolutional autoencoder to map the high-dimensional complex plume extents onto low-dimensional latent space. Next, we use neural networks to learn the relationship between the observation variables and the prediction latent variables. And then for given observation data, we infer the prediction values directly. This one-step direct forecasting is computationally efficient which requires a few number of parallelizable reservoir simulations and it can provide accurate predictions with limited observations by learning the observation-prediction relationship in the reduced dimension. Therefore, our proposed method enables an in-time forecast of dynamic CO 2 plume distributions. In this work, we demonstrate the effectiveness and accuracy of our method in predicting the CO 2 plume migration using four metrics such as plume area, centroid movement distance, and plume spreading in the primary and secondary directions. And the spatio-temporal evolution patterns of plume migration under diverse geological complexities are also accurately quantified.

15 GEOTHERMAL ENERGY↗

DOE OSTI2025

Exact block encoding of imaginary time evolution with universal quantum neural networks

We develop a constructive approach to generate quantum neural networks capable of representing the exact thermal states of all many-body qubit Hamiltonians. The Trotter expansion of the imaginary time propagator is implemented through an exact block encoding by means of a unitary, restricted Boltzmann machine architecture. Marginalization over the hidden-layer neurons (auxiliary qubits) creates the nonunitary action on the visible layer. Then, we introduce a unitary deep Boltzmann machine architecture in which the hidden-layer qubits are allowed to couple laterally to other hidden qubits. We prove that this wave-function is closed under the action of the imaginary time propagator and, more generally, can represent the action of a universal set of quantum gate operations. We provide analytic expressions for the coefficients for both architectures, thus enabling exact network representations of thermal states without stochastic optimization of the network parameters. In the limit of large imaginary time, the yields the ground state of the system. The number of qubits grows linearly with the number of interactions and total imaginary time for a fixed interaction order. Both networks can be readily implemented on quantum hardware via midcircuit measurements of auxiliary qubits. If only one auxiliary qubit is measured and reset, the circuit depth scales linearly with imaginary time and number of interactions, while the width is constant. Alternatively, one can employ a number of auxiliary qubits linearly proportional to the number of interactions, and circuit depth grows linearly with imaginary time only. Every midcircuit measurement has a postselection success probability, and the overall success probability is equal to the product of the probabilities of the midcircuit measurements.

97 MATHEMATICS AND COMPUTING↗