Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Deep Operator Networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Ultrafast Bragg coherent diffraction imaging of epitaxial thin films using deep complex-valued neural networks

Abstract Domain wall structures form spontaneously due to epitaxial misfit during thin film growth. Imaging the dynamics of domains and domain walls at ultrafast timescales can provide fundamental clues to features that impact electrical transport in electronic devices. Recently, deep learning based methods showed promising phase retrieval (PR) performance, allowing intensity-only measurements to be transformed into snapshot real space images. While the Fourier imaging model involves complex-valued quantities, most existing deep learning based methods solve the PR problem with real-valued based models, where the connection between amplitude and phase is ignored. To this end, we involve complex numbers operation in the neural network to preserve the amplitude and phase connection. Therefore, we employ the complex-valued neural network for solving the PR problem and evaluate it on Bragg coherent diffraction data streams collected from an epitaxial La 2-x Sr x CuO 4 (LSCO) thin film using an X-ray Free Electron Laser (XFEL). Our proposed complex-valued neural network based approach outperforms the traditional real-valued neural network methods in both supervised and unsupervised learning manner. Phase domains are also observed from the LSCO thin film at an ultrafast timescale using the complex-valued neural network.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2025

Deep-learning-enhanced assessment of wellbore barrier effectiveness in geologic storage systems with intermediate aquifers

For geologic systems where carbon dioxide (CO 2 ) is injected underground, existing wells represent potential pathways for fluid migration. Here, this study introduces a novel deep learning model to quantify the likelihood and potential magnitude of fluid migration through wellbores at sites with intermediate aquifers or thief zones between the injection units and underground drinking water sources. Synthetic datasets, generated using reservoir simulations, captured a wide range of subsurface conditions, well attributes, operational parameters, and fluid migration scenarios. Among the regression models developed to predict brine and CO 2 leakage rates and CO 2 saturations along leaky wellbores, convolutional neural network (CNN) outperformed both Light Gradient Boosting Machine and deep neural network. Additionally, a CNN-based classification model was created to predict whether brine and CO 2 would leak along a wellbore, further improving performance over regression alone. The best models were integrated into the National Risk Assessment Partnership Open-source Integrated Assessment Model for rapid, stochastic assessment of storage system containment and leakage risks. A case study demonstrated the model’s ability to simulate fluid migration through existing wells with multiple intermediate aquifers. This computationally efficient wellbore model offers value in support of site performance evaluation and risk-informed decision making by stakeholders.

CO2 leakage↗
DOE OSTI2021

Hybrid Imitation Learning for Real-Time Service Restoration in Resilient Distribution Systems

Self-healing capability is a critical factor for a resilient distribution system, which requires intelligent agents to automatically perform service restoration online, including network reconfiguration and reactive power dispatch. Here, the article proposes the imitation learning framework for training such an agent, where the agent will interact with an expert built based on the mixed-integer program to learn its optimal policy, and therefore significantly improve the training efficiency compared with exploration-dominant reinforcement learning (RL) methods. This significantly improved training efficiency makes the training problem under N-k scenarios tractable. A hybrid policy network is proposed to handle tie-line operations and reactive power dispatch simultaneously to further improve the restoration performance. The 33-bus and 119-bus systems with N-k disturbances are employed to conduct the training. The results indicate that the proposed method outperforms traditional RL algorithms such as the deep-Q network.

42 ENGINEERING↗
DOE OSTI2020

Approximating Trajectory Constraints With Machine Learning – Microgrid Islanding With Frequency Constraints

Here, we introduce deep earning aided constraint encoding to tackle the frequency-constraint microgrid scheduling problem. The nonlinear function between system operating condition and frequency nadir is approximated by using a neural network, which admits an exact mixed-integer formulation (MIP). This formulation is then integrated with the scheduling problem to encode the frequency constraint. With the stronger representation power of the neural network, the resulting commands can ensure adequate frequency response in a realistic setting in addition to islanding success. The proposed method is validated on a modified 33-node system. Successful islanding with a secure response is simulated under the scheduled commands using a detailed three-phase model in Simulink. The advantages of our model are particularly remarkable when the inertia emulation functions from wind turbine generators are considered.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Numerical modeling based machine learning approach for the optimization of falling - film evaporator in thermal desalination application

Scale formation that drastically increases thermal resistance and reduces freshwater production remains a critical challenge in thermal desalination. Novel designs of falling film evaporator and optimal operating condition hold great promise to mitigate scale formation, and increase heat transfer performance and fresh water production. In this work, CFD simulation based machine learning and multi-objective optimization are performed to identify optimal conditions and tube arrangement for evaporator. Non-dominated sorting genetic algorithm is adopted to determine and analyze the optimal pareto front for multiple objectives in desalination criteria. The errors of training, validation, and testing set are computed to identify an optimal hyperparameter set. For performance ratio, fouling resistance, and water production rate, the average relative error is 2.26%, 3.67%, and 3.24%. At pareto front, both performance ratio and water production rate increase at high temperature with fouling resistance (thermal resistance of the fouling layer) increasing as well. Tradeoffs between mitigating scale formation and enhancing desalination performance are evaluated in optimizations for different objectives. Finally, potential optima are identified and can be applied as guidelines to determine evaporator design and system operating conditions.

42 ENGINEERING↗
DOE OSTI2025

Continuous operation of a coherent 3,000-qubit system

Neutral atoms are a promising platform for quantum science, enabling advances in areas ranging from quantum simulations and computation to metrology, atomic clocks and quantum networking. Although atom losses typically limit these systems to a pulsed mode, continuous operation could substantially enhance cycle rates, remove bottlenecks in metrology and enable deep-circuit quantum evolution through quantum error correction. Here we demonstrate an experimental architecture for high-rate reloading and continuous operation of a large-scale atom-array system while realizing coherent storage and manipulation of quantum information. Our approach utilizes a series of two optical lattice conveyor belts to transport atom reservoirs into the science region, where atoms are repeatedly extracted into optical tweezers without affecting the coherence of qubits stored nearby. Using a reloading rate of 300,000 atoms in tweezers per second, we create over 30,000 initialized qubits per second, which we leverage to assemble and maintain an array of over 3,000 atoms for more than 2 hours. Furthermore, we demonstrate persistent refilling of the array with atomic qubits in either a spin-polarized or a coherent superposition state while preserving the quantum state of stored qubits. Our results pave the way for the realization of large-scale continuously operated atomic clocks, sensors and fault-tolerant quantum computers.

atomic and molecular physics↗
DOE OSTI2023

Solving multiphysics-based inverse problems with learned surrogates and constraints

Abstract Solving multiphysics-based inverse problems for geological carbon storage monitoring can be challenging when multimodal time-lapse data are expensive to collect and costly to simulate numerically. We overcome these challenges by combining computationally cheap learned surrogates with learned constraints. Not only does this combination lead to vastly improved inversions for the important fluid-flow property, permeability, it also provides a natural platform for inverting multimodal data including well measurements and active-source time-lapse seismic data. By adding a learned constraint, we arrive at a computationally feasible inversion approach that remains accurate. This is accomplished by including a trained deep neural network, known as a normalizing flow, which forces the model iterates to remain in-distribution, thereby safeguarding the accuracy of trained Fourier neural operators that act as surrogates for the computationally expensive multiphase flow simulations involving partial differential equation solves. By means of carefully selected experiments, centered around the problem of geological carbon storage, we demonstrate the efficacy of the proposed constrained optimization method on two different data modalities, namely time-lapse well and time-lapse seismic data. While permeability inversions from both these two modalities have their pluses and minuses, their joint inversion benefits from either, yielding valuable superior permeability inversions and CO 2 plume predictions near, and far away, from the monitoring wells.

Yin, Ziyi (ORCID:0000000250248771)↗
DOE OSTI2023

Scalable Graph Analytics and HPC Operational Enhancement: Parallel Computing and ML/DL Innovations

Parallel computing plays a pivotal role in the efficient processing of large-scale graphs. Complex network analysis stands as a capti- vating research frontier, holding promise across diverse scientific domains such as sociology, biology, online media, and recommenda- tion systems. In this era, Machine Learning (ML) and Deep Learning (DL) have emerged as indispensable tools, underpinning remarkable technological achievements. Within this dynamic landscape, my research revolves around advancing parallel algorithms tailored for large-scale graph operations. To achieve this, I harness the power of cutting-edge technologies including OpenMP, MPI, HIP, and CUDA, on the High-Performance Computing (HPC) platforms to unlock optimal performance. I also apply ML/DL techniques to HPC operational data, to streamline the monitoring and maintenance of supercomputers, alleviating the complexities associated with their upkeep and enhancing user support. My research echoes the syn- ergy between parallel computing, large-scale graph analysis, and ML/DL, improving computational efficiency and user experience.

Sattar, Naw Safrin↗
DOE OSTI2023

Recurrent Convolutional Deep Neural Networks for Modeling Time-Resolved Wildfire Spread Behavior

The increasing incidence and severity of wildfires underscores the necessity of accurately predicting their behavior. While high-fidelity models derived from first principles offer physical accuracy, they are too computationally expensive for use in real-time fire response. Low-fidelity models sacrifice some physical accuracy and generalizability via the integration of empirical measurements, but enable real-time simulations for operational use in fire response. Machine learning techniques have demonstrated the ability to bridge these objectives by learning first-principles physics while achieving computational speedups. While deep learning approaches have demonstrated the ability to predict wildfire propagation over large time periods, time-resolved fire-spread predictions are needed for active fire management. Here, in this work, we evaluate the ability of deep learning approaches in accurately modeling the time-resolved dynamics of wildfires. We use an autoregressive process in which a convolutional recurrent deep learning model makes predictions that propagate a wildfire over 15 min increments. We apply the model to four simulated datasets of increasing complexity, containing both field fires with homogeneous fuel distribution as well as real-world topologies sampled from the California region of the United States. We show that even after 100 autoregressive predictions representing more than 24 h of simulated fire spread, the resulting models generate stable and realistic propagation dynamics, achieving a Jaccard score between 0.89 and 0.94 when predicting the resulting fire scar. The inference time of the deep learning models are examined and compared, and directions for future work are discussed.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

A physics-informed deep learning description of Knudsen layer reactivity reduction

A physics-informed neural network (PINN) is used to evaluate the fast ion distribution in the hot spot of an inertial confinement fusion target. The use of tailored input and output layers to the neural network is shown to enable a PINN to learn the parametric solution to the Vlasov–Fokker–Planck equation in the absence of any synthetic or experimental data. As an explicit demonstration of the approach, the specific problem of Knudsen layer fusion yield reduction is treated. Here, the predictions from the Vlasov–Fokker–Planck PINN are used to provide a non-perturbative solution of the fast ion tail in the vicinity of the hot spot, thus allowing the spatial profile of the fusion reactivity to be evaluated for a range of collisionalities and hot spot conditions. Excellent agreement is found between the predictions of the Vlasov–Fokker–Planck PINN and the results from traditional numerical solvers with respect to both the energy and spatial distribution of fast ions and the fusion reactivity profile, demonstrating that the Vlasov–Fokker–Planck PINN provides an accurate and efficient means of determining the impact of Knudsen layer yield reduction across a broad range of plasma conditions.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

An LSTM Deep Learning Network for ¿Background Radiation Prediction

Determination of appropriate background radiation is important in any measurement application. Environmental radiation monitors and monitors used to assess dose to individuals outside of controlled areas are particularly susceptible to changes in readings due to fluctuations in the environmental conditions. These fluctuations (e.g. radon progeny concentrations) lead to changes in the detector response in the actual radiation environment, and they need to be taken into account when extracting the net operational doses. Work has been ongoing to apply advances in Deep Learning and Artificial Intelligence to account for changes in detector responses based on environmental parameters; in particular, a Long-Short Term Memory (LSTM) Deep Learning architecture has been utilized to incorporate time-series data into a prediction model. In this work, we present the current status of the project to predict radiation measurements based on meteorological conditions and air packet trajectories extracted using the National Oceanic and Atmospheric Administration's (NOAA) Hybrid Single-Particle Lagrangian Integrated Trajectory (HYSPLIT4) model.

Degtiarenko, Pavel↗
DOE OSTI2024

Cybersecurity for the Operational Technology Environment (CyOTE) (Final Technical Report)

Electric grids have historically been susceptible to both physical attacks and environmental hazards but the implementation of smart grids, remote management, and self-healing networks, has now made the grid vulnerable to cyber attacks. To address risks introduced by routable connectivity, utilities must establish dynamic solutions to identify, protect, detect, respond to, and recover from cyber security threats and vulnerabilities. In response to the evolving threat landscape U.S. Department of Energy-Office of Cybersecurity, Energy Security, and Emergency Response (DOE CESER) initiated the Cybersecurity for the OT Environment (CyOTE) pilot program, a U.S. Department of Energy (DOE) effort designed to leverage U.S. intelligence capabilities to prevent, detect, or mitigate a cyber attack on utility operational technology (OT) networks. As part of the CyOTE pilot, The Southern Company (Southern Company or Southern) researched, evaluated and deployed emerging Commercial off the Shelf (COTS) technologies and cyber security monitoring architectures to provide previously unrealized network visibility and situational awareness through deep packet inspection and data analytics. This Final Scientific/Technical Report documents the objectives, methodology, lessons learned, and results of Southern Company’s participation in the CyOTE pilot from December 2018 to September 2023.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Hydrogen and Battery Storage Technologies for Low-Cost Energy Decarbonization in Distribution Networks

Deep energy decarbonization cannot be achieved without high penetration of renewables. At higher renewable energy penetrations, the variability and intermittent nature of solar photovoltaic (PV) electricity can cause ramping issues with existing fossil fuel generation, requiring longer term energy storage to increase the reliability of grid operation. Here, a proton exchange membrane electrolyzer can produce H 2 and serves as a utility controllable load. The produced H 2 can then be stored and converted back into electricity, or mixed with natural gas, or used as transportation fuel, or chemical feedstock. This paper considers the perspective of the distribution system operator that operates the distributed energy resources on a standard IEEE 33-node distribution network considering the technical and physical constraints with the goal of minimizing total investment and operation cost. Different case studies, at very high PV penetrations are considered to show the challenges and path to net-zero emission energy production using H 2 energy. Sensitivity of utility PV costs and electrolyzer capital costs on producing H 2 at $\$1$ /kg are presented showing that the distribution network could produce 100% renewable electricity and H 2 could be produced at a cost of $\$1$ /kg by 2050 with conservative cost estimates and by 2030 with accelerated cost declines.

25 ENERGY STORAGE↗
DOE OSTI2022

Deep Reinforcement Learning From Demonstrations to Assist Service Restoration in Islanded Microgrids

Microgrids can be operated in island mode during utility grid outages to support service restoration and improve system resilience. To schedule and dispatch distributed energy resources (DERs) in an islanded microgrid, conventional model-based methods rely on accurate distribution network models and lack generalization and adaptability. Data-driven methods are promising for DER coordination but face practical challenges such as potential hazards to microgrids during online training and insufficient online training opportunities due to low outage rates. This paper presents a novel two-stage learning framework that builds on the deep deterministic policy gradient from demonstrations to identify an optimal restoration strategy. At the pre-training stage, imitation learning is applied to equip the control agent with expert experiences to guarantee acceptable initial performance. At the online training stage, action clipping, reward shaping, and expert demonstrations are leveraged to ensure safe exploration while accelerating the training process. In conclusion, the proposed method is illustrated using the IEEE 123-node system and compared with a representative model-based method and the standard deep deterministic policy gradient method to prove solution accuracy and demonstrate increased computational efficiency.

Du, Yan↗
DOE OSTI2022

PRODeepSyn: predicting anticancer synergistic drug combinations by embedding cell lines with protein–protein interaction network

Abstract Although drug combinations in cancer treatment appear to be a promising therapeutic strategy with respect to monotherapy, it is arduous to discover new synergistic drug combinations due to the combinatorial explosion. Deep learning technology holds immense promise for better prediction of in vitro synergistic drug combinations for certain cell lines. In methods applying such technology, omics data are widely adopted to construct cell line features. However, biological network data are rarely considered yet, which is worthy of in-depth study. In this study, we propose a novel deep learning method, termed PRODeepSyn, for predicting anticancer synergistic drug combinations. By leveraging the Graph Convolutional Network, PRODeepSyn integrates the protein–protein interaction (PPI) network with omics data to construct low-dimensional dense embeddings for cell lines. PRODeepSyn then builds a deep neural network with the Batch Normalization mechanism to predict synergy scores using the cell line embeddings and drug features. PRODeepSyn achieves the lowest root mean square error of 15.08 and the highest Pearson correlation coefficient of 0.75, outperforming two deep learning methods and four machine learning methods. On the classification task, PRODeepSyn achieves an area under the receiver operator characteristics curve of 0.90, an area under the precision–recall curve of 0.63 and a Cohen’s Kappa of 0.53. In the ablation study, we find that using the multi-omics data and the integrated PPI network’s information both can improve the prediction results. Additionally, the case study demonstrates the consistency between PRODeepSyn and previous studies.

Wang, Xiaowen↗
DOE OSTI2024

Extraction of Drell-Yan Angular Parameters in $pp$ Collisions with a 120 GeV Beam Energy Using a Deep-Learning Unfolding Algorithm

Dilepton production in pp collisions through the Drell-Yan process provides a crucial tool for studying the internal quark-gluon structure of the nucleon. By precisely measuring the $\cos2\phi$ asymmetry, where $\phi$ represents the azimuthal angle of the $l^{+}l^{-}$ pair in the Collins-Soper frame, we can gain valuable insights into the proton’s structure and the transverse momentum ($q_{T}$) dependence of the $\cos2\phi$ asymmetry. SeaQuest, a fixed-target Drell-Yan experiment at Fermilab, involved an unpolarized proton beam colliding with unpolarized LH$_{2}$ and LD$_{2}$ targets. Measurements obtained from experiments typically require corrections for detector inefficiencies, smearing, and acceptance. Traditionally, these corrections involve “unfolding” the detector-level measurements through matrix operations. However, in higher-dimensional phase space, these conventional methods fail to scale effectively. To overcome these limitations, we employ an unbinned unfolding method that utilizes deep neural networks for unfolding higher-dimensional phase space. In this presentation, we will explain the design of the neural network architecture, our training strategies, and outline our plans to achieve conclusive results.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Permeability Prediction Using Vision Transformers

Accurate permeability predictions remain pivotal for understanding fluid flow in porous media, influencing crucial operations across petroleum engineering, hydrogeology, and related fields. Traditional approaches, while robust, often grapple with the inherent heterogeneity of reservoir rocks. With the advent of deep learning, convolutional neural networks (CNNs) have emerged as potent tools in image-based permeability estimation, capitalizing on micro-CT scans and digital rock imagery. This paper introduces a novel paradigm, employing vision transformers (ViTs)—a recent advancement in computer vision—for this crucial task. ViTs, which segment images into fixed-sized patches and process them through transformer architectures, present a promising alternative to CNNs. We present a methodology for implementing ViTs for permeability prediction, its results on diverse rock samples, and a comparison against conventional CNNs. The prediction results suggest that, with adequate training data, ViTs can match or surpass the predictive accuracy of CNNs, especially in rocks exhibiting significant heterogeneity. This study underscores the potential of ViTs as an innovative tool in permeability prediction, paving the way for further research and integration into mainstream reservoir characterization workflows.

58 GEOSCIENCES↗
DOE OSTI2023

Galaxy Morphology Classification Using Bayesian Neural Networks for LSST

Within the decade, many new ground and space-based observatories will become operational, generating massive amounts of data on short timescales. New surveys like Rubin Observatory's Legacy Survey of Space and Time (LSST) will be capable of observing objects with greater resolution than ever before, but processing and analyzing these datasets optimally will pose a significant challenge. In an effort to prepare for this, we explore how incorporating Deep Neural Networks can better support future data-intensive Astrophysics tasks such as galaxy morphology classification.

79 ASTRONOMY AND ASTROPHYSICS↗