Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Deep Operator Networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Meta-optic accelerators for object classifiers

Rapid advances in deep learning have led to paradigm shifts in a number of fields, from medical image analysis to autonomous systems. These advances, however, have resulted in digital neural networks with large computational requirements, resulting in high energy consumption and limitations in real-time decision-making when computation resources are limited. Here, we demonstrate a meta-optic–based neural network accelerator that can off-load computationally expensive convolution operations into high-speed and low-power optics. In this architecture, metasurfaces enable both spatial multiplexing and additional information channels, such as polarization, in object classification. End-to-end design is used to co-optimize the optical and digital systems, resulting in a robust classifier that achieves 93.1% accurate classification of handwriting digits and 93.8% accuracy in classifying both the digit and its polarization state. This approach could enable compact, high-speed, and low-power image and information processing systems for a wide range of applications in machine vision and artificial intelligence.

42 ENGINEERING↗
DOE OSTI2026

Extreme Temperature Cryptography Based On Nitrogen-Incorporated Ultrananocrystalline Diamond

Physical entropy sources that remain stable under extreme temperatures are essential for cryptography in emerging technological frontiers in deep space exploration, geothermal energy harvesting, and nuclear energy. However, conventional semiconductor platforms fail to generate stable and reliable cryptographic keys above 200 degrees C due to performance degradation. Here, we report a diamond-based cryptographic primitive that exploits the defect-rich sp 2 -bonded grain boundary network in nitrogen-incorporated ultrananocrystalline diamond (n-UNCD) film as a robust entropy source to generate cryptographic keys that remain operationally stable even after enduring extreme temperatures of 700 degrees C for 54 h while also surviving thermal cycling between room temperature and 700 degrees C for 48 h. The strength of the generated keys is assessed through several cryptographic metrics such as bit uniformity, entropy, hamming distances, and correlation coefficients, all of which are found to be near their respective ideal values. Moreover, the generated keys pass the NIST SP 800 and SP 800-90B tests and are also resilient to supply bias variations and a regression-based machine learning attack model based on the Fourier series. The robustness of the keys is attributed to the better thermal stability and chemical inertness of the n-UNCD film. This is supported by high-resolution energy-dispersive X-ray spectroscopy (EDS), which shows no significant lateral diffusion of metal atoms into the n-UNCD layer, and by Raman spectroscopy, which reveals no significant changes in the bonding configuration of the n-UNCD structure. Our findings highlight the remarkable potential of n-UNCD film for extreme environment cryptography by expanding the operational limits of conventional hardware security platforms.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Application of deep learning methods for beam size control during user operation at the Advanced Light Source

Past research at the Advanced Light Source (ALS) provided a proof-of-principle demonstration that deep learning methods could be effectively employed to compensate for the significant perturbations to the transverse electron beam size induced by user-controlled adjustments of the insertion devices. However, incorporating these methods into the ALS’ daily operations has faced notable challenges. The complexity of the system’s operational requirements and the significant upkeep demands has restricted their sustained application during user operation. Here, we introduce the development of a more robust neural network (NN)-based algorithm that utilizes a novel online fine-tuning approach and its systematic integration into the day-to-day machine operations. Our analysis emphasizes the process of NN model selection, demonstrates the superior performance of the NN-based method over traditional feedback methods, and examines the effectiveness and resilience of the new algorithm during user-operation scenarios. Published by the American Physical Society 2024

43 PARTICLE ACCELERATORS↗
DOE OSTI2023

Bayesian GAN-Based False Data Injection Attack Detection in Active Distribution Grids With DERs

Advancements in information and communication technologies have revolutionized monitoring and control capabilities within smart grids. However, it also brings new vulnerabilities to data acquisition systems and state estimation functions, which attackers can subtly tamper with the measurement data through compromising the communication network. Moreover, the high penetration of renewable energy sources with the inherited characteristics of uncertainty and variability further complicates the design of effective intrusion detection systems. In this paper, a Bayesian deep learning-based approach is developed to detect cyber attacks and maintain the security of smart grids. Our method specifically addresses the prevalent issue of imbalanced data in real power systems, which arises from the predominance of normal system operations over compromised or attacked states. Employing a novel Bayesian GAN-based technique, our approach successfully discriminates between secure and compromised measurement data, even in scenarios with significant data imbalance. Furthermore, the proposed method accommodates various practical application factors, ensuring accurate intrusion detection despite the presence of measurement noise. The feasibility and effectiveness of the proposed detection mechanism are validated by testing on IEEE 13-node and 123-node test systems. Simulation results and comparisons with literature methods demonstrate the superiority of proposed cybersecurity solutions.

Bayesian GAN↗
DOE OSTI2024

Graph neural networks for CO 2 solubility predictions in Deep Eutectic Solvents

Deep Eutectic Solvents (DESs) are a promising class of solvents for CO 2 capture. DESs are complex mixtures that can be designed to optimize CO solubility and overall capture process efficiency. However, the vast design landscape of DES mixtures makes experimental investigation prohibitive; as such, there is a need for computational models that can quickly and efficiently navigate the design space and inform data collection efforts. In this work, we propose Graph Neural Network (GNN) models for predicting CO 2 solubility for DESs; the GNN leverages a mixture graph representation that captures the molecular structure of the DES components as well as their intermolecular interactions. Here, we compare the GNN framework against alternative architectures (neural networks, graph convolution networks, and random forests) and data representations (molecular fingerprints, sigma profiles, and graphs). We show that the proposed approach offers superior predictive performance; specifically, we show that solubility can be predicted reliably directly from molecular structure (without the need of using sigma profiles as proposed in previous studies). This result is important, as obtaining sigma profiles requires expensive density functional theory computations. We also explored the ability of GNNs to predict solubility for new DES mixtures and operating conditions. We found that the model extrapolates across temperature reliably. However, we also found deficiencies in the ability of the model to predict solubility for DES mixtures, pressures, and molar ratio not included in the training sets; we show that this is due to an inherent lack of chemical diversity in datasets available in the literature. The proposed computational capabilities can thus help navigate the design space of DES and inform data collection efforts. Our models, data, and benchmarks are shared as Python code implemented in Jupyter notebooks.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Throughput-Oriented and Accuracy-Aware DNN Training with BFloat16 on GPU

Deep Neural Networks (DNNs) have transformed the field of artificial intelligence and achieved extraordinary success in many areas. The training of DNNs is commonly compute and memory-intensive, which has resulted in several optimizations in the training phase. Among them, reduced precision is a typical and widely used technique to accelerate DNN training and reduce memory requirements. However, applying a widely adopted reduced precision format such as Float16 to all involved operations in DNN training is not optimal as the use of Float16 in some operations can hurt model accuracy. Meanwhile, additional optimizations including loss scaling and autocast techniques can mitigate the accuracy loss but lead to inherent overhead and inadequate use of reduced precision. In this work, we leverage another reduced precision format, BFloat16, and introduce a throughput-oriented and accuracy-aware approach to maximize the performance potential of DNN training. Since the high throughput provided by BFloat16 format is accompanied by low precision of the floating-point representation, this approach achieves high throughput by using BFloat16 on all DNN operations and avoids the accuracy loss through a customized accuracy-aware normalization. Results show that our approach outperforms the state-of-the-art mixed-precision training by 1.21x on an NVIDIA A100 GPU.

Xie, Zhen↗
DOE OSTI2021

A hub and spoke approach to optimizing energy wheeling of renewable resources

The deployment of zero carbon renewable energy sources needs to increase significantly to support the goal of net zero greenhouse gas emissions by 2050. At the same time energy end use needs to decarbonize. This will change both energy supply and energy demand patterns, requiring the energy delivery infrastructure (grid-based transmission circuits) to become increasingly flexible to maintain security of supply everywhere and always. The integration of zero carbon renewable energy requires cross-border and cross energy system coupling and a fit-for-purpose design. Nowadays, energy systems are planned, designed and operated in silos with a strong national focus. However, large-scale offshore wind production needs to be transported to deep inland locations, across country borders. The increased peak generation capacity of renewable energy sources will, at times, significantly exceed demand (Matthew Langholtz, 2020). The traditional solution of continuously reinforcing and extending the electricity grid is not sustainable from a cost and societal perspective. This paper will, however, propose a deterministic approach on how networked (interconnected grid) Points of receipt (POR) to Points of Delivery (POD) can be optimized for wheeling renewable energy resources while minimizing energy cost with a hub and spoke approach. The statistical approach will be done via using existing daily energy market clearing prices, available transmission capacity and firm daily transmission prices in open access energy markets. Renewable energy targets, including specific offshore wind targets, need to be in line with the ramp-up as implied by the Paris Agreement. These targets are required to provide industry with a secure market outlook that allows them to build up supply chains accordingly. Optimizing wheeled energy paths from carbon neutral resources such as renewables make them not only cost competitive on the unit commitment stack, but also more accessible on the dispatch stack to other carbon heavy forms of generation such as coal and natural gas turbines (Matthew Langholtz, 2020). This correlates to maximizing renewable resource inertia (wind, solar, biomass) within an interconnected grid without having to consider additional expansion of resources via land purchases and de-forestation.

Mukherjee, Srijib↗
DOE OSTI2022

Structure-Informed Graph Learning of Networked Dependencies for Online Prediction of Power System Transient Dynamics

Online transient analysis plays an increasingly important role in dynamic power grids as the renewable generation continues growing. Traditional numerical methods for transient analysis not only are computationally intensive but also require precise contingency information as input, and therefore, are not suitable for online applications. Existing online transient assessment studies focus on the determination of post-contingency system stability or stability margin. Here, this paper develops a novel graph-learning framework, Deep-learning Neural Representation or DNR, for online prediction, of the time-series trajectories of the system states using initial system responses that can be measured by phasor measurement units (PMUs). The proposed DNR framework consists of two sequential modules: a Network Constructor that captures network dependencies among generators, and a Dynamics Predictor that predicts the system trajectories. The key to improved prediction performance is the introduction of the spatio-temporal message-passing operations into graph neural networks with structural knowledge. Its effectiveness and scalability are validated through comparative studies, demonstrating the prediction performance under different contingency scenarios for systems of different sizes. This framework provides a solution to online predicting post-fault system dynamics based on real-time PMU measurements. Additionally, it can also be applied to facilitate the offline transient simulation without simulating the entire trajectories.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

L-VISP: LSTM Visualization for Interpretable Symptom Prediction in Patient Cohorts

Symptom modelling in head and neck cancer is challenged by the complexity of heterogeneous patient data, leading to an interest in deep learning approaches. Although Long Short-Term Memory Networks (LSTMs) have shown great results in patient risk prediction, their low interpretability requires data modellers to collaborate with clinical experts to validate the results. We present L-VISP, a human–machine solution that uses visual analytics for LSTM modelling in clinical research. L-VISP uses custom visual encodings to make multiple LSTM variants interpretable, supporting a full range of analysis, from understanding model operations and evaluating performance to interpreting results in a clinical context. We evaluate L-VISP with data modellers and a clinical oncologist and present the takeaways from this multidisciplinary collaboration.

LSTM modeling↗
DOE OSTI2023

Optical information transfer through random unknown diffusers using electronic encoding and diffractive decoding

Free-space optical information transfer through diffusive media is critical in many applications, such as biomedical devices and optical communication, but remains challenging due to random, unknown perturbations in the optical path. We demonstrate an optical diffractive decoder with electronic encoding to accurately transfer the optical information of interest, corresponding to, e.g., any arbitrary input object or message, through unknown random phase diffusers along the optical path. This hybrid electronic-optical model, trained using supervised learning, comprises a convolutional neural network-based electronic encoder and successive passive diffractive layers that are jointly optimized. After their joint training using deep learning, our hybrid model can transfer optical information through unknown phase diffusers, demonstrating generalization to new random diffusers never seen before. The resulting electronic-encoder and optical-decoder model was experimentally validated using a 3D-printed diffractive network that axially spans <70λ, where λ = 0.75 mm is the illumination wavelength in the terahertz spectrum, carrying the desired optical information through random unknown diffusers. The presented framework can be physically scaled to operate at different parts of the electromagnetic spectrum, without retraining its components, and would offer low-power and compact solutions for optical information transfer in free space through unknown random diffusive media.

36 MATERIALS SCIENCE↗
DOE OSTI2024

A Deep Learning Based Framework to Identify Undocumented Orphaned Oil and Gas Wells from Historical Maps: A Case Study for California and Oklahoma

Undocumented Orphaned Wells (UOWs) are wells without an operator that have limited or no documentation with regulatory authorities. An estimated 310,000 to 800,000 UOWs exist in the United States (US), whose locations are largely unknown. These wells can potentially leak methane and other volatile organic compounds to the atmosphere, and contaminate groundwater. In this study, we developed a novel framework utilizing a state-of-the-art computer vision neural network model to identify the precise locations of potential UOWs. The U-Net model is trained to detect oil and gas well symbols in georeferenced historical topographic maps, and potential UOWs are identified as symbols that are further than 100 m from any documented well. A custom tool was developed to rapidly validate the potential UOW locations. We applied this framework to four counties in California and Oklahoma, leading to the discovery of 1301 potential UOWs across >40,000 km 2 . We confirmed the presence of 29 UOWs from satellite images and 15 UOWs from magnetic surveys in the field with a spatial accuracy on the order of 10 m. This framework can be scaled to identify potential UOWs across the US since the historical maps are available for the entire nation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Predicting Elastic Properties of Materials from Electronic Charge Density Using 3D Deep Convolutional Neural Networks

Materials representation plays a key role in machine learning-based prediction of materials properties and new materials discovery. Currently both graph and three-dimensional (3D) voxel representation methods are based on the heterogeneous elements of the crystal structures. Here, we propose to use electronic charge density (ECD) as a generic unified 3D descriptor for materials property prediction with the advantage of possessing close relation with the physical and chemical properties of materials. We developed an ECD-based 3D convolutional neural networks (CNNs) for predicting the elastic properties of materials, in which CNNs can learn effective hierarchical features with multiple convolving and pooling operations. Extensive benchmark experiments over 2170 $Fm\bar3m$ face-centered-cubic materials show that our ECD-based CNNs can achieve good performance for elasticity prediction. Especially, our CNN models based on the fusion of elemental Materials-Agnostic Platform for Informatics and Exploration features and ECD descriptors achieved the best fivefold cross-validation performance. More importantly, we showed that our ECD-based CNN models can achieve significantly better extrapolation performance when evaluated over nonredundant data sets, where there are few neighbor-training samples around test samples. As an additional validation, we evaluated the predictive performance of our models on 329 materials of space group $Fm\bar3m$ by comparing to density functional theory calculated values, which shows a better prediction power of our model for bulk modulus than shear modulus. Because of the unified representation power of ECD, it is expected that our ECD-based CNN approach can also be applied to predict other physical and chemical properties of crystalline materials.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Development of a plug-and-play anti-noise module for fault diagnosis of rotating machines in nuclear power plants

The health of rotating machines is crucial to the stable operation and safety of nuclear power plants. However, research on machine learning-based fault diagnosis of rotating machines in the nuclear industry is still in its infancy. The signal noise generated in the plant may negatively affect the effectiveness of the analysis. In this paper, a plug-and-play anti-noise machine learning module is proposed to fill the knowledge and capability gap. The modules are loaded into a convolutional neural network called deep residual network (ResNet) to obtain a new model with noise reduction capability. The basic idea is that the module is able to identify noise features and include them in the subsequent analysis, effectively filtering noise at the feature level of the network. Nine variants of the new model are compared with the original ResNet as well as four classical machine learning models to test the effectiveness of the module and to examine the impact of the module's loading modes on the performance of the new model. Finally, this research helps facilitate the application of machine learning in the plant noise environment.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2024

Mid-IR UAV-based sensing platform with deep learning to Identify and Quantify Gaseous Emission in Gas Flares

This report details the development and evaluation of a Mid-Infrared (Mid-IR) Unmanned Aerial Vehicle (UAV)-based sensing platform integrated with deep learning algorithms for the identification and quantification of gaseous emissions in gas flares. The project, spearheaded by Omega Optics, Inc., aimed to address environmental monitoring challenges by leveraging advanced photonic technologies and autonomous UAV operations. The research focused on designing, optimizing, and fabricating photonic crystal waveguides and grating couplers to enhance the sensitivity and accuracy of gas detection. A comprehensive drone-based system was developed, featuring a miniaturized sensor, GPS module, and microcontroller communication network for real-time gas concentration monitoring. The system's adaptive sampling algorithm, implemented using the Robot Operating System (ROS), enables autonomous detection and localization of gas emission sources. Preliminary results demonstrate the platform's capability to detect and monitor gas emissions with high precision, cost-effectiveness, and scalability. Future work will expand upon this foundation by introducing 3D wind model-based learning for dynamic environmental conditions and further enhancing the user interface and data processing algorithms to support broader environmental monitoring applications. Overall, this project represents a significant step forward in UAV-based environmental sensing technologies, offering robust solutions for detecting and mitigating the impacts of gaseous emissions on public health and safety.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

FALCON: Framework for Anomaly Detection in Industrial Control Systems

Industrial Control Systems (ICS) are used to control physical processes in critical infrastructure. These systems are used in a wide variety of operations such as water treatment, power generation and distribution, and manufacturing. While the safety and security of these systems are of serious concern, recent reports have shown an increase in targeted attacks aimed at manipulating physical processes to cause catastrophic consequences. This trend emphasizes the need for algorithms and tools that provide resilient and smart attack detection mechanisms to protect ICS. In this paper, we propose an anomaly detection framework for ICS based on a deep neural network. The proposed methodology uses dilated convolution and long short-term memory (LSTM) layers to learn temporal as well as long term dependencies within sensor and actuator data in an ICS. The sensor/actuator data are passed through a unique feature engineering pipeline where wavelet transformation is applied to the sensor signals to extract features that are fed into the model. Additionally, this paper explores four variations of supervised deep learning models, as well as an unsupervised support vector machine (SVM) model for this problem. The proposed framework is validated on Secure Water Treatment testbed results. This framework detects more attacks in a shorter period of time than previously published methods.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A Novel Method for Controlling Crud Deposition in Nuclear Reactors Using Optimization Algorithms and Deep Neural Network Based Surrogate Models

This work presents the use of a high-fidelity neural network surrogate model within a Modular Optimization Framework for treatment of crud deposition as a constraint within light-water reactor core loading pattern optimization. The neural network was utilized for the treatment of crud constraints within the context of an advanced genetic algorithm applied to the core design problem. This proof-of-concept study shows that loading pattern optimization aided by a neural network surrogate model can optimize the manner in which crud distributes within a nuclear reactor without impacting operational parameters such as enrichment or cycle length. Several analysis methods were investigated. Analysis found that the surrogate model and genetic algorithm successfully minimized the deviation from a uniform crud distribution against a population of solutions from a reference optimization in which the crud distribution was not optimized. Strong evidence is presented that shows boron deposition in crud can be optimized through the loading pattern. This proof-of-concept study shows that the methods employed provide a powerful tool for mitigating the effects of crud deposition in nuclear reactors.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Deep learning models for interpretation of point of care ultrasound in military working dogs

Introduction: Military working dogs (MWDs) are essential for military operations in a wide range of missions. With this pivotal role, MWDs can become casualties requiring specialized veterinary care that may not always be available far forward on the battlefield. Some injuries such as pneumothorax, hemothorax, or abdominal hemorrhage can be diagnosed using point of care ultrasound (POCUS) such as the Global FAST® exam. This presents a unique opportunity for artificial intelligence (AI) to aid in the interpretation of ultrasound images. In this article, deep learning classification neural networks were developed for POCUS assessment in MWDs. Methods: Images were collected in five MWDs under general anesthesia or deep sedation for all scan points in the Global FAST® exam. For representative injuries, a cadaver model was used from which positive and negative injury images were captured. A total of 327 ultrasound clips were captured and split across scan points for training three different AI network architectures: MobileNetV2, DarkNet-19, and ShrapML. Gradient class activation mapping (GradCAM) overlays were generated for representative images to better explain AI predictions. Results: Performance of AI models reached over 82% accuracy for all scan points. The model with the highest performance was trained with the MobileNetV2 network for the cystocolic scan point achieving 99.8% accuracy. Across all trained networks the diaphragmatic hepatorenal scan point had the best overall performance. However, GradCAM overlays showed that the models with highest accuracy, like MobileNetV2, were not always identifying relevant features. Conversely, the GradCAM heatmaps for ShrapML show general agreement with regions most indicative of fluid accumulation. Discussion: Overall, the AI models developed can automate POCUS predictions in MWDs. Preliminarily, ShrapML had the strongest performance and prediction rate paired with accurately tracking fluid accumulation sites, making it the most suitable option for eventual real-time deployment with ultrasound systems. Further integration of this technology with imaging technologies will expand use of POCUS-based triage of MWDs.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Dynamic Role-Based Access Control Policy for Smart Grid Applications: An Offline Deep Reinforcement Learning Approach

Role-based access control (RBAC) is adopted in the information and communication technology domain for authentication purposes. However, due to a very large number of entities within organizational access control (AC) systems, static RBAC management can be inefficient, costly, and can lead to cybersecurity threats. In this paper, a novel hybrid RBAC model is proposed, based on the principles of offline deep reinforcement learning (RL) and Bayesian belief networks. The considered framework utilizes a fully offline RL agent, which models the behavioral history of users as a Bayesian belief-based trust indicator. Thus, the initial static RBAC policy is improved in a dynamic manner through off-policy learning while guaranteeing compliance of the internal users with the security rules of the system. By deploying our implementation within the smart grid domain and specifically within a Distributed Energy Resources (DER) ecosystem, we provide an end-to-end proof of concept of our model. Finally, detailed analysis and evaluation regarding the offline training phase of the RL agent are provided, while the online deployment of the hybrid RL-based RBAC model into the DER ecosystem highlights its key operation features and salient benefits over traditional RBAC models.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗