Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “deep learning method”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 649 records · Page 36
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Algorithms to Improve Training for Deep Learning with Diabetic Retinopathy Images

This is a minisymposium presentation I plan to give at the SIAM conference on Computational Science and Engineering on March 1st, 2021. The work is based on research from my SDRD this year and describes a new pooling method we've developed called variable stride. The goal of the minisymposium is to highlight some of the ground-breaking work being done across the National Nuclear Security Administration by graduate students and Postdocs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Flavor Classification in ICARUS Using Convolutional Visual Networks

In this work, we adapt the Convolutional Visual Network (CVN) approach [1] to the ICARUS detector by incorporating TPC stitching methods inspired by NuGraph[8]. The stitching technique used here is unique to ICARUS, designed specifically to handle its distinct detector segmentations. We then retrain the network using ICARUS-specific data. This study underscores the flexibility of deep learning models in high-energy physics and the importance of accounting for detector-specific features when transferring machine learning techniques between experiments. This poster presents the methods, classification performance, and insights gained from applying CVN to ICARUS data.

Wieler, Felipe [Tech. Fed. Parana U.]↗
DOE OSTI2026

Neutrino Flavor Classification in ICARUS Experiment Using Convolutional Visual Networks

In this work, I adapt the Convolutional Visual Network (CVN) approach to the ICARUS detector by incorporating TPC stitching methods inspired by NuGraph. The stitching technique used here is unique to ICARUS, designed specifically to handle its distinct detector segmentation s. I then retrain the network using ICARUS-specific data. This study underscores the flexibility of deep learning models in high-energy physics and the importance of accounting for detector-specific features when transferring machine learning techniques between experiments. This dissertation presents the methods, classification performance, and insights gained from applying CVN to ICARUS data.

Wieler, Felipe Andre [Parana Tech. Fed. U., Toledo↗
DOE OSTI2026

Transplatformer: translating toxicogenomic profiles between generations of platforms

Background Transcriptomic profiling technologies have advanced the analysis of biological and toxicological responses. However, substantial differences in probe design, dynamic range, gene coverage, and preprocessing pipelines across platforms introduce artifacts that limit cross-study integration and hinder the reuse of historical datasets. We aim to develop computational methods for accurate cross-platform translation to maximize the value of legacy resources. Results We present TransPlatformer a deep learning framework for translating gene expression profiles across heterogeneous toxicogenomics platforms. TransPlatformer employs a novel attention-based architecture to map high-dimensional fold-change vectors from legacy microarray technologies to current platforms. Models are trained and evaluated using DrugMatrix, spanning three technological generations. We investigate mixed-tissue, single-tissue, and cross-tissue training paradigms and benchmark performance against multilayer perceptron and matrix-completion baselines. In mixed-tissue training, TransPlatformer achieves a greater than 50% reduction in mean absolute error (0.043 vs. 0.09) and nearly doubles Pearson correlation ( ≈ 0.71 vs. 0.37) relative to baseline methods. Importantly, TransPlatformer preserves rare but biologically meaningful over- and under-expressed signals, with mean absolute error below 0.22. Single-tissue models yield further improvements for well-represented organs, such as a 10% reduction in liver mean absolute error, while underscoring the need for data augmentation strategies in low-sample tissues.ra Conclusions TransPlatformer provides an effective and scalable computational solution for cross-platform transcriptomic translation. By enabling biologically faithful harmonization of gene expression data, the proposed approach facilitates the reuse of legacy toxicogenomics datasets, enhances downstream biomarker discovery, and supports more reproducible predictive modeling in toxicology.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

AIMSim : An accessible cheminformatics platform for similarity operations on chemicals datasets

The recent advances in deep learning, generative modeling, and statistical learning have ushered in a renewed interest in traditional cheminformatics tools and methods. Quantifying molecular similarity is essential in molecular generative modeling, exploratory molecular synthesis campaigns, and drug-discovery applications to assess how new molecules differ from existing ones. Further, most tools target advanced users and lack general implementations accessible to the larger community. In this work, we introduce Artificial Intelligence Molecular Similarity (AIMSim), an accessible cheminformatics platform for performing similarity operations on collections of molecules called molecular datasets. AIMSim provides a unified platform to perform similarity-based tasks on molecular datasets, such as diversity quantification, outlier and novelty analysis, clustering, dimensionality reduction, and inter-molecular comparisons. AIMSim implements all major binary similarity metrics and molecular fingerprints and is provided as a Python package that includes support for command-line use as well as a Graphical User Interface for code-free utilization with fully interactive plots.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Uncertainty quantification for Multiphase-CFD simulations of bubbly flows: a machine learning-based Bayesian approach supported by high-resolution experiments

In this paper, we developed a machine learning-based Bayesian approach to inversely quantify and reduce the uncertainties of multiphase computational fluid dynamics (MCFD) simulations for bubbly flows. The proposed approach is supported by high-resolution two-phase flow measurements, including those by double-sensor conductivity probes, high-speed imaging, and particle image velocimetry. Local distributions of key physical quantities of interest (QoIs), including the void fraction and phasic velocities, are obtained to support the Bayesian inference. In the process, the epistemic uncertainties of the closure relations are inversely quantified while the aleatory uncertainties from stochastic fluctuations of the system are evaluated based on experimental uncertainty analysis. The combined uncertainties are then propagated through the MCFD solver to obtain uncertainties of the QoIs, based on which probability-boxes are constructed for validation. The proposed approach relies on three machine learning methods: feedforward neural networks and principal component analysis for surrogate modeling, and Gaussian processes for model form uncertainty modeling. The whole process is implemented within the framework of an open-source deep learning library PyTorch with graphics processing unit (GPU) acceleration, thus ensuring the efficiency of the computation. The results demonstrate that with the support of high-resolution data, the uncertainties of MCFD simulations can be significantly reduced. The proposed approach has the potential for other applications that involve numerical models with empirical parameters.

42 ENGINEERING↗
DOE OSTI2024

Uncertainty-aware particle segmentation for electron microscopy at varied length scales

Electron microscopy is indispensable for examining the morphology and composition of solid materials at the sub-micron scale. To study the powder samples that are widely used in materials development, scanning electron microscopes (SEMs) are increasingly used at the laboratory scale to generate large datasets with hundreds of images. Parsing these images to identify distinct particles and determine their morphology requires careful analysis, and automating this process remains challenging. In this work, we enhance the Mask R-CNN architecture to develop a method for automated segmentation of particles in SEM images. We address several challenges inherent to measurements, such as image blur and particle agglomeration. Moreover, our method accounts for prediction uncertainty when such issues prevent accurate segmentation of a particle. Recognizing that disparate length scales are often present in large datasets, we use this framework to create two models that are separately trained to handle images obtained at low or high magnification. By testing these models on a variety of inorganic samples, our approach to particle segmentation surpasses an established automated segmentation method and yields comparable results to the predictions of three domain experts, revealing comparable accuracy while requiring a fraction of the time. These findings highlight the potential of deep learning in advancing autonomous workflows for materials characterization.

36 MATERIALS SCIENCE↗
DOE OSTI2020

CoSim: A Simulator for Co-Scheduling of Batch and On-Demand Jobs in HPC Datacenters

The increasing scale and complexity of scientific applications are rapidly transforming the ecosystem of tools, methods, and workflows adopted by the high-performance computing (HPC) community. Big data analytics and deep learning are gaining traction as essential components in this ecosystem in a variety of scenarios, such as, steering of experimental instruments , acceleration of high-fidelity simulations through surrogate computations, and guided ensemble searches. In this context, the batch job model traditionally adopted by the supercomputing in-frastructures needs to be complemented with support to schedule opportunistic on-demand analytics jobs, leading to the problem of efficient preemption of batch jobs with minimum loss of progress. In this paper, we design and implement a simulator, CoSim, that enables on-the-fly analysis of the trade-offs arising between delaying the start of opportunistic on-demand jobs, which leads to longer analytics latency, and loss of progress due to preemption of batch jobs, which is necessary to make room for on-demand jobs. To this end, we propose an algorithm based on dynamic programming with predictable performance and scalability that enables supercomputing infrastructure schedulers to analyze the aforementioned trade-off and take decisions in near real-time. Compared with other state-of-art approaches using traces of the Theta pre-Exascale machine, our approach is capable of finding the optimal solution, while achieving high performance and scalability.

batch job preemption↗
DOE OSTI2021

Characterizing Machine Learning I/O Workloads on Leadership Scale HPC Systems

High performance computing (HPC) is no longer solely limited to traditional workloads such as simulation and modeling. With the increase in the popularity of machine learning (ML) and deep learning (DL) technologies, we are observing that an increasing number of HPC users are incorporating ML methods into their workflow and scientific discovery processes, across a wide spectrum of science domains such as biology, earth science, and physics. This gives rise to a diverse set of I/O patterns than the traditional checkpoint/restart-based HPC I/O behavior. The details of the I/O characteristics of such ML I/O workloads have not been studied extensively for large-scale leadership HPC systems. This paper aims to fill that gap by providing an in-depth analysis to gain an understanding of the I/O behavior of ML I/O workloads using darshan - an I/O characterization tool designed for lightweight tracing and profiling. We study the darshan logs of more than 23, 000 HPC ML I/O jobs over a time period of one year running on Summit - the second-fastest supercomputer in the world. This paper provides a systematic I/O characterization of ML I/O jobs running on a leadership scale supercomputer to understand how the I/O behavior differs across science domains and the scale of workloads, and analyze the usage of parallel file system and burst buffer by ML I/O workloads.

Paul, Arnab↗
DOE OSTI2025

Explainable and Differentiable Reinforcement Learning for Multi-objective Optimization in Particle Accelerators

Operating particle accelerators involves optimizing multiple goals simultaneously, which can be challenging due to trade-offs among objectives. While evolutionary algorithms like the genetic algorithm (GA) have been used for various Multi-Objective Optimization (MOO) tasks, they are not inherently suited for complex control problems. This talk highlights two variations of Reinforcement Learning (RL) for concurrently optimizing heat load and trip rates at the Continuous Electron Beam Accelerator Facility (CEBAF). The problem involves strict constraints on individual states, actions, and overall energy requirements of the beam. First, this talk highlights how differentiability can be harnessed through a Deep Differentiable Reinforcement Learning (DDRL) approach to address MOO issues within particle accelerators. We examine the DDRL method alongside Model Free Reinforcement Learning (MFRL), GA, and Bayesian Optimization (BO). The performance of these methods is assessed by generating a Pareto-front for two objectives. Our findings indicate that DDRL excels in handling high-dimensional problems more effectively than MFRL, BO, and GA. Next, we will show integration of explainable physics-based constraints into RL algorithms to enhance trans- parency and trust in decision-making processes by enabling users to verify that agents adhere to established physical principles. This surrogate function can be modeled using neural networks or sparse dictionary mod- els. By examining the mathematical form of the learned constraint function, we are able to confirm the agent has learned to use the established physics of each environment provided but the surrogate model. In addi- tion, we find that the introduction of a mathematical functional dictionary based surrogate model enables our reinforcement learning algorithms to reliably converge for difficult high-dimensional accelerator controls environments.

Rajput, Kishansingh [Thomas Jefferson National Acc↗
DOE OSTI2025

Protein-ligand binding affinity prediction using multi-instance learning with docking structures

Recent advances in 3D structure-based deep learning approaches demonstrate improved accuracy in predicting protein-ligand binding affinity in drug discovery. These methods complement physics-based computational modeling such as molecular docking for virtual high-throughput screening. Despite recent advances and improved predictive performance, most methods in this category primarily rely on utilizing co-crystal complex structures and experimentally measured binding affinities as both input and output data for model training. Nevertheless, co-crystal complex structures are not readily available and the inaccurate predicted structures from molecular docking can degrade the accuracy of the machine learning methods. We introduce a novel structure-based inference method utilizing multiple molecular docking poses for each complex entity. Our proposed method employs multi-instance learning with an attention network to predict binding affinity from a collection of docking poses. We validate our method using multiple datasets, including PDBbind and compounds targeting the main protease of SARS-CoV-2. The results demonstrate that our method leveraging docking poses is competitive with other state-of-the-art inference models that depend on co-crystal structures. This method offers binding affinity prediction without requiring co-crystal structures, thereby increasing its applicability to protein targets lacking such data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Physics-constrained Deep Recurrent Neural Models of Building Thermal Dynamics

We develop physics-constrained and control-oriented predictive deep learning models for the thermal dynamics of a real-world commercial office building. The proposed method is based on the systematic encoding of physics-based prior knowledge into a structured recurrent neural architecture. Specifically, our model mimics the structure of the building thermal dynamics model and leverages penalty methods to model inequality constraints. Additionally, we use constrained matrix parameterization based on the Perron-Frobenius theorem to bound the eigenvalues of the learned network weights. We interpret the stable eigenvalues as dissipativeness of the learned building thermal model. We demonstrate the effectiveness of the proposed approach on a dataset obtained from an office building with $20$ thermal zones.

Building Energy, structured neural networks↗
DOE OSTI2023

Strictly Enforcing Invertibility and Conservation in CNN-Based Super Resolution for Scientific Datasets

Abstract Recently, deep convolutional neural networks (CNNs) have revolutionized image “super resolution” (SR), dramatically outperforming past methods for enhancing image resolution. They could be a boon for the many scientific fields that involve imaging or any regularly gridded datasets: satellite remote sensing, radar meteorology, medical imaging, numerical modeling, and so on. Unfortunately, while SR-CNNs produce visually compelling results, they do not necessarily conserve physical quantities between their low-resolution inputs and high-resolution outputs when applied to scientific datasets. Here, a method for “downsampling enforcement” in SR-CNNs is proposed. A differentiable operator is derived that, when applied as the final transfer function of a CNN, ensures the high-resolution outputs exactly reproduce the low-resolution inputs under 2D-average downsampling while improving performance of the SR schemes. The method is demonstrated across seven modern CNN-based SR schemes on several benchmark image datasets, and applications to weather radar, satellite imager, and climate model data are shown. The approach improves training time and performance while ensuring physical consistency between the super-resolved and low-resolution data. Significance Statement Recent advancements in using deep learning to increase the resolution of images have substantial potential across the many scientific fields that use images and image-like data. Most image super-resolution research has focused on the visual quality of outputs, however, and is not necessarily well suited for use with scientific data where known physics constraints may need to be enforced. Here, we introduce a method to modify existing deep neural network architectures so that they strictly conserve physical quantities in the input field when “super resolving” scientific data and find that the method can improve performance across a wide range of datasets and neural networks. Integration of known physics and adherence to established physical constraints into deep neural networks will be a critical step before their potential can be fully realized in the physical sciences.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

DeepSZ: identification of Sunyaev–Zel’dovich galaxy clusters using deep learning

Galaxy clusters identified via the Sunyaev–Zel’dovich (SZ) effect are a key ingredient in multiwavelength cluster cosmology. In this paper, we present and compare three methods of cluster identification: the standard matched filter (MF) method in SZ cluster finding, a convolutional neural networks (CNN), and a ‘combined’ identifier. We apply the methods to simulated millimeter maps for several observing frequencies for a survey similar to SPT-3G, the third-generation camera for the South Pole Telescope. The MF requires image pre-processing to remove point sources and a model for the noise, while the CNN requires very little pre-processing of images. Additionally, the CNN requires tuning of hyperparameters in the model and takes cut-out images of the sky as input, identifying the cut-out as cluster-containing or not. We compare differences in purity and completeness. The MF signal-to-noise ratio depends on both mass and redshift. Our CNN, trained for a given mass threshold, captures a different set of clusters than the MF, some with signal-to-noise-ratio below the MF detection threshold. However, the CNN tends to mis-classify cut-out whose clusters are located near the edge of the cut-out, which can be mitigated with staggered cut-out. We leverage the complementarity of the two methods, combining the scores from each method for identification. The purity and completeness are both 0.61 for MF, and 0.59 and 0.61 for CNN. The combined method yields 0.60 and 0.77, a significant increase for completeness with a modest decrease in purity. We advocate for combined methods that increase the confidence of many low signal-to-noise clusters.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Quantum computing based hybrid deep learning for fault diagnosis in electrical power systems

Quantum computing (QC) and deep learning have shown promise of supporting transformative advances and have recently gained popularity in a wide range of areas. Here, this paper proposes a hybrid QC-based deep learning framework for fault diagnosis of electrical power systems that combine the feature extraction capabilities of conditional restricted Boltzmann machine with an efficient classification of deep networks. Computational challenges stemming from the complexities of such deep learning models are overcome by QC-based training methodologies that effectively leverage the complementary strengths of quantum assisted learning and classical training techniques. The proposed hybrid QC-based deep learning framework is tested on a simulated electrical power system with 30 buses and wide variations of substation and transmission line faults, to demonstrate the framework’s applicability, efficiency, and generalization capabilities. High computational efficiency is enjoyed by the proposed hybrid approach in terms of computational effort required and quality of diagnosis performance over classical training methods. In addition, superior and reliable fault diagnosis performance with faster response time is achieved over state-of-the-art pattern recognition methods based on artificial neural networks (ANN) and decision trees (DT).

42 ENGINEERING↗
DOE OSTI2022

Risk-Informed Operations and Maintenance Decision Making Using Deep Reinforcement Learning

A challenge for operating nuclear power plants is the significant cost of operations and maintenance, at times consuming up to 66% of annual operating costs. This project aims to build a framework for a risk-informed asset-management tool that integrates inspections, repairs, spare-part inventory, supply chain, and business choices to lower overall O&M costs. Our approach uses a combination of data-driven modeling and deep reinforcement learning to create and implement optimal maintenance policies for the existing nuclear fleet, as well as new advanced reactors. The creation of an asset management tool that uses these advanced methods will give operators new capabilities to help reduce the burden of O&M spending in nuclear power plants.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Branching Dueling Q-Network Based Online Scheduling of a Microgrid With Distributed Energy Storage Systems

This letter investigates a Branching Dueling Q-Network (BDQ) based online operation strategy for a microgrid with distributed battery energy storage systems (BESSs) operating under uncertainties. Additionally, the developed deep reinforcement learning (DRL) based microgrid online optimization strategy can achieve a linear increase in the number of neural network outputs with the number of distributed BESSs, which overcomes the curse of dimensionality caused by the charge and discharge decisions of multiple BESSs. Numerical simulations validate the effectiveness of the proposed method.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Grid-Connected Modular Soft-Switching Solid State Transformers (M-S4T)

The objective of this project is to develop and verify the concept of a flexible and modular soft-switching solid-state transformer (M-S4T) for direct grid-connected applications. The ability to directly connect power electronics converters to the medium voltage grid (4 kV – 13 kV), and to potentially replace the passive and bulky, but ubiquitous 60 hertz service transformer in the 25 kVA to 100 kVA range, with a more flexible and controllable device, has been regarded as the ‘holy grail’ in grid control. However, this has proven to be extremely difficult. This project has developed the solutions to several key challenges of the direct grid-connected power electronics and realized a 7.2 kV M-S4T prototype. First, a protection method to protect the M-S4T from the high voltages (110 kV for the 13 kV system) that occur on the grid due to transients and lightning strikes have been developed and experimentally verified. Second, the realization and the operation of the M-S4T based on high-voltage SiC devices (>3.3 kV) and a medium-frequency medium-voltage low-leakage transformer in a single-stage solid-state transformer with zero-voltage switching, low dv/dt, and low electromagnetic interference has been successfully demonstrated up to 7.5 kV peak. Third, an oil-cooling system and stable communication and distributed control system for converter module voltage sharing have been developed and experimentally verified. The developed M-S4T has realized a modular universal high-performance power conversion system. This conversion system is scalable to different voltage and power levels and adaptable to four-quadrant bidirectional operation. Moreover, the use of passive cooling techniques meets the equipment life requirements, and the lightning protection scheme fulfills the basic insulation level specifications for direct grid connection. Such power conversion system opens up near-term opportunities, including energy storage, solar PV, or electric vehicle charging with significant cost and footprint savings. In the longer term, the possibility of replacing the utility distribution transformer with an M-S4T will be transformative for future distribution grids with a compact footprint and full controllability to enable high renewable energy and storage penetration. In addition to the main project, this report expands on the Plus-Up projected including as part of the main award. This project developed and demonstrated the technology for autonomous collaborative inverters that can be connected in an ad hoc manner to the grid. The aim of the project was to: (1) evaluate the existing techniques for grid-connected inverters and find their limitations; (2) develop detailed requirements for grid-connected inverters in the modern grid with millions of active nodes; (3) design a unified control strategy that brings more autonomy and intelligence to grid-connected inverters, and addresses parts of the issues with the existing techniques. The proposed technique, called UniCon, enables inverters to 1) connect/disconnect to/from the grid in an ad hoc manner; (2) work based on local sensing. Slow communication could be used for a more optimized behavior; (3) work automatically in both grid-forming/grid-following mode; (4) handle large disturbances, e.g., big load step and fault, in an oscillation-free manner; (5) work collaboratively with other inverters in steady-state and during transients. UniCon can be implemented in the middle-level control; hence it is agnostic to the vendor and to the implementation of the inner voltage/current and protection loops. Furthermore, a new synchronization scheme, based on deep learning, was developed that can extract the grid voltage phase and amplitude in a stable manner. The method is cheap to implement can improve the dynamic performance of the grid-connected inverters during fast transients, e.g., fault. The proposed control scheme was validated by (1) MATLAB/Simulink; (2) hardware-in-the-loop results, and; (3) experimental results using three inverters that form a microgrid in a down-scaled feeder. Lastly, both the M-S4T and UniCon have achieved promising tangible paths to markets. In the case of the M-S4T, the underlying technology — the Soft Switching Solid State Transformer (S4T) developed at the Georgia Tech Center for Distributed Energy (GT-CDE) has been licensed by GridBlock from the Georgia Tech Research Corporation, and GridBlock has been working with manufacturing partner Jabil (one of the largest US-based contract manufacturers) and system integrator Power Secure (largest deployer of microgrids in the US with 4.7 GW under management), to meet the strong initial demand. Similarly, GridBlock has an exclusive license to the UniCon technology, developed under this award by GT-CDE. The UniCon provides an intermediate control layer that enables the implementation of the higher-level ‘transactive’ control commands for the system. The architecture of the system - slow communications with the cloud for system optimization and setpoints, and the use of locally measured quantities for real-time control, provide a very robust and secure way of implementing a real-time must-run grid that is also secure and stable. This is a brand-new functionality that is critical for the future grid and key to GridBlock’s business model.

24 POWER TRANSMISSION AND DISTRIBUTION↗