Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “deep learning method”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

DeepThermo: Deep Learning Accelerated Parallel Monte Carlo Sampling for Thermodynamics Evaluation of High Entropy Alloys

Since the introduction of Metropolis Monte Carlo (MC) sampling, it and its variants have become standard tools used for thermodynamics evaluations of physical systems. However, a long-standing problem that hinders the effectiveness and efficiency of MC sampling is the lack of a generic method (a.k.a. MC proposal) to update the system configurations. Consequently, current practices are not scalable. Here we propose a parallel MC sampling framework for thermodynamics evaluation—DeepThermo. By using deep learning–based MC proposals that can globally update the system configurations, we show that DeepThermo can effectively evaluate the phase transition behaviors of high entropy alloys, which have an astronomical configuration space. For the first time, we directly evaluate a density of states expanding over a range of ~e 10,000 for a real material. We also demonstrate DeepThermo’s performance and scalability up to 3,000 GPUs on both NVIDIA V100 and AMD MI250X-based supercomputers.

Yin, Junqi↗
DOE OSTI2022

Deep Learning for Spectral Filling in Radio Frequency Applications

Due to the Internet of Things (IoT) proliferation, Radio Frequency (RF) channels are increasingly congested with new kinds of devices, which carry unique and diverse communication needs. This poses complex challenges in modern digital communications, and calls for the development of technological innovations that (i) optimize capacity (bitrate) in limited bandwidth environments, (ii) integrate cooperatively with already-deployed RF protocols, and (iii) are adaptive to the ever-changing demands in modern digital communications. In this paper we present methods for applying deep neural networks for spectral filling. Given an RF channel transmitting digital messages with a pre-established modulation scheme, we automatically learn novel modulation schemes for sending extra information, in the form of additional messages, “around” the fixed-modulation signals (i.e., without interfering with them). In so doing, we effectively increase channel capacity without increasing bandwidth. We further demonstrate the ability to generate signals that closely resemble the original modulations, such that the presence of extra messages is undetectable to third-party listeners. We present three computational experiments demonstrating the efficacy of our methods, and conclude by discussing the implications of our results for modern RF applications.

Setzler, Matthew D.↗
DOE OSTI2025

Large-scale deep learning for metastasis detection in pathology reports

Objectives No existing algorithm can reliably identify metastasis from pathology reports across multiple cancer types and the entire US population. In this study, we develop a deep learning model that automatically detects patients with metastatic cancer by using pathology reports from many laboratories and of multiple cancer types. Materials and Methods We use 60 471 unstructured pathology reports from 4 Surveillance, Epidemiology, and End Results (SEER) registries. The reports were coded into 1 of 3 labels: metastasis negative, metastases positive, or metastasis undetermined. We utilize a task-specific deep neural network trained from scratch and compare its performance with a widely used large language model (LLM). Results Our deep learning architecture trained on task-specific data outperforms a general-purpose LLM, with a recall of 0.894 compared to 0.824. We quantified model uncertainty and used it to defer reports for human review. We found that retaining 72.9% of reports increased recall from 0.894 to 0.969. Discussion A smaller deep learning architecture trained on task-specific data outperforms a general LLM. Equally critical to model performance is the incorporation of uncertainty quantification, achieved here through an abstention mechanism. Conclusions This study’s finding demonstrate the feasibility of developing algorithms to automatically identify metastatic cancer cases from unstructured pathology reports.

machine learning↗
DOE OSTI2021

Versatile feature learning with graph convolutions and graph structures

Graphs represent real world relationships, and graph embedding projects nodes in a graph to a latent space that can help simplify downstream tasks. Recent development of graph convolutions in deep learning significantly improves the performance of many learning tasks on graphs. Unfortunately, prior embedding methods either do not embed graphs with node features, or fail to produce high-quality embeddings for downstream learning tasks that result in large performance gap in comparison to direct learning on graphs.We present a versatile and effective embedding method, Conv2Vec, for embedding graphs with or without node features. It is based on graph convolutions with objective functions motivated by concepts and structures from classical graph algorithms. Conv2Vec produce high-quality embedding for both plain graphs and graphs with node features for downstream tasks.We evaluate the embeddings generated by Conv2Vec with a transductive node classification task. With the generated embeddings and very simple machine learning approaches, we are able to achieve accuracies similar to those achieved by direct learning with graph convolutions. Interestingly, if we strip the node features from the graph and thus learning an embedding has to rely entirely on the graph topology, node classification with our embedding significantly outperforms direct learning with various graph convolutions. This suggests that structures from classical graph algorithms may play an important role in learning on graphs.

Cong, Guojing↗
DOE OSTI2023

Knowledge Distillation for Anomaly Detection

Unsupervised deep learning techniques are widely used to identify anomalous behaviour. The performance of such methods is a product of the amount of training data and the model size. However, the size is often a limiting factor for the deployment on resource-constrained devices. Here, we present a novel procedure based on knowledge distillation for compressing an unsupervised anomaly detection model into a supervised deployable one and we suggest a set of techniques to improve the detection sensitivity. Compressed models perform comparably to their larger counterparts while significantly reducing the size and memory footprint.

Pol, Adrian Alan↗
DOE OSTI2021

An Overview of the Usefulness of Machine Learning Techniques on Network Packet Data

Understanding the health and behavior of a computer network allows for better network efficiency and security. We present an overview of various machine learning techniques for classifying network packet data via packet metadata. While some classical machine learning approaches achieve reasonable results, the most accurate classification can be achieved with deep learning. On the four data sets studied herein, a basic deep learning model achieved at or near 100\% classification accuracy. We also propose a method for determining variable importance as a means for potential transfer learning applications to classifying yet unseen network packet data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Deep Learning Explicit Differentiable Predictive Control Laws for Buildings

We present a differentiable predictive control (DPC) methodology for learning constrained control laws for unknown nonlinear systems. DPC poses an approximate solution to multiparametric programming problems emerging from explicit nonlinear model predictive control (MPC). Contrary to approximate MPC, DPC does not require supervision by an expert controller. Instead, a system dynamics model is learned from a small dataset of recorded observations of the perturbed system's dynamics and the control law is optimized offline by interaction with the learned system model. The DPC method is based on two sequential steps, i) system identification using a constrained neural state-space model, and ii) optimization of an explicit control law parametrized by another neural network in closed-loop simulation with the identified neural state-space model. The combination of a differentiable closed-loop system and penalty methods for constraint handling of system outputs and inputs allows us to optimize the control law's parameters directly by backpropagating economic MPC loss through the learned system model. By incorporating domain knowledge and leveraging established techniques from optimal control, our method leverages deep neural networks as nonlinear function approximators for system identification and control while avoiding concomitant costs of intractably large datasets, and computationally expensive over-parametrized models. The scalability, data efficiency, and constrained optimal control capability of the proposed DPC method are demonstrated in simulation using a multi-zone building emulator.

Drgona, Jan↗
DOE OSTI2023

TopFusion: Using Topological Feature Space for Fusion and Imputation in Multi-Modal Data

We present a novel multi-modal data fusion technique using topological features. The method, TopFusion, leverages the flexibility of topological data analysis tools (namely persistent homology and persistence images) to map multi-modal datasets into a common feature space by forming a new multi-channel persistence image. Each channel in the image is representative of a view of the data from a modality-dependent filtration. We demonstrate that the topological perspective we take allows for more effective data reconstruction, i.e. imputation. In particular, by performing imputation in topological feature space we are able to outperform the same imputation techniques applied to raw data or alternatively derived features. We show that TopFusion representations can be used as input to downstream deep learning-based computer vision models and doing so achieves comparable performance to other fusion methods for classification on two multi-modal datasets.

Myers, Audun D.↗
DOE OSTI2026

Introduction to and comparison of deep learning and optimization approaches to analytical wake modeling of a tilted wind turbine

This paper introduces innovative optimization and deep learning techniques to enhance the prediction of complex wake dynamics in the downstream wind velocity of tilted wind turbines. Traditional methods for calibrating the Bastankhah wake model often lead to increased errors in wind velocity distribution due to overfitting of the local wake characteristics. To address this issue, we propose an additional global optimization step to reduce errors in wind velocity predictions with respect to various wake parameters. Despite this improvement, the Bastankhah model's axisymmetric Gaussian wake shape limits its accuracy for complex wake structures. Therefore, we also propose a deep learning approach, which demonstrates promising results by accurately modeling complex wake shapes across a broader range of tilt angles with minimal computational cost. The deep learning approach achieves near-identical predictions to high-fidelity large-eddy simulations, representing a promising advancement in wake modeling.

17 WIND ENERGY↗
DOE OSTI2021

DeepDist: real-value inter-residue distance prediction with deep residual convolutional network

Abstract Background Driven by deep learning, inter-residue contact/distance prediction has been significantly improved and substantially enhanced ab initio protein structure prediction. Currently, most of the distance prediction methods classify inter-residue distances into multiple distance intervals instead of directly predicting real-value distances. The output of the former has to be converted into real-value distances to be used in tertiary structure prediction. Results To explore the potentials of predicting real-value inter-residue distances, we develop a multi-task deep learning distance predictor (DeepDist) based on new residual convolutional network architectures to simultaneously predict real-value inter-residue distances and classify them into multiple distance intervals. Tested on 43 CASP13 hard domains, DeepDist achieves comparable performance in real-value distance prediction and multi-class distance prediction. The average mean square error (MSE) of DeepDist’s real-value distance prediction is 0.896 Å 2 when filtering out the predicted distance ≥ 16 Å, which is lower than 1.003 Å 2 of DeepDist’s multi-class distance prediction. When distance predictions are converted into contact predictions at 8 Å threshold (the standard threshold in the field), the precision of top L/5 and L/2 contact predictions of DeepDist’s multi-class distance prediction is 79.3% and 66.1%, respectively, higher than 78.6% and 64.5% of its real-value distance prediction and the best results in the CASP13 experiment. Conclusions DeepDist can predict inter-residue distances well and improve binary contact prediction over the existing state-of-the-art methods. Moreover, the predicted real-value distances can be directly used to reconstruct protein tertiary structures better than multi-class distance predictions due to the lower MSE. Finally, we demonstrate that predicting the real-value distance map and multi-class distance map at the same time performs better than predicting real-value distances alone.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

Multi-Task Reinforcement Learning for Distribution System Voltage Control With Topology Changes

This letter proposes a multi-task deep reinforcement learning (DRL) approach for distribution system voltage regulation considering topology changes via PV smart inverter control. The key idea is to encode the topology as an additional state for the DRL and leverage the multi-task learning scheme for joint learning of all task control policies. Unlike other DRL-based methods, our approach is robust to different topologies. Comparison results on the modified IEEE 123-node system demonstrate the enhanced robustness of the proposed method.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Quantitative Insight to Fission Gas Bubble Distribution in Irradiated Annular U-10Zr Metallic Fuel Using Deep Learning

U-10Zr Metal fuel is a promising nuclear fuel candidate for next-generation sodium-cooled fast spectrum reactors. Porosity is one of the key facts to impact the performance of metallic fuel. Additionally, a mechanical understanding of fission gas bubbles evolution behavior is a prerequisite for fuel development and qualification. Previous study of fission gas bubbles relied on a simple threshold method working on low resolution optical microscopy images, which has challenges in recognizing bubble boundaries, and caused inaccurate statistics of bubble properties. In this paper, a pre-trained deep learning model on Scanning Electron Microscopy (SEM) images from an annular U-10Zr fuel (AF1), was applied to another U-10Zr annular fuel (AF2). More accurate fission gas bubble segmentation results were generated, which leads to more precise qualitative analysis on the morphology, size, density, and orientation of bubbles. Furthermore, we investigated the lanthanide movement along the radial temperature gradient and obtained conclusive findings.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Deep-Learning Electron Diffractive Imaging

Here, we report the development of deep-learning coherent electron diffractive imaging at subangstrom resolution using convolutional neural networks (CNNs) trained with only simulated data. We experimentally demonstrate this method by applying the trained CNNs to recover the phase images from electron diffraction patterns of twisted hexagonal boron nitride, monolayer graphene, and a gold nanoparticle with comparable quality to those reconstructed by a conventional ptychographic algorithm. Fourier ring correlation between the CNN and ptychographic images indicates the achievement of a resolution in the range of 0.70 and 0.55 Å. We further develop CNNs to recover the probe function from the experimental data. The ability to replace iterative algorithms with CNNs and perform real-time atomic imaging from coherent diffraction patterns is expected to find applications in the physical and biological sciences.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Efficient proactive vehicle relocation for on-demand mobility service with recurrent neural networks

One major challenge for on-demand mobility service (OMS) providers is to seamlessly match empty vehicles with trip requests so that the total vacant mileage is minimized. In this work, we develop an innovative data-driven approach for devising efficient vehicle relocation policy for OMS that (1) proactively relocates vehicles before the demand is observed and (2) reduces the inequality among drivers' income so that the proactive relocation policy is fair and is likely to be followed by drivers. Our approach represents the fusion of optimization and machine learning methods, which comprises three steps: First, we formulate the optimal proactive relocation as an optimal/stable matching problems and solve for global optimal solutions based on historical data. Second, the optimal solutions are then grouped and fed to train the deep learning models which consist of fully connected layers and long short-term memory networks. Low rank approximation is introduced to reduce the model complexity and improve the training performances. Finally, we use the trained model to predict the relocation policy which can be implemented in real time. We conduct comprehensive numerical experiments and sensitivity analyses to demonstrate the performances of the proposed method using New York City taxi data. Here, the results suggest that our method will reduce empty mileage per trip by 54-70% under optimal matching strategy, and a 25-32% reduction can also be achieved by following stable matching strategy. We also validate that the predicted relocation policies are robust in the presence of uncertain passenger demand level and passenger trip-requesting behavior.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2023

Coarse-to-fine Task-driven Inpainting for Geoscience Images

The processing and recognition of geoscience images have wide applications. Most of existing researches focus on understanding the high-quality geoscience images by assuming that all the images are clear. However, in many real-world cases, the geoscience images might contain occlusions during the image acquisition. This problem actually implies the image inpainting problem in computer vision and multimedia. As far as we know, all the existing image inpainting algorithms learn to repair the occluded regions for a better visualization quality, they are excellent for natural images but not good enough for geoscience images, and they never consider the following geoscience task when developing inpainting methods. Here, this paper aims to repair the occluded regions for a better geoscience task performance and advanced visualization quality simultaneously, without changing the current deployed deep learning based geoscience models. Because of the complex context of geoscience images, we propose a coarse-to-fine encoder-decoder network with the help of designed coarse-to-fine adversarial context discriminators to reconstruct the occluded image regions. Due to the limited data of geoscience images, we propose a MaskMix based data augmentation method, which augments inpainting masks instead of augmenting original images, to exploit the limited geoscience image data. The experimental results on three public geoscience datasets for remote sensing scene recognition, cross-view geolocation and semantic segmentation tasks respectively show the effectiveness and accuracy of the proposed method. The code is available at: https://github.com/HMS97/Task-driven-Inpainting.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Emerging anomaly detection techniques for electronic health records: A survey

Background Anomaly detection in electronic health records (EHRs) is a cornerstone of biomedical informatics, with direct implications for patient safety, clinical decision-making, and the prevention of healthcare fraud. Once guided primarily by simple rule-based methods, the field has advanced rapidly, driven by increased computing power, richer and more detailed health data, and the rise of machine learning and deep learning techniques. The objective of this paper is to provide a comprehensive overview of modern approaches to detecting anomalies in EHRs, outlining their strengths, limitations, and relevance to key healthcare challenges. We review traditional statistical methods alongside newer ML- and DL-based strategies and hybrid models, with particular attention to how these techniques support transparency and build clinical trust. Methods This paper presents a thorough and critical survey through systematic review (PRISMA-based) of the latest anomaly detection strategies in time-sequence data domains within electronic health record systems. Results We explore a broad spectrum of methodologies, including statistical models, supervised and unsupervised learning approaches, hybrid frameworks, and state-of-the-art ML-based techniques that collectively advance the precision and scalability of detecting anomalies in complex clinical datasets. In addition to mapping current capabilities, we address the enduring challenges that hinder widespread implementation and provide a forward-looking perspective on the future of anomaly detection in the data-rich landscape of modern healthcare. Summary The advancement in AI-based approaches is reported along with the basic principles of the individual approaches and their applicability. The increased availability of high-quality data, advancements in DL approaches, and enhanced computation power are leading to more frequent adaptation of DL-based approaches. Emerging DL-based approaches that have been adapted in other domains or recently applied in the EHR domain are also discussed in detail. Although DL-based approaches can improve model predictions by incorporating comorbidities, their application is limited in low-frequency data domains (e.g., when the total available data remains in the single digits). Therefore, the user must carefully consider the application based on data availability.

Anomaly detection↗
DOE OSTI2020

Computing Poincaré maps using physics-informed deep learning

Outline: Describe principle of magnetically-confined nuclear fusion; Explain the role of Poincaré maps in fusion reactor design; Describe a new method for rapidly computing Poincaré maps using physics-informed machine learning

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

A deep learning approach to identify missing is-a relations in SNOMED CT

Abstract Objective SNOMED CT is the largest clinical terminology worldwide. Quality assurance of SNOMED CT is of utmost importance to ensure that it provides accurate domain knowledge to various SNOMED CT-based applications. In this work, we introduce a deep learning-based approach to uncover missing is-a relations in SNOMED CT. Materials and Methods Our focus is to identify missing is-a relations between concept-pairs exhibiting a containment pattern (ie, the set of words of one concept being a proper subset of that of the other concept). We use hierarchically related containment concept-pairs as positive instances and hierarchically unrelated containment concept-pairs as negative instances to train a model predicting whether an is-a relation exists between 2 concepts with containment pattern. The model is a binary classifier leveraging concept name features, hierarchical features, enriched lexical attribute features, and logical definition features. We introduce a cross-validation inspired approach to identify missing is-a relations among all hierarchically unrelated containment concept-pairs. Results We trained and applied our model on the Clinical finding subhierarchy of SNOMED CT (September 2019 US edition). Our model (based on the validation sets) achieved a precision of 0.8164, recall of 0.8397, and F1 score of 0.8279. Applying the model to predict actual missing is-a relations, we obtained a total of 1661 potential candidates. Domain experts performed evaluation on randomly selected 230 samples and verified that 192 (83.48%) are valid. Conclusions The results showed that our deep learning approach is effective in uncovering missing is-a relations between containment concept-pairs in SNOMED CT.

97 MATHEMATICS AND COMPUTING↗