Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Network Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,495 records · Page 83
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

3D-ReG: A 3D ReRAM-based Heterogeneous Architecture for Training Deep Neural Networks

Deep neural network (DNN) models are being expanded to a broader range of applications. The computational capability of traditional hardware platforms cannot accommodate the growth of model complexity. Among recent technologies to accelerate DNN, resistive memory (ReRAM)-based processing-in-memory (PIM) emerged as a promising solution for DNN inference due to its high efficiency for matrix-based computation. We face two major technical challenges in extending the use of ReRAM-based accelerators for training: (1) full-precision data is essential in back-propagation; (2) the need to support both feed-forward and back-propagation aggravates the data-movement burden. We propose a heterogeneous architecture named as 3D-ReG, which leverages full-precision GPU to ensure training accuracy and low-overhead 3D integration to provide low-cost data movements. Moreover, we introduce conservative and aggressive task-mapping schemes, which partition the computation phases in different ways to balance execution efficiency and training accuracy. We evaluate 3D-ReG implemented with two 3D integration technologies, through-silicon vias (TSVs) and monolithic inter-tier vias (MIVs), and compare them with GPU-only and PIM-only counterparts. Various GPU-only platforms using two main-memory technologies (DRAM, ReRAM) and three interconnect technologies (2D, TSV, MIV) are evaluated as well. Experimental results show that 3D-ReG can achieve on average 5.64× training speedup and 3.56× higher energy efficiency compared with the GPU with DRAM as main memory, at the cost of 0.05%–3.39% accuracy drop. We define a new metric, gain-loss ratio (GLR), which quantitatively evaluates the capability of a DNN training hardware in terms of the model accuracy and hardware efficiency. The results of our comparison show that the aggressive task-mapping scheme on MIV-based 3D-ReG outperforms the other methods.

Computer Science↗
DOE OSTI2022

VelocityGAN

VelocityGAN is a software to solve subsurface imaging problems. It leverages a generative adversarial network to model the correspondence from seismic data to subsurface velocity structures.

Lin, Youzuo↗
DOE OSTI2022

Deep Learning Denoiser

SAND2022-14744 O The Deep Learning Denoiser uses a trained deep convolutional neural network (CNN) model to decompose an input waveform into a signal and noise waveform. In the approach, the CNN provides a signal mask and a noise mask for an input signal. The Short-Time Fourier Transform (STFT) of the estimated signal is obtained by multiplying the signal mask with the STFT of the input signal. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Koch, Clinton↗
DOE OSTI2023

IdahoLabResearch/INLIDD

This code along with the included data allows users to train machine learning models on network packet data.

Wadsworth, JaredC. (0000000191873672)↗
DOE OSTI2024

Defect diffusion graph neural networks (d2gnn)

SAND2025-01004O Defect Diffusion Graph Neural Networks (d2gnn) is a software tool that assists in the discovery of new materials for high-temperature, clean-energy applications. It uses advanced graph neural networks to model the relationship between material structures and their defect properties. The application helps predict how materials will behave under different conditions and accelerates the development of innovative materials. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Witman, Matthew [Sandia National Lab. (SNL-CA), Li↗
DOE OSTI2022

TULIP: An RNA-seq-based Primary Tumor Type Prediction Tool Using Convolutional Neural Networks

Background: With cancer as one of the leading causes of death worldwide, accurate primary tumor type prediction is critical in identifying genetic factors that can inhibit or slow tumor progression. There have been efforts to categorize primary tumor types with gene expression data using machine learning, and more recently with deep learning, in the last several years. Methods In this paper, we developed four 1-dimensional (1D) Convolutional Neural Network (CNN) models to classify RNA-seq count data as one of 17 highly represented primary tumor types or 32 primary tumor types regardless of imbalanced representation. Additionally, we adapted the models to take as input either all Ensembl genes (60,483) or protein coding genes only (19,758). Unlike previous work, we avoided selection bias by not filtering genes based on expression values. RNA-seq count data expressed as FPKM-UQ of 9,025 and 10,940 samples from The Cancer Genome Atlas (TCGA) were downloaded from the Genomic Data Commons (GDC) corresponding to 17 and 32 primary tumor types respectively for training and validating the models. Results: All 4 1D-CNN models had an overall accuracy of 94.7% to 97.6% on the test dataset. Further evaluation indicates that the models with protein coding genes only as features performed with better accuracy compared to the models with all Ensembl genes for both 17 and 32 primary tumor types. For all models, the accuracy by primary tumor type was above 80% for most primary tumor types. Conclusions: We packaged all 4 models as a Python-based deep learning classification tool called TULIP (TUmor CLassIfication Predictor) for performing quality control on primary tumor samples and characterizing cancer samples of unknown tumor type. Further optimization of the models is needed to improve the accuracy of certain primary tumor types.

Jones, Sara↗
DOE OSTI2024

Variation in forest root image annotation by experts, novices, and AI

Abstract Background The manual study of root dynamics using images requires huge investments of time and resources and is prone to previously poorly quantified annotator bias. Artificial intelligence (AI) image-processing tools have been successful in overcoming limitations of manual annotation in homogeneous soils, but their efficiency and accuracy is yet to be widely tested on less homogenous, non-agricultural soil profiles, e.g., that of forests, from which data on root dynamics are key to understanding the carbon cycle. Here, we quantify variance in root length measured by human annotators with varying experience levels. We evaluate the application of a convolutional neural network (CNN) model, trained on a software accessible to researchers without a machine learning background, on a heterogeneous minirhizotron image dataset taken in a multispecies, mature, deciduous temperate forest. Results Less experienced annotators consistently identified more root length than experienced annotators. Root length annotation also varied between experienced annotators. The CNN root length results were neither precise nor accurate, taking ~ 10% of the time but significantly overestimating root length compared to expert manual annotation ( p = 0.01). The CNN net root length change results were closer to manual ( p = 0.08) but there remained substantial variation. Conclusions Manual root length annotation is contingent on the individual annotator. The only accessible CNN model cannot yet produce root data of sufficient accuracy and precision for ecological applications when applied to a complex, heterogeneous forest image dataset. A continuing evaluation and development of accessible CNNs for natural ecosystems is required.

Handy, Grace↗
DOE OSTI2021

Data for Adoption of Perennial Energy Crops in the U.S. Midwest, Causal and Heterogeneous Determinants

The success of a bioenergy policy relies largely on the wide adoption of perennial energy crops at the farm scale. This study uses survey data to examine potential adoption decisions by farmers in the U.S. Midwest and the causal effects of various direct and indirect influencing factors, especially heterogeneous preferences of farmers. A Bayesian network (BN) model is developed to delineate the causal relationship between farmers adoption decisions and the influencing factors. We find a dominating role of economic factors and a non-negligible impact of non-economic factors, such as the perceived environmental benefits and the extent of familiarity with perennial energy crops. To examine the effect of heterogeneity in farmer preferences, we classify the surveyed farmers into four categories based on their attitudes toward the economic, social, and environmental dimensions of perennial energy crops. We identified statistically significant between-group differences in the responses of the four types of farmers to the various influencing factors. Our findings contribute to disentangling the complicated motivations that will influence perennial energy crop adoption decisions and provide implications for more targeted policy development that need to consider the heterogeneous drivers of farmer decisions about land use.

Modeling↗
DOE OSTI2025

Fiats: Functional inference and training for surrogates

Fiats provides a platform for research on the training and deployment of neural-network surrogate models for computational science. Fiats also supports exploring, advancing, and combining functional, object-oriented, and parallel programming patterns in Fortran 2023. As such, the Fiats name has dual expansions: “Functional Inference And Training for Surrogates” or “Fortran Inference And Training for Science.” Fiats inference and training procedures are pure and therefore satisfy a language constraint imposed on procedure invocations inside Fortran’s parallel loop construct: do concurrent. Furthermore, the Fiats training procedures are built around a do concurrent parallel reduction. Several compilers can automatically parallelize do concurrent on Central Processing Units (CPUs) or Graphics Processing Units (GPUs). Fiats thus aims to achieve performance portability through standard language mechanisms.

Rouson, Damian [Lawrence Berkeley National Laborat↗
DOE OSTI2023

LDRD23-0730: Invoking Multilayer Networks to Develop a Paradigm for Security Science—Summary Report

Current approaches to securing high consequence facilities (HCF) and critical assets are linear and static and therefore struggle to adapt to emerging threats (e.g., unmanned aerial systems) and changing environmental conditions (e.g., decreasing operational control). The pace of change in technological, organizational, societal, and political dynamics necessitates a move toward codifying underlying scientific principles to better characterize the rich interactions observed between HCF security technology, infrastructure, digital assets, and human or organizational components. The promising results of Laboratory Directed Research and Development (LDRD) 20-0373—“Developing a Resilient, Adaptive, and Systematic Paradigm for Security Analysis”—suggest that when compared to traditional security analysis, invoking multilayer network (MLN) modeling for HCF security system components captures unexpected failure cases and unanticipated interactions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

DECOVALEX-2023: Task F2 Salt Final Report

The subject of Task F of DECOVALEX-2023 concerns performance assessment modelling of radioactive waste disposal in deep mined repositories. The primary objectives of Task F are to build confidence in the models, methods, and software used for performance assessment (PA) of deep geologic nuclear waste repositories, and/or to bring to the fore additional research and development needed to improve PA methodologies. In Task F2- (salt), these objectives have been accomplished through staged development and comparison of the models and methods used by participating teams in their PA frameworks. Coupled-process submodels and deterministic simulations of the entire PA model for a reference scenario for waste disposal in domal salt have been conducted. The task specification has been updated continuously since the initiation of the project to reflect the staged development of the conceptual repository model and performance metrics. Thermal, hydrological, mechanical, and chemical properties of individual components of the engineered and natural system were chosen for relevance by participating teams. The salt reference case system was characterized using data and measurements collected at relevant underground research laboratories (URLs), field sites, and simulation results from teams with specialized modelling capability. Participating teams made a wide range of model assumptions from compartmentalized networks to full 3D models of the salt formation. No single contributed model includes full-fidelity representation of all the features, events, and processes (FEPs) detailed in the task specification, but almost all features and processes are represented in at least one model. Despite differences in the modelling strategies developed by participating teams, all models indicate that salt compaction and radionuclide diffusion are key processes in the repository, and for the FEPs and model scenario considered, little of the disposed radionuclides will migrate beyond the repository seal over the 100,000 year simulations. In general, the model output quantities have the largest differences over the short term and near the waste. The models tend to be more similar further from waste and at later time. Disparities between the models are believed to be due to differing simplifications from the task specification, some of which are chosen simplifications to reduce complexity, and some are restrictions imposed by the modelling tools. A second round of this task has been accepted for DECOVALEX-2027 in conjunction with Task F1 on crystalline PA modelling. The future round includes waste package heating, improved modelling of salt creep closure, additional comparisons of coupled-process sub-models, and the impact of repository engineering design on radionuclide migration in the repository. Participants will also propose and finalize a set of uncertain inputs for the reference case simulations, propagate these uncertainties in a set of realizations, and conduct sensitivity analyses on the simulation results.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2025

Energy-efficient, Large-scale Molecular Dynamics Simulations via Hardware- and Algorithm-level Optimization

This work aims to develop a framework for energy-efficient computing that will enable molecular dynamics (MD) simulations of large-scale phenomena with atomic precision and simultaneously remove computational bottlenecks limiting the speed of MD simulations. We seek to implement such an approach through the development of surrogate models for the interatomic force calculation combined with the use of mixed numerical precision formats. For a model system of neutral atoms (only pairwise interactions), significant force calculation efficiency improvements were achieved, without detrimental effects on atomic structures or average energies, using single precision, by developing a surrogate model (deep neural network), and by quantizing this surrogate model. For a model system of charged atoms, the reciprocal-space calculation of electrostatic interactions was identified as the main bottleneck, and the development of a surrogate model should be pursued to achieve an estimated one-order-of-magnitude additional speedup.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2026

PIP-II LLRF Master Oscillator and Precision Reference Line - Station Level Design and Testing

The PIP-II superconducting linac at Fermilab requires a highly stable RF Reference Line to maintain phase syn- chronization throughout the accelerator. Temperature- induced changes in the electrical length of long coaxial cables can introduce phase drift and measurement errors. The reference-line architecture mitigates these effects by phase averaging the forward and reflected RF signals, while a phase-locked loop anchors the system to the mas- ter oscillator. This work focuses on the characterization, validation, and mechanical integration of station-level RF assemblies using CAD modeling, vector network analyzer measurements, and spectrum analyzer testing. PID-controlled thermal plates will stabilize critical RF components and further reduce temperature-dependent phase and amplitude variations. These methods sup- port repeatable, standardized designs that can be reliably integrated across the different reference-line stations.

Mosher, Alexander [U. Illinois, Chicago]↗
DOE OSTI2024

Addressing GPU memory limitations for Graph Neural Networks in High-Energy Physics applications

Introduction Reconstructing low-level particle tracks in neutrino physics can address some of the most fundamental questions about the universe. However, processing petabytes of raw data using deep learning techniques poses a challenging problem in the field of High Energy Physics (HEP). In the Exa.TrkX Project, an illustrative HEP application, preprocessed simulation data is fed into a state-of-art Graph Neural Network (GNN) model, accelerated by GPUs. However, limited GPU memory often leads to Out-of-Memory (OOM) exceptions during training, due to the large size of models and datasets. This problem is exacerbated when deploying models on High-Performance Computing (HPC) systems designed for large-scale applications. Methods We observe a high workload imbalance issue during GNN model training caused by the irregular sizes of input graph samples in HEP datasets, contributing to OOM exceptions. We aim to scale GNNs on HPC systems, by prioritizing workload balance in graph inputs while maintaining model accuracy. Our paper introduces diverse balancing strategies aimed at decreasing the maximum GPU memory footprint and avoiding the OOM exception, across various datasets. Results Our experiments showcase memory reduction of up to 32.14% compared to the baseline. We also demonstrate the proposed strategies can avoid OOM in application. Additionally, we create a distributed multi-GPU implementation using these samplers to demonstrate the scalability of these techniques on the HEP dataset. Discussion By assessing the performance of these strategies as data loading samplers across multiple datasets, we can gauge their effectiveness in both single-GPU and distributed environments. Our experiments, conducted on datasets of varying sizes and across multiple GPUs, broaden the applicability of our work to various GNN applications that handle input datasets with irregular graph sizes.

Lee, Claire Songhyun↗
DOE OSTI2022

Embedded, Real-Time, and Distributed Traveling Wave Fault Location Method Using Graph Convolutional Neural Networks

This work proposes and develops an implementation of a fault location method to provide a fast and resilient protection scheme for power distribution systems. The method analyzes the transient dynamics of traveling waves (TWs) to generate features using the discrete wavelet transform (DWT), which are then used to train several graph convolutional network (GCN) models. Faults are simulated in the IEEE 34-node system, which is divided into three protection zones (PZs). The goal is to identify the PZ in which the fault occurs. The GCN models create a distributed protection scheme, as all nodes are able to retrieve a prediction. Given that message-passing between nodes occurs both during training and in the execution of the model, the resiliency of such schemes to communication losses was analyzed and demonstrated. One of the models, which only uses voltage measurements, was implemented on a Texas Instruments F28379D development board. The execution times were monitored to assess the speed of the protection scheme. It is shown that the proposed method can be executed in approximately a millisecond, which is comparable to existing TW protection in the transmission system. For experimental purposes, a DWT-based detection method is employed. A design of a setup to playback TWs using two development boards is also addressed.

Jiménez-Aparicio, Miguel (ORCID:000000016864461X)↗