Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “train”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 775 records · Page 43
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Deep residual networks for crystallography trained on synthetic data

The use of artificial intelligence to process diffraction images is challenged by the need to assemble large and precisely designed training data sets. To address this, a codebase called Resonet was developed for synthesizing diffraction data and training residual neural networks on these data. Here, two per-pattern capabilities of Resonet are demonstrated: (i) interpretation of crystal resolution and (ii) identification of overlapping lattices. Resonet was tested across a compilation of diffraction images from synchrotron experiments and X-ray free-electron laser experiments. Crucially, these models readily execute on graphics processing units and can thus significantly outperform conventional algorithms. While Resonet is currently utilized to provide real-time feedback for macromolecular crystallography users at the Stanford Synchrotron Radiation Lightsource, its simple Python-based interface makes it easy to embed in other processing frameworks. This work highlights the utility of physics-based simulation for training deep neural networks and lays the groundwork for the development of additional models to enhance diffraction collection and analysis.

36 MATERIALS SCIENCE↗
DOE OSTI2019

Adversarial Training for Privacy-Preserving Deep Learning Model Distribution

Collaboration among cancer registries is essential to develop accurate, robust, and generalizable deep learning models for automated information extraction from cancer pathology reports. Sharing data presents a serious privacy issue, especially in biomedical research and healthcare delivery domains. Distributing pretrained deep learning (DL) models has been proposed to avoid critical data sharing. However, there is growing recognition that collaboration among clinical institutes through DL model distribution exposes new security and privacy vulnerabilities. These vulnerabilities increase in natural language processing (NLP) applications, in which the dataset vocabulary with word vector representations needs to be associated with the other model parameters. In this paper, we propose a novel privacy-preserving DL model distribution across cancer registries for information extraction from cancer pathology reports with privacy and confidentiality considerations. The proposed approach exploits the adversarial training framework to distinguish private features from shared features among different datasets. It only shares registry-invariant model parameters, without sharing raw data nor registry-specific model parameters among cancer registries. Thus, it protects both the data and the trained model simultaneously. We compare our proposed approach to single-registry models, and a model trained on centrally hosted data from different cancer registries. The results show that the proposed approach significantly outperforms the single-registry models and achieves statistically indistinguishable micro and macro F1-score as compared to the centralized model.

Alawad, Mohammed↗
DOE OSTI2022

Adaptive Generation of Training Data for ML Reduced Model Creation

Machine learning proxy models are often used to speed up or completely replace complex computational models. The greatly reduced and deterministic computational costs enable new use cases such as digital twin control systems and global optimization. The challenge of building these proxy models is generating the training data. A naive uniform sampling of the input space can result in a non-uniform sampling of the output space of a model. This can cause gaps in the training data coverage that can miss finer scale details resulting in poor accuracy. While larger and larger data sets could eventually fill in these gaps, the computational burden of full-scale simulation codes can make this prohibitive. In this paper, we present an adaptive data generation method that utilizes uncertainty estimation to identify regions where training data should be augmented. By targeting data generation to areas of need, representative data sets can be generated efficiently. The effectiveness of this method will be demonstrated on a simple one-dimensional function and a complex multidimensional physics model.

Cianciosa, Mark↗
DOE OSTI2019

A Quantitative Study of Deep Learning Training on Heterogeneous Supercomputers

Deep learning (DL) has become a key technique for solving complex problems in scientific research and discovery. DL training for science is substantially challenging because it has to deal with massive quantities of multi-dimensional data. High- performance computing (HPC) supercomputers are increasingly being employed for meeting the exponentially growing demand for DL. Multiple GPUs and high-speed interconnect network are needed for supporting DL on HPC systems. However, the excessive use of GPUs without considering effective benefits leads to inefficient resource utilization of these expensive setups. In this paper, we conduct a quantitative analysis to gauge the efficacy of DL workloads on the latest HPC system and identify viability of next-generation DL-optimized heterogeneous supercomputers for enabling researchers to develop more efficient resource management and distributed DL middleware. We evaluate well- known DL models with large-scale datasets using the popular TensorFlow framework, and provide a thorough evaluation in- cluding scalability, accuracy, variability, storage resource, GPU- GPU/GPU-CPU data transfer, and GPU utilization. Our analysis reveals that the latest heterogeneous supercomputing cluster shows varying performance trend as compared to the existing literature for single- and multi-node training. To the best of our knowledge, this is the first work to conduct such a quantitative and comprehensive study of DL training on a supercomputing system with multiple GPUs.

Han, Jingoo↗
DOE OSTI2017

Hashed Binary Search Sampling for Convolutional Network Training with Large Overhead Image Patches

Very large overhead imagery associated with ground truth maps has the potential to generate billions of training image patches for machine learning algorithms. However, random sampling selection criteria often leads to redundant and noisy-image patches for model training. With minimal research efforts behind this challenge, the current status spells missed opportunities to develop supervised learning algorithms that generalize over wide geographical scenes. In addition, much of the computational cycles for large scale machine learning are poorly spent crunching through noisy and redundant image patches. We demonstrate a potential framework to address these challenges specifically, while evaluating a human settlement detection task. A novel binary search tree sampling scheme is fused with a kernel based hashing procedure that maps image patches into hash-buckets using binary codes generated from image content. The framework exploits inherent redundancy within billions of image patches to promote mostly high variance preserving samples for accelerating algorithmic training and increasing model generalization.

Lunga, Dalton↗
DOE OSTI2024

Multiport Converter based Auxiliary Power Supply for Heavy Duty Fuel Cell Power Train

This paper presents a multiport converter (MPC) based power supply to charge the 12 V and 24 V auxiliary batteries in heavy duty (HD) fuel cell (FC) electric vehicle (EV) power train. Compared to conventional auxiliary power supply architectures, the proposed architecture shares the power electronic components of the main power train yet preserves the isolation and all other desirable features of auxiliary power supplies. The proposed auxiliary power supply architecture reduces the requirements of the devices, gate drivers and thermal requirements thereby having potential benefits towards improving the power density, cost and weight of the overall power train. The topology, design parameters and control methodology are discussed, and results are presented to validate the proposed architecture.

Mukherjee, Subho↗
DOE OSTI2024

MDLoader: A Hybrid Model-Driven Data Loader for Distributed Graph Neural Network Training

Scalable data management is essential for processing large scientific dataset on HPC platforms for distributed deep learning. In-memory distributed storage is preferred for its speed, enabling rapid, random, and frequent data access required by stochastic optimizers. Processes use one-sided or collective communication to fetch remote data, with optimal performance depending on (i) dataset characteristics, (ii) training scale, and (iii) interconnection network. Empirical analysis shows collective communication excels with larger mini-batch sizes and/or fewer processes, whereas one-sided communication outperforms at larger scales. We propose MDLoader, a hybrid in-memory data loader for distributed graph neural network training. MDLoader features a model-driven performance estimator that dynamically selects between one-sided and collective communication at the beginning of training using Tree of Parzen Estimators (TPE). Evaluations on NERSC Perlmutter and OLCF Summit show MDLoader outperforms single-backend loaders by up to 2.83 × and predicts the suitable communication method with 96.3% (Perlmutter) and 94.3% (Summit) success rate.

Bae, Jonghyun↗
DOE OSTI2017

Analysis of the Training Behavior of the MICE Spectrometer Solenoid

Muon ionization cooling is theoretically well understood but has never been fully demonstrated experimentally. Ionization cooling is regarded as an important technology both in terms of cost and performance for a Neutrino Factory and is absolutely essential for a Muon Collider. The MICE experiment (Muon Ionization Cooling Experiment), based at the Rutherford Appleton Laboratory, U.K., is currently collecting data to prove the concept. Part of MICE are two spectrometer magnets, each consisting of five superconducting large bore solenoids. The spectrometer solenoid is designed for a peak field of 4T. Both spectrometer magnets required about 15 quenches to reach the design current. However, it was discovered that both spectrometers do not remember their training; after a warm-up the spectrometers have to be retrained, following a very similar training curve. The MICE spectrometer was analyzed using 2-D and 3-D finite element software to understand the quench and training behavior; the analysis revealed a clamping mechanism, leading to a stick-slip situation for one of the coils in its coil pocket. This paper summarizes the results and makes suggestions how to improve the design.

43 PARTICLE ACCELERATORS↗
DOE OSTI2019

Training a Neural Network on Analog TaO x ReRAM Devices Irradiated With Heavy Ions: Effects on Classification Accuracy Demonstrated With CrossSim

The image classification accuracy of a TaO x ReRAM-based neuromorphic computing accelerator is evaluated after intentionally inducing a displacement damage up to a fluence of 10 14 2.5-MeV Si ions/cm 2 on the analog devices that are used to store weights. Results are consistent with a radiation-induced oxygen vacancy production mechanism. When the device is in the high-resistance state during heavy ion radiation, the device resistance, linearity, and accuracy after training are only affected by high fluence levels. Here, the findings in this paper are in accordance with the results of previous studies on TaO x -based digital resistive random access memory. When the device is in the low-resistance state during irradiation, no resistance change was detected, but devices with a 4-kΩ inline resistor did show a reduction in accuracy after training at 10 14 2.5-MeV Si ions/cm 2 . This indicates that changes in resistance can only be somewhat correlated with changes to devices’ analog properties. This paper demonstrates that TaO x devices are radiation tolerant not only for high radiation environment digital memory applications but also when operated in an analog mode suitable for neuromorphic computation and training on new data sets.

42 ENGINEERING↗
DOE OSTI2024

An Incremental Tensor Train Decomposition Algorithm

We present a new algorithm for incrementally updating the tensor train decomposition of a stream of tensor data. This new algorithm, called the tensor train incremental core expansion (TT-ICE) improves upon the current state-of-the-art algorithms for compressing in tensor train format by developing a new adaptive approach that incurs significantly slower rank growth and guarantees compression accuracy. This capability is achieved by limiting the number of new vectors appended to the TT-cores of an existing accumulation tensor after each data increment. These vectors represent directions orthogonal to the span of existing cores and are limited to those needed to represent a newly arrived tensor to a target accuracy. We provide two versions of the algorithm: TT-ICE and TT-ICE accelerated with heuristics (TT-ICE*). Here, we provide a proof of correctness for TT-ICE and empirically demonstrate the performance of the algorithms in compressing large-scale video and scientific simulation datasets. Compared to existing approaches that also use rank adaptation, TT-ICE* achieves 57× higher compression and up to 95% reduction in computational time.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Development of systematic uncertainty-aware neural network trainings for binned-likelihood analyses at the LHC

We propose a neural network training method capable of accounting for the effects of systematic variations of the data model in the training process and describe its extension towards neural network multiclass classification. The procedure is evaluated on the realistic case of the measurement of Higgs boson production via gluon fusion and vector boson fusion in the τ τ decay channel at the CMS experiment. The neural network output functions are used to infer the signal strengths for inclusive production of Higgs bosons as well as for their production via gluon fusion and vector boson fusion. We observe improvements of 12 and 16% in the uncertainty in the signal strengths for gluon and vector-boson fusion, respectively, compared with a conventional neural network training based on cross-entropy.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

QuantumNAT: quantum noise-aware training with noise injection, quantization and normalization

Parameterized Quantum Circuits (PQC) are promising towards quantum advantage on near-term quantum hardware. However, due to the large quantum noises (errors), the performance of PQC models has a severe degradation on real quantum devices. Take Quantum Neural Network (QNN) as an example, the accuracy gap between noise-free simulation and noisy results on IBMQ-Yorktown for MNIST-4 classification is over 60%. Existing noise mitigation methods are general ones without leveraging unique characteristics of PQC; on the other hand, existing PQC work does not consider noise effect. To this end, we present QuantumNAT, a PQC-specific framework to perform noise-aware optimizations in both training and inference stages to improve robustness. We experimentally observe that the effect of quantum noise to PQC measurement outcome is a linear map from noise-free outcome with a scaling and a shift factor. Motivated by that, we propose post-measurement normalization to mitigate the feature distribution differences between noise-free and noisy scenarios. Furthermore, to improve the robustness against noise, we propose noise injection to the training process by inserting quantum error gates to PQC according to realistic noise models of quantum hardware. Lastly, post-measurement quantization is introduced to quantize the measurement outcomes to discrete values, achieving the denoising effect. Extensive experiments on 8 classification tasks using 6 quantum devices demonstrate that QuantumNAT improves accuracy by up to 43%, and achieves over 94% 2-class, 80% 4-class, and 34% 10-class classification accuracy measured on real quantum computers. The code for construction and noise-aware training of PQC is available in the TorchQuantum library.

Wang, Hanrui↗
DOE OSTI2023

Multiobjective Hyperparameter Optimization for Deep Learning Interatomic Potential Training Using NSGA-II

Deep neural network (DNN) potentials are an emerging tool for simulation of dynamical atomistic systems, with the promise of quantum mechanical accuracy at speedups of 10000$\times$. As with other DNN methods, hyperparameters used during training can make a substantial difference in model accuracy, and optimal settings vary with dataset. To enable rapid tuning of hyperparameters for DNN potential training, we developed a scalable multiobjective optimization evolutionary algorithm for supercomputers and tested it on the Summit system at the Oak Ridge Leadership Computing Facility (OLCF). The multiobjective approach is required due to the coupling of two learned values defining the potential: the energy and force. Using a large-scale implementation of the NSGA-II algorithm adapted for training DNN potentials, we discovered several optimal multiobjective combinations, including best choices of activation functions, learning rate scaling scheme, and pairing of the two radial cutoffs used in the three dimensional descriptor function.

Coletti, Mark↗
DOE OSTI2025

Intelligent Sampling of Extreme-Scale Turbulence Datasets for Accurate and Efficient Spatiotemporal Model Training

With the end of Moore’s law and Dennard scaling, efficient training increasingly requires rethinking data volume. Can we train better models with significantly less data via intelligent subsampling? To explore this, we develop SICKLE, a sparse intelligent curation framework for efficient learning, featuring a novel maximum entropy (MaxEnt) sampling approach, scalable training, and energy benchmarking. We compare MaxEnt with random and phase-space sampling on large direct numerical simulation (DNS) datasets of turbulence. Evaluating SICKLE at scale on Frontier, we show that subsampling as a preprocessing step can, in many cases, improve model accuracy and substantially lower energy consumption, with observed reductions of up to 38×.

Brewer, Wes [ORNL] (ORCID:0000000236393956)↗
DOE OSTI2021

Humming Trains in Seismology: An Opportune Source for Probing the Shallow Crust

Abstract Seismologists are eagerly seeking new and preferably low-cost ways to map and track changes in the complex structure of the top few kilometers of the crust. By understanding it better, they can build on what is known regarding important, practical issues. These include telling us whether imminent earthquakes and volcanic eruptions are generating telltale underground signs of hazard, about mitigation of induced seismicity such as from deep injection of wastewater, how the Earth and its atmosphere couple, and where accessible natural resources are. Passive seismic imaging usually relies on blind correlations within extended recordings of Earth’s ceaseless “hum” or coda of well-mixed, small vibrations. In this article, we propose a complementary approach. It is seismic interferometry using opportune sources—specifically ones not stationary in time and moving in a well-understood configuration. Its interpretation relies on an accurate understanding of how these sources radiate seismic waves, precise timing, careful placement of pairs of listening stations, and seismic phase differentiation (surface and body waves). Massive freight trains were only recently recognized as such a persistent, powerful cultural (human activity-caused) seismic source. One train passage may generate a tremor with an energy output of a magnitude 1 earthquake and be detectable for up to 100 km from the track. We discuss the source mechanisms of train tremors and review the basic theory on sources. Finally, we present case studies of body- and surface-wave retrieval as an aid to mineral exploration in Canada and to monitoring of a southern California fault zone. We believe noise recovery from this new signal source, together with dense data acquisition technologies such as nodes or distributed acoustic sensing, will deeply transform our ability to monitor activity in the shallow crust at sharpened resolution in time and space.

Geochemistry & Geophysics↗
DOE OSTI2017

Biosafety Level 3 Recon Training

The Biosafety Level 3 Recon training is a 3D virtual tool developed for the Counter WMD Analysis Cell (CWAC) and the Asymmetric Warfare Group (AWG) by the Application Modeling and Development Team within the NEN-3 International Threat Reduction Group. The training simulates a situation where friendly forces have secured from hostile forces a suspected bioweapons development laboratory. The trainee is a squad member tasked to investigate the facility, locate laboratories within the facility, and identify hazards to entrants and the surrounding area. Before beginning the 3D simulation, the trainee must select the appropriate MOPP level for entering the facility. The items in the simulation, including inside and outside the bioweapon facility, are items that are commonly used by scientists in Biosafety Level (BSL) laboratories. Each item has clickable red tags that, when activated, give the trainee a brief description of the item and a controllable turn-around view. The descriptions also contain information about potential hazards the item can present. Trainees must find all tagged items in order to complete the simulation, but can also reference descriptions and turn-around view of the items in a glossary menu. Training is intended to familiarize individuals whom have little or no biology or chemistry background with technical equipment used in BSL laboratories. The revised edition of this simulation (Biosafety Level 3 Virtual Lab) changes the trainee into a investigator instead of a military combatant. Many doors now require a virtual badge swipe to open. Airlock doors may come in sets such that the open door must be closed before the next door in the set can be opened. A user interface was added so that the instructor can edit the information about the items (the brief descriptions mentioned above) using the simulation software instead of the previous method of manually entering the material in xml settings files. Facility labels, such as "No Parking" and "Men's room", were changed from Korean, into English. No other changes were made.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2017

Improving Insider Threat Training Awareness and Mitigation Programs at Nuclear Facilities.

In recent years, insider threat programs have become an important aspect of nuclear security, and nuclear security training courses. However, many nuclear security insider threat programs fail to address the insider threat attack and monitoring potential that exists on information technology (IT) systems. This failure is critical because of the importance of information technology and networks in today’s world. IT systems offer an opportunity to perpetrate dangerous insider attacks, but they also present an opportunity to monitor for them and prevent them. This paper suggests a number of best practices for monitoring and preventing insider attacks on IT systems, and proposes the development of a new IT insider threat tabletop that can be used to help train nuclear security practitioners on how best to implement IT insider threat prevention best practices. The development of IT insider threat best practices and a practical tabletop exercise will allow nuclear security practitioners to improve nuclear security trainings as it integrates a critical part of insider threat prevention into the broader nuclear security system.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗