Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “TRAINING”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Reliable Determination of Pulses and Pulse-Shape Instability in Ultrashort Laser Pulse Trains Using Polarization-Gating and Transient-Grating Frequency-Resolved Optical Gating Using the RANA Approach

Devices that measure the presence of instability in the pulse shapes in trains of ultrashort laser pulses do not exist, so this task necessarily falls to pulse-measurement devices, like Frequency-Resolved Optical Gating (FROG) and its variations, which have proven to be a highly reliable class of techniques for measuring stable trains of ultrashort laser pulses. Fortunately, multi-shot versions of FROG have also been shown to sensitively distinguish trains of stable from those of unstable pulse shapes by displaying readily visible systematic discrepancies between the measured and retrieved traces in the presence of unstable pulse trains. However, the effects of pulse-shape instability and algorithm stagnation can be indistinguishable, so a never-stagnating algorithm—even when instability is present—is required and is generally important. In previous work, we demonstrated that our recently introduced Retrieved-Amplitude N-grid Algorithmic (RANA) approach produces highly reliable (100%) pulse-retrieval in the second-harmonic-generation (SHG) version of FROG for thousands of sample trains of pulses with stable pulse shapes. Further, it does so even for trains of unstable pulse shapes and thus both reliably distinguishes between the two cases and provides a rough measure of the degree of instability as well as a reasonable estimate of most typical pulse parameters. Here, we perform the analogous study for the polarization-gating (PG) and transient-grating (TG) versions of FROG, which are often used for higher-energy pulse trains. We conclude that PG and TG FROG, coupled with the RANA approach, also provide reliable indicators of pulse-shape instability. In addition, for PG and TG FROG, the RANA approach provides an even better estimate of a typical pulse in an unstable pulse train than SHG FROG does, even in cases of significant pulse-shape instability.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Propagating Uncertainties in the SALT3 Model-training Process to Cosmological Constraints

Type Ia supernovae (SNe Ia) are standardizable candles that must be modeled empirically to yield cosmological constraints. To understand the robustness of this modeling to variations in the model-training procedure, we build an end-to-end pipeline to test the recently developed SALT3 model. We explore the consequences of removing pre-2000s low-z or poorly calibrated U-band data, adjusting the amount and fidelity of SN Ia spectra, and using a model-independent framework to simulate the training data. We find that the SALT3 model surfaces are improved by having additional spectra and U-band data, and can be shifted by ~5% if host-galaxy contamination is not sufficiently removed from SN spectra. We find that resulting measurements of w are consistent to within 2.5% for all of the training variants explored in this work, with the largest shifts coming from variants that add color-dependent calibration offsets or host-galaxy contamination to the training spectra and those that remove pre-2000s low-z data. These results demonstrate that the SALT3 model-training procedure is largely robust to reasonable variations in the training data, but that additional attention must be paid to the treatment of spectroscopic data in the training process. We also find that the training procedure is sensitive to the color distributions of the input data—the resulting w measurement can be biased by ~2% if the color distribution is not sufficiently wide. Future low-z data, particularly u-band observations and high signal-to-noise ratio SN Ia spectra, will help to significantly improve SN Ia modeling in the coming years.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Geographic Information System Based Emergency Response Training Assessments for DOE Radioactive Materials Transport - 20027

Safety and security are priorities of U.S. Department of Energy (DOE) radioactive materials shipping campaigns. In the more than 70-year history of domestic transport of spent nuclear fuel (SNF), there has never been a transportation-related radiological injury. To support transportation planning, among the tools that DOE uses is the Stakeholder Tool for Assessing Radioactive Transportation (START). START contains geospatial data and transportation route analyses capabilities designed to support a range of DOE transportation planning initiatives. One of those functions is the capability to support emergency response planning and training for State and Tribal jurisdictions located on routes used for DOE shipments of radioactive materials. As part of the Department's commitment to public safety, DOE provides federally-funded radiological response training to emergency responders along DOE radioactive materials transportation corridors through its Transportation Emergency Preparedness Program (TEPP). START contains spatial data representing the locations and emergency-response capabilities of fire departments, police, hospitals, State emergency response centers, and where TEPP-trained personnel are based. The START tool supports State and Tribal users' ability to evaluate emergency-response coverage on active and potential DOE radioactive materials transport routes through their jurisdictions, provide expected response times to reach the scene of an incident, identify equipment available to support a response, and identify the number of response personnel and their respective training levels. In addition, START can be used to identify gaps in coverage along a transportation corridor where additional radiological emergency response training may be needed. This paper describes the data, features, and functionality DOE uses to provide a resource for emergency response training needs assessments for States and Tribes along active and potential routes for transporting radioactive materials, and illustrates its use. It also discusses future plans to integrate TEPP and Federal Emergency Management Agency (FEMA) radiological training data to provide a more comprehensive source of geospatial information on personnel who have received equivalent radiological response training. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2023

Evaluation of pre-training large language models on leadership-class supercomputers

Large language models (LLMs) have arisen rapidly to the center stage of artificial intelligence as the foundation models applicable to many downstream learning tasks. However, how to effectively build, train, and serve such models for many high-stake and first-principle-based scientific use cases are both of great interests and of great challenges. Moreover, pre-training LLMs with billions or even trillions of parameters can be prohibitively expensive not just for academic institutions, but also for well-funded industrial and government labs. Furthermore, the energy cost and the environmental impact of developing LLMs must be kept in mind. Here, in this work, we conduct a first-of-its-kind performance analysis to understand the time and energy cost of pre-training LLMs on the Department of Energy (DOE)’s leadership-class supercomputers. Employing state-of-the-art distributed training techniques, we evaluate the computational performance of various parallelization approaches at scale for a range of model sizes, and establish a projection model for the cost of full training. Our findings provide baseline results, best practices, and heuristics for pre-training such large models that should be valuable to HPC community at large. We also offer insights and optimization strategies for using the first exascale computing system, Frontier, to train models of the size of GPT-3 and beyond.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Current Status of the DOE/NNSA Nuclear Criticality Safety Program Hands-on Criticality Safety Training Courses

In 2011, the US Department of Energy/National Nuclear Security Administration (DOE/NNSA) Nuclear Criticality Safety Program (NCSP) developed and piloted a 2-week nuclear criticality safety (NCS) practitioner course to support training and qualification of new NCS staff. The course was developed in accordance with the American National Standard Institute/American Nuclear Society (ANSI/ANS) standard for NCS training and qualifications (ANSI/ANS-8.26-2007). In 2013, an NCS manager’s course was developed for process supervisors, managers, regulators, and other professionals with NCS-related responsibilities. These courses consist of classroom education, facility training, and hands-on subcritical and critical experiments training. Each course is currently offered twice per year. The 2-week practitioner course offers a week of classroom training, with practical workshops and exercises focused on teaching students how to perform an NCS evaluation. The second week of training involves hands-on critical and subcritical experiments and measurements. The first week is offered in Las Vegas, Nevada, at the DOE Nevada Field Office or the National Atomic Testing Museum. Depending on the student’s clearance level, the second week is offered at Sandia National Laboratory (SNL) (uncleared and L-cleared students) or at the National Criticality Experiments Research Center (NCERC) (Q-cleared students). The 1-week manager’s course is offered at SNL or NCERC, depending on clearance or interest, and includes classroom and hands-on critical and subcritical experiments and measurements. This paper provides an overview and status report for the DOE/NNSA NCSP training courses in NCS and provides information about future course offerings.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Nuclear Criticality Safety Training: Needs and Efforts

Nuclear criticality safety (NCS) training is essential and required for NCS professionals to ensure that fissionable materials outside of nuclear reactors are safely processed, handled, stored, and transported. Furthermore, the requirement of training in NCS is mandated in several international and domestic documents. In this regard, management has an obligation to establish an NCS training program that provides confidence in the continuing proficiency of personnel, and supervisors must ensure that their staff is suitably trained. Training of all personnel in nuclear facilities is required to support the robust performance of operations. There is a continuing need to maintain, enhance, and develop new training courses to support the NCS community. This paper provides an outline of some of the ongoing and future training courses in the United States to support the field of NCS.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2020

Accounting for Training Data Error in Machine Learning Applied to Earth Observations

Remote sensing, or Earth Observation (EO), is increasingly used to understand Earth system dynamics and create continuous and categorical maps of biophysical properties and land cover, especially based on recent advances in machine learning (ML). ML models typically require large, spatially explicit training datasets to make accurate predictions. Training data (TD) are typically generated by digitizing polygons on high spatial-resolution imagery, by collecting in situ data, or by using pre-existing datasets. TD are often assumed to accurately represent the truth, but in practice almost always have error, stemming from (1) sample design, and (2) sample collection errors. The latter is particularly relevant for image-interpreted TD, an increasingly commonly used method due to its practicality and the increasing training sample size requirements of modern ML algorithms. TD errors can cause substantial errors in the maps created using ML algorithms, which may impact map use and interpretation. Despite these potential errors and their real-world consequences for map-based decisions, TD error is often not accounted for or reported in EO research. Here we review the current practices for collecting and handling TD. We identify the sources of TD error, and illustrate their impacts using several case studies representing different EO applications (infrastructure mapping, global surface flux estimates, and agricultural monitoring), and provide guidelines for minimizing and accounting for TD errors. To harmonize terminology, we distinguish TD from three other classes of data that should be used to create and assess ML models: training reference data, used to assess the quality of TD during data generation; validation data, used to iteratively improve models; and map reference data, used only for final accuracy assessment. We focus primarily on TD, but our advice is generally applicable to all four classes, and we ground our review in established best practices for map accuracy assessment literature. EO researchers should start by determining the tolerable levels of map error and appropriate error metrics. Next, TD error should be minimized during sample design by choosing a representative spatio-temporal collection strategy, by using spatially and temporally relevant imagery and ancillary data sources during TD creation, and by selecting a set of legend definitions supported by the data. Furthermore, TD error can be minimized during the collection of individual samples by using consensus-based collection strategies, by directly comparing interpreted training observations against expert-generated training reference data to derive TD error metrics, and by providing image interpreters with thorough application-specific training. We strongly advise that TD error is incorporated in model outputs, either directly in bias and variance estimates or, at a minimum, by documenting the sources and implications of error. TD should be fully documented and made available via an open TD repository, allowing others to replicate and assess its use. To guide researchers in this process, we propose three tiers of TD error accounting standards. Finally, we advise researchers to clearly communicate the magnitude and impacts of TD error on map outputs, with specific consideration given to the likely map audience.

58 GEOSCIENCES↗
DOE OSTI2021

Industrial Energy Assessment Training Effectiveness Evaluation: An Eye-Tracking Study

It is essential to understand the effectiveness of any training program so it can be improved accordingly. Various studies have applied standard metrics for the evaluation of visual behavior to recognize the areas of interest that attract individuals’ attention as there is a high correlation between attentional behavior and where one is focusing on. However, through reviewing the literature, we believe that studies that applied eye-tracking technologies for training purposes are still limited, especially in the industrial energy assessment training field. In this paper, the effectiveness of industrial energy assessment training was quantitatively evaluated by measuring the attentional allocation of trainees using eye-tracking technology. Moreover, this study identifies the areas that require more focus based on evaluating the performance of subjects after receiving the training. Additionally, this research was conducted in a controlled environment to remove the distractions that may be caused by environmental factors to only concentrate on variables that influence the learning behavior of subjects. The experiment results showed that after receiving the training, the subjects’ performance in energy assessment was significantly improved in two areas: production, and recycling and waste management, and the designed training program enhanced the knowledge of participants in identifying energy-saving opportunities to the knowledge level of experienced participants.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

Overview and Current Progress of the DOE/NNSA Nuclear Criticality Safety Program Training and Education Program

Since 2011, the US Department of Energy/National Nuclear Security Administration (DOE/NNSA) Nuclear Criticality Safety Program (NCSP) has been providing a two-week nuclear criticality safety (NCS) practitioner course to support the training and qualification of new NCS staff. In 2013, an NCS Manager’s course was developed for process supervisors, managers, regulators, and other professionals with NCS-related responsibilities, and, in 2017, an additional course was proposed for Criticality Safety Officers (CSOs). In addition to hands-on training, other NCSP-funded tasks include training on sensitivity/uncertainty methods, NCS resource pipeline tasks, development of NCS guides, maintenance and development of Nuclear Criticality Safety Education Training (NCSET) modules and designing of a new subcritical assembly for hands-on training purposes. This paper provides an overview of the NCSP training and education program. The status of the NCSP hands-on training program will also be provided.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2021

When and why PINNs fail to train: A neural tangent kernel perspective

Physics-informed neural networks (PINNs) have lately received great attention thanks to their flexibility in tackling a wide range of forward and inverse problems involving partial differential equations. However, despite their noticeable empirical success, little is known about how such constrained neural networks behave during their training via gradient descent. More importantly, even less is known about why such models sometimes fail to train at all. Here in this work, we aim to investigate these questions through the lens of the Neural Tangent Kernel (NTK); a kernel that captures the behavior of fully-connected neural networks in the infinite width limit during training via gradient descent. Specifically, we derive the NTK of PINNs and prove that, under appropriate conditions, it converges to a deterministic kernel that stays constant during training in the infinite-width limit. This allows us to analyze the training dynamics of PINNs through the lens of their limiting NTK and find a remarkable discrepancy in the convergence rate of the different loss components contributing to the total training error. To address this fundamental pathology, we propose a novel gradient descent algorithm that utilizes the eigenvalues of the NTK to adaptively calibrate the convergence rate of the total training error. Finally, we perform a series of numerical experiments to verify the correctness of our theory and the practical effectiveness of the proposed algorithms.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Training data selection for accuracy and transferability of interatomic potentials

Abstract Advances in machine learning (ML) have enabled the development of interatomic potentials that promise the accuracy of first principles methods and the low-cost, parallel efficiency of empirical potentials. However, ML-based potentials struggle to achieve transferability, i.e., provide consistent accuracy across configurations that differ from those used during training. In order to realize the promise of ML-based potentials, systematic and scalable approaches to generate diverse training sets need to be developed. This work creates a diverse training set for tungsten in an automated manner using an entropy optimization approach. Subsequently, multiple polynomial and neural network potentials are trained on the entropy-optimized dataset. A corresponding set of potentials are trained on an expert-curated dataset for tungsten for comparison. The models trained to the entropy-optimized data exhibited superior transferability compared to the expert-curated models. Furthermore, the models trained to the expert-curated set exhibited a significant decrease in performance when evaluated on out-of-sample configurations.

36 MATERIALS SCIENCE↗
DOE OSTI2020

A Look Inside the Black Box: Using graph-theoretical descriptors to interpret a Continuous-Filter Convolutional Neural Network (CF-CNN) trained on the global and local minimum energy structures of neutral water clusters

A Continuous Filter Convolutional Neural Network (CF-CNN) was trained to predict the potential energy of water cluster networks \ce{(H2O)_{\textit{N}}}, \textit{N}=10--30, corresponding to local minima lying within 5 kcal/mol from the putative minima taken from a newly published database containing over 5 million unique networks. The chemical sampling space of the database was characterized using chemical descriptors derived from graph theory, which led to the identification of important trends in the topology, connectivity, polygon structures associated with the various networks as a function of cluster size. The resulting graphs are available alongside the original database at \url{https://sites.uw.edu/wdbase/}. The CF-CNN trained on a subset of 500,000 networks for (\textit{N}=10, 30) yielded a mean absolute error of 0.002$\pm$0.002 kcal/mol per water molecule, giving the trained CF-CNN the highest accuracy of any neural network-based surrogate model to date. In addition, clusters of sizes not included in the training set exhibited errors of the same magnitude, indicating that the CF-CNN ptotocol is general enough to accurately predict energies of networks for both smaller and larger sizes than those used during training. The graph-theoretical descriptors were developed in order to analyze the properties of the full database and interpret the predictive power of the CF-CNN. Using topology measures, such as the Wiener index and the average shortest path length along with two similarity measures, we showed that all networks from the test set were within the range of the ones from the training set, suggesting that the training set covered the chemical space of interest quite well. Our graph analysis suggests that the mean degree and number of polygons for networks with larger errors tend to lie further from the mean than those with lower errors. The generality of the used CF-CNN was thus demonstrated, while the use of the graph-theoretical descriptors assisted in interpreting the predicted results.

Bilbrey, Jenna A.↗
DOE OSTI2021

A Software Framework for Comparing Training Approaches for Spiking Neuromorphic Systems

There are a wide variety of training approaches for spiking neural networks for neuromorphic deployment. However, it is often not clear how these training algorithms perform or compare when applied across multiple neuromorphic hardware platforms and multiple datasets. In this work, we present a software framework for comparing performance across four neuromorphic training algorithms across three neuromorphic simulators and four simple classification tasks. We introduce an approach for training a spiking neural network using a decision tree, and we compare this approach to training algorithms based on evolutionary algorithms, back-propagation, and reservoir computing. We present a hyperparameter optimization approach to tune the hyperparameters of the algorithm, and show that these optimized hyperparameters depend on the processor, algorithm, and classification task. Finally, we compare the performance of the optimized algorithms across multiple metrics, including accuracy, training time, and resulting network size, and we show that there is not one best training algorithm across all datasets and performance metrics.

Schuman, Catherine↗
DOE OSTI2023

Improving Deep Neural Networks’ Training for Image Classification With Nonlinear Conjugate Gradient-Style Adaptive Momentum

Momentum is crucial in stochastic gradient-based optimization algorithms for accelerating or improving training deep neural networks (DNNs). In deep learning practice, the momentum is usually weighted by a well-calibrated constant. However, tuning the hyperparameter for momentum can be a significant computational burden. In this article, we propose a novel adaptive momentum for improving DNNs training; this adaptive momentum, with no momentum-related hyperparame- ter required, is motivated by the nonlinear conjugate gradient (NCG) method. Stochastic gradient descent (SGD) with this new adaptive momentum eliminates the need for the momentum hyperparameter calibration, allows using a significantly larger learning rate, accelerates DNN training, and improves the final accuracy and robustness of the trained DNNs. For example, SGD with this adaptive momentum reduces classification errors for training ResNet110 for CIFAR10 and CIFAR100 from 5.25% to 4.64% and 23.75% to 20.03%, respectively. Furthermore, SGD, with the new adaptive momentum, also benefits adversarial training and, hence, improves the adversarial robustness of the trained DNNs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Towards maintaining canine training aid integrity: Effects of environmental factors and operational use on the triacetone triperoxide polymer odor capture‐and‐release system

Abstract There are many factors that may affect the longevity of or guide the use of canine training aids. Literature to date has mainly focused on identifying the headspace volatiles associated with training aids or odors and only minimal research exists into how different variables may alter those volatiles. The current study examines several factors affecting canine training aids: humidity, air flow, transportation, and operational deployment, using the triacetone triperoxide polymer odor capture‐and‐release canine training aid (TATP POCR) as the target. The TATP POCR is an absorption‐based canine training aid developed to be used to safely train canines to detect the odor of the explosive TATP in operational settings. Comparisons of the TATP POCR to neat TATP are made throughout the manuscript. First, humidity increased the background components of the POCR matrix, as well as the amount of TATP recovered was above the POCR. Humidity thus affected the amount of TATP detected but did not prevent detection. Second, air flow lessened the lifetime of the TATP POCR. Third, the practice of using primary and secondary containment successfully prevented contamination, cross‐contamination, and significant target loss, thereby maintaining kit integrity. Finally, the absorption of background odors from training environments was not observed. TATP headspace concentrations between a Deployed and Control POCR kit were not significantly different at time 0 (i.e., upon opening), which suggests that the operational use does not affect the function of the TATP POCR system. This information provides pivotal evidence for explosives detection canine handlers or trainers who utilize the TATP POCR.

Legal Medicine↗
DOE OSTI2022

Training reinforcement learning models via an adversarial evolutionary algorithm

When training for control problems, more episodes used in training usually leads to better generalizability, but more episodes also requires significantly more training time. There are a variety of approaches for selecting the way that training episodes are chosen, including fixed episodes, uniform sampling, and stochastic sampling, but they can all leave gaps in the training landscape. In this work, we describe an approach that leverages an adversarial evolutionary algorithm to identify the worst performing states for a given model. We then use information about these states in the next cycle of training; this process can be repeated until the desired level of model performance is met. We demonstrate this approach with the OpenAI Gym cart-pole problem. With this problem, we show that the adversarial evolutionary algorithm did not reduce the number of episodes required in training needed to attain model generalizability when compared with stochastic sampling, and actually performed slightly worse.

Coletti, Mark↗
DOE OSTI2024

RAP: Resource-aware Automated GPU Sharing for Multi-GPU Recommendation Model Training and Input Preprocessing

Ensuring high-quality recommendations for newly onboarded users requires the continuous retraining of Deep Learning Recommendation Models (DLRMs) with freshly generated data. To serve the online DLRM retraining, existing solutions use hundreds of CPU computing nodes designated for input preprocessing, causing significant power consumption that surpasses even the power usage of GPU trainers. To this end, we propose RAP, an end-to-end DLRM training framework that supports Resource-aware Automated GPU sharing for DLRM input Preprocessing and Training. The core idea of RAP is to accurately capture the remaining GPU computing resources during DLRM training for input preprocessing, achieving superior training efficiency without requiring additional resources. Specifically, RAP utilizes a co-running cost model to efficiently assess the costs of various input preprocessing operations, and it implements a resource-aware horizontal fusion technique that adaptively merges smaller kernels according to GPU availability, circumventing any interference with DLRM training. In addition, RAP leverages a heuristic searching algorithm that jointly optimizes both the input preprocessing graph mapping and the co-running schedule to maximize the end-to-end DLRM training throughput. The comprehensive evaluation shows that RAP achieves 78.3× speedup on average over CPU-based DLRM input preprocessing frameworks. In addition, the end-to-end training throughput of RAP is only 2.04% lower than the ideal case, which has no input preprocessing overhead.

Wang, Zheng↗
DOE OSTI2019

Troubleshooting deep-learner training data problems using an evolutionary algorithm on Summit

Architectural and hyper-parameter design choices can influence deep-learner (DL) model fidelity but can also be affected by malformed training and validation data. However, practitioners may spend significant time refining layers and hyper-parameters before discovering that distorted training data was impeding training progress. We found that an evolutionary algorithm (EA) can be used to diagnose this kind of DL problem. An EA evaluated thousands of DL configurations on Summit that yielded no overall improvement in DL performance, which suggested problems with the training and validation data. We suspected that Contrast Limited Adaptive Histogram Equalization (CLAHE) enhancement that was applied to previously generated digital surface models (DSMs), for which we were training DLs to find errors, had damaged the training data. Subsequent runs with an alternative global normalization yielded significantly improved DL performance. However, the DL Intersection Over Union (IOU) still exhibited consistent sub-par performance, which suggested further problems with the training data and DL approach. Nonetheless, we were able to diagnose this problem within a 12-hour span via Summit runs, which prevented several weeks of unproductive trial-and-error DL configuration refinement, and allowed for a more timely convergence on an ultimately viable solution.

97 MATHEMATICS AND COMPUTING↗