Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “training”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Improving Deep Neural Networks’ Training for Image Classification With Nonlinear Conjugate Gradient-Style Adaptive Momentum

Momentum is crucial in stochastic gradient-based optimization algorithms for accelerating or improving training deep neural networks (DNNs). In deep learning practice, the momentum is usually weighted by a well-calibrated constant. However, tuning the hyperparameter for momentum can be a significant computational burden. In this article, we propose a novel adaptive momentum for improving DNNs training; this adaptive momentum, with no momentum-related hyperparame- ter required, is motivated by the nonlinear conjugate gradient (NCG) method. Stochastic gradient descent (SGD) with this new adaptive momentum eliminates the need for the momentum hyperparameter calibration, allows using a significantly larger learning rate, accelerates DNN training, and improves the final accuracy and robustness of the trained DNNs. For example, SGD with this adaptive momentum reduces classification errors for training ResNet110 for CIFAR10 and CIFAR100 from 5.25% to 4.64% and 23.75% to 20.03%, respectively. Furthermore, SGD, with the new adaptive momentum, also benefits adversarial training and, hence, improves the adversarial robustness of the trained DNNs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Towards maintaining canine training aid integrity: Effects of environmental factors and operational use on the triacetone triperoxide polymer odor capture‐and‐release system

Abstract There are many factors that may affect the longevity of or guide the use of canine training aids. Literature to date has mainly focused on identifying the headspace volatiles associated with training aids or odors and only minimal research exists into how different variables may alter those volatiles. The current study examines several factors affecting canine training aids: humidity, air flow, transportation, and operational deployment, using the triacetone triperoxide polymer odor capture‐and‐release canine training aid (TATP POCR) as the target. The TATP POCR is an absorption‐based canine training aid developed to be used to safely train canines to detect the odor of the explosive TATP in operational settings. Comparisons of the TATP POCR to neat TATP are made throughout the manuscript. First, humidity increased the background components of the POCR matrix, as well as the amount of TATP recovered was above the POCR. Humidity thus affected the amount of TATP detected but did not prevent detection. Second, air flow lessened the lifetime of the TATP POCR. Third, the practice of using primary and secondary containment successfully prevented contamination, cross‐contamination, and significant target loss, thereby maintaining kit integrity. Finally, the absorption of background odors from training environments was not observed. TATP headspace concentrations between a Deployed and Control POCR kit were not significantly different at time 0 (i.e., upon opening), which suggests that the operational use does not affect the function of the TATP POCR system. This information provides pivotal evidence for explosives detection canine handlers or trainers who utilize the TATP POCR.

Legal Medicine↗
DOE OSTI2022

Training reinforcement learning models via an adversarial evolutionary algorithm

When training for control problems, more episodes used in training usually leads to better generalizability, but more episodes also requires significantly more training time. There are a variety of approaches for selecting the way that training episodes are chosen, including fixed episodes, uniform sampling, and stochastic sampling, but they can all leave gaps in the training landscape. In this work, we describe an approach that leverages an adversarial evolutionary algorithm to identify the worst performing states for a given model. We then use information about these states in the next cycle of training; this process can be repeated until the desired level of model performance is met. We demonstrate this approach with the OpenAI Gym cart-pole problem. With this problem, we show that the adversarial evolutionary algorithm did not reduce the number of episodes required in training needed to attain model generalizability when compared with stochastic sampling, and actually performed slightly worse.

Coletti, Mark↗
DOE OSTI2024

RAP: Resource-aware Automated GPU Sharing for Multi-GPU Recommendation Model Training and Input Preprocessing

Ensuring high-quality recommendations for newly onboarded users requires the continuous retraining of Deep Learning Recommendation Models (DLRMs) with freshly generated data. To serve the online DLRM retraining, existing solutions use hundreds of CPU computing nodes designated for input preprocessing, causing significant power consumption that surpasses even the power usage of GPU trainers. To this end, we propose RAP, an end-to-end DLRM training framework that supports Resource-aware Automated GPU sharing for DLRM input Preprocessing and Training. The core idea of RAP is to accurately capture the remaining GPU computing resources during DLRM training for input preprocessing, achieving superior training efficiency without requiring additional resources. Specifically, RAP utilizes a co-running cost model to efficiently assess the costs of various input preprocessing operations, and it implements a resource-aware horizontal fusion technique that adaptively merges smaller kernels according to GPU availability, circumventing any interference with DLRM training. In addition, RAP leverages a heuristic searching algorithm that jointly optimizes both the input preprocessing graph mapping and the co-running schedule to maximize the end-to-end DLRM training throughput. The comprehensive evaluation shows that RAP achieves 78.3× speedup on average over CPU-based DLRM input preprocessing frameworks. In addition, the end-to-end training throughput of RAP is only 2.04% lower than the ideal case, which has no input preprocessing overhead.

Wang, Zheng↗
DOE OSTI2019

Troubleshooting deep-learner training data problems using an evolutionary algorithm on Summit

Architectural and hyper-parameter design choices can influence deep-learner (DL) model fidelity but can also be affected by malformed training and validation data. However, practitioners may spend significant time refining layers and hyper-parameters before discovering that distorted training data was impeding training progress. We found that an evolutionary algorithm (EA) can be used to diagnose this kind of DL problem. An EA evaluated thousands of DL configurations on Summit that yielded no overall improvement in DL performance, which suggested problems with the training and validation data. We suspected that Contrast Limited Adaptive Histogram Equalization (CLAHE) enhancement that was applied to previously generated digital surface models (DSMs), for which we were training DLs to find errors, had damaged the training data. Subsequent runs with an alternative global normalization yielded significantly improved DL performance. However, the DL Intersection Over Union (IOU) still exhibited consistent sub-par performance, which suggested further problems with the training data and DL approach. Nonetheless, we were able to diagnose this problem within a 12-hour span via Summit runs, which prevented several weeks of unproductive trial-and-error DL configuration refinement, and allowed for a more timely convergence on an ultimately viable solution.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Analyzing and Exploring Training Recipes for Large-Scale Transformer-Based Weather Prediction

Abstract The rapid rise of deep learning (DL) in numerical weather prediction (NWP) has led to a proliferation of models which forecast atmospheric variables with comparable or superior skill than traditional physics-based NWP. However, among these leading DL models, there is a wide variance in both the training settings and architecture used. Further, the lack of thorough ablation studies makes it hard to discern which components are most critical to success. In this work, we show that it is possible to attain high forecast skill even with relatively off-the-shelf architectures, simple training procedures, and moderate compute budgets. Specifically, we train a minimally modified Swin Transformer V2 (SwinV2) on ERA5 data and find that it attains superior skill in terms of mean-square errors of deterministic forecasts when compared against the European Centre for Medium-Range Weather Forecasts’ Integrated Forecasting System (IFS). Almost all DL–NWP systems share a core set of hyperparameters and design decisions. To aid and expedite future DL–NWP research, we present an in-depth, systematic exploration of different loss functions, model sizes and depths, patch sizes, and multistep training objectives. We also examine the model performance with metrics beyond the typical accuracy (ACC) and RMSE and investigate how the performance scales with model size. Through our open-source code, scoring pipelines, and models, we share our findings on key aspects of the training pipeline. These ablations reduce the necessity for expensive hyperparameter tuning and lower the barrier to entry for future DL–NWP research. Significance Statement This study investigates the potential of using large-scale transformer-based models for weather prediction, showing that it is possible to achieve high forecast accuracy with simpler, off-the-shelf architectures. By training a minimally modified SwinV2 transformer on ERA5 data, we show that the model achieves competitive forecast skill in terms of mean-square error for key variables, outperforming the European Centre for Medium-Range Weather Forecasts’ Integrated Forecasting System (IFS) at all lead times. Our findings suggest that effective training strategies, such as multistep fine-tuning and channel-weighted losses, significantly enhance the model’s performance. However, we also highlight that these improvements come with trade-offs in other areas, such as ensemble spread and high-frequency spatial detail. This work highlights the promise of deep learning in improving weather forecasts, which could lead to better preparedness and response to weather events, ultimately benefiting society by providing more reliable weather predictions.

Willard, Jared D. [Lawrence Berkeley National Labo↗
DOE OSTI2015

Project Management Plan/Progress Report: UT/GTKS Training Program Development for Commercial Building Operators (Final Report)

Universidad del Turabo (UT), in a collaborative effort with Global Turn Key Services, Inc. (GTKS), proposed to develop a training program and a commercialization plan for the development of Commercial Building Operators (CBOs). The CBOs will operate energy efficient buildings to help maintain existing buildings up to their optimal energy performance level, and ensure that net-zero-energy buildings continuously operate at design specifications, thus helping achieve progress towards meeting BTP Strategic Goals of creating technologies and design approaches that enable net-zero-energy buildings at low incremental costs by 2025. The proposed objectives were then: (1) Develop a Commercial Building Operator (CBO) training program and accreditation that will in turn provide a certification to participants recognized by Accreditation Boards such as the North American Board of Certified Energy Practitioners (NABCEP) and Leadership in Energy & Environmental Designs (LEED). (2) Develop and implement a commercialization and sustainability plan that details marketing, deployment, financial characterization, job placement, and other goals required for long-term sustainability of the project after the funding period. (3) After program development and deployment, provide potential candidates with the knowledge and skill sets to obtain employment in the commercial building green energy (net-zero-energy building) job market. The developed CBO training program will focus on providing skills for participants, such as displaced and unemployed workers, to enter the commercial building green energy (net-zeroenergy building) job market. This course was designed to allow a participant with minimal to no experience in commercial building green technology to obtain the required skill sets to enter the job market in as little as 12 weeks of intensive multi-faceted learning. After completion of the course, the CBO staff concluded the participant will meet minimum established accreditation standards established by UT and will complete the contact hours required of training to apply to the Certification on Energy Management (CEM) offered by the Association of Energy Engineers (AEE). The CBO training program consists of a combination of theory (classroom), online & computer simulation, laboratory & hands on (onsite) training lessons. The training is addressed four basic learning elements: (1) Learn the Technology; (2) Practice Skills with hands-on the Energy Simulation Builder program; (3) Final Project and Presentation; and, (4) Accreditation and Certifications.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2020

Simple and efficient algorithms for training machine learning potentials to force data

Machine learning models, trained on data from ab initio quantum simulations, are yielding molecular dynamics potentials with unprecedented accuracy. One limiting factor is the quantity of available training data, which can be expensive to obtain. A quantum simulation often provides all atomic forces, in addition to the total energy of the system. These forces provide much more information than the energy alone. It may appear that training a model to this large quantity of force data would introduce significant computational costs. Actually, training to all available force data should only be a few times more expensive than training to energies alone. Here, we present a new algorithm for efficient force training, and benchmark its accuracy by training to forces from real-world datasets for organic chemistry and bulk aluminum.

74 ATOMIC AND MOLECULAR PHYSICS↗
DOE OSTI2026

Custom-trained Machine-learning Interatomic Potentials: ZnCl2 Aqueous Solution

This dataset was generated using an iterative active-learning strategy implemented in the ArcaNN software package (https://github.com/arcann-chem/arcann_training) to train machine-learning interatomic potentials for aqueous ZnCl2 solutions. Each active-learning cycle consisted of three stages: training, exploration, and labeling. The initial training set combined configurations generated in this work from enhanced-sampling ab initio molecular dynamics simulations with configurations from a previously reported neural-network-potential study of aqueous ZnCl2. The enhanced-sampling ab initio molecular dynamics simulations involved Zn–Cl separation and the chloride coordination number around Zn²? as collective variables. These configurations served as the seed dataset. Subsequent active-learning cycles expanded the training set by identifying and labeling configurations that were poorly represented by the current models, thereby improving coverage of ion-association states and changes in local coordination and charge-state environments relevant to the solution free-energy landscape. For all selected configurations, single-point calculations of the total energies and atomic forces were performed within density functional theory using the CP2K Quickstep module. Reference calculations employed the revPBE-D3 and r2SCAN exchange-correlation functionals. Motivated by recent work on aqueous Zn²?, the main revPBE calculations omitted D3 dispersion contributions involving Zn²?, while retaining the D3 correction for water and chloride. For comparison, fully dispersion-corrected revPBE-D3 reference calculations were also performed, with D3 applied to all species, including Zn²?. Valence electrons were treated explicitly, while core electrons were represented using norm-conserving Goedecker–Teter–Hutter pseudopotentials. The wave functions were expanded using the mixed Gaussian-and-plane-wave scheme with TZV2P-MOLOPT basis sets for all elements and a 600 Ry auxiliary plane-wave cutoff for the electron density. Self-consistent-field convergence was accelerated using the orbital-transformation and Direct Inversion in the Iterative Subspace algorithms, with a convergence threshold of 10?6. All single-point calculations were performed in periodic orthorhombic cells. The CELL_REF keyword in CP2K was used to define a fixed reference cell with a box length of 25 Å. This treatment ensured a consistent reference for configurations extracted from NpT trajectories with fluctuating cell dimensions. The resulting DFT energies and atomic forces constitute the ground-truth labels used to train the MLIPs. The resulting MLIP was trained for aqueous ZnCl2 solutions spanning concentrations from 0 to 30 molal and a broad pH range, from strongly acidic to strongly basic conditions. Representative examples of configurations included in the MLIP training dataset are provided below. These include 1) Representative configurations from the dataset labeled at the revPBE-D3 level, with D3 dispersion interactions involving Zn2+ excluded (revPBE-wo-D3). 2) Representative configurations from the dataset labeled at the fully dispersion-corrected revPBE-D3 level, with D3 interactions applied to all species, including Zn2+ (revPBE-D3). 3) Representative configurations from the dataset labeled at the r2SCAN level of theory (r2SCAN).

Dinpajooh, Mohammadhasan [Pacific Northwest Nation↗
NASA NTRS1975

The Texas Remote Sensing Training Project

The project was designed to train federal, state and regional agency managers, scientists and engineers. A one-week seminar was designed and implemented to build vocabulary, introduce technical subject areas and give students enough training to allow them to relate remote sensing technology to operational agency projects. The seminar was designed to perform the dual function of conveying enough remote sensing information to be of value as a stand-alone and preparing students for detailed pattern recognition training. The LARSYS III portion of the training project was executed exactly as designed in the LARSYS training materials package; the LARSYS package did not contain a LANDSAT training module. Two LANDSAT training modules were developed using Texas LANDSAT data. One module contained central Texas data and the second module contained coastal zone data.

Wells, J. B.↗
NASA NTRS1983

Effect of confinement in small space flight size cages on insulin sensitivity of exercise-trained rats

The effect of confinement in small cages (simulating the size to be used in future space Shuttle missions) on insulin sensitivity was studied in rats having an increased insulin sensitivity due to exercise training prior to confinement. Oral glucose tolerance tests (OGTT) were given to both control and exercise-trained rats before and after placement in the small cages for 7 days. The insulin resistance was assessed by the product of the area of the insulin and glucose curves of the OGTT (IG index). Results show that the values obtained before confinement were one-half as high in exercise-trained rats as those in control rats, reflecting an increased sensitivity to insulin with exercise training. After 7 days confinement, the IG index was found to be not significantly different from initial values for both control and exercise-trained rats. These findings suggest that increased insulin sensitivity in exercise-trained rats persists 7 days after cessation of running activity. The data also indicate that exercise training, before flight, may be beneficial in minimizing the loss of insulin sensitivity expected with decreased use of gravity dependent muscles during exposure to hypogravity in space flight.

Mondon, C. E.↗
NASA NTRS1984

Scientific and technical training in the Soviet Union

The Soviet Union recognizes that the foundation of their system depends upon complete dedication of the people to the state through thorough psychological training as well as through military training, and through specialized education in the broad fields of engineering, natural sciences, life sciences, social sciences, and education. An outline of the U.S.S.R. educational system indicates the extent of academic training, coupled with on-the-job and military training, that can produce a highly skilled, dedicated, and matured person. Observations on the coupling of political, economic, and psychological training along with the technical training are made, along with some mention of positive and negative aspects of the training.

Spearman, M. L.↗
NASA NTRS1990

Payload training methodology study

The results of the Payload Training Methodology Study (PTMS) are documented. Methods and procedures are defined for the development of payload training programs to be conducted at the Marshall Space Flight Center Payload Training Complex (PCT) for the Space Station Freedom program. The study outlines the overall training program concept as well as the six methodologies associated with the program implementation. The program concept outlines the entire payload training program from initial identification of training requirements to the development of detailed design specifications for simulators and instructional material. The following six methodologies are defined: (1) The Training and Simulation Needs Assessment Methodology; (2) The Simulation Approach Methodology; (3) The Simulation Definition Analysis Methodology; (4) The Simulator Requirements Standardization Methodology; (5) The Simulator Development Verification Methodology; and (6) The Simulator Validation Methodology.

Source record↗
NASA NTRS1991

Intelligent Computerized Training System

Intelligent computer-aided training system gives trainees same experience gained from best on-the-job training. Automated system designed to emulate behavior of experienced teacher devoting full time and attention to training novice. Proposes challenging training scenarios, monitors and evaluates trainee's actions, makes meaningful comments in response to errors, reponds to requests for information, gives hints when appropriate, and remembers strengths and weaknesses so it designs suitable exercises. Used to train flight-dynamics officers in deploying satellites from Space Shuttle. Adapted to training for variety of tasks and situations, simply by modifying one or at most two of its five modules. Helps to ensure continuous supply of trained specialists despite scarcity of experienced and skilled human trainers.

Wang, Lui↗
NASA NTRS1990

Training effectiveness of an intelligent tutoring system for a propulsion console trainer

A formative evaluation was conducted on an Intelligent Tutoring System (ITS) developed for tasks performed on the Propulsion Console. The ITS, which was developed primarily as a research tool, provides training on use of the Manual Select Keyboard (MSK). Three subjects completed three phases of training using the ITS: declarative, speed, and automaticity training. Data were collected on several performance dimensions, including training time, number of trials performed in each training phase, and number of errors. Information was also collected regarding the user interface and content of training. Suggestions for refining the ITS are discussed. Further, future potential uses and limitations of the ITS are discussed. The results provide an initial demonstration of the effectiveness of the Propulsion Console ITS and indicate the potential benefits of this form of training tool for related tasks.

Johnson, Debra Steele↗
NASA NTRS1991

MITT writer and MITT writer advanced development: Developing authoring and training systems for complex technical domains

MITT Writer is a software system for developing computer based training for complex technical domains. A training system produced by MITT Writer allows a student to learn and practice troubleshooting and diagnostic skills. The MITT (Microcomputer Intelligence for Technical Training) architecture is a reasonable approach to simulation based diagnostic training. MITT delivers training on available computing equipment, delivers challenging training and simulation scenarios, and has economical development and maintenance costs. A 15 month effort was undertaken in which the MITT Writer system was developed. A workshop was also conducted to train instructors in how to use MITT Writer. Earlier versions were used to develop an Intelligent Tutoring System for troubleshooting the Minuteman Missile Message Processing System.

Wiederholt, Bradley J.↗
NASA NTRS1991

Applications of intelligent computer-aided training

Intelligent computer-aided training (ICAT) systems simulate the behavior of an experienced instructor observing a trainee, responding to help requests, diagnosing and remedying trainee errors, and proposing challenging new training scenarios. This paper presents a generic ICAT architecture that supports the efficient development of ICAT systems for varied tasks. In addition, details of ICAT projects, built with this architecture, that deliver specific training for Space Shuttle crew members, ground support personnel, and flight controllers are presented. Concurrently with the creation of specific ICAT applications, a general-purpose software development environment for ICAT systems is being built. The widespread use of such systems for both ground-based and on-orbit training will serve to preserve task and training expertise, support the training of large numbers of personnel in a distributed manner, and ensure the uniformity and verifiability of training experiences.

Loftin, R. B.↗
NASA NTRS1993

Forecasting the impact of virtual environment technology on maintenance training

To assist NASA and the Air Force in determining how and when to invest in virtual environment (VE) technology for maintenance training, we identified possible roles for VE technology in such training, assessed its cost-effectiveness relative to existing technologies, and formulated recommendations for a research agenda that would address instructional and system development issues involved in fielding a VE training system. In the first phase of the study, we surveyed VE developers to forecast capabilities, maturity, and estimated costs for VE component technologies. We then identified maintenance tasks and their training costs through interviews with maintenance technicians, instructors, and training developers. Ten candidate tasks were selected from two classes of maintenance tasks (seven aircraft maintenance and three space maintenance) using five criteria developed to identify types of tasks most likely to benefit from VE training. Three tasks were used as specific cases for cost-benefit analysis. In formulating research recommendations, we considered three aspects of feasibility: technological considerations, cost-effectiveness, and anticipated R&D efforts. In this paper, we describe the major findings in each of these areas and suggest research efforts that we believe will help achieve the goal of a cost-effective VE maintenance training system by the next decade.

Schlager, Mark S.↗