SEARCH · Engineering Papers

Results for “Training Time”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 883 records · Page 49

All sources NASA NTRS DOE OSTI

DOE OSTI2020

DEEP CELLULAR RECURRENT NEURAL ARCHITECTURE FOR EFFICIENT MULTIDIMENSIONAL TIME-SERIES DATA PROCESSING

Efficient processing of time series data is a fundamental yet challenging problem in pattern recognition. Though recent developments in machine learning and deep learning have enabled remarkable improvements in processing large scale datasets in many application domains, most are designed and regulated to handle inputs that are static in time. Many real-world data, such as in biomedical, surveillance and security, financial, manufacturing and engineering applications, are rarely static in time, and demand models able to recognize patterns in both space and time. Current machine learning (ML) and deep learning (DL) models adapted for time series processing tend to grow in complexity and size to accommodate the additional dimensionality of time. Specifically, the biologically inspired learning based models known as artificial neural networks that have shown extraordinary success in pattern recognition, tend to grow prohibitively large and cumbersome in the presence of large scale multi-dimensional time series biomedical data such as EEG. Consequently, this work aims to develop representative ML and DL models for robust and efficient large scale time series processing. First, we design a novel ML pipeline with efficient feature engineering to process a large scale multi-channel scalp EEG dataset for automated detection of epileptic seizures. With the use of a sophisticated yet computationally efficient time-frequency analysis technique known as harmonic wavelet packet transform and an efficient self-similarity computation based on fractal dimension, we achieve state-of-the-art performance for automated seizure detection in EEG data. Subsequently, we investigate the development of a novel efficient deep recurrent learning model for large scale time series processing. For this, we first study the functionality and training of a biologically inspired neural network architecture known as cellular simultaneous recurrent neural network (CSRN). We obtain a generalization of this network for multiple topological image processing tasks and investigate the learning efficacy of the complex cellular architecture using several state-of-the?art training methods. Finally, we develop a novel deep cellular recurrent neural network (CDRNN) architecture based on the biologically inspired distributed processing used in CSRN for processing time series data. The proposed DCRNN leverages the cellular recurrent architecture to promote extensive weight sharing and efficient, individualized, synchronous processing of multi-source time series data. Experiments on a large scale multi-channel scalp EEG, and a machine fault detection dataset show that the proposed DCRNN offers state-of-the-art recognition performance while using substantially fewer trainable recurrent units.

Vidyaratne, Lasitha S.↗

NASA NTRS2016

Simulation of Mission Phases

This position with the Simulation and Graphics Branch (ER7) at Johnson Space Center (JSC) provided an introduction to vehicle hardware, mission planning, and simulation design. ER7 supports engineering analysis and flight crew training by providing high-fidelity, real-time graphical simulations in the Systems Engineering Simulator (SES) lab. The primary project assigned by NASA mentor and SES lab manager, Meghan Daley, was to develop a graphical simulation of the rendezvous, proximity operations, and docking (RPOD) phases of flight. The simulation is to include a generic crew/cargo transportation vehicle and a target object in low-Earth orbit (LEO). Various capsule, winged, and lifting body vehicles as well as historical RPOD methods were evaluated during the project analysis phase. JSC core mission to support the International Space Station (ISS), Commercial Crew Program (CCP), and Human Space Flight (HSF) influenced the project specifications. The simulation is characterized as a 30 meter +V Bar and/or -R Bar approach to the target object's docking station. The ISS was selected as the target object and the international Low Impact Docking System (iLIDS) was selected as the docking mechanism. The location of the target object's docking station corresponds with the RPOD methods identified. The simulation design focuses on Guidance, Navigation, and Control (GNC) system architecture models with station keeping and telemetry data processing capabilities. The optical and inertial sensors, reaction control system thrusters, and the docking mechanism selected were based on CCP vehicle manufacturer's current and proposed technologies. A significant amount of independent study and tutorial completion was required for this project. Multiple primary source materials were accessed using the NASA Technical Report Server (NTRS) and reference textbooks were borrowed from the JSC Main Library and International Space Station Library. The Trick Simulation Environment and User Training Materials version 2013.0 release was used to complete the Trick tutorial. Multiple network privilege and repository permission requests were required in order to access previous simulation models. The project was also an introduction to computer programming and the Linux operating system. Basic C++ and Python syntax was used during the completion of the Trick tutorial. Trick's engineering analysis and Monte Carlo simulation capabilities were observed and basic space mission planning procedures were applied in the conceptual design phase. Multiple professional development opportunities were completed in addition to project duties during this internship through the System for Administration, Training, and Education Resources for NASA (SATERN). Topics include: JSC Risk Management Workshop, CCP Risk Management, Basic Radiation Safety Training, X-Ray Radiation Safety, Basic Laser Safety, JSC Export Control, ISS RISE Ambassador, Basic SharePoint 2013, Space Nutrition and Biochemistry, and JSC Personal Protective Equipment. Additionally, this internship afforded the opportunity for formal project presentation and public speaking practice. This was my first experience at a NASA center. After completing this internship I have a much clearer understanding of certain aspects of the agency's processes and procedures, as well as a deeper appreciation from spaceflight simulation design and testing. I will continue to improve my technical skills so that I may have another opportunity to return to NASA and Johnson Space Center.

Carlstrom, Nicholas Mercury↗

NASA NTRS2017

Prediction of Cognitive States During Flight Simulation Using Multimodal Psychophysiological Sensing

The Commercial Aviation Safety Team found the majority of recent international commercial aviation accidents attributable to loss of control inflight involved flight crew loss of airplane state awareness (ASA), and distraction was involved in all of them. Research on attention-related human performance limiting states (AHPLS) such as channelized attention, diverted attention, startle/surprise, and confirmation bias, has been recommended in a Safety Enhancement (SE) entitled "Training for Attention Management." To accomplish the detection of such cognitive and psychophysiological states, a broad suite of sensors was implemented to simultaneously measure their physiological markers during a high fidelity flight simulation human subject study. Twenty-four pilot participants were asked to wear the sensors while they performed benchmark tasks and motion-based flight scenarios designed to induce AHPLS. Pattern classification was employed to predict the occurrence of AHPLS during flight simulation also designed to induce those states. Classifier training data were collected during performance of the benchmark tasks. Multimodal classification was performed, using pre-processed electroencephalography, galvanic skin response, electrocardiogram, and respiration signals as input features. A combination of one, some or all modalities were used. Extreme gradient boosting, random forest and two support vector machine classifiers were implemented. The best accuracy for each modality-classifier combination is reported. Results using a select set of features and using the full set of available features are presented. Further, results are presented for training one classifier with the combined features and for training multiple classifiers with features from each modality separately. Using the select set of features and combined training, multistate prediction accuracy averaged 0.64 +/- 0.14 across thirteen participants and was significantly higher than that for the separate training case. These results support the goal of demonstrating simultaneous real-time classification of multiple states using multiple sensing modalities in high fidelity flight simulators. This detection is intended to support and inform training methods under development to mitigate the loss of ASA and thus reduce accidents and incidents.

Harrivel, Angela R.↗

NASA NTRS1997

Alternative Perspectives on Risk

The goal of the commercial air transport system is to provide air transportation to the flying public at an acceptable cost with minimal risk. in an ideal situation these three goals would support each other. In fact, it is sometimes the case that the goals conflict: getting passengers to their destinations on time may conflict with fixing a minor mechanical malfunction that may or may not impact safety; flying a route that will avoid turbulence, thereby providing passengers with a more comfortable ride, may consume more fuel; managing traffic density may mean aircraft are delayed or must use an approach that will result in a long taxi to their gates, costing time and fuel. Various players in the system--pilots, dispatchers, controllers, as well as managers in the airline carriers and traffic management system--make decisions every day that involve trade-offs of benefits and costs. The prospect of revisions in the air traffic management system, with shifts in responsibilities from controllers to users, including airline operations center personnel and pilots, means that individuals may be performing either new jobs or old jobs under new guidance. It will be essential to know how the various players (a) perceive the risks and benefits associated with the decisions they will make under the old and new control structures, and (b) how much risk they are willing to accept in making decisions. Risk is here defined as the probability and magnitude of negative events (after Slovic, 1987). Of primary interest are risks associated with traffic, weather, and operational factors such as schedule, fuel consumption, and passenger service. Previous research has documented differences between groups in perceptions of risks associated with both everyday and aviation related situations. Risk perception varies as a function of familiarity with the situation, degree to which one is potentially affected by the risk, the level of control one has over the situation, and one's level of experience and responsibility in the situation. In our presentation we will consider several factors that may influence differences in risk perception, risk tolerance and risk management among the three major categories of participants in the aviation system (pilots, dispatchers, and controllers). Primary factors that may affect risk attitudes (the collective term we will use for the three components of risk) include: expertise or job-specific knowledge or training, personal involvement or vulnerability to consequences, goals, time horizon (imminence of consequences), span of control and type of control, and information or technology support. These will be considered in the context of five different types of risk (physical threat, economic, social, legal, and ethical). A study designed to study differences between pilots and controllers in their perceptions and responses to traffic risks under the present and a user-preferred control situation will be described. Future studies involving dispatchers' risk perceptions regarding various types of risk will be discussed.

Davison, Jeannie↗

NASA NTRS2006

Adaptive Changes In Postural Equilibrium And Motion Sickness Following Repeated Exposures To Virtual Environments

Virtual environments offer unique training opportunities, particularly for training astronauts and preadapting them to the novel sensory conditions of microgravity. Two unresolved human factors issues in virtual reality (VR) systems are: 1) potential "cybersickness", and 2) maladaptive sensorimotor performance following exposure to VR systems. Interestingly, these aftereffects are often quite similar to adaptive sensorimotor responses observed in astronauts during and/or following space flight. Changes in the environmental sensory stimulus conditions and the way we interact with the new stimuli may result in motion sickness, and perceptual, spatial orientation and sensorimotor disturbances. Initial interpretation of novel sensory information may be inappropriate and result in perceptual errors. Active exploratory behavior in a new environment, with resulting feedback and the formation of new associations between sensory inputs and response outputs, promotes appropriate perception and motor control in the new environment. Thus, people adapt to consistent, sustained alterations of sensory input such as those produced by microgravity, unilateral labyrinthectomy and experimentally produced stimulus rearrangements. Adaptation is revealed by aftereffects including perceptual disturbances and sensorimotor control disturbances. The purpose of the current study was to compare disturbances in postural control produced by dome and head-mounted virtual environment displays, and to examine the effects of exposure duration, and repeated exposures to VR systems. Forty-one subjects (21 men, 20 women) participated in the study with an age range of 21-49 years old. One training session was completed in order to achieve stable performance on the posture and VR tasks before participating in the experimental sessions. Three experimental sessions were performed each separated by one day. The subjects performed a navigation and pick and place task in either a dome or head-mounted display (HMD) VR system for either 30 or 60 min. The environment was a square room with 15 pedestals on two opposite walls. The objects appeared on one set of pedestals and the subject s objective was to move the objects to the other set of pedestals. After the subject picked up an object, a pathway appeared and they were required to follow the pathway to the other side of the room. The subject was instructed to perform the task as quickly and accurately as possible, avoiding hitting walls and other any obstacles and placing the object on the center of the pedestal. Postural equilibrium was measured (using the Equitest CDP balance system, Neurocom, International) before, immediately after, and at 1 hr, 2 hr, 4 hr and 6 hr following exposure to VR. Postural equilibrium was measured during quiet stance with eyes open, eyes closed and vision and/or ankle proprioceptive inputs selectively altered by servo-controlling the visual surround and/or support surface to the subject s center of mass sway. Posture data was normalized using a log transformation and motion sickness data were normalized using the square root. In general, we found that exposure to VR resulted in decrements in postural stability. The largest decrements were observed in the tests performed immediately following exposure to VR and showed a fairly rapid recovery across the remaining test sessions. In addition, subjects generally showed improvement across days. We found significant main effects for day and time for the composite equilibrium score and for sensory organization tests (SOT) 1, 2 and 6. Significant main effects were observed for day for SOT 3 and 5. Although we found no significant main effects for gender (when center of gravity was used as a covariate), we did observe significant gender X time interaction effects for composite equilibrium and for SOT 1, 3, 4 and 5. Women appeared to show larger decrements in postural stability immediately after exposure to VR than men, but recover more quickly than n. Finally, we found no significant main effects for type of VR device or for exposure duration, however, these factors did interact with other factors during some of the SOTs. Subjects exhibited rapid recovery of motion sickness symptoms across time following exposure to VR and significantly less severe symptoms across days. We did not observe main effects for gender, type of device or duration of exposure. Individuals recovered from the detrimental effects of exposure to virtual reality on postural control and motion sickness within one hour. Sickness severity and initial decrements in postural equilibrium decreases over days, which suggests that subjects become dual-adapted over time. These findings provide some direction for developing training schedules for VR users that facilitate adaptation, and support the idea that preflight training of astronauts may serve as useful countermeasure for the sensorimotor effects of space flight.

Harm, D. L.↗

DOE OSTI2023

A rotorcraft in-flight ice detection framework using computational aeroacoustics and Bayesian neural networks

Abstract This work develops a novel ice detection framework specifically suitable for rotorcraft using computational aeroacoustics and Bayesian neural networks. In an offline phase of the work, the acoustic signature of glaze and rime ice shapes on an oscillating wing are computed. In addition, the aerodynamic performance indicators corresponding to the ice shapes are also monitored. These performance indicators include the lift, drag, and moment coefficients. A Bayesian neural network is subsequently trained using projected Stein variational gradient descent to create a mapping from the acoustic signature generated by the iced wings to predict their performance indicators along with quantified uncertainty that is highly important for time- and safety-critical decision-making scenarios. While the training is carried out fully offline, usage of the Bayesian neural network to make predictions can be conducted rapidly online allowing for an ice detection system that can be used in real time and in-flight.

42 ENGINEERING↗

DOE OSTI2024

Unpaired image translation to mitigate domain shift in liquid argon time projection chamber detector responses

Deep learning algorithms often are developed and trained on a training dataset and deployed on test datasets. Any systematic difference between the training and a test dataset may severely degrade the final algorithm performance on the test dataset—what is known as the domain shift problem . This issue is prevalent in many scientific domains where algorithms are trained on simulated data but applied to real-world datasets. Typically, the domain shift problem is solved through various domain adaptation (DA) methods. However, these methods are often tailored for a specific downstream task, such as classification or semantic segmentation, and may not easily generalize to different tasks. This work explores the feasibility of using an alternative way to solve the domain shift problem that is not specific to any downstream algorithm. The proposed approach relies on modern Unpaired Image-to-Image (UI2I) translation techniques, designed to find translations between different image domains in a fully unsupervised fashion. In this study, the approach is applied to a domain shift problem commonly encountered in Liquid Argon Time Projection Chamber (LArTPC) detector research when seeking a way to translate samples between two differently distributed LArTPC detector datasets deterministically. This translation allows for mapping real-world data into the simulated data domain where the downstream algorithms can be run with much less domain-shift-related performance degradation. Conversely, using the translation from the simulated data to a real-world domain can increase the realism of the simulated dataset and reduce the magnitude of any systematic uncertainties. To evaluate the quality of the translations, we use both pixel-wise metrics and a downstream task to measure the effectiveness of UI2I methods for mitigating the domain shift problem. We adapted several popular UI2I translation algorithms to work on scientific data and demonstrated the viability of these techniques for solving the domain shift problem with LArTPC detector data. To facilitate further development of DA techniques for scientific datasets, the ‘Simple Liquid-Argon Track Samples’ dataset used in this study is also published.

97 MATHEMATICS AND COMPUTING↗

NASA NTRSDate not supplied

Collision Avoidance Approach Using Deep Reinforcement Learning

A method to enable autonomous robots moving in a 2D space collision free motivates the purposed approach for collision avoidance for autonomous UAM vehicles. Challenges of autonomous collision free navigation for both problems are similar. Agents in each environment do not know the intent, or goal, of the other. Finding the time efficient paths require some level of anticipation with neighboring agents which is computationally expensive. In the original work, these obstacles were overcome with a novel application of deep reinforcement learning which offloads the online computation to an offline learning algorithm. A value network that encodes the estimated time to the goal given the agent’s state and the observable portion of the other agent’s state is trained on a baseline policy and further refined with reinforcement learning to promote time efficient collision free navigation. Online, the value network efficiently informs the agent’s decision making in the face of uncertainty of the other agent’s next move. In this paper, challenges extending this methodology to the 3D environment of autonomous UAM vehicles with kinematic constraints are discussed and initial results shown.

Collision Avoidance↗

NASA NTRSDate not supplied

Collision Avoidance Approach Using Deep Reinforcement Learning

Collision Avoidance↗

NASA NTRS1991

Diminished Baroreflex Control of Forearm Vascular Resistance Following Training

The stimulus-response characteristics of cardiopulmonary baroreflex control of forearm vascular resistance (FVR units in mm Hg x min x I00 ml/ml) were studied in 14 volunteers before and after 10 wk of endurance training. We assessed the relationship betaleen reflex stimulus (changes in central venous pressure, CVP) and response (FVR) during unloading of cardiopulmonary baroreceptors with lower body negative pressure (LBNP, 0 to - 2O mm Hg). Changes in CVP during LBNP were estimated from pressure changes in a large peripheral vein in the dependent arm of the subject in the right lateral decubitus position. Maximal oxygen uptake (VO(sub 2max)) and total blood volume increased with endurance training from 37.8 +/- 1.4 ml/min x kg and 63.6 +/- 2.1 ml/kg to 45.3 +/- 1.4 ml/ min x kg and 69.3 +/- 2.8 ml/kg respectively (P less than 0.05). Reflex forearm vasoconstriction occurred in response to a reduction in estimated CVP, and the absolute change in FVR per unit of CVP was reduced from -5.96 +/- 0.79 to -4.06 +/- 0.52 units x mm/ Hg (P less than 0.05) following exercise training but was unchanged from -6.10 to 0.57 to -6.22 +/- 0.94 units x mm/ Hg for the time control group (N = 7). Resting values for FVR were similar before and after exercise training; however, resting estimated CVP was elevated from 9.5 +/- 0.5 mm x Hg before training to 11.3 +/- 0.6 mm x Hg after training. The reduction in sensitivity of the cardiopulmonary baroreflex control of FVR was linearly related to the increase in blood volume (r = 0.65, P less than 0.05). suggesting that diminished cardiopulmonary baroreflex control of FVR in physically fit individuals is related, in part, to a training-induced blood volume expansion.

Mack, G. W.↗

NASA NTRS1990

Terminal weather information management

Since the mid-1960's, microburst/windshear events have caused at least 30 aircraft accidents and incidents and have killed more than 600 people in the United States alone. This study evaluated alternative means of alerting an airline crew to the presence of microburst/windshear events in the terminal area. Of particular interest was the relative effectiveness of conventional and data link ground-to-air transmissions of ground-based radar and low-level windshear sensing information on microburst/windshear avoidance. The Advanced Concepts Flight Simulator located at Ames Research Center was employed in a line oriented simulation of a scheduled round-trip airline flight from Salt Lake City to Denver Stapleton Airport. Actual weather en route and in the terminal area was simulated using recorded data. The microburst/windshear incident of July 11, 1988 was re-created for the Denver area operations. Six experienced airline crews currently flying scheduled routes were employed as test subjects for each of three groups: (1) A baseline group which received alerts via conventional air traffic control (ATC) tower transmissions; (2) An experimental group which received alerts/events displayed visually and aurally in the cockpit six miles (approx. 2 min.) from the microburst event; and (3) An additional experimental group received displayed alerts/events 23 linear miles (approx. 7 min.) from the microburst event. Analyses of crew communications and decision times showed a marked improvement in both situation awareness and decision-making with visually displayed ground-based radar information. Substantial reductions in the variability of decision times among crews in the visual display groups were also found. These findings suggest that crew performance will be enhanced and individual differences among crews due to differences in training and prior experience are significantly reduced by providing real-time, graphic display of terminal weather hazards.

Lee, Alfred T.↗

DOE OSTI2020

Deep-Learning-Enabled Fast Optical Identification and Characterization of 2D Materials

Not provided.

Chemistry↗

DOE OSTI2025

Structure-preserving neural networks for the regularized entropy-based closure of a linear, kinetic, radiative transport equation

The main challenge of large-scale numerical simulation of radiation transport is the high memory and computation time requirements of discretization methods for kinetic equations. In this work, we derive and investigate a neural network-based approximation to the entropy-based closure method to accurately compute the solution of the multi-dimensional moment system with a low memory footprint and competitive computational time. We extend methods developed for the standard entropy-based closure to the regularized entropy-based closures. The main idea is to interpret structure-preserving neural network approximations of the regularized entropy-based closure as a two-stage approximation to the original entropy-based closure. We conduct a numerical analysis of this approximation and investigate optimal parameter choices. Our numerical experiments demonstrate that the method has a much lower memory footprint than traditional methods with competitive computation times and simulation accuracy. The code and all trained networks are provided on GitHub.

entropy closure↗

DOE OSTI2022

Deep compressed seismic learning for fast location and moment tensor inferences with natural and induced seismicity

Fast detection and characterization of seismic sources is crucial for decision-making and warning systems that monitor natural and induced seismicity. However, besides the laying out of ever denser monitoring networks of seismic instruments, the incorporation of new sensor technologies such as Distributed Acoustic Sensing (DAS) further challenges our processing capabilities to deliver short turnaround answers from seismic monitoring. In response, this work describes a methodology for the learning of the seismological parameters: location and moment tensor from compressed seismic records. In this method, data dimensionality is reduced by applying a general encoding protocol derived from the principles of compressive sensing. The data in compressed form is then fed directly to a convolutional neural network that outputs fast predictions of the seismic source parameters. Thus, the proposed methodology can not only expedite data transmission from the field to the processing center, but also remove the decompression overhead that would be required for the application of traditional processing methods. An autoencoder is also explored as an equivalent alternative to perform the same job. We observe that the CS-based compression requires only a fraction of the computing power, time, data and expertise required to design and train an autoencoder to perform the same task. Implementation of the CS-method with a continuous flow of data together with generalization of the principles to other applications such as classification are also discussed.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Hydra: An AI-Based Framework for Interpretable and Portable Data Quality Monitoring

Hydra is an advanced framework designed for training and managing AI models for near real time data quality monitoring at Jefferson Lab. Deployed in all four experimental halls, Hydra has analyzed over 2 million images and has extended its capabilities to offline monitoring and validation. Hydra utilizes computer vision to continually analyze sets of images of monitoring plots generated 24/7 during experiments. Generally, these sets of images are produced at a rate and quantity that is exceedingly difficult for shift crews to effectively monitor. Significant effort has been devoted to enhancing Hydra’s user interface, to ensure that it provides clear, actionable insights for shift workers and other users. Gradient Weighted Class Activation Maps (GradCAM) provide added interpretability, allowing users to visualize important regions of the image for classification. Hydra has been containerized to enable the creation of portable demos and seamless integration with container-based technologies such as Kubernetes and Docker. With the user interface enhancements and containerization, Hydra can be rapidly deployed for new use cases and experiments. This talk will describe the Hydra framework, its user interface and experience, and the challenges inherent in its design and deployment.

Britton, Thomas [Thomas Jefferson National Acceler↗

DOE OSTI2021

Atomic photoionization by multiple temporal pairs of slits

In this work, we study interactions between multiple temporal pairs of slits in coherent control of photoionization of S-state atoms using a pulse train of N+1 pairs evenly delayed in time by τ, in which the two pulses in a pair with a delay τ0 are counter-rotating circularly polarized. For interacting two double-slit experiments, while Ramsey interference between two identical Archimedean spirals yields pairs of principal spirals, interference of two spirals with opposite handedness does not lead to spirals, but instead to crocodile-eye-like patterns with nictares. For more than two interacting experiments, the resulting patterns turn out just to be those two reference patterns modulated by different kinds of N-dependent time-energy Fraunhofer functions exhibiting diffraction-grating-like patterns.

74 ATOMIC AND MOLECULAR PHYSICS↗

DOE OSTI2025

Temporal Forecasting of Distributed Temperature Sensing in a Thermal Hydraulic System With Machine Learning and Statistical Models

We benchmark performance of long-short term memory (LSTM) network machine learning model and autoregressive integrated moving average (ARIMA) statistical model in temporal forecasting of distributed temperature sensing (DTS). Data in this study consists of fluid temperature transient measured with two co-located Rayleigh scattering fiber optic sensors (FOS) in a forced convection mixing zone of a thermal tee. We treat each gauge of a FOS as an independent temperature sensor. We first study prediction of DTS time series using Vanilla LSTM and ARIMA models trained on prior history of the same FOS that is used for testing. The results yield maximum absolute percentage error (MaxAPE) and root mean squared percentage error (RMSPE) of 1.58% and 0.06% for ARIMA, and 3.14% and 0.44% for LSTM, respectively. Next, we investigate zero-shot forecasting (ZSF) with LSTM and ARIMA trained on history of the co-located FOS only, which is advantageous when limited training data is available. The ZSF MaxAPE and RMSPE values for ARIMA are comparable to those of the Vanilla use case, while the error values for LSTM increase. We show that in ZSF, performance of LSTM network can be improved by training on most correlated gauges between the two FOS, which are identified by calculating the Pearson correlation coefficient. The improved ZSF MaxAPE and RMSPE for LSTM are 4.4% and 0.33%, respectively. Performance of ZSF LSTM can be further enhanced through transfer learning (TL), where LSTM is re-trained on a subset of the FOS that is the target of forecasting. We show that LSTM pre-trained on correlated dataset and re-trained on 30% of testing target dataset achieves MaxAPE and RMSPE values of 2.32% and 0.28%, respectively.

ARIMA↗

DOE OSTI2024

An Efficient Checkpointing System for Large Machine Learning Model Training

As machine learning models increase in size and complexity rapidly, the cost of checkpointing in ML training became a bottleneck in storage and performance (time). For example, the latest GPT-4 model has massive parameters at the scale of 1.76 trillion. It is highly time and storage consuming to frequently writes the model to checkpoints with more than 1 trillion floating point values to storage. This work aims to understand and attempt to mitigate this problem. First, we characterize the checkpointing interface in a collection of representative large machine learning/language models with respect to storage consumption and performance overhead. Second, we propose the two optimizations: i) A periodic cleaning strategy that periodically cleans up outdated checkpoints to reduce the storage burden; ii) A data staging optimization that coordinates checkpoints between local and shared file systems for performance improvement.

machine learning, artificial intelligence↗