Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Training Time”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 829 records · Page 46
All sourcesNASA NTRSDOE OSTI
NASA NTRS2018

A Deterministic Self-Organizing Map Approach and its Application on Satellite Data based Cloud Type Classification

A self-organizing map (SOM) is a type of competitive artificial neural network, which projects the high dimensional input space of the training samples into a low dimensional space with the topology relations preserved. This makes SOMs supportive of organizing and visualizing complex data sets and have been pervasively used among numerous disciplines with different applications. Notwithstanding its wide applications, the self-organizing map is perplexed by its inherent randomness, which produces dissimilar SOM patterns even when being trained on identical training samples with the same parameters every time, and thus causes usability concerns for other domain practitioners and precludes more potential users from exploring SOM based applications in a broader spectrum. Motivated by this practical concern, we propose a deterministic approach as a supplement to the standard self-organizing map. In accordance with the theoretical design, the experimental results with satellite cloud data demonstrate the effective and efficient organization as well as simplification capabilities of the proposed approach.

Initialization method↗
NASA NTRS2018

BioSentinel: Optimizing Growth Conditions for Improved Yeast Cell Viability After Long-Term Desiccation

NASA's BioSentinel mission is one of thirteen secondary payloads to be deployed on the Space Launch System Exploration Mission-1 (SLS EM-1). The BioSentinel nanosatellite will be sent into a heliocentric orbit beyond Low Earth Orbit (LEO), to study the effects of deep space radiation on the budding yeast, Saccharomyces cerevisiae. Ionizing radiation encountered in deep space can create damaging lesions in DNA, including double strand breaks (DSBs). Budding yeast is suitable as a biological model to study these effects, as it is eukaryotic, and can be desiccated for prolonged periods while retaining viability, thus serving as a robust analog for human cells. On the ground, yeast cells are grown in liquid medium, then loaded into the wells of microfluidic cards and air dried prior to integration into the payload. Once the spacecraft reaches its target heliocentric orbit, a mixture of growth medium and metabolic indicator dye will be pumped into the microwells at specific time points to rehydrate the cells and allow them to grow. A 3-color LED detection system will measure changes in growth and metabolism resulting from ionizing radiation exposure. BioSentinel contains a wild type control strain and a rad51 mutant that is defective for DNA damage repair. In this study, we will determine the optimal amount of time to grow diploid yeast cells in liquid culture before they are desiccated for space flight. After an extended time in stationary phase, they become more tolerant to desiccation due to stress caused by nitrogen starvation. However, excessive exposure can lead to loss of viability and to a heterogeneous cell population due to sporulation. Since viability loss during desiccation poses a risk to mission success, a stress preconditioning process during initial growth may increase long-term cell viability. To determine the growth period that improves desiccation tolerance but allows for retention of uniform radiation sensitivity, we will grow both strains in liquid medium for a varying number of days (4 to 7), desiccate the cells, and then observe changes to cell viability and ionizing radiation sensitivity over time. Supported by the Space Life Sciences Training Program at NASA Ames Research Center.

Cui, Kaixin↗
NASA NTRSDate not supplied

Variance Decomposition of MEDLI2 Reconstructed Heating Using Neural Networks

The Mars Entry, Descent, and Landing Instrumentation (MEDLI2) sensor suite collected data during entry of the Mars 2020 Perseverance rover into Mars’ atmosphere. An inverse estimation of the backshell and heatshield surface aeroheating was performed, using the data from the MEDLI2 Instrumented Sensor Plugs, a network of thermocouples embedded within the thermal protection system across the aeroshell. Monte Carlo analysis was conducted to assess the sensitivity of the surface heat rate, temperature, and heat load to uncertainties in thermocouple depth and material properties. In this paper, a variance decomposition method using Sobol indices was employed to understand the relative contributions of each uncertainty parameter. Performing this analysis using results from the inverse analysis tool FIAT_Opt was found to require incredibly high computation time, and thus machine learning models were trained and evaluated as a surrogate model for FIAT_Opt. This paper demonstrates that machine learning models can be an efficient, accurate alternative to state-of-the-art inverse analysis tools like FIAT_Opt, especially for computationally-expensive processes. Using these models, the sensitivity analysis showed that uncertainties in heat capacity and thermal conductivity were the main drivers for the overall uncertainty in peak reconstructed heating and heat load.

H S Alpert↗
DOE OSTI2020

AGC-4 Experiment Irradiation Monitoring Data Qualification Final Report

The Graphite Technology Development Program ran a series of six experiments to quantify the effects of irradiation on nuclear grade graphite. This report focuses on the fourth experiment, Advanced Graphite Creep 4 (AGC 4). The Advanced Reactor Development (ARD) Technology Development Office (TDO) Program for research and development activities require documentation of qualified monitoring data to design and license the first high-temperature reactor nuclear plant. Qualified data meets the requirements for use as described in the experiment planning and quality assurance documents. Failed data do not meet the requirements and provide no useable information. Trend data may not meet all requirements but still provide some useable information. Use of Trend data requires assessment of how any deficiencies affect a particular use of the data. AGC-4 began with Advanced Test Reactor (ATR) Cycle 157D on May 30, 2015. After irradiating the graphite for two cycles, the capsule was removed from the reactor after ATR Cycle 158A, which ended on January 2, 2016, due to interference with another experiment. Irradiation was resumed with Cycle 162A on October 7, 2017 after the interfering experiment was removed from the reactor. AGC-4 irradiation completed by the end of Cycle 166B on January 10, 2020. Between Cycles 162A and 166B, AGC-4 capsule was removed from the reactor core during two Powered Axial Locator Mechanism (PALM) Cycles 163A and 165A to avoid overheating. All thermocouples (TCs) have functioned throughout the AGC-4 experiment. A total of 9,256,184 out of all 11,167,213 temperature records (or 82.9%) are Qualified for use by the ARD Program and 1,911,029 missing values are Failed. Argon, helium, and total gas flow data were within expected ranges, except only eight out-of-range values occurred during outages. A total of 21,240,627 out of all 22,288,5103 gas flow rates (or 92.8%) are Qualified for use by the ARD Program and 1,644,476 are Failed records mostly due to missing values. Discharge gas line moisture values were consistently low during full ATR power, except for 162B, when moisture content was increasing to more than 200 ppmv by the end of this cycle. During outages, moisture content reached as high as 700 ppmv. 1,089,941 out of a total of 1,107,331 moisture values (or 98.4%) are within the measurement range of the instrument and are Qualified for use by the ARD Program and 17,390 missing moisture values are Failed. Graphite creep specimens were subjected to one of three loads: 393, 491, or 589 lbf. For a brief period during Cycle 157D between 12:19 on June 2, 2015, and 08:23 on June 11, 2015, the load cells were wired incorrectly, resulting in missing stack load data. Missing stack loads were estimated from measured ram pressures using regression equations developed from the existing data from Cycle 157D. Estimated stack loads during this period are considered to be an accurate representation of actual load applied to the stacks. These loads deviate slightly from the planned loads. This deviation does not prevent the data from being Qualified for use but must be taken into account when analyzing the effect of load on creep. 6,095,728 out of a total of 6,403,985 moisture values (or 95.2%) are within the measurement range of the instrument and are Qualified for use by the ARD Program and 308,257 missing load values are Failed. Stack displacement increased consistently throughout the eight cycles, with total displacement reached highest value of 2.4 in by the end of irradiation. During ATR outages, a set of pneumatic rams raised the stacks of graphite creep specimens to ensure the specimens were not stuck within the test train. This stack raising was performed seven times throughout irradiation. All stacks were raised successfully each time. 4,744,974 out of a total of 6,094,513 displacement values (or 77.9%) are within the measurement range of the instrument and are Qualified for use by the ARD Program and 1,349,539 displacement values are Failed mostly due to missing values. Analyses were conducted on correlations between TCs to look for trends and step changes that might indicate instrument degradation or failure. Correlation analysis was used to identify instances when TCs form short circuits, referred to as virtual junctions, which result in TCs reporting temperatures from some location in the capsule other than the location where they were intended to read. No evidence of virtual junctions was found. Analyses were also conducted on control charts of temperature differences of between two TCs, which are expected to behave consistently throughout the entire irradiation period. Upward or downward trend over time indicates at least one TC in the pair was drifted. Examining control charts for all 66 possible pairs out of twelve TCs installed in the AGC-4 capsules reveal no clear drift failures occurred, except unstable behavior of two TCs in Zone 3, TC-7 and TC-8, over irradiation time. In conclus

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

Photopolymerizable Sol-gel Synthesis for Faster Manufacturing of Surrogate Nuclear Debris

Simulated debris is used to train first responders for potential nuclear detonations. Sol gel chemistry has been used to produce surrogate material for these training exercises, but the current average production time is 4 hours. Optimization using UV-polymerizable sol gel precursors can potentially decrease time as well as cost of the production process. FT-IR spectroscopy was used to measure and compare gelation kinetics of the standard and photopolymerizable sol gel synthetic procedures. Addition of the UV-polymerizable precursors and controlling the aqueous phase volume resulted in a 120-fold decrease in vitrification time.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗
DOE OSTI2025

Transformers and Long Short-Term Memory Transfer Learning for GenIV Reactor Temperature Time Series Forecasting

Automated monitoring of the coolant temperature can enable autonomous operation of generation IV reactors (GenIV), thus reducing their operating and maintenance costs. Automation can be accomplished with machine learning (ML) models trained on historical sensor data. However, the performance of ML usually depends on the availability of large amount of training data, which is difficult to obtain for GenIV, as this technology is still under development. We propose the use of transfer learning (TL), which involves utilizing knowledge across different domains, to compensate for this lack of training data. TL can be used to create pre-trained ML models with data from small-scale research facilities, which can then be fine-tuned to monitor GenIV reactors. In this work, we develop pre-trained Transformer and long short-term memory (LSTM) networks by training them on temperature measurements from thermal hydraulic flow loops operating with water and Galinstan fluids at room temperature at Argonne National Laboratory. The pre-trained models are then fine-tuned and re-trained with minimal additional data to perform predictions of the time series of high temperature measurements obtained from the Engineering Test Unit (ETU) at Kairos Power. The performance of the LSTM and Transformer networks is investigated by varying the size of the lookback window and forecast horizon. The results of this study show that LSTM networks have lower prediction errors than Transformers, but LSTM errors increase more rapidly with increasing lookback window size and forecast horizon compared to the Transformer errors.

LSTM↗
NASA NTRS2004

Medical/Scientific Illustration And Production Of Otological Health Awareness Materials

Over the past year, I have worked for my mentor, Beth Cooper, on a large variety of projects. Beth is the Manager of the Acoustical Testing Laboratory, which tests the acoustical emissions of payloads destined for the International Space Station. She is also responsible for educating, and developing new methods of educating, people of all occupational and educational backgrounds in hearing conservation. Beth spends much of her time developing new materials and strategies with which to train people and teach other people to train people in hearing conservation and noise emissions control. I have been helping Beth develop and market these materials by way of graphic design and scientific illustration. Last summer, I spent much of my time creating educational illustrations that visually explained particular concepts in Beth's presentations. Sometimes these illustrations were small "comics" while, at other times, they were an instructional series of illustrations. Since then, Beth and her lab have been developing and updating some materials which will be distributed free to hearing conservation and noise control professionals and others in related fields. I have helped with these projects by designing their packaging. In each instance, it was my responsibility to develop an aesthetically appealing package that would also, through its imagery, describe or summarize the contents of the product. I did this for 3 CD's (Auditory Demonstrations 11, MACSUG, and JeopEARdy) and saw them through their actual production and distribution. In addition to working with Beth, I work with the Imaging Technology Center on various imaging projects. Some of my activities include photo retouching and manipulation for videos and print. This summer, I also had the opportunity to develop a screen saver that would show of some of the photography contained on the soon-to-be-released "Highlights of the GRC Image Archives, vol. 2". I was also able to utilize my medical training to help several of ITC s videographers identify the best histological examples of cancerous cells for incorporation in one of their videos. Over the last part of this summer and then throughout the school year, I will be working with Beth to develop a "pre-packaged" lecture series about the physics of acoustics in the context of hearing conservation. These lectures will be used to teach people of all backgrounds the fundamental concepts involved in acoustical physics so they might be better aware of their own and others auditory health in and out of the work place, and, in the case of payload developers, to design and build more quiet science experiments for the ISS. Even though it may not seem as such, this project is precisely what I am learning to do as a student of the Cleveland Institute of Art's Medical Illustration Department. From my perspective, this project is about taking technical information and translating it into terms that anyone, regardless of background, can understand.

Hawes, Nicholas E.↗
DOE OSTI2022

3DBFSVBF (3D BatFinder Smart Video BioFilter and Multi-class BatFinder Smart Video BioFilter) [SWR-22-88]

Bats are notoriously difficult to study, therefore, identifying specific behavioral trends and the precise environmental conditions at the time of collision requires a monitoring solution that can reliably collect relevant data. To date, thermal infrared video surveillance has been extensively applied to study bats and has proven to be a powerful yet cumbersome tool. Current analytical approaches are time consuming because data processing data has not been fully automated. In the past, steps have been taken to record avian and bat activity in conjunction with complicated image processing techniques that separate species from other moving objects within the field of view (i.e. clouds and portions of the wind turbine). Once the videos are collected, the post-processing does not allow real time monitoring and identification, leading to a delay in both studying the behavior of these species and determining the effectiveness of any impact reduction strategy being studied. Moreover, object identification capability is lacking, thus limiting the usefulness of video data. To resolve these issues, we are using open source 3D computer vision and machine learning techniques allowing for automatic detection of objects in real-time with the ability to correlate these objects with environmental variables and recording the flight paths of each object. The machine learning has been trained on 3D data and allows for automated real-time data collection, identification and tracking, thereby eliminating the need for long and tedious post-analysis processing of the videos. This machine learning model is an added feature to the previous BatFinder Smart Video BioFilter and increases the accuracy of that systems classification by increasing the accuracy of identifying bats (90% accuracy) and insects (69% accuracy) to a 97% accuracy. There are two object classifier machine learning models, Binary and multi-classification. Binary object classifier labeled BatFinder_Smart_Video_BioFilter.h5 distinguishes between biological objects and non-biological objects. The main goal of this object classifier is to ignore the turbine blades while detecting biological object flying withing the rotor swept area of the turbine. Non-biological objects have a probability of 0 and biological objects have a probability of 1. Multi-classifier labeled Multiclass_BatFinder_Smart_Video_BioFilter.h5 distinguishes between bats, birds, insects and non-biological.

Yarbrough, John↗
NASA NTRS1993

Controlling a truck with an adaptive critic temporal difference CMAC design

In this study, CMAC (Cerebellar Model Articulated Controller) neural architectures are shown to be viable for the purposes of real-time learning and control. An adaptive critic temporal difference neurocontrol design has been implemented that learns in real-time how to back up a trailer truck along a fixed straight line trajectory. The truck backer-upper experiment is a standard performance measure in the neural network literature, but previously the training of the controllers was done off-line. With the CMAC neural architectures, it was possible to train the neurocontrollers on-line in real-time on a MS-DOS PC 386.

Shelton, Robert O.↗
DOE OSTI2021

Educating HPC Users in the use of advanced computing technology

We examine a multi-modal approach to educating and training users of an advanced computing technology testbed at the Institute for Advanced Computational Science at Stony Brook University. Ookami provides researchers worldwide with access to 176 Fujitsu A64FX compute nodes, this being the same processor technology powering the Japanese Fugaku supercomputer, the fastest computer in the world since June 2020. However, achieving high-performance on this Arm-based, leadership computing technology requires that users be familiar with details of computer architecture, performance analysis and modeling, and high-performance programming models that are commonly omitted in introductory programming courses. Indeed, regardless of their seniority, many of the testbed users are surprisingly unfamiliar with basic concepts such as vectorization, pipelining, latency/bandwidth, roofline models, computing energy/power, threads, and non-uniform memory access. These same concepts also pervade mainstream x86 technologies, so this is of widespread concern. Due to the national/global nature of our user community that is also very diverse in both discipline and experience, the inability to offer formal classes, and our experience that most people do not tend to read online documentation or training materials in sufficient depth, we have consciously employed multiple approaches that heavily emphasize (online) personal interactions and transfer of skills. Online documentation has been organized around best-practices and FAQs; twice-weekly hackathons and office hours via Zoom enable deep dives by both the team and the user community with multiple broad benefits; a Slack channel provides both real time and archived answers and discussions; and workshops, training and webinars target community needs as they arise. Furthermore, the perspective that these tools are being used in an educational setting rather than just for project communication makes them more effective and contributes to community success.

A64FX↗
NASA NTRS2017

Training, Retention, and Transfer of Data Entry Perceptual and Motoric Processes Over Long Retention Intervals

Subjects trained in a standard data entry task, which involved typing numbers (e.g., 5421) using their right hands. At test (6 months post-training), subjects completed the standard task, followed by a left-hand variant (typing with their left hands) that involved the same perceptual, but different motoric, processes as the standard task. At a second test (8 months post-training), subjects completed the standard task, followed by a code variant (translating letters into digits, then typing the digits with their right hands) that involved different perceptual, but the same motoric, processes as the standard task. For each of the three tasks, half the trials were trained numbers (old) and half were new. Repetition priming (faster response times to old than new numbers) was found for each task. Repetition priming for the standard task reflects retention of trained numbers; for the left-hand variant reflects transfer of perceptual processes; and for the code variant reflects transfer of motoric processes. There was thus evidence for both specificity and generalizability of training data entry perceptual and motoric processes over very long retention intervals.

dat↗
NASA NTRS2017

Robonaut 2 and You: Specifying and Executing Complex Operations

Crew time is a precious resource due to the expense of trained human operators in space. Efficient caretaker robots could lessen the manual labor load required by frequent vehicular and life support maintenance tasks, freeing astronaut time for scientific mission objectives. Humanoid robots can fluidly exist alongside human counterparts due to their form, but they are complex and high-dimensional platforms. This paper describes a system that human operators can use to maneuver Robonaut 2 (R2), a dexterous humanoid robot developed by NASA to research co-robotic applications. The system includes a specification of constraints used to describe operations, and the supporting planning framework that solves constrained problems on R2 at interactive speeds. The paper is developed in reference to an illustrative, typical example of an operation R2 performs to highlight the challenges inherent to the problems R2 must face. Finally, the interface and planner is validated through a case-study using the guiding example on the physical robot in a simulated microgravity environment. This work reveals the complexity of employing humanoid caretaker robots and suggest solutions that are broadly applicable.

Baker, William↗
NASA NTRS1994

Exercise detraining: Applicability to microgravity

Physical training exposes the various systems of the body to potent physiologic stimuli. These stimuli induce specific adaptations that enhance an individual's tolerance for the type of exercise encountered in training. The level of adaptation and the magnitude of improvement in exercise tolerance is proportional to the potency of the physical training stimuli. Likewise, our bodies are stimulated by gravity, which promotes adaptations of both the cardiovascular and skeletal muscles. Exposure to microgravity removes normal stimuli to these systems, and the body adapts to these reduced demands. In many respects the cessation of physical training in athletes and the transition from normal gravity to microgravity represent similar paradigms. Inherent to these situations is the concept of the reversibility of the adaptations induced by training or by exposure to normal gravity. The reversibility concept holds that when physical training is stopped (i.e., detraining) or reduced, or a person goes from normal gravity to microgravity, the bodily systems readjust in accordance with the diminished physiologic stimuli. The focus of this chapter is on the time course of loss of the adaptations to endurance training as well as on the possibility that certain adaptations persist, to some extent, when training is stopped. Because endurance exercise training generally improves cardiovascular function and promotes metabolic adaptations within the exercising skeletal musculature, the reversibility of these specific adaptations is considered. These observations have some applicability to the transition from normal to microgravity.

Coyle, Edward F.↗
DOE OSTI2024

Learning stochastic dynamics and predicting emergent behavior using transformers

We show that a neural network originally designed for language processing can learn the dynamical rules of a stochastic system by observation of a single dynamical trajectory of the system, and can accurately predict its emergent behavior under conditions not observed during training. We consider a lattice model of active matter undergoing continuous-time Monte Carlo dynamics, simulated at a density at which its steady state comprises small, dispersed clusters. We train a neural network called a transformer on a single trajectory of the model. The transformer, which we show has the capacity to represent dynamical rules that are numerous and nonlocal, learns that the dynamics of this model consists of a small number of processes. Forward-propagated trajectories of the trained transformer, at densities not encountered during training, exhibit motility-induced phase separation and so predict the existence of a nonequilibrium phase transition. Transformers have the flexibility to learn dynamical rules from observation without explicit enumeration of rates or coarse-graining of configuration space, and so the procedure used here can be applied to a wide range of physical systems, including those with large and complex dynamical generators.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Neural units with time-dependent functionality

We show that the time-resolved dynamics of an underdamped harmonic oscillator can be used to do multifunctional computation, performing distinct computations at distinct times within a single dynamical trajectory. We consider the amplitude of an oscillator whose inputs influence its frequency. The activity of the oscillator at fixed times is a nonmonotonic function of its inputs, so it can solve problems such as XOR that are not linearly separable. The activity of the oscillator at fixed input is a nonmonotonic function of time, so it is multifunctional in a temporal sense, and able to carry out distinct nonlinear computations at distinct times within the same dynamical trajectory. We show that a single oscillator, observed at different times, can act as all of the elementary logic gates and perform binary addition, the latter usually implemented in hardware using five logic gates. We show that a set of n oscillators, observed at different times, can perform an arbitrary number of analog-to-n-bit digital conversions. We also show that oscillators can be trained by gradient descent to perform distinct classification tasks at distinct times. Computing with time-dependent functionality can be done in or out of equilibrium, and suggests a way of reducing the number of parameters or devices required to do nonlinear computations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Sequential Inference of Hospitalization Electronic Health Records Using Probabilistic Models

In the dynamic hospital setting, decision support can be a valuable tool for improving patient outcomes. Data-driven inference of future outcomes is challenging in this dynamic setting, where long sequences such as laboratory tests and medications are updated frequently. This is due in part to heterogeneity of data types and mixed-sequence types contained in variable length sequences. In this work we design a probabilistic unsupervised model for multiple arbitrary-length sequences contained in hospitalization Electronic Health Record (EHR) data. The model uses a latent variable structure and captures complex relationships between medications, diagnoses, laboratory tests, neurological assessments, and medications. It can be trained on original data, without requiring any lossy transformations or time binning. Inference algorithms are derived that use partial data to infer properties of the complete sequences, including their length and presence of specific values. We train this model on data from subjects receiving medical care in the Kaiser Permanente Northern California integrated healthcare delivery system. The results are evaluated against held-out data for predicting the length of sequences and presence of Intensive Care Unit (ICU) in hospitalization bed sequences. Our method outperforms a baseline approach, showing that in these experiments the trained model captures information in the sequences that is informative of their future values.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Operator learning for predicting multiscale bubble growth dynamics

We report simulating and predicting multiscale problems that couple multiple physics and dynamics across many orders of spatiotemporal scales is a great challenge that has not been investigated systematically by deep neural networks (DNNs). Herein, we develop a framework based on operator regression, the so-called deep operator network (DeepONet), with the long-term objective to simplify multiscale modeling by avoiding the fragile and time-consuming “hand-shaking” interface algorithms for stitching together heterogeneous descriptions of multiscale phenomena. To this end, as a first step, we investigate if a DeepONet can learn the dynamics of different scale regimes, one at the deterministic macroscale and the other at the stochastic microscale regime with inherent thermal fluctuations. Specifically, we test the effectiveness and accuracy of the DeepONet in predicting multirate bubble growth dynamics, which is described by a Rayleigh–Plesset (R–P) equation at the macroscale and modeled as a stochastic nucleation and cavitation process at the microscale by dissipative particle dynamics (DPD). First, we generate data using the R–P equation for multirate bubble growth dynamics caused by randomly time-varying liquid pressures drawn from Gaussian random fields (GRFs). Our results show that properly trained DeepONets can accurately predict the macroscale bubble growth dynamics and can outperform long short-term memory networks. We also demonstrate that the DeepONet can extrapolate accurately outside the input distribution using only very few new measurements. Subsequently, we train the DeepONet with DPD data corresponding to stochastic bubble growth dynamics. Although the DPD data are noisy and we only collect sparse data points on the trajectories, the trained DeepONet model is able to predict accurately the mean bubble dynamics for time-varying GRF pressures. Taken together, our findings demonstrate that DeepONets can be employed to unify the macroscale and microscale models of the multirate bubble growth problem, hence providing new insight into the role of operator regression via DNNs in tackling realistic multiscale problems and in simplifying modeling with heterogeneous descriptions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Layer-Parallel Training of Deep Residual Neural Networks

Residual neural networks (ResNets) are a promising class of deep neural networks that have shown excellent performance for a number of learning tasks, e.g., image classification and recognition. Mathematically, ResNet architectures can be interpreted as forward Euler discretizations of a nonlinear initial value problem whose time-dependent control variables represent the weights of the neural network. Hence, training a ResNet can be cast as an optimal control problem of the associated dynamical system. For similar time-dependent optimal control problems arising in engineering applications, parallel-in-time methods have shown notable improvements in scalability. This paper demonstrates the use of those techniques for efficient and effective training of ResNets. The proposed algorithms replace the classical (sequential) forward and backward propagation through the network layers with a parallel nonlinear multigrid iteration applied to the layer domain. This adds a new dimension of parallelism across layers that is attractive when training very deep networks. From this basic idea, we derive multiple layer-parallel methods. The most efficient version employs a simultaneous optimization approach where updates to the network parameters are based on inexact gradient information in order to speed up the training process. Finally, using numerical examples from supervised classification, we demonstrate that the new approach achieves a training performance similar to that of traditional methods, but enables layer-parallelism and thus provides speedup over layer-serial methods through greater concurrency.

97 MATHEMATICS AND COMPUTING↗