Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Training Time”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Transformers and Long Short-Term Memory Transfer Learning for GenIV Reactor Temperature Time Series Forecasting

Automated monitoring of the coolant temperature can enable autonomous operation of generation IV reactors (GenIV), thus reducing their operating and maintenance costs. Automation can be accomplished with machine learning (ML) models trained on historical sensor data. However, the performance of ML usually depends on the availability of large amount of training data, which is difficult to obtain for GenIV, as this technology is still under development. We propose the use of transfer learning (TL), which involves utilizing knowledge across different domains, to compensate for this lack of training data. TL can be used to create pre-trained ML models with data from small-scale research facilities, which can then be fine-tuned to monitor GenIV reactors. In this work, we develop pre-trained Transformer and long short-term memory (LSTM) networks by training them on temperature measurements from thermal hydraulic flow loops operating with water and Galinstan fluids at room temperature at Argonne National Laboratory. The pre-trained models are then fine-tuned and re-trained with minimal additional data to perform predictions of the time series of high temperature measurements obtained from the Engineering Test Unit (ETU) at Kairos Power. The performance of the LSTM and Transformer networks is investigated by varying the size of the lookback window and forecast horizon. The results of this study show that LSTM networks have lower prediction errors than Transformers, but LSTM errors increase more rapidly with increasing lookback window size and forecast horizon compared to the Transformer errors.

LSTM↗
DOE OSTI2022

3DBFSVBF (3D BatFinder Smart Video BioFilter and Multi-class BatFinder Smart Video BioFilter) [SWR-22-88]

Bats are notoriously difficult to study, therefore, identifying specific behavioral trends and the precise environmental conditions at the time of collision requires a monitoring solution that can reliably collect relevant data. To date, thermal infrared video surveillance has been extensively applied to study bats and has proven to be a powerful yet cumbersome tool. Current analytical approaches are time consuming because data processing data has not been fully automated. In the past, steps have been taken to record avian and bat activity in conjunction with complicated image processing techniques that separate species from other moving objects within the field of view (i.e. clouds and portions of the wind turbine). Once the videos are collected, the post-processing does not allow real time monitoring and identification, leading to a delay in both studying the behavior of these species and determining the effectiveness of any impact reduction strategy being studied. Moreover, object identification capability is lacking, thus limiting the usefulness of video data. To resolve these issues, we are using open source 3D computer vision and machine learning techniques allowing for automatic detection of objects in real-time with the ability to correlate these objects with environmental variables and recording the flight paths of each object. The machine learning has been trained on 3D data and allows for automated real-time data collection, identification and tracking, thereby eliminating the need for long and tedious post-analysis processing of the videos. This machine learning model is an added feature to the previous BatFinder Smart Video BioFilter and increases the accuracy of that systems classification by increasing the accuracy of identifying bats (90% accuracy) and insects (69% accuracy) to a 97% accuracy. There are two object classifier machine learning models, Binary and multi-classification. Binary object classifier labeled BatFinder_Smart_Video_BioFilter.h5 distinguishes between biological objects and non-biological objects. The main goal of this object classifier is to ignore the turbine blades while detecting biological object flying withing the rotor swept area of the turbine. Non-biological objects have a probability of 0 and biological objects have a probability of 1. Multi-classifier labeled Multiclass_BatFinder_Smart_Video_BioFilter.h5 distinguishes between bats, birds, insects and non-biological.

Yarbrough, John↗
DOE OSTI2021

Educating HPC Users in the use of advanced computing technology

We examine a multi-modal approach to educating and training users of an advanced computing technology testbed at the Institute for Advanced Computational Science at Stony Brook University. Ookami provides researchers worldwide with access to 176 Fujitsu A64FX compute nodes, this being the same processor technology powering the Japanese Fugaku supercomputer, the fastest computer in the world since June 2020. However, achieving high-performance on this Arm-based, leadership computing technology requires that users be familiar with details of computer architecture, performance analysis and modeling, and high-performance programming models that are commonly omitted in introductory programming courses. Indeed, regardless of their seniority, many of the testbed users are surprisingly unfamiliar with basic concepts such as vectorization, pipelining, latency/bandwidth, roofline models, computing energy/power, threads, and non-uniform memory access. These same concepts also pervade mainstream x86 technologies, so this is of widespread concern. Due to the national/global nature of our user community that is also very diverse in both discipline and experience, the inability to offer formal classes, and our experience that most people do not tend to read online documentation or training materials in sufficient depth, we have consciously employed multiple approaches that heavily emphasize (online) personal interactions and transfer of skills. Online documentation has been organized around best-practices and FAQs; twice-weekly hackathons and office hours via Zoom enable deep dives by both the team and the user community with multiple broad benefits; a Slack channel provides both real time and archived answers and discussions; and workshops, training and webinars target community needs as they arise. Furthermore, the perspective that these tools are being used in an educational setting rather than just for project communication makes them more effective and contributes to community success.

A64FX↗
DOE OSTI2024

Learning stochastic dynamics and predicting emergent behavior using transformers

We show that a neural network originally designed for language processing can learn the dynamical rules of a stochastic system by observation of a single dynamical trajectory of the system, and can accurately predict its emergent behavior under conditions not observed during training. We consider a lattice model of active matter undergoing continuous-time Monte Carlo dynamics, simulated at a density at which its steady state comprises small, dispersed clusters. We train a neural network called a transformer on a single trajectory of the model. The transformer, which we show has the capacity to represent dynamical rules that are numerous and nonlocal, learns that the dynamics of this model consists of a small number of processes. Forward-propagated trajectories of the trained transformer, at densities not encountered during training, exhibit motility-induced phase separation and so predict the existence of a nonequilibrium phase transition. Transformers have the flexibility to learn dynamical rules from observation without explicit enumeration of rates or coarse-graining of configuration space, and so the procedure used here can be applied to a wide range of physical systems, including those with large and complex dynamical generators.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Neural units with time-dependent functionality

We show that the time-resolved dynamics of an underdamped harmonic oscillator can be used to do multifunctional computation, performing distinct computations at distinct times within a single dynamical trajectory. We consider the amplitude of an oscillator whose inputs influence its frequency. The activity of the oscillator at fixed times is a nonmonotonic function of its inputs, so it can solve problems such as XOR that are not linearly separable. The activity of the oscillator at fixed input is a nonmonotonic function of time, so it is multifunctional in a temporal sense, and able to carry out distinct nonlinear computations at distinct times within the same dynamical trajectory. We show that a single oscillator, observed at different times, can act as all of the elementary logic gates and perform binary addition, the latter usually implemented in hardware using five logic gates. We show that a set of n oscillators, observed at different times, can perform an arbitrary number of analog-to-n-bit digital conversions. We also show that oscillators can be trained by gradient descent to perform distinct classification tasks at distinct times. Computing with time-dependent functionality can be done in or out of equilibrium, and suggests a way of reducing the number of parameters or devices required to do nonlinear computations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Sequential Inference of Hospitalization Electronic Health Records Using Probabilistic Models

In the dynamic hospital setting, decision support can be a valuable tool for improving patient outcomes. Data-driven inference of future outcomes is challenging in this dynamic setting, where long sequences such as laboratory tests and medications are updated frequently. This is due in part to heterogeneity of data types and mixed-sequence types contained in variable length sequences. In this work we design a probabilistic unsupervised model for multiple arbitrary-length sequences contained in hospitalization Electronic Health Record (EHR) data. The model uses a latent variable structure and captures complex relationships between medications, diagnoses, laboratory tests, neurological assessments, and medications. It can be trained on original data, without requiring any lossy transformations or time binning. Inference algorithms are derived that use partial data to infer properties of the complete sequences, including their length and presence of specific values. We train this model on data from subjects receiving medical care in the Kaiser Permanente Northern California integrated healthcare delivery system. The results are evaluated against held-out data for predicting the length of sequences and presence of Intensive Care Unit (ICU) in hospitalization bed sequences. Our method outperforms a baseline approach, showing that in these experiments the trained model captures information in the sequences that is informative of their future values.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Operator learning for predicting multiscale bubble growth dynamics

We report simulating and predicting multiscale problems that couple multiple physics and dynamics across many orders of spatiotemporal scales is a great challenge that has not been investigated systematically by deep neural networks (DNNs). Herein, we develop a framework based on operator regression, the so-called deep operator network (DeepONet), with the long-term objective to simplify multiscale modeling by avoiding the fragile and time-consuming “hand-shaking” interface algorithms for stitching together heterogeneous descriptions of multiscale phenomena. To this end, as a first step, we investigate if a DeepONet can learn the dynamics of different scale regimes, one at the deterministic macroscale and the other at the stochastic microscale regime with inherent thermal fluctuations. Specifically, we test the effectiveness and accuracy of the DeepONet in predicting multirate bubble growth dynamics, which is described by a Rayleigh–Plesset (R–P) equation at the macroscale and modeled as a stochastic nucleation and cavitation process at the microscale by dissipative particle dynamics (DPD). First, we generate data using the R–P equation for multirate bubble growth dynamics caused by randomly time-varying liquid pressures drawn from Gaussian random fields (GRFs). Our results show that properly trained DeepONets can accurately predict the macroscale bubble growth dynamics and can outperform long short-term memory networks. We also demonstrate that the DeepONet can extrapolate accurately outside the input distribution using only very few new measurements. Subsequently, we train the DeepONet with DPD data corresponding to stochastic bubble growth dynamics. Although the DPD data are noisy and we only collect sparse data points on the trajectories, the trained DeepONet model is able to predict accurately the mean bubble dynamics for time-varying GRF pressures. Taken together, our findings demonstrate that DeepONets can be employed to unify the macroscale and microscale models of the multirate bubble growth problem, hence providing new insight into the role of operator regression via DNNs in tackling realistic multiscale problems and in simplifying modeling with heterogeneous descriptions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Layer-Parallel Training of Deep Residual Neural Networks

Residual neural networks (ResNets) are a promising class of deep neural networks that have shown excellent performance for a number of learning tasks, e.g., image classification and recognition. Mathematically, ResNet architectures can be interpreted as forward Euler discretizations of a nonlinear initial value problem whose time-dependent control variables represent the weights of the neural network. Hence, training a ResNet can be cast as an optimal control problem of the associated dynamical system. For similar time-dependent optimal control problems arising in engineering applications, parallel-in-time methods have shown notable improvements in scalability. This paper demonstrates the use of those techniques for efficient and effective training of ResNets. The proposed algorithms replace the classical (sequential) forward and backward propagation through the network layers with a parallel nonlinear multigrid iteration applied to the layer domain. This adds a new dimension of parallelism across layers that is attractive when training very deep networks. From this basic idea, we derive multiple layer-parallel methods. The most efficient version employs a simultaneous optimization approach where updates to the network parameters are based on inexact gradient information in order to speed up the training process. Finally, using numerical examples from supervised classification, we demonstrate that the new approach achieves a training performance similar to that of traditional methods, but enables layer-parallelism and thus provides speedup over layer-serial methods through greater concurrency.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

HydraGNN_Predictive_GFM_2024 - Ensemble of predictive graph foundation models for ground state atomistic materials modeling

We provide the ensemble of fifteen pre-trained graph foundation models (GFMs) for atomistic materials modeling applications. Each one of the fifteen GFMs has been trained on five open-source datasets that (once aggregated) amount to over 154 million atomistic structures, which cover over two-thirds of the natural elements of the periodic table and that comprises a broad set of organic and inorganic compounds. This vast set of atomistic structures comprises ground state configurations that are dynamically stable (i.e., equilibrated structures with atomic forces approximately close to zero values) as well as dynamically unstable structures (i.e., non-equilibrium structures with non-negligible non-zero values of atomic forces). The ensemble of datasets aggregated does NOT include excited states. The datasets have been curated to remove atomistic structures with spectral norm of the force tensor above 100 eV/angstrom. Moreover, a linear term of the energy was computed for each dataset using a linear regression model that uses the chemical concentration of each natural element as regressor. The linear term predicted by the linear regression model has been subtracted from each original energy value to perform a re-alignment of the energy values across different electronic structures approximation theories performed to generate the diverse multi-source, multi-fidelity datasets. The folder "ADIOS_files" contains the set of pre-processed datasets in Adaptable I/O System (ADIOS) format (https://www.exascaleproject.org/research-project/adios/) that have been used for the development and training of GFMs in this work. The "ADIOS_files" directory contains 6 sub-directories named as follows: - ANI1x-v3.bp - MPTrj-v3.bp - OC2020-20M-v3.bp - OC2020-v3.bp - OC2022-v3.bp - qm7x-v3.bp Each sub-directory contains the pre-processed datasets converted in Adaptable I/O System (ADIOS) format (https://www.exascaleproject.org/research-project/adios/) that have been used to the development, training, and performance testing of the ensemble go predictive graph foundation models. Each GFM was developed using HydraGNN (https://github.com/ORNL/HydraGNN) as underlying graph neural network (GNN) architecture. The multi-task learning (MTL) capability of HydraGNN was used to simultaneously train the GFMs on labeled values for direct predictions of energy (a total system property of an atomistic structure that measures the chemical stability) and atomic forces (an atomic level property of an atomistic structure that measures the dynamical stability). The hyper parameters of the GFM have been tuned using scalable hyperparameter optimization (HPO) algorithms implemented in the software DeepHyper (https://github.com/deephyper/deephyper). The pre-training of each HPO trial was performed using distributed data parallelism (DDP) to scale the training across 128 compute nodes of the exascale OLCF supercomputer Frontier. Each HPO trial was trained only for 10 epochs and an early stopping was performed to avoid wasting significant computational resources on GNN architectures that were clearly underperforming. For each HPO trial, the 'omnistat' tool developed by (AMD Research - Advanced Micro Device) was used to measure the total energy consumption in kWh. The ensemble of GFMs was obtained by selecting the fifteen best performing HPO trials. Four models have been selected for their clear advantage in accuracy, and these are the GFMs with IDs 229, 156, 147, 260. Additional eleven models have been selected based on judicious balance between accuracy and energy consumption needed for training, and these are the GFMs with IDs 165, 78, 137, 1, 175, 171, 181, 67, 179, 167, 351. Each selected GFM of the ensemble was continued to cumulate a total of at most 30 epochs. In some cases, the total number of epochs actually performed was les than 30 due to two combined factors: (1) the size of the GFM (i.e., the number of model parameters to train) and (2) the total wall-clock time for which the computational resources could be allocated on OLCF-Frontier. The "Ensemble_of_models" directory contains 15 sub-directories named as follows: - gfm_0.229 - gfm_0.156 - gfm_0.147 - gfm_0.260 - gfm_0.165 - gfm_0.78 - gfm_0.137 - gfm_0.1 - gfm_0.175 - gfm_0.171 - gfm_0.181 - gfm_0.67 - gfm_0.179 - gfm_0.167 - gfm_0.351 Each one of these sub-directories refers to one of the fifteen HPO trials that have been selected to continue the pre-training with at most 30 epochs. With each sub-directory associated with a specific HPO trial, the following files can be found: - config.json: file for argument parsing to develop and train an HydraGNN architecture - gfm_0.ID_epoch_N.pk: file with model parameters for HPO ID trial after N epochs of training The ensemble of fifteen GFM architectures was used for (1) ensemble averaging to stabilize the predictions of energy and atomic forces after pre-training for post-processing analysis and (2) ensemble uncertainty quantification (UQ). The code used to develop, pre-train, and load the pre-trained models for post-processing analysis is available on the ORNL-GitHub at the following link: https://github.com/ORNL/HydraGNN/tree/Predictive_GFM_2024

36 MATERIALS SCIENCE↗
DOE OSTI2024

Application of Modified Meshgraphnets for Subsurface Prediction during CO2 Sequestration

In the face of the increasingly dire consequences of anthropogenic climate change, capturing and storing carbon dioxide is paramount. However, several impediments exist to the safe and effective subsurface storage of CO2, such as cost of transport, identification of suitable sites for subsurface storage, and assessment of long-term risk from storage in subsurface aquifers. Accurate subsurface modeling is necessary to ensure that CO2 storage is both safe and effective. Still, such modeling has traditionally required either substantial time and computational power (numerical simulation) or a substantial amount of pre-existing data for training (machine learning models). Additionally, these models lack flexibility in dealing with both changes in discretization of the input data and generalizability beyond the data on which they are trained. In order to address these issues, this research applies graph neural networks (GNNs) to predict subsurface saturation and pressure during CO₂ injection in a model of the Illinois Basin-Decatur Project (IBDP). GNNs provide a flexible, intuitive method for representing and manipulating complex unstructured data, which is often found in many practical domain problems such as fluid flow and subsurface characterization. These unstructured grids are easily represented in GNNs by representing spatially-localized features such as permeability, porosity, saturation, and pressure as nodes in a graph and relationships between these properties as edges connecting these nodes. This research applies a specific GNN model called MeshGraphNets (MGN) to model the change in CO2 saturation and pressure over a 50-month time period (36 months of injection, 14 months post-injection). The MGN model leverages a message passing process that allows the network to learn both the spatial and temporal dynamics of this system simultaneously. Additionally, training on a limited dataset (64 realizations, 20 time points each) resulted in a high degree of accuracy in saturation prediction both within the same timeframe as the training (20 months, 0.039 average RMSE) and when projecting out to the end of injection (36 months, 0.053 average RMSE). Temporal predictions such as those generated by MGNs and other similar models are prone to accumulated error over time; in order to address this, a multi-step rollout (MSR) training process was applied to calculate training loss. This method mimics the forward prediction during inference by “rolling out” multiple time points in a single training step using the previous prediction as input to the MGN model. By calculating the loss several time steps forward from the current prediction, the model is forced to find a more stable state over time. Application of MSR to the MGN model resulted in an average 15% reduction in inference error over time during forward prediction. This study showcases the immense potential of GNNs as a game-changing methodology for predicting pressure and saturation evolution in CCS projects, ultimately paving the way for more sustainable and effective carbon storage solutions. Presentation prepared for the 2024 AiChE Annual Meeting, October 27 to November 1 2024, San Diego, CA.

Holcomb, Paul↗
DOE OSTI2020

Root identification in minirhizotron imagery with multiple instance learning

In this study, multiple instance learning (MIL) algorithms to automatically perform root detection and segmentation in minirhizotron imagery using only image-level labels are proposed. Root and soil characteristics vary from location to location, and thus, supervised machine learning approaches that are trained with local data provide the best ability to identify and segment roots in minirhizotron imagery. However, labeling roots for training data (or otherwise) is an extremely tedious and time-consuming task. This paper aims to address this problem by labeling data at the image level (rather than the individual root or root pixel level) and train algorithms to perform individual root pixel level segmentation using MIL strategies. Three MIL methods (multiple instance adaptive cosine coherence estimator, multiple instance support vector machine, multiple instance learning with randomized trees) were applied to root detection and compared to non-MIL approaches. The results show that MIL methods improve root segmentation in challenging minirhizotron imagery and reduce the labeling burden. In our results, multiple instance support vector machine outperformed other methods. The multiple instance adaptive cosine coherence estimator algorithm was a close second with an added advantage that it learned an interpretable root signature which identified the traits used to distinguish roots from soil and did not require parameter selection.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Generalization of Deep-Learning Models for Classification of Local Distance Earthquakes and Explosions across Various Geologic Settings

Although accurately classifying signals from earthquakes and explosions at local distance (<250 km) remains an important task for seismic network operations, the growing volume of available seismic data presents a challenge for analysts using traditional source discrimination techniques. In recent years, deep-learning models have proven effective at discriminating between low-magnitude earthquakes and explosions measured at local distances, but it is not clear how well these models are capable of generalizing across different geological settings. To address the issue of generalization between regions, we train deep-learning models (convolutional neural networks [CNNs]) on time–frequency representations (scalograms) of three-component earthquake and explosion signals from eight different regions in the continental United States. We explore scenarios where models are trained on data from all regions, individual regions, or all but one region. We find that although CNN models trained on individual regions do not necessarily generalize well across different settings, models trained on multiple regions that include diverse path coverage generalize to new regions, with station-level accuracy of up to 90% or more for data sets from unseen regions. In general, CNN-based discrimination models significantly outperform models based on uncorrected P/S ratio (measured in the 10–18 Hz frequency band), even when CNN models are tested on data from entirely unseen regions.

58 GEOSCIENCES↗
DOE OSTI2020

A Fully Automatic Method for Rapidly Mapping Impacted Area by Natural Disaster

Deep learning based change detection methods have achieved the state-of-the-art performance in several recent studies. However, such methods usually are supervised, and therefore a large number of training samples is often a requisite. Manually preparing those training samples is not only expensive but also time-consuming, which does not fit the need of rapidly mapping the impacted area caused by nature disaster for further rescue mission and damage assessment. In this study, a fully automatic method was proposed to address the issue by automating training sample generation for mapping the impacted area caused by nature disaster. We used the 2011 tornado event in Joplin, Missouri, US, as an example of its application. The generated impacted area map was both visually and quantitatively evaluated against the ground truth data collected by US Federal Emergency Management Agency (FEMA). The results show that the map matches well with the FEMA ground truth data with 86% of major-damaged and destroyed buildings identified by FEMA on the ground also detected by this fully automatic framework using very high resolution (VHR) satellite images.

Liu, Tao↗
DOE OSTI2021

Conditioned Simulation of Ground-Motion Time Series at Uninstrumented Sites Using Gaussian Process Regression

Ground-motion time series are essential input data in seismic analysis and performance assessment of the built environment. Because instruments to record free-field ground motions are generally sparse, methods are needed to estimate motions at locations with no available ground-motion recording instrumentation. In this study, given a set of observed motions, ground-motion time series at target sites are constructed using a Gaussian process regression (GPR) approach, which treats the real and imaginary parts of the Fourier spectrum as random Gaussian variables. Model training, verification, and applicability studies are carried out using the physics-based simulated ground motions of the 1906 Mw 7.9 San Francisco earthquake and Mw 7.0 Hayward fault scenario earthquake in northern California. Additionally, the method’s performance is further evaluated using the 2019 Mw 7.1 Ridgecrest earthquake ground motions recorded by the Community Seismic Network stations located in southern California. These evaluations indicate that the trained GPR model is able to adequately estimate the ground-motion time series for frequency ranges that are pertinent for most earthquake engineering applications. The trained GPR model exhibits proper performance in predicting the long-period content of the ground motions as well as directivity pulses.

58 GEOSCIENCES↗
DOE OSTI2021

Recovering the phase and amplitude of X-ray FEL pulses using neural networks and differentiable models

Dynamics experiments are an important use-case for X-ray free-electron lasers (XFELs), but time-domain measurements of the X-ray pulses themselves remain a challenge. Shot-by-shot X-ray diagnostics could enable a new class of simpler and potentially higher-resolution pump-probe experiments. Here, we report training neural networks to combine low-resolution measurements in both the time and frequency domains to recover X-ray pulses at high-resolution. Critically, we also recover the phase, opening the door to coherent-control experiments with XFELs. The model-based generative neural-network architecture can be trained directly on unlabeled experimental data and is fast enough for real-time analysis on the new generation of MHz XFELs.

42 ENGINEERING↗
DOE OSTI2021

QUBO formulations for training machine learning models

Abstract Training machine learning models on classical computers is usually a time and compute intensive process. With Moore’s law nearing its inevitable end and an ever-increasing demand for large-scale data analysis using machine learning, we must leverage non-conventional computing paradigms like quantum computing to train machine learning models efficiently. Adiabatic quantum computers can approximately solve NP-hard problems, such as the quadratic unconstrained binary optimization (QUBO), faster than classical computers. Since many machine learning problems are also NP-hard, we believe adiabatic quantum computers might be instrumental in training machine learning models efficiently in the post Moore’s law era. In order to solve problems on adiabatic quantum computers, they must be formulated as QUBO problems, which is very challenging. In this paper, we formulate the training problems of three machine learning models—linear regression, support vector machine (SVM) and balanced k-means clustering—as QUBO problems, making them conducive to be trained on adiabatic quantum computers. We also analyze the computational complexities of our formulations and compare them to corresponding state-of-the-art classical approaches. We show that the time and space complexities of our formulations are better (in case of SVM and balanced k-means clustering) or equivalent (in case of linear regression) to their classical counterparts.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Leveraging Multitime Hamilton–Jacobi PDEs for Certain Scientific Machine Learning Problems

Hamilton-Jacobi partial differential equations (HJ PDEs) have deep connections with a wide range of fields, including optimal control, differential games, and imaging sciences. By considering the time variable to be a higher dimensional quantity, HJ PDEs can be extended to the multi-time case. In this paper, we establish a novel theoretical connection between specific optimization problems arising in machine learning and the multi-time Hopf formula, which corresponds to a representation of the solution to certain multi-time HJ PDEs. Through this connection, we increase the interpretability of the training process of certain machine learning applications by showing that when we solve these learning problems, we also solve a multi-time HJ PDE and, by extension, its corresponding optimal control problem. As a first exploration of this connection, we develop the relation between the regularized linear regression problem and the Linear Quadratic Regulator (LQR). We then leverage our theoretical connection to adapt standard LQR solvers (namely, those based on the Riccati ordinary differential equations) to design new training approaches for machine learning. Lastly, we provide some numerical examples that demonstrate the versatility and possible computational advantages of our Riccati-based approach in the context of continual learning, post-training calibration, transfer learning, and sparse dynamics identification.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

The effects of 8-week complex training on lower-limb strength and power of Chinese elite female modern pentathlon athletes

Complex training (CT) is a combination training method that alternates between performing high-load resistance training (RT) and plyometric training within one single session. The study aimed to examine the effects of CT on lower-limb strength and power of elite female modern pentathlon athletes under the new modern pentathlon format and competition rules. Ten female participants (age: 23.55 ± 2.22 years, weight: 60.59 ± 3.87 kg, height: 169.44 ± 4.57 cm, and training experience: 6.90 ± 2.08 years) of the national modern pentathlon team completed 8 weeks of RT as followed by 8 weeks of CT, with 2 weeks of break. Then, the participants conducted 8 weeks of CT, which included RT combined with plyometric training (e.g., drop jump and continuous jump). All stages of training were designed by the linear strength training period theories, requiring participants to train twice for the first 4 weeks and three times for the second 4 weeks. The one-repetition maximum (1RM) of squat, isometric mid-thigh pull (IMTP), counter-movement jump (CMJ), squat jump (SJ), pre-stretch augmentation percentage (PSAP), and reaction strength index (RSI) were assessed before and after both RT and CT training. One-way repeated-measure ANOVA models revealed that the 1RM of squat was significantly improved ( p < 0.001) after RT as compared to pre-RT. No significant improvement in IMTP ( p = 0.055), CMJ ( p = 0.194), SJ ( p = 0.692), PSAP ( p = 0.087), and RSI ( p = 0.238) was not observed. After CT, 1RM of squat ( p < 0.001), IMTP ( p < 0.035), CMJ ( p < 0.001), SJ ( p < 0.008), RSI ( p < 0.006) were significant improved as compared to pre-RT, post-RT and pre-CT, while significant improvements in PSAP were observed as compared to pre-RT ( p = 0.003) and pre-CT ( p = 0.027), but not to post-RT ( p = 0.156). This pilot study showed the promise of CT following RT to improve lower-limb strength and power in elite female modern pentathlon athletes. The findings are worthwhile to be confirmed in future studies with larger sample size and randomized design.

Qiao, Zining↗