Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “train”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,477 records · Page 82

A domain wall-magnetic tunnel junction artificial synapse with notched geometry for accurate and efficient training of deep neural networks

Inspired by the parallelism and efficiency of the brain, several candidates for artificial synapse devices have been developed for neuromorphic computing, yet a nonlinear and asymmetric synaptic response curve precludes their use for backpropagation, the foundation of modern supervised learning. Spintronic devices—which benefit from high endurance, low power consumption, low latency, and CMOS compatibility—are a promising technology for memory, and domain-wall magnetic tunnel junction (DW-MTJ) devices have been shown to implement synaptic functions such as long-term potentiation and spike-timing dependent plasticity. In this work, we propose a notched DW-MTJ synapse as a candidate for supervised learning. Using micromagnetic simulations at room temperature, we show that notched synapses ensure the non-volatility of the synaptic weight and allow for highly linear, symmetric, and reproducible weight updates using either spin transfer torque (STT) or spin–orbit torque (SOT) mechanisms of DW propagation. We use lookup tables constructed from micromagnetics simulations to model the training of neural networks built with DW-MTJ synapses on both the MNIST and Fashion-MNIST image classification tasks. Accounting for thermal noise and realistic process variations, the DW-MTJ devices achieve classification accuracy close to ideal floating-point updates using both STT and SOT devices at room temperature and at 400 K. Our work establishes the basis for a magnetic artificial synapse that can eventually lead to hardware neural networks with fully spintronic matrix operations implementing machine learning.

42 ENGINEERING↗

Training self-learning circuits for power-efficient solutions

As the size and ubiquity of artificial intelligence and computational machine learning models grow, the energy required to train and use them is rapidly becoming economically and environmentally unsustainable. Recent laboratory prototypes of self-learning electronic circuits, such as “physical learning machines,” open the door to analog hardware that directly employs physics to learn desired functions from examples at a low energy cost. In this work, we show that this hardware platform allows for an even further reduction in energy consumption by using good initial conditions and a new learning algorithm. Using analytical calculations, simulations, and experiments, we show that a trade-off emerges when learning dynamics attempt to minimize both the error and the power consumption of the solution—greater power reductions can be achieved at the cost of decreasing solution accuracy. Finally, we demonstrate a practical procedure to weigh the relative importance of error and power minimization, improving the power efficiency given a specific tolerance to error.

Stern, Menachem (ORCID:0000000302158082)↗

Periodic training of creeping solids

Significance It is well appreciated that many disordered materials deform their shape irreversibly (plastically) under an external load (e.g., memory foam). Here, we show that this plasticity can be exploited to train materials to develop novel elastic responses by straining them periodically. By applying different periodic strains to a common viscoelastic material, we are able to design a number of different responses. These include a maximally negative Poisson’s ratio, bistable behavior, and nonlocal bond-specific responses. In contrast to computer-aided design, we rely on plasticity to self-organize the system in response to local stresses. This approach shows promise to achieve an unprecedented control over behavior at large strains well beyond the linear-response regime.

36 MATERIALS SCIENCE↗

Learning to learn by using nonequilibrium training protocols for adaptable materials

Evolution in time-varying environments naturally leads to adaptable biological systems that can easily switch functionalities. Advances in the synthesis of environmentally responsive materials therefore open up the possibility of creating a wide range of synthetic materials which can also be trained for adaptability. We consider high-dimensional inverse problems for materials where any particular functionality can be realized by numerous equivalent choices of design parameters. By periodically switching targets in a given design algorithm, we can teach a material to perform incompatible functionalities with minimal changes in design parameters. We exhibit this learning strategy for adaptability in two simulated settings: elastic networks that are designed to switch deformation modes with minimal bond changes and heteropolymers whose folding pathway selections are controlled by a minimal set of monomer affinities. The resulting designs can reveal physical principles, such as nucleation-controlled folding, that enable such adaptability.

Science & Technology - Other Topics↗

Statistical mentoring at early training and career stages

At Los Alamos National Laboratory (LANL), statistical scientists develop solutions for a variety of national security challenges through scientific excellence, typically as members of interdisciplinary teams. At LANL, mentoring is actively encouraged and practiced to develop statistical skills and positive career-building behaviors. Mentoring activities targeted at different career phases from student to junior staff are an important catalyst for both short and long term career development. This article discusses mentoring strategies for undergraduate and graduate students through internships as well as for postdoctoral research associates and junior staff. Topics addressed include project selection, progress, and outcome; intellectual and social activities that complement the student internship experience; key skills/knowledge not typically obtained in academic training; and the impact of such internships on students’ careers. Experiences and strategies from a number of successful mentorships are presented. Feedback from former mentees obtained via a questionnaire is incorporated. As a result, these responses address some of the benefits the respondents received from mentoring, helpful contributions and advice from their mentors, key skills learned, and how mentoring impacted their later careers.

97 MATHEMATICS AND COMPUTING↗

Trains of electron micro-bunches in plasma wake-field acceleration

Plasma-based charged particle accelerators have been intensively investigated in the past three decades due to their capability to open up new horizons in accelerator science and particle physics yielding electric field accelerating gradient more than three orders of magnitudes higher than in conventional devices. At the current stage the most advanced and reliable mechanism for accelerating electrons is based on the propagation of an intense laser pulse or a relativistic electron beam in a low density gaseous target. In this paper we concentrate on the electron beam-driven plasma wake-field acceleration and demonstrate using 3D PiC simulations that a train of electron micro-bunches with ~10 fs period can be generated behind the driving beam propagating in a density down-ramp. We will discuss the conditions and properties of the micro-bunches generated aiming at understanding and study of multi-bunch mechanism of injection. It is show that the periodicity and duration of micro-bunches can be controlled by adjusting the plasma density gradient and driving beam charge.

Physics↗

Training models using forces computed by stochastic electronic structure methods

Abstract Quantum Monte Carlo (QMC) can play a very important role in generating accurate data needed for constructing potential energy surfaces. We argue that QMC has advantages in terms of a smaller systematic bias and an ability to cover phase space more completely. The stochastic noise can ease the training of the machine learning model. We discuss how stochastic errors affect the generation of effective models by analyzing the errors within a linear least squares procedure, finding that there is an advantage to having many relatively imprecise data points for constructing models. We then analyze the effect of noise on a model of many-body silicon finding that noise in some situations improves the resulting model. We then study the effect of QMC noise on two machine learning models of dense hydrogen used in a recent study of its phase diagram. The noise enables us to estimate the errors in the model. We conclude with a discussion of future research problems.

Ceperley, David M. (ORCID:0000000150826271)↗

Evaluation of the Radioactive Material Released in the Harborview Research and Training Building and Some Implications for Emergency Response

On 2 May 2019, during the 137 Cs source recovery operation, a source capsule in a research irradiator containing approximately 77.1 TBq was breached. Based on a geometric reconstruction analysis of the damage to the capsule, approximately 46.3 GBq (0.04%) was impacted by the chop saw (grinder) inside a mobile hot cell on the loading dock at the University of Washington Harborview Research and Training (HRT) Building. A very small fraction of the material impacted, less than 1%, was released from the mobile hot cell and then to the rest of the HRT Building. The objectives of this project were to assess the accidental release of 137 CsCl and its implications related to emergency response methods and the ramifications of 137 CsCl transport. The phenomenology of this event was also compared with past alkali halide dispersal events. The vast number of measurements and samples collected by the remediation contractors, the Department of Energy’s Nuclear Emergency Support Team, and the small number of retrospective samples collected by the authors informed the analysis. The techniques included (1) autoradiography and electron microscopy of samples collected from the HRT Building and the irradiator, (2) 3D visualization of deposition on surfaces and within the ventilation system, and (3) a study of the damage to the source capsule to evaluate the Cs particle size and particle composition due to the grinding accident. Subsequently, the cesium contaminant transport through the numerous pathways in the building was reconstructed to assess the deposition on surfaces as a function of particle size. Furthermore, the implications for emergency response are relevant to data quality and management. A Data Quality Objective guides data collection methods so that they have appropriate accuracy and precision for the intended application. Recommendations were made with respect to the sample collection protocols and archiving of samples.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗

Optimizing temperature distributions for training neural quantum states using parallel tempering

Parametrized artificial neural networks (ANNs) can be very expressive ansatzes for variational algorithms, reaching state-of-the-art energies on many quantum many-body Hamiltonians. Nevertheless, the training of the ANN can be slow and stymied by the presence of local minima in the parameter landscape. One approach to mitigate this issue is to use parallel tempering methods, and in this work, we focus on the role played by the temperature distribution of the parallel tempering replicas. Using an adaptive method that adjusts the temperatures in order to equate the exchange probability between neighboring replicas, we show that this temperature optimization can significantly increase the success rate of the variational algorithm with negligible computational cost by eliminating bottlenecks in the replicas' random walk. Furthermore, we demonstrate this using two different neural networks, a restricted Boltzmann machine and a feedforward network, which we use to study a toy problem based on a permutation invariant Hamiltonian with a pernicious local minimum and the 𝐽 1 −𝐽 2 model on a rectangular lattice.

Neural network simulations↗

Stable Solid Molecular Hydrogen above 900 K from a Machine-Learned Potential Trained with Diffusion Quantum Monte Carlo

Here, we survey the phase diagram of high-pressure molecular hydrogen with path integral molecular dynamics using a machine-learned interatomic potential trained with quantum Monte Carlo forces and energies. Besides the HCP and C2/c–24 phases, we find two new stable phases both with molecular centers in the Fmmm–4 structure, separated by a molecular orientation transition with temperature. The high temperature isotropic Fmmm–4 phase has a reentrant melting line with a maximum at higher temperature (1450 K at 150 GPa) than previously estimated and crosses the liquid-liquid transition line around 1200 K and 200 GPa.

08 HYDROGEN↗

Dynamic pulse-to-pulse thermal load effects in pulse-train-mode self-seeded X-ray free-electron laser

Thermal load has been a haunting factor that undermines the brightness and coherence of the self-seeded X-ray free-electron laser. Additionally, different from uniformly pulsed mode, in pulse train mode a thermal quasi-steady state of the crystal monochromator may not be reached. This leads to a dynamic thermal distortion of the spectral transmission curves and seed quality degradation. In this paper, the pulse-to-pulse thermal load effects on the spectral transmission curves and seed quality are shown, and some instructive information for the tuning process is provided.

47 OTHER INSTRUMENTATION↗

Efficient I/O for Neural Network Training with Compressed Data

FanStore is a shared object store that enables efficient and scalable neural network training on supercomputers. By providing a global cache layer on node-local burst buffers using a compressed representation, it significantly enhances the processing capability of deep learning (DL) applications on existing hardware. In addition, FanStore allows POSIX-compliant file access to the compressed data in user space. We investigate the tradeoff between runtime overhead and data compression ratio using real-world datasets and applications, and propose a compressor selection algorithm to maximize storage capacity given performance constraints. We consider both asynchronous (i.e., with prefetching) and synchronous I/O strategies, and propose mechanisms for selecting compressors for both approaches. Using FanStore, the same storage hardware can host 2-13x more data for example applications without significant runtime overhead. Empirically, our experiments show that FanStore scales to 512 compute nodes with near linear performance scalability.

Zhang, Zhao↗

Adaptive Quantum Generative Training using an Unbounded Loss Function

We propose a generative quantum learning algorithm using the Adaptive Derivative-Assembled Problem Tailored ansatz (ADAPT) framework in which the loss function to be minimized is the maximal quantum Rényi divergence of order two, an unbounded function that mitigates barren plateaus which inhibit training variational circuits. We benchmark this method against other state-of-the-art adaptive algorithms by learning random two-local thermal states. We perform numerical experiments of up to 12 qubits comparing our method learning algorithms that use linear objective functions and show that Rényi-ADAPT is capable of constructing shallow quantum circuits competitive with existing methods, while the gradients remain favorable resulting from the maximal Rényi divergence loss function.

quantum algorithms, quantum machine learning, quan↗

An Efficient Checkpointing System for Large Machine Learning Model Training

As machine learning models increase in size and complexity rapidly, the cost of checkpointing in ML training became a bottleneck in storage and performance (time). For example, the latest GPT-4 model has massive parameters at the scale of 1.76 trillion. It is highly time and storage consuming to frequently writes the model to checkpoints with more than 1 trillion floating point values to storage. This work aims to understand and attempt to mitigate this problem. First, we characterize the checkpointing interface in a collection of representative large machine learning/language models with respect to storage consumption and performance overhead. Second, we propose the two optimizations: i) A periodic cleaning strategy that periodically cleans up outdated checkpoints to reduce the storage burden; ii) A data staging optimization that coordinates checkpoints between local and shared file systems for performance improvement.

machine learning, artificial intelligence↗

Assessment of Training Performance, Degradation and Robustness of Paraffin-Wax Impregnated Nb 3 Sn Demonstrator Under High Magnetic Field

In the context of high-energy physics, the use of Nb 3 Sn superconducting magnets as a cost-effective and reliable technology depends on improvements in the following areas: long development and manufacturing cycles, conductor degradation after thermal cycling, long training, as well as a demonstration in accelerator magnets with a beam aperture of the full potential of modern Nb 3 Sn conductors. In short, performance, robustness, and cost are the three issues to be addressed. The Magnet Development project (MagDev) of the Swiss Accelerator Research and Technology initiative (CHART) at the Paul Scherrer Institute (PSI) aims to contribute to the solutions to each of these issues, re-thinking the manufacturing and design process. Here in our program, every innovation is to be validated by means of a panoply of fast-turnaround tools: from non-powered and powered samples and coils, tested under background field, to low-field subscale magnets and high field short prototypes. This work presents one element in this panoply of R&D vehicles: a stress-managed Nb 3 Sn coil called BigBOX, impregnated with paraffin wax, and tested, through a collaboration with the Magnet Development Program of the United States (US-MDP), in the background field of Brookhaven National Laboratory (BNL)’s common coils dipole DCC17.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Sparse Coding for N-Gram Feature Extraction and Training for File Fragment Classification

File fragment classification is an important step in the task of file carving in digital forensics. In file carving, files must be reconstructed based on their content as a result of their fragmented storage on disk or in memory. Existing methods for classification of file fragments typically use hand-engineered features such as byte histograms or entropy measures. In this paper, we propose an approach using sparse coding that enables automated feature extraction. Sparse coding, or sparse dictionary learning, is an unsupervised learning algorithm, and is capable of extracting features based simply on how well those features can be used to reconstruct the original data. With respect to file fragments, we learn sparse dictionaries for n-grams, continuous sequences of bytes, of different sizes. These dictionaries may then be used to estimate n-gram frequencies for a given file fragment, but for significantly larger n-gram sizes than are typically found in existing methods which suffer from combinatorial explosion. To demonstrate the capability of our sparse coding approach, we used the resulting features to train standard classifiers such as support vector machines (SVMs) over multiple file types. Experimentally, we achieved significantly better classification results with respect to existing methods, especially when the features were used in supplement to existing hand-engineered features.

97 MATHEMATICS AND COMPUTING↗

SU-E-I-62: Reduction of Susceptibility Artifacts by Increasing the Bandwidth (BW) and Echo Train Length (ETL)

Purpose: The aim of this present study is to increase bandwidth (BW) and echo train length (ETL) in Proton Density Turbo Spin Echo (PD TSE) sequences with and without fat saturation (FS) as well as in Turbo Inversion Recovery Magnitude sequences (TIRM) in order to assess whether these sequences are capable of reducing susceptibility artifacts. Methods: We compared 1) TIRM coronal (COR) with the same sequence with increased both BW and ETL 2) Conventional PD TSE sagittal (SAG) with FS with an increased BW 3) Conventional PD TSE SAG without FS with an increased BW 4) Conventional PD TSE SAG without FS with increased both BW and ETL. A quantitative analysis was performed to measure the extent of the susceptibility artifacts. Furthermore, a qualitative analysis was performed by two radiologists in order to evaluate the susceptibility artifacts, image distortion and fat suppression. The depiction of cartilage, menisci, muscles, tendons and bone marrow were also qualitatively analyzed. Results: The quantitative analysis found that the modified TIRM sequence is significantly superior to the conventional one regarding the extent of the susceptibility artifacts. In the qualitative analysis, the modified TIRM sequence was superior to the corresponding conventional one in eight characteristics out of ten that were analyzed. The modified PD TSE with FS was superior to the corresponding conventional one regarding the susceptibility artifacts, image distortion and depiction of bone marrow and cartilage while achieving effective fat saturation. The modified PD TSE sequence without FS with a high (H) BW was found to be superior corresponding to the conventional one in the case of cartilage. Conclusion: Consequently, TIRM sequence with an increased BW and ETL is proposed for producing images of high quality and modified PD TSE with H BW for smaller metals, especially when FS is used.

60 APPLIED LIFE SCIENCES↗