Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

ChemEcho v1.0

ChemEcho is a tool that converts tandem mass spectra into embeddings used to build machine learning (ML) models with fully explainable predictions. It provides an API for transforming raw tandem mass spectral data into embeddings, along with functions for training and validating ML models. Additionally, it includes utilities for retrieving and cleaning training data. ChemEcho is broadly applicable in ML pipelines that use tandem mass spectra for a variety of tasks, such as chemical classification or bioactivity mining. While there are existing methods to generate embeddings from fragmentation data, ChemEcho's approach ensures that predictions remain interpretable, enabling experts to evaluate results and generate hypotheses about the underlying data.

Harwood, Thomas [Lawrence Berkeley National Labora↗
DOE OSTI2023

In Situ Infrared Spectroscopy of a Plasma Jet and Data-Driven Solution of Multi-Scale Plasma Chemistry Problems

Multi-scale problems are commonly known in many scientific and engineering fields where microscopic behaviors are coupled with macroscopic processes. This is also an unsolved problem in low-temperature plasma chemistry where hundreds of chemical species are involved in thousands of chemical reactions. To address this problem, a physics-informed data-driven modeling is developed to solve such a multi-scale problem using the experimental Fourier-transform infrared spectroscopy (FTIR) measurements of several species’ concentrations. The modeling based on modern machine learning techniques provides concentrations of other relevant species along with the electron temperature and gas temperature at the location of FTIR measurements. For example, the concentrations O, OH, and H 2 O 2 play key roles in plasma-based cancer therapy. This approach overcomes the multi-scale difficulties of microscopic low-temperature plasma chemistry coupling with macroscopic gas flow and allows the acquisition of a full picture of output species concentrations. Presented here for the helium-air jet at atmospheric pressure, the ML-based modeling can be used to describe and possibly control multiscale systems using partial experimental data sets.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Tuning martensitic transformations via coherent second phases in nanolaminates using free energy landscape engineering

In this work we explore the possibilities and limitations of using a coherent second phase to engineer the thermo-mechanical properties of a martensitic alloy by modifying the underlying free energy landscape that controls the transformation. We use molecular dynamics simulations of a model atomistic system where the properties of a coherent, nanoscale second phase can be varied systematically. With a base martensitic material that undergoes a temperature-induced transformation from a cubic austenite to a monoclinic martensite, simulations show significant ability to engineer the transformation temperatures, from a ~50% reduction to a ~200% increase, with 50 at.% of the cubic second phase. We establish correlations between the properties of the second phase, the transformation characteristics, and the microstructure via the free energy landscape of the twophase systems. Coherency stresses have a strong influence on the martensitic variants observed and can even cause the non-martensitic second phase to undergo a transformation. Reducing the stiffness of the second phase increases the transformation strain and modifies the martensitic microstructure, increasing the volume fraction of the transformed material. This increase in transformation strain is accompanied by a significant increase in Af and thermal hysteresis, while Ms remains unaltered. Our findings on the tunability of martensitic transformations can be used for informed searches of second phases to achieve desired material properties, such as achieving room temperature, lightweight shape memory alloys.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Structure of the thermal boundary layer in turbulent channel flows at transcritical conditions

Enhanced fluctuations, steep gradients, and intensified heat transfer are characteristics of wall-bounded turbulence at transcritical conditions. Although such conditions are prevalent in numerous technical applications, the structure of the thermal boundary layer under realistic density gradients and heating conditions remains poorly understood. Specifically, statistical descriptions of the temperature field in such flows are provided inconsistently using existing models. To address this issue, direct numerical simulations are performed by considering fully developed transcritical turbulent channel flow at pressure and temperature conditions that cause density changes of a factor of up to $O(20)$ between the hot and cold walls. As a consequence of the proximity of the Widom line to the hot wall, significant asymmetries are observed when comparing regions near the cold wall and near the hot wall. Previous transformations that attempt to collapse the near-wall mean temperature profiles among different cases to a single curve are examined. By addressing model deficiencies of these transformations, a formulation for an improved mean temperature transformation is proposed, with appropriate considerations for real fluid effects that involve strong variations in thermodynamic quantities. Our proposed transformation is shown to perform well in collapsing the slope of the logarithmic region to a single universal value with reduced uncertainty. Coupled with a predictive framework to estimate the non-universal shift parameter of the logarithmic region using a priori information, our transformation provides an analytic profile to model the near-wall mean temperature. These results thus provide a framework to guide the development of models for wall-bounded transcritical turbulence.

42 ENGINEERING↗
DOE OSTI2026

Hybrid Quantum–Classical Graph Transformers for Efficient Sentiment Analysis

Quantum Machine Learning (QML) offers a promising paradigm that leverages quantum computing principles to develop efficient and expressive models for learning from complex and structured data. Recent advances in natural language processing (NLP) and artificial intelligence (AI) have demonstrated capabilities in understanding, generating, and reasoning over linguistic and multimodal information. In this work, we present the Quantum Graph Transformer (QGT), a hybrid quantum–classical architecture that extends graph transformer capabilities through quantum self-attention. The QGT models variable-length sentences as token graphs, where both the embedding encoding and the self-attention mechanisms are implemented using parameterized quantum circuits (PQCs), enabling efficient contextual learning with significantly fewer trainable parameters. We train QGT using both fully connected and 𝑘 -nearest-neighbor graph structures and evaluate it on five benchmark sentiment-classification datasets. Experimental results show that QGT consistently achieves higher or comparable accuracy to existing quantum NLP models and outperforms a Classical Graph Transformer (CGT) baseline with identical architecture, achieving 29.4 × fewer parameters while requiring 3–5 × fewer samples to reach comparable performance. These findings highlight the potential of graph-based quantum models as scalable and data-efficient architectures for natural language understanding.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

3D CAD Modeling and Mechanical Integration at the Mu2e Experiment at Fermilab

This abstract encapsulates a transformative summer internship at Fermilab, focusing on the Mu2e Experiment within the Mechanical Integration and 3D CAD Modeling group. The internship was characterized by a two-phase approach, with the initial month dedicated to the design of an aluminum platform tailored for DS Trench Maintenance. This phase encompassed a comprehensive exploration of structural integrity, material selection, and precision engineering principles, demonstrating an adeptness in CAD modeling and a keen eye for detail. The subsequent month saw a seamless transition into a dynamic phase of the project, wherein the intern played a pivotal role in effecting numerous adjustments across diverse sections of the 3D experiment structure. This phase demanded a nuanced understanding of the intricacies of the Mu2e Experiment and a high level of proficiency in CAD modeling.

3D CAD Modeling↗
DOE OSTI2024

Modeling of frequency-sweeping Alfvén modes in the TJ-II stellarator

Alfvénic activity has been observed in the TJ-II stellarator which resembles the frequency sweeping demonstrated by Alfvén cascade modes in tokamaks. A numerical validation study was conducted using a reduced magnetohydrodynamic (MHD) model to show that such modes could only have been observed in discharges where the rotational transform profile was non-monotonic. During experiments, coil current was varied which resulted in shifting of the minimum value of the rotational transform profile. To mimic this effect, we study the Alfvénic activity predicted by the reduced MHD model for a set of input rotational transform profiles with varying minima. A mode is found whose toroidal and poloidal mode numbers match those predicted in experiments which sweeps downward/upward in frequency as the minimum value of the rotational transform profile is increased/decreased. The results serve as a demonstration of the validity and utility of MHD spectroscopy.

Alfvén cascades↗
DOE OSTI2025

A Preferences Corpus and Annotation Scheme for Human-Guided Alignment of Time-Series GPTs

The process of time-series forecasting such as predicting trajectories of silicon content in blast furnaces is a difficult task. Most time-series approaches today focus on scalar-type MSE loss optimization. This optimization approach, while widely common, could benefit from the use of human expert or process-level preferences. In this paper, we introduce a novel alignment and fine-tuning approach that involves learning from a corpus of preferred and dis-preferred time-series prediction trajectories. Our contributions include (1) a preference annotation pipeline for time-series forecasts, (2) the application of Score-based Preference Optimization (SPO) to train decoder-only transformers from preferences, and (3) results showing improvements in forecast quality. The approach is validated on both proprietary blast furnace data and the UCI Appliances Energy dataset. The proposed preference corpus and training strategy offer a new option for fine-tuning sequence models in industrial settings.

DPO↗
DOE OSTI2025

Sequence modeling of higher-order wave modes of quasi-circular, spinning, non-precessing binary black hole mergers

Higher-order gravitational wave modes from quasi-circular, spinning, non-precessing binary-black-hole (BBH) mergers encode rich information about the nonlinear dynamics of strong-field gravity. We present a transformer-based sequence-completion surrogate that, given an early-inspiral segment, forecasts the subsequent late inspiral, merger, and ringdown. The intended applications are (i) patching or completing expensive or interrupted numerical-relativity (NR) simulations and (ii) providing late-time cross-checks and rapid hybridization studies. The training set is built from the NRHybSur3dq8 surrogate, which provides spherical-harmonic modes up to $\ell$ ≤ 4 (excluding (4, 0) and (4,±1), and including (5, 5)) for mass ratios q ≤ 8, dimensionless spin components s$^{z}_{1,2}$ ϵ[–0.8, 0.8], and inclination angles θ ϵ [0, π]. Waveforms are supplied on the interval t ϵ [–5000M, –100 M) and the model autoregressively generates the plus and cross polarizations (h + , h x ) on t ϵ [–100 M, 130M]. Training on the Delta supercomputer with 16 NVIDIA A100 GPUs required ~15 h on more than 14 million hybrid waveforms. Evaluation on a held-out test set of 840,000 samples yields mean and median overlaps of 0.996 and 0.997, respectively, with respect to the surrogate ground truth.

black-hole merger↗
DOE OSTI2020

The impact of the fiducial cosmology assumption on BAO distance scale measurements

ABSTRACT Standard analysis pipelines for measurements of Baryon Acoustic Oscillations (BAO) in galaxy surveys make use of a fiducial cosmological model to guide the data compression required to transform from observed redshifts and angles to the measured angular and radial BAO peak positions. In order to remove any dependence on the fiducial cosmology from the results, all models compared to the data should mimic the compression and its dependence on the fiducial model. In practice, approximations are made when testing models: (1) There is assumed to be no residual dependence on the fiducial cosmology after reconstruction, (2) differences in the distance–redshift relationship are assumed to match a linear scaling, and (3) differences in clustering between true and fiducial models are assumed to be removed by the free parameters used to null the non-BAO signal. We test these approximations using the current standard measurement procedure with a set of halo catalogues from the aemulus suite of N-body simulations, which span a range of wCDM cosmological models. We focus on reconstruction of the primordial BAO and locating the BAO. For the range of wCDM cosmologies covered by the aemulus suite, we find no evidence for systematic errors in the measured BAO shift parameters α∥ and α⊥ to $\lt 0.1\%$. However, the measured errors $\sigma _{\alpha _{\parallel }}$ and $\sigma _{\alpha _{\bot }}$ show a notable absolute increase by up to +0.001 and +0.002, respectively, in the case that the fiducial cosmology does not match the truth. These effects on the inferred BAO scale will be important, given the precision of measurements expected from future surveys including DESI, Euclid, and WFIRST.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Pre-training Vision Models for the Classification of Alerts from Wide-field Time-domain Surveys

Modern wide-field time-domain surveys facilitate the study of transient, variable and moving phenomena by conducting image differencing and relaying alerts to their communities. Machine learning tools have been used on data from these surveys and their precursors for more than a decade, and convolutional neural networks (CNNs), which make predictions directly from input images, saw particularly broad adoption through the 2010s. Since then, continually rapid advances in computer vision have transformed the standard practices around using such models. It is now commonplace to use standardized architectures pre-trained on large corpora of everyday images (e.g., ImageNet). In contrast, time-domain astronomy studies still typically design custom CNN architectures and train them from scratch. Here, we explore the effects of adopting various pre-training regimens and standardized model architectures on the performance of alert classification. We find that the resulting models match or outperform a custom, specialized CNN like what is typically used for filtering alerts. Moreover, our results show that pre-training on galaxy images from Galaxy Zoo tends to yield better performance than pre-training on ImageNet or training from scratch. We observe that the design of standardized architectures are much better optimized than the custom CNN baseline, requiring significantly less time and memory for inference despite having more trainable parameters. On the eve of the Legacy Survey of Space and Time and other image-differencing surveys, these findings advocate for a paradigm shift in the creation of vision models for alerts, demonstrating that greater performance and efficiency, in time and in data, can be achieved by adopting the latest practices from the computer vision field.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Bayesian force fields from active learning for simulation of inter-dimensional transformation of stanene

Abstract We present a way to dramatically accelerate Gaussian process models for interatomic force fields based on many-body kernels by mapping both forces and uncertainties onto functions of low-dimensional features. This allows for automated active learning of models combining near-quantum accuracy, built-in uncertainty, and constant cost of evaluation that is comparable to classical analytical models, capable of simulating millions of atoms. Using this approach, we perform large-scale molecular dynamics simulations of the stability of the stanene monolayer. We discover an unusual phase transformation mechanism of 2D stanene, where ripples lead to nucleation of bilayer defects, densification into a disordered multilayer structure, followed by formation of bulk liquid at high temperature or nucleation and growth of the 3D bcc crystal at low temperature. The presented method opens possibilities for rapid development of fast accurate uncertainty-aware models for simulating long-time large-scale dynamics of complex materials.

Chemistry↗
DOE OSTI2020

On the elastocaloric effect in CuAlBe shape memory alloys: A quantitative phase-field modeling approach

We report the reversible stress-induced phase transformation in shape memory alloys (SMAs) is a dissipative process during which heat is absorbed or released. The inherent temperature variations inside the material has an elastocaloric effect (eCE) with appealing applications in solid-state cooling technology such as compact and efficient on-board refrigeration system for eletronic devices. In this manuscript, we conduct the first study of eCE of CuAlBe SMAs utilizing phase-field modeling. For an applied stress of 500 MPa, the results for polycrystalline Cu-Al11-2Be (at. %) show a minimum adiabatic unloading temperature change of -10 K over a pseudoelastic window of 40 K. In the absence of plastic deformation, the material demonstrates good reproducibility of the eCE over a few loading–unloading cycles. The presence of plastic deformation is found to cause functional fatigue that deteriorates the cooling capacity; however, the coefficient of performance only decreases from 9.04 to 8.03, which is still a very good value. These results place CuAlBe as a frontrunner SMA for solid-state cooling compared to the expensive NiTi.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Phase transformation mechanism in irradiation-induced superlattice formation

Atomic kinetic Monte Carlo simulations were used to model void superlattice formation under irradiation in molybdenum, driven by anisotropic diffusion of self-interstitial atoms. A change in the phase transformation mechanism from nucleation and growth to spinodal decomposition occurred with increasing dose rate, with both mechanisms leading to superlattice formation. Analysis of a rate-theory based analytical model showed that an observed change in the kinetics of vacancy accumulation, the appearance of a region of positive second derivative in the plot of average vacancy concentration versus time, was caused by the onset of spinodal instability. Further, the analytical model showed that for molybdenum and several other metals where void superlattice formation is commonly observed, the phase transformation likely occurs by nucleation and growth. However, nickel may offer the possibility of experimental observation of the transition between phase transformation mechanisms.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Transforming Energy through Computational Excellence. Exascale Computing: Combustion; Deep Learning for Presumed Probability Density Function (PDF) Models

NREL researchers use advanced machine learning techniques to define improved methods using deep learning models to resolve reacting flows in turbulent combustion flows, reducing the computational burden, increasing computational speed, and improving accuracy. These advancements reduce cost and improve fidelity of rapid-turn-around engineering calculations.

combustion↗
DOE OSTI2021

PyOECP: A flexible open-source software for the measurement and modeling of the dielectric spectrum from the reflection coefficient

We present PyOECP, a Python-based flexible open-source software for estimating and modeling the complex permittivity obtained from the open-ended coaxial probe (OECP) technique. The transformation of the measured reflection coefficient to complex permittivity is performed based on two different models, including the capacitance model and the antenna model. The software library contains dielectric spectra of common reference liquids, which can be used to transform the reflection coefficient into the dielectric spectra. Several Python routines that are commonly employed (e.g., SciPy and NumPy) are only required so that the users can alter the software structure depending on their needs. The modeling algorithm exploits the Markov chain Monte Carlo method for the data regression. The discrete relaxation models can be built by a proper combination of well-known relaxation models. In addition to these models, the electrode polarization, which is a common measurement artifact for interpreting the dielectric spectra, can be incorporated in the modeling algorithm. A continuous relaxation model, which solves the Fredholm integral equation of the first kind (a mathematically ill-posed problem) is also included.

Yoon, Tae-Jun↗
DOE OSTI2025

Dynamic Model of a supercritical CO2 10MW Recompression Closed Brayton Cycle

This model of the 10MW recompression closed Brayton cycle (RCBC) was developed in conjunction with the DOE’s Supercritical Transformational Electric Power (STEP) project. A high-fidelity dynamic model was used extensively to study the dynamic characteristics of the cycle and develop the process control architecture and strategies for start-up and shutdown procedures. This version of the model has been simplified from the original version to be more accessible for a variety of applications and research. The controllers developed for the original model have been maintained for this version and tuned to give a similar performance to the original model. This model can be used to perform similar studies as those performed in Liese et al (2020). The overall cycle performance will not be identical but will perform similarly.

Controls,Power Cycles,Process Systems Engineering,↗
DOE OSTI2020

Predictive Skill of Deep Learning Models Trained on Limited Sequence Data

In this report we investigate the utility of one-dimensional convolutional neural network (CNN) models in epidemiological forecasting. Deep learning models, especially variants of recurrent neural networks (RNNs) have been studied for influenza forecasting, and have achieved higher forecasting skill compared to conventional models such as ARIMA models. In this study, we adapt two neural networks that employ one-dimensional temporal convolutional layers as a primary building block temporal convolutional networks and simple neural attentive meta-learner for epidemiological forecasting and test them with influenza data from the US collected over 2010-2019. We find that epidemiological forecasting with CNNs is feasible, and their forecasting skill is comparable to, and at times, superior to, RNNs. Thus CNNs and RNNs bring the power of nonlinear transformations to purely data-driven epidemiological models, a capability that heretofore has been limited to more elaborate mechanistic/compartmental disease models.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗