Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Simple Autoignition Model for the Derived Cetane Number of Oxygenated Compounds and Fuel Blends

A four-step autoignition model was used to derive an expression for the ignition delay and derived cetane number (DCN) measured by ignition quality testers (IQT) for oxygenated compounds and blends as a function of each compound’s global initiation and net chain branching rate constants. Expressions for these values were determined assuming they could be related to the functional groups present in a compound. Measurements for 124 compounds and 94 binary and ternary blends, gathered from the literature and performed at UW-Madison, were used to obtain the dependence of the measured ignition delay on each functional group. The new model was able to describe the ignition behavior of both the compounds and blends, with an average DCN error of 5.0 (22%) and 3.5 (15%), respectively. Additionally, the blend model was transformed into a mixing rule by incorporating existing IQT ignition delay data for each compound. Use of the mixing rule was found to offer improvements over the full prediction, with an average DCN error of 3.2 (14%).

Witkowski, Dustin↗
DOE OSTI2022

Learning to simulate high energy particle collisions from unlabeled data

In many scientific fields which rely on statistical inference, simulations are often used to map from theoretical models to experimental data, allowing scientists to test model predictions against experimental results. Experimental data is often reconstructed from indirect measurements causing the aggregate transformation from theoretical models to experimental data to be poorly-described analytically. Instead, numerical simulations are used at great computational cost. We introduce Optimal-Transport-based Unfolding and Simulation (OTUS), a fast simulator based on unsupervised machine-learning that is capable of predicting experimental data from theoretical models. Without the aid of current simulation information, OTUS trains a probabilistic autoencoder to transform directly between theoretical models and experimental data. Identifying the probabilistic autoencoder’s latent space with the space of theoretical models causes the decoder network to become a fast, predictive simulator with the potential to replace current, computationally-costly simulators. Here, we provide proof-of-principle results on two particle physics examples, Z-boson and top-quark decays, but stress that OTUS can be widely applied to other fields.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Dynamic mass accountancy modeling of a molten salt reactor using equilibrium thermodynamics

A mechanistic-based mass accountancy model in the context of liquid-fueled molten salt reactors was implemented in the dynamic systems modeling software library TRANSFORM by way of coupling with the equilibrium thermodynamics code Thermochimica. Liquid-fueled molten salt reactors present new challenges for mass accountancy because of the dissolution of fuel and evolved fission products, which may be soluble in the salt, off-gas, or precipitate. Two cases of mass loss from the molten salt were addressed: off-gassing and precipitation. The software implementation was tested through a series of increasingly complex demonstration problems, culminating in a model of the primary fuel and primary coolant loops of the molten salt demonstration reactor. Analysis shows that negligible mass was lost from the salt under normal operating conditions, but an overheating event caused by partial loss of fuel loop cooling resulted in release of measurable amounts of uranium (among other elements) via off-gassing. The tools developed here are primarily aimed at capability development but are readily available for use in further modeling of molten salt reactor concepts. As a result, these tools have not yet been validated, and future experimental work to perform this validation is recommended.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2024

Improving streamflow predictions across CONUS by integrating advanced machine learning models and diverse data

Accurate streamflow prediction is crucial to understand climate impacts on water resources and develop effective adaption strategies. A global long short-term memory (LSTM) model, using data from multiple basins, can enhance streamflow prediction, yet acquiring detailed basin attributes remains a challenge. To overcome this, we introduce the Geo-vision transformer (ViT)-LSTM model, a novel approach that enriches LSTM predictions by integrating basin attributes derived from remote sensing with a ViT architecture. Applied to 531 basins across the Contiguous United States, our method demonstrated superior prediction accuracy in both temporal and spatiotemporal extrapolation scenarios. Geo-ViT-LSTM marks a significant advancement in land surface modeling, providing a more comprehensive and effective tool for better understanding the environment responses to climate change.

Tayal, Kshitij↗
DOE OSTI2025

Deformable phrase level attention: A flexible approach for improving AI based medical coding

Objective: Improving the AI-driven automated medical encoding of clinical text plays a vital role in gathering information on the occurrence of diseases to improve population-level health. This work presents a novel attention mechanism designed to enhance text classification models and ensure appropriate classification of medical concepts in unstructured electronic health records. Materials and Methods: We developed a deformable, phrase-level attention mechanism to identify important lexical word-level and contextual phrase-level information from clinical text documents. We evaluated conventional and transformer-based deep learning models that we extended with our attention mechanism on the extraction of critical cancer information (e.g., site, subsite, laterality, histology, behavior) from 629,908 electronic pathology reports and on the automated medical encoding of 52,722 hospital discharge summaries. Results: Transformer-based models with the deformable, phrase-level attention mechanism achieved the best performance on the extraction of critical cancer information from pathology reports. Conventional- and transformer-based models show similar or better performance than their baseline counterparts on the automated medical encoding of clinical documents. Discussion: The addition of phrase-level information allowed models extended with our proposed method to outperform standard word-level attention. Our method showed favorable properties for the real-world application in terms of model robustness and phenotyping. These results indicate that our method is promising for automated data harmonization for common data models. Conclusion: This work proposes a novel deformable, phrase-level attention mechanism that enhances text classification models in the extraction of medical concepts from clinical text documents. We demonstrate strong performances on two clinical text datasets and showcase real-world deployability of our method.

Automated medical encoding↗
DOE OSTI2021

Cross-national analysis of food security drivers: comparing results based on the Food Insecurity Experience Scale and Global Food Security Index

Abstract The second UN Sustainable Development Goal establishes food security as a priority for governments, multilateral organizations, and NGOs. These institutions track national-level food security performance with an array of metrics and weigh intervention options considering the leverage of many possible drivers. We studied the relationships between several candidate drivers and two response variables based on prominent measures of national food security: the 2019 Global Food Security Index (GFSI) and the Food Insecurity Experience Scale’s (FIES) estimate of the percentage of a nation’s population experiencing food security or mild food insecurity (FI ). We compared the contributions of explanatory variables in regressions predicting both response variables, and we further tested the stability of our results to changes in explanatory variable selection and in the countries included in regression model training and testing. At the cross-national level, the quantity and quality of a nation’s agricultural land were not predictive of either food security metric. We found mixed evidence that per-capita cereal production, per-hectare cereal yield, an aggregate governance metric, logistics performance, and extent of paid employment work were predictive of national food security. Household spending as measured by per-capita final consumption expenditure (HFCE) was consistently the strongest driver among those studied, alone explaining a median of 92% and 70% of variation (based on out-of-sample R 2 ) in GFSI and FI , respectively. The relative strength of HFCE as a predictor was observed for both response variables and was independent of the countries used for model training, the transformations applied to the explanatory variables prior to model training, and the variable selection technique used to specify multivariate regressions. The results of this cross-national analysis reinforce previous research supportive of a causal mechanism where, in the absence of exceptional local factors, an increase in income drives increase in food security. However, the strength of this effect varies depending on the countries included in regression model fitting. We demonstrate that using multiple response metrics, repeated random sampling of input data, and iterative variable selection facilitates a convergence of evidence approach to analyzing food security drivers.

42 ENGINEERING↗
DOE OSTI2024

Understanding Line Losses and Transformer Losses in Rural Isolated Distribution Systems

Rural, isolated power systems in the mainland U.S. and in states like Alaska and Hawaii are powered by assets like diesel generators. These rural, isolated power systems also cannot operate at the higher band of medium voltage (like 69kV). They are primarily in the 12 to 14 kV range to keep the cost of the distribution investments lower. Because of this mid-band medium voltage range, the line losses and distribution transformers losses consume significant diesel consumption (almost 10 percent of the peak load). This work considers one such power system powering an isolated system and presents key findings online losses, and transformer losses. Understanding and documenting the impacts is critical for these communities operating their power systems and take actions to reduce expensive diesel consumption. In this paper, we will present one such typical grid and model it in electromagnetic transients (EMT) domain. We used the tower structure, under ground cabling installation to develop high fidelity models of lines. We also used high fidelity models of distribution transformers to present the no-load losses and full load loses. We will also present technical solutions available commercially off-the-shelf to reduce these losses and reduce diesel consumption. This work will be a primer for communities to understand the technical challenges and to understand the possible solution available to solve such challenges for rural, isolated power system operators.

blackstart↗
DOE OSTI2025

Generalizable machine learning potentials for quantum-accurate predictions of non-equilibrium behavior in 2D materials

Machine learning interatomic potentials (ML-IAPs) are emerging as transformative tools in materials modeling, promising quantum-level accuracy at a fraction of the computational cost. However, their ability to generalize beyond equilibrium configurations and to reliably capture defect- and temperature-driven behavior remains underexplored. Here, we develop and benchmark two state-of-the-art ML-IAPs, Spectral Neighbor Analysis Potential (SNAP) and Allegro, on a comprehensive dataset for monolayer MoSe₂. Using density functional theory (DFT) as the reference, we evaluate their performance in capturing stress–strain behavior, phase transition energetics, defect evolution, edge stability, and fracture toughness. Allegro, a deep equivariant neural network potential, surpasses both SNAP and the classical Tersoff potential in accuracy, efficiency, and transferability. Importantly, both ML potentials accurately reproduce experimental fracture measurements and ab initio predictions of inversion domain formation—phenomena well beyond their training sets. Our findings establish ML-IAPs as viable replacements for traditional force fields in the study of non-equilibrium mechanical phenomena, enabling large-scale, high-fidelity simulations in 2D materials and beyond. In conclusion, this work provides a broadly applicable framework for data-driven modeling of structural and functional transformations under extreme conditions.

2D materials↗
DOE OSTI2022

Overcoming the disconnect between energy system and climate modeling

Energy system models underpin decisions by energy system planners and operators. Energy system modeling faces a transformation: accounting for changing meteorological conditions imposed by climate change. To enable that transformation, a community of practice in energy-climate modeling has started to form that aims to better integrate energy system models with weather and climate models. In this work, we evaluate the disconnects between the energy system and climate modeling communities, then lay out a research agenda to bridge those disconnects. In the near-term, we propose interdisciplinary activities for expediting uptake of future climate data in energy system modeling. In the long-term, we propose a transdisciplinary approach to enable development of (1) energy-system-tailored climate datasets for historical and future meteorological conditions and (2) energy system models that can effectively leverage those datasets. This agenda increases the odds of meeting ambitious climate mitigation goals by systematically capturing and mitigating climate risk in energy sector decision-making.

17 WIND ENERGY↗
DOE OSTI2022

Image transformers for classifying acute lymphoblastic leukemia

Cancer is the leading cause of death by disease in American children. Each year, nearly 16,000 children in the United States and over 300,000 children globally are diagnosed with cancer. Leukemia is a form of blood cancer that originates in the bone marrow and accounts for one-third of pediatric cancers. This disease occurs when the bone marrow contains 20% or more immature white blood cell blasts. Acute lymphoblastic leukemia is the most prevalent leukemia type found in children, with half of all annual cases in the U.S. diagnosed for subjects under 20 years of age. To diagnose acute lymphoblastic leukemia, pathologists often conduct a morphological bone marrow assessment. This assessment determines whether the immature white blood cell blasts in bone marrow display the correct morphological characteristics, such as size and appearance of nuclei. Pathologists also use immunophenotyping via multi-channel flow cytometry to test whether certain antigens are present on the surface of blast cells; the antigens are used to identify the cell lineage of acute lymphoblastic leukemia. These manual processes require well-trained personnel and medical professionals, thus being costly in time and expenses. Computerized decision support via machine learning can accelerate the diagnosis process and reduce the cost. Training a reliable classification model to distinguish between mature and immature white blood cells is essential to the decision support system. Here, we adopted the Vision Transformer model to classify white blood cells. The Vision Transformer achieved superb classification performance compared to state-of-the-art convolutional neural networks while requiring less computational resources for training. Additionally, the latent self-attention architecture provided attention maps for a given image, providing clues as to which portion(s) of the image were significant in decision-making. We applied the Vision Transformer model and a convolutional neural network model to an acute lymphoblastic leukemia classification dataset of 12,528 samples and achieved accuracies of 88.4% and 86.2%.

Cho, Priscilla↗
DOE OSTI2023

A mesoscale crystal plasticity model to predict room-temperature deformation and martensitic transformation of high-strength Quenching and Partitioning (Q&P) Steels and validation with synchrotron X-ray diffraction

Renowned for the superior mechanical properties and adeptness at cold-forming, Quenching and Partitioning (QP) steels have gained prominence as a promising candidate material in fabricating safety-critical components in various industries. The pertinent research on QP steels focus on the martensitic transformation of the Retained Austenite (RA) phase during cold-forming, a crucial mechanism that substantially influences the overall strength and ductility of QP steels. The austenite stability and transformation rate heavily rely on the local strain path and the initial microstructure, which is challenging for analytical prediction. In this paper, a mesoscale model is developed to capture the deformation and transformation kinetics of QP steels inside the microstructure. The model integrates the detailed explicit microstructure, acquired from characterization experiments, into a high-resolution finite element (FE) mesh. It distinctly model the deformation and interaction between the various phases and the effect on the transformation of RA. The model is validated with high energy X-ray diffraction (HEXRD) data, and shows excellent capability in predicting the asymmetric stress-strain behavior under uniaxial tension and compression, as well as the martensitic transformation rate. The model is used to investigate the strain and load partitioning effect of surrounding matrix to the transformation of RA, offering insights into the complex behavior of QP980 and facilitates further material development.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Non-intrusive data-driven model reduction for differential–algebraic equations derived from lifting transformations

In this paper we present a non-intrusive data-driven approach for model reduction of nonlinear systems. The approach considers the particular case of nonlinear partial differential equations (PDEs) that form systems of partial differential–algebraic equations (PDAEs) when lifted to polynomial form. Such systems arise, for example, when the governing equations include Arrhenius reaction terms (e.g., in reacting flow models) and thermodynamic terms (e.g., the Helmholtz free energy terms in a phase-field solidification model). Using the known structured form of the lifted algebraic equations, the approach computes the reduced operators for the algebraic equations explicitly, using straightforward linear algebra operations on the basis matrices. The reduced operators for the differential equations are inferred from lifted snapshot data using operator inference, which solves a linear least squares regression problem. The approach is illustrated for the nonlinear model of solidification of a pure material. The lifting transformations reformulate the solidification PDEs as a system of PDAEs that have cubic structure. The operators of the lifted system for this solidification example have affine dependence on key process parameters, permitting us to learn a parametric reduced model with operator inference. Numerical experiments show the effectiveness of the resulting reduced models in capturing key aspects of the solidification dynamics.

42 ENGINEERING↗
DOE OSTI2023

Near-wall model for compressible turbulent boundary layers based on an inverse velocity transformation

In this work, a near-wall model, which couples the inverse of a recently developed compressible velocity transformation (Griffin et al., Proc. Natl Acad. Sci., vol. 118, 2021, p. 34) and an algebraic temperature–velocity relation, is developed for high-speed turbulent boundary layers. As input, the model requires the mean flow state at one wall-normal height in the inner layer of the boundary layer and at the boundary-layer edge. As output, the model can predict mean temperature and velocity profiles across the entire inner layer, as well as the wall shear stress and heat flux. The model is tested in an a priori sense using a wide database of direct numerical simulation high-Mach-number turbulent channel flows, pipe flows and boundary layers (48 cases, with edge Mach numbers in the range 0.77–11, and semi-local friction Reynolds numbers in the range 170–5700). The present model is significantly more accurate than the classical ordinary differential equation (ODE) model for all cases tested. The model is deployed as a wall model for large-eddy simulations in channel flows with bulk Mach numbers in the range 0.7–4 and friction Reynolds numbers in the range 320–1800. When compared to the classical framework, in the a posteriori sense, the present method greatly improves the predicted heat flux, wall stress, and temperature and velocity profiles, especially in cases with strong heat transfer. In addition, the present model solves one ODE instead of two, and has a computational cost and implementation complexity similar to that of the commonly used ODE model.

42 ENGINEERING↗
DOE OSTI2021

A Novel Data-based Stochastic Distribution Control for Non-Gaussian Stochastic Systems

In this note, we present a novel data-based approach to investigate the non-Gaussian stochastic distribution control problem. As the motivation of this note, the existing methods have been summarised regarding to the drawbacks, for example, neural network weights training for unknown stochastic distribution and so on. To overcome these disadvantages, a new transformation for dynamic probability density function is given by kernel density estimation using interpolation. Based upon this transformation, a representative model has been developed while the stochastic distribution control problem has been transformed into an optimisation problem. Then, data-based direct optimisation and identification-based indirect optimisation have been proposed. In addition, the convergences of the presented algorithms are analysed and the effectiveness of these algorithms has been evaluated by numerical examples. In summary, the contributions of this note are as follows: 1) a new data-based probability density function transformation is given; 2) the optimisation algorithms are given based on the presented model; and 3) a new research framework is demonstrated as the potential extensions to the existing stochastic distribution control.

42 ENGINEERING↗
DOE OSTI2024

De novo atomic protein structure modeling for cryoEM density maps using 3D transformer and HMM

Accurately building 3D atomic structures from cryo-EM density maps is a crucial step in cryo-EM-based protein structure determination. Converting density maps into 3D atomic structures for proteins lacking accurate homologous or predicted structures as templates remains a significant challenge. Here, we introduce Cryo2Struct, a fully automated de novo cryo-EM structure modeling method. Cryo2Struct utilizes a 3D transformer to identify atoms and amino acid types in cryo-EM density maps, followed by an innovative Hidden Markov Model (HMM) to connect predicted atoms and build protein backbone structures. Cryo2Struct produces substantially more accurate and complete protein structural models than the widely used ab initio method Phenix. Additionally, its performance in building atomic structural models is robust against changes in the resolution of density maps and the size of protein structures.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Beyond interpolation: Physics-inspired gating transformers for extrapolating irradiation conditions to novel nuclear fuels

The qualification of advanced nuclear fuels relies on irradiation experiments in test reactors that emulate commercial conditions. Designing these tests requires accurate prediction of key irradiation quantities, particularly heat generation rate and burnup, yet obtaining them typically involves computationally expensive multi-step simulation workflows. We propose a physics-inspired gating transformer (PIGT) that integrates an inverse-square, distance-based attenuation into the encoder representation to bias attention toward physically relevant spatial relationships while retaining data-driven flexibility. Using MiniFuel irradiation data from the High Flux Isotope Reactor at Oak Ridge National Laboratory, we benchmark against ensemble methods, feedforward and recurrent networks, convolutional models, and standard transformers. While baseline models perform well under interpolation, they exhibit a pronounced generalization gap when evaluated on fuels not included in the training set. The proposed model consistently improves extrapolative accuracy and stability, yielding the strongest performance on unseen fuel configurations. These results indicate that a lightweight physics structure embedded within attention mechanisms can substantially improve robustness, enabling more reliable surrogate predictions to accelerate the design of nuclear fuel irradiation experiments.

Fuel qualification↗
DOE OSTI2025

Direct observations of transient weakening during phase transformations in quartz and olivine

Phase transformations are widely invoked as a source of rheological weakening during subduction, continental collision, mantle convection and various other geodynamic phenomena. However, despite more than half a century of research, the likelihood and magnitude of such weakening in nature remain poorly constrained. Here we use experiments performed on a synchrotron beamline to reveal transient weakening of up to three orders of magnitude during the polymorphic quartz to coesite (SiO 2 ) and olivine to ringwoodite (Fe 2 SiO 4 ) phase transitions. Weakening becomes increasingly prominent as the transformation outpaces deformation. We suggest that this behaviour is broadly applicable among silicate minerals undergoing first-order phase transitions and examine the likelihood of weakening due to the olivine-spinel, (Mg,Fe) 2 SiO 4 , transformation during subduction. Modelling suggests that cold, wet slabs are most susceptible to transformational weakening, consistent with geophysical observations of slab stagnation in the mantle transition zone beneath the western Pacific. In conclusion, our study highlights the importance of incorporating transformational weakening into geodynamic simulations and provides a quantitative basis for doing so.

Cross, Andrew J. [Woods Hole Oceanographic Institu↗
DOE OSTI2020

Steady-State Analysis of Microgrid Distributed Control Under Denial of Service Attacks

Distributed control is an effective method to coordinate multiple components in a microgrid. It usually has mandatory requirements for communication graph topology; changing the topology unexpectedly could disrupt the operation of a distributed controller. A denial of service (DoS) attack blocks communication channels to cause variations in communication graph. Unforeseen topology changes render a controller to receive incomplete system information. Control decisions made based on incorrect information can lead to significantly reduced grid supporting capability. As a result, the vital power balance between generation and load may be challenged. Determining power balance requires steady-state power flow analysis; however, the cyber-induced steady-state analysis is not well studied in the literature. Here, in this article, we show that a cyber–physical microgrid with unknown DoS attacks has augmented uncertain power flow equations, whose special structure creates new computational issue to determining power balance. To deal with the issue, we develop a novel cyber-induced microgrid model: the cyber–physical system is transformed into a pure physical model, where the disruption effects of DoS in the cyber layer are shown to be equivalent to the variations of the physical model parameters. With the transformed system, we develop a sufficient condition to ascertain the system power balance under unknown DoS attacks. In addition, practical insights for resilient distributed control design are shown as well.

24 POWER TRANSMISSION AND DISTRIBUTION↗