Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Systematic improvement of redox potential calculation of Fe(III)/Fe(II) complexes using a three-layer micro-solvation model

Electrochemical transformations of metal ions in aqueous media are challenging to model accurately due to the dynamic solvation structure surrounding ions at different charge states. Predictive modeling at the atomistic scale is essential for understanding these solvation architectures but is often computationally prohibitive. In this contribution, we present a simple, fast, and accurate three-layer micro-solvation model to evaluate the redox potential of metal ions in aqueous solutions. Our model, developed and validated for Fe 3+ /Fe 2+ redox potentials, combines the DFT-based geometry optimizations of the octahedral Fe complex with two layers of explicit water molecules to capture solute–solvent interactions and an implicit solvation model to account for bulk solvent effects. This approach yields accurate predictions for Fe 3+ /Fe 2+ redox potentials in water, achieving errors of 0.02 V with ωB97X-V, 0.01 V with ωB97X-D3, 0.04 V with ωB97M-V, and 0.02 V with B3LYP-D3 functionals. We further demonstrate the generality of our model by applying it to additional metal complexes, including the challenging Fe(CN) 6 3−/4− system, where our model successfully achieves close agreement with experimental values, with an error of 0.07 V and an average error of 0.21 V for all five systems. In summary, the presented simple solvation model has broad applicability and potential for enhancing computational efficiency in redox potential predictions across various chemical and industrial processes of metal ions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

A Digital Twin Framework Utilizing Machine Learning for Robust Predictive Maintenance: Enhancing Tire Health Monitoring

We introduce a novel digital twin (DT) framework for the predictive maintenance of long-term physical systems. Using monitoring tire health as an application, we show how the DT framework can be used to enhance automotive safety and efficiency, and how the technical challenges can be overcome using a three-step approach. First, to manage the data complexity over a long operation span, we employ data reduction techniques to concisely represent physical tires using historical performance and usage data. Relying on these data, for fast real-time prediction, we train a transformer-based model offline on our concise dataset to predict future tire health over time, represented as remaining casing potential (RCP). Based on our architecture, our model quantifies both epistemic and aleatoric uncertainties, providing reliable confidence intervals around predicted RCP. Second, to incorporate real-time data, we update the predictive model in the DT framework, ensuring its accuracy throughout its lifespan with the aid of hybrid modeling and the use of the discrepancy function. Third, to assist decision-making in predictive maintenance, we implement a tire state decision algorithm, which strategically determines the optimal timing for tire replacement based on RCP forecasted by our transformer model. This approach ensures that our DT accurately predicts system health, continually refines its digital representation, and supports predictive maintenance decisions. Furthermore, our framework effectively embodies a physical system, leveraging big data and machine learning (ML) for predictive maintenance, model updates, and decision-making.

advanced computing infrastructure↗
DOE OSTI2026

Computing the Critical Temperature of the Affine-Transformed $D=3$ Ising Model Using Masked Autoregressive Flow

The simple Ising model provides a rich environment to build and study lattice field theories. As part of an ongoing project to construct a conformal field theory (CFT) on an arbitrarily curved manifold, in this work we develop methods to measure the critical temperature $β_c$ of the affine-transformed Ising model on the face-centered cubic (FCC) lattice. The main challenge in this endeavor is finding a computationally efficient and accurate method of interpolating and extrapolating Monte Carlo observables with respect to coupling coefficients and temperature. Herein, we compare two such methods. A traditional statistical approach uses the multiple histogram (MH) method, while a newer machine learning approach uses a masked autoregressive flow (MAF) to estimate the underlying probability density function of a set of observables. While the MH method is specifically designed to interpolate and extrapolate Monte Carlo observables, we find that MAF is a viable alternative for measuring $β_c$ with a computational cost that scales more favorably. Furthermore, we comment on additional advantages of MAF relevant to our work, such as extrapolating in system volume.

Svenson, Kai [Texas U.]↗
DOE OSTI2025

Water Mass Transformation Budgets in Finite‐Volume Generalized Vertical Coordinate Ocean Models

Water Mass Transformation (WMT) theory provides conceptual tools that in principle enable innovative analyses of numerical ocean models; in practice, however, these methods can be challenging to implement and interpret, and therefore remain under-utilized. Our aim is to demonstrate the feasibility of diagnosing all terms in the water mass budget and to exemplify their usefulness for scientific inquiry and model development by quantitatively relating water mass changes, overturning circulations, boundary fluxes, and interior mixing. We begin with a pedagogical derivation of key results of classical WMT theory. We then describe best practices for diagnosing each of the water mass budget terms from the output of Finite-Volume Generalized Vertical Coordinate (FV-GVC) ocean models, including the identification of a non-negligible remainder term as the spurious numerical mixing due to advection scheme discretization errors. We illustrate key aspects of the methodology through the analysis of a polygonal region of the Greater Baltic Sea in a regional demonstration simulation using the Modular Ocean Model v6 (MOM6). We verify the convergence of our WMT diagnostics by brute-force, comparing time-averaged (“offline”) diagnostics on various vertical grids to timestep-averaged (“online”) diagnostics on the native model grid. Finally, we briefly describe a stack of xarray-enabled Python packages for evaluating WMT budgets in FV-GVC models (culminating in the new xwmb package), which is intended to be model-agnostic and available for community use and development.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Towards Trustworthy and Interpretable Deep Learning-assisted Ecohydrological Models

The transformational science question we plan to address is: How do we leverage in-situ observations and simulations from process-based ecohydrological model to construct interpretable and trustworthy deep learning (DL) models for improved reliability of prediction of quantities of interest (QoIs) under hydro-climatic extremes?

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Transformer Masked Autoencoders for RF Device Fingerprinting

Machine learning methods for RF device fingerprinting typically rely on CNN-based models. Transformer-based models have outperformed CNNs for modulation classification tasks, but there are few implementations for device fingerprinting. We train a transformer for device fingerprinting with the largest device count to date and explore several variations of the architecture. Additionally, we demonstrate that pre-training an RF transformer as a Masked Autoencoder improves classification accuracy, as has been observed for CNN fingerprinting models and vision transformers.

artificial intelligence↗
DOE OSTI2023

Transferring a Molecular Foundation Model for Polymer Property Predictions

Transformer-based large language models have remarkable potential to accelerate design optimization for applications such as drug development and material discovery. Self-supervised pretraining of transformer models requires large-scale data sets, which are often sparsely populated in topical areas such as polymer science. Further, state-of-the-art approaches for polymers conduct data augmentation to generate additional samples but unavoidably incur extra computational costs. In contrast, large-scale open-source data sets are available for small molecules and provide a potential solution to data scarcity through transfer learning. In this work, we show that using transformers pretrained on small molecules and fine-tuned on polymer properties achieves comparable accuracy to those trained on augmented polymer data sets for a series of benchmark prediction tasks.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

GenAI4UQ: A software for forward and inverse uncertainty quantification using conditional generative AI

We introduce GenAI4UQ, a software package for forward and inverse uncertainty quantification in model calibration, parameter estimation, and ensemble forecasting. GenAI4UQ leverages a generative AI-based conditional modeling framework to address limitations of traditional inverse modeling techniques, such as Markov Chain Monte Carlo (MCMC) methods. By replacing computationally intensive iterative processes with a direct, learned mapping, GenAI4UQ enables efficient calibration of input parameters and generation of predictions directly from observations. The software supports rapid ensemble forecasting with robust uncertainty quantification while maintaining computational and storage efficiency. Built-in auto-tuning of hyperparameters simplifies model training, ensuring accessibility for users with varying expertise. Its versatile conditional generative framework is applicable across diverse scientific domains. While GenAI4UQ offers significant advantages in flexibility and efficiency, users should interpret its uncertainty estimates with caution in data-sparse scenarios, as the model may overestimate uncertainty—an effect common to all surrogate-based approaches including MCMC with surrogate models. Despite this, GenAI4UQ transforms inverse modeling by providing a fast, reliable, and user-friendly solution. It empowers researchers and practitioners to quickly estimate parameter distributions and generate model predictions for new observations, facilitating efficient decision-making and advancing the state of uncertainty quantification in computational modeling.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Assessment of Current MACCS Capabilities for Modeling Atmospheric Physical and Chemical Transformations

The physical and chemical transformation during atmospheric transport of radionuclides released into the environment has the possibility of impacting consequence modeling results. Accordingly, this report identifies physical and chemical transformations that may occur following release of chemically reactive radioactive species, how those transformations may affect modeling of consequences of release to the atmosphere and identifies current capabilities – in both MACCS and other state-of-practice atmospheric transport and dispersions models– to model those transformations. It was found that the inclusion of physical and chemical transformations is currently very limited in current state-of-practice codes for atmospheric dispersion of radionuclides. State-of-practice atmospheric dispersion codes appear to be typically limited to simulating either physical-chemical transformations or radioactive transformations, but not both. A state-of-practice atmospheric dispersion code capable of performing parallel physical, chemical, and radioactive transformation was not identified. A few atmospheric dispersion codes capable of modeling physical and chemical transport of specific species such as tritium or uranium hexafluoride were identified. Consequently, there is currently no information available that clearly suggests updates to the MACCS code are needed to bring it up to state-of-practice. However, investigations concluded that the MACCS computational framework can currently accommodate multiple physical/chemical forms in one simulation. Additionally, with some major assumptions, the computational framework in MACCS can accommodate parallel physical-chemical and radioactive transformations.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗
DOE OSTI2020

Inference offers a metric to constrain dynamical models of neutrino flavor transformation

The multimessenger astrophysics of compact objects presents a vast range of environments where neutrino flavor transformation may occur and may be important for nucleosynthesis, dynamics, and a detected neutrino signal. Development of efficient techniques for surveying flavor evolution solution spaces in these diverse environments, which augment and complement existing sophisticated computational tools, could leverage progress in this field. To this end we continue our exploration of statistical data assimilation (SDA) to identify solutions to a small-scale model of neutrino flavor transformation. SDA is a machine learning formula wherein a dynamical model is assumed to generate any measured quantities. Specifically, we use an optimization formulation of SDA wherein a cost function is extremized via the variational method. Regions of state space in which the extremization identifies the global minimum of the cost function will correspond to parameter regimes in which a model solution can exist. Our example study seeks to infer the flavor transformation histories of two monoenergetic neutrino beams coherently interacting with each other and with a matter background. We require that the solution be consistent with measured neutrino flavor fluxes at the point of detection, and with constraints placed upon the flavor content at various locations along their trajectories, such as the point of emission, and the locations of the Mikheyev-Smirnov-Wolfenstein resonances. We show how the procedure efficiently identifies solution regimes and rules out regimes where solutions are infeasible. Overall, results in this work intimate the promise of this “variational annealing” methodology to efficiently probe an array of fundamental questions that traditional numerical simulation codes render difficult to access.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Simplified Model and Approach to Transform Infrared Surface Temperature to Film Effectiveness in a Conjugate Heat Transfer Experiment

This paper describes a simplified engineering model based on a one-dimensional thermal resistance network. The model is used to develop a new method to relate film cooling effectiveness and heat transfer augmentation to local overall cooling effectiveness in a conjugate flat plate experiment. This paper presents experimental proof-of-concept data to demonstrate the potential for this model. In contrast to previous approaches, neither the wall heat flux nor the adiabatic wall temperature is required to estimate the local film cooling performance parameters. The model predicts surface temperatures that are within the experimental uncertainties over the range for which the model is trained, and to within five percent when the model is extrapolated to higher coolant channel Reynolds numbers. This paper is relevant to conjugate test rigs that can measure the hot surface temperature distribution with and without film cooling. This information may also be relevant to designers as a method to approximate surface temperatures or used as an approximate heat transfer model for optimization studies.

advanced gas turbines↗
DOE OSTI2025

Computational Analysis and Optimized Modeling of Geomagnetically Induced Currents in Power Transformers

In this project we aim to better understand the effect of geomagnetically induced currents (GIC) on power transformers. Expanding upon our previous work focused on producing a methodology for accuracy-enhanced computation of GIC signatures (i.e., time-domain current magnitude variation for the event duration) from a combination of physics-based and data-driven computational tools, we propose the use of these GIC signatures as inputs for a physically-detailed and optimized model of the power transformer to investigate how GIC determination and transformer modeling influence the evaluation of GIC effects on the transformer operation, as well as in its interaction with the power grid.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Hybrid Quantum Vision Transformers for Event Classification in High Energy Physics

Models based on vision transformer architectures are considered state-of-the-art when it comes to image classification tasks. However, they require extensive computational resources both for training and deployment. The problem is exacerbated as the amount and complexity of the data increases. Quantum-based vision transformer models could potentially alleviate this issue by reducing the training and operating time while maintaining the same predictive power. Although current quantum computers are not yet able to perform high-dimensional tasks, they do offer one of the most efficient solutions for the future. In this work, we construct several variations of a quantum hybrid vision transformer for a classification problem in high-energy physics (distinguishing photons and electrons in the electromagnetic calorimeter). We test them against classical vision transformer architectures. Our findings indicate that the hybrid models can achieve comparable performance to their classical analogs with a similar number of parameters.

Unlu, Eyup B. (ORCID:0000000266836463)↗
DOE OSTI2024

A Compound Data Poisoning Technique with Significant Adversarial Effects on Transformer-based Sentiment Classification Tasks

Transformer-based models have demonstrated much success in various natural language processing tasks. However, they are often vulnerable to adversarial attacks, such as data poisoning, which can intentionally fool the model into generating incorrect results. In this article, we present a novel, compound variant of a data poisoning attack on a transformer-based model that maximizes the poisoning effect while minimizing the scope of poisoning. Here we do so by combining the established data poisoning technique (label flipping) with a novel adversarial artifact selection and insertion technique aimed at minimizing detectability and the scope of the poisoning footprint. We find that by using a combination of these two techniques, we achieve a state-of-the-art attack success rate of approximately 90% while poisoning only 0.5% of the original training set, thus minimizing the scope and detectability of the poisoning action. These findings have the potential to advance the development of better data poisoning detection methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Molecular dynamics simulation of vacancy and void effects on strain-induced martensitic transformations in Fe-50 at.% Ni model concentrated solid solution alloy

Strain-induced martensitic transformations can improve the strength and ductility of some face centered cubic (FCC) metals and alloys. Irradiation-induced defects such as vacancies, dislocation loops, and voids activate martensitic transformations over a wider range of mechanical loading conditions than in pristine material. Furthermore, the mechanisms underlying irradiation-enabled martensite transformations remain unclear. In this work, we use molecular dynamics simulations to study the effect of vacancies and voids on strain-induced martensitic transformations in a model concentrated solid solution alloy Fe-50 at.% Ni. It has been found that single vacancies have no resolvable effect on the transformation because they reduce the stacking fault energy by a relatively insignificant margin and do so only if the vacancy is located on the stacking fault plane. Voids, however, activate the martensite transformation through shear strain accumulation around the void due to dislocation pileup. The larger the void, the more pronounced this effect.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Simplified Model and Approach to Transform Infrared Surface Temperature to Film Effectiveness in a Conjugate Heat Transfer Experiment

In the pursuit of more efficient gas turbines, film cooling is a critical technology. This article describes a simplified engineering model based on a one-dimensional thermal resistance network. The model is used to relate film-cooling effectiveness and heat transfer augmentation to local overall cooling effectiveness in a conjugate flat plate experiment. Here, this article presents experimental proof-of-concept data to demonstrate the potential for this model. In contrast to previous approaches, neither the wall heat flux nor the adiabatic wall temperature is required to estimate the local film-cooling performance parameters. The model predicts surface temperatures that are within the experimental uncertainties over the range for which the model is trained and to within five percent when the model is extrapolated to higher coolant channel Reynolds numbers. This article is relevant to conjugate test rigs that can measure the hot-surface temperature distribution with and without film cooling. This information may also be relevant to designers as a method to approximate surface temperatures or used as an approximate heat transfer model for optimization studies.

advanced turbines↗
DOE OSTI2025

Developing a digital twin framework for remotely monitoring nuclear reactor facilities

A digital twin must seek to represent all applicable functional components of the system of interest. Different expertise is required for understanding the physical system being modeled than the skills needed for transforming those models into a functional digital twin through physics modeling, machine learning analysis, and visualization. The diversity of knowledge requires a multi-disciplinary team to ensure all system details are captured. Team members also need a method to verify that the data they generate within their domain can be effectively communicated to professionals in other fields. To address this challenge, this work provides an approach for developing a digital twin framework to remotely monitoring nuclear facilities. Through this, general knowledge of the framework is presented along with two examples to solidify the process. The AGN-201 digital twin and microreactor digital twins provide varying levels of complexity in a potential nuclear facility, where common threads are identified and lessons learned are provided. The goal of this research is to aid future researchers by providing a formula for a successful digital twin and in turn reducing the development time of nuclear system digital twins, specifically for remote monitoring.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Understanding the deformation behavior of the γ rich transformative Fe 38.5 Mn 20 Co 20 Cr 15 Si 5 Cu 1.5 complex concentrated alloy using in situ synchrotron diffraction

In situ tensile testing coupled with synchrotron x-ray diffraction was used to study the deformation behavior of metastability-engineered Fe 38.5 Mn 20 Co 20 Cr 15 Si 5 Cu 1.5 complex concentrated alloy. Monitoring the evolution of phase fraction and strain hardening response allowed the determination of true critical stress for the onset of the transformation-induced plasticity (TRIP) to be ∼375 MPa, preceded by slip starting at ∼255 MPa. In situ EBSD was used to validate the critical stress for transformation at the microstructure level and observe slip traces to confirm prior slip activity before the transformation. Further, a modeling framework based on stacking fault energy (SFE) was developed to predict the critical stress for transformation. Modeling suggested the SFE of the alloy to fall nearly 15 mJ/m 2 , which agrees well with SFE values calculated using synchrotron peak shifting (12 mJ/m 2 ) and thermodynamic calculation (11 mJ/m 2 ). As a result of γ-fcc to ε-hcp phase transformation, new {0002} ε planes emerged parallel to unaligned {111} γ planes with the tensile loading following S-N orientation relationship. Such selective emergence of new diffraction rings corresponding to ε phase is understood based on the reorientation of γ crystals with reference to tensile axis. In conclusion, this approach can be extended to effectively design alloys based on critical stress required for activating different deformation mechanisms to further push the limits of the strength-ductility envelope.

Complex concentrated alloy↗