Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 775 records · Page 43
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Linear Solvers for Collector Systems of Generalized Large-scale Inverter-Based Resources

Collector systems for inverter-based resources (IBRs) are typically represented by equivalent circuits for electromagnetic transient (EMT) simulations. Recent studies have revealed that modeling a detailed collector system is essential to accurately represent the behavior of IBRs, especially when dealing with partial tripping during external disturbances. However, there are several challenges in simulating a detailed EMT model of a collector system due to the time required to simulate such systems. Thus, this paper investigates the modeling of a detailed collector system, taking into account its configuration and components as defined in IEEE standard 2800. The configurations include the collector systems of generalized large-scale IBR plants. The components include the main IBR transformer, collector bus, and feeders with lines and/or cables. The EMT model of the collector system is represented by differential algebraic equations (DAEs) that are discretized to form linear equations that are solved using linear solvers. In this paper, linear solvers are proposed based on the Schur complement method, which are utilized for simulation of the EMT model of collector systems of generalized large-scale IBRs to accelerate simulation speed while maintaining the accuracy of the results. The proposed solvers are verified by comparing the performance to that of linear solvers provided in MATLAB.

Choi, Jongchan↗
DOE OSTI2025

Computational Modeling of Molten Salt Infiltration and Oxidation in Nuclear Graphite

Graphite is utilized as a moderator and reflector in advanced nuclear reactor designs due to its high thermal conductivity, neutron moderation properties, and resistance to radiation damage. However, its longterm performance and reliability are challenged by degradation mechanisms such as molten salt infiltration in molten salt reactors (MSRs) and oxidation in gas-cooled reactors (GCRs). These mechanisms can compromise the structural integrity and operational lifetime of graphite components, necessitating a more detailed assessment of their physical behavior. This report focuses on the development of computational models for molten salt infiltration and oxidation of graphite to aid the design and performance analysis of graphite components. For molten salt infiltration, a computational framework is developed that couples incompressible Navier-Stokes and phase-field model to simulate the penetration of molten salt into graphite?s interconnected pore structure. Initial model verification is performed using two-phase flows in two dimensions, demonstrating the models ability to capture fundamental physical behavior and agree with analytical solution. This framework is then applied to a realistic IG110 nuclear graphite , where a computed tomography extracted pore geometry is used to analyse the infiltration behavior of FLiNaK molten salt. This model provides insights into how the microstructure and other relevant parameters influence the transport pathways of molten salt into graphite, potentially offering a means to rapidly evaluate a graphite grade?s resistance to infiltration. For oxidation, the report details pore-scale mass and heat transport models, describing the diffusion of gases, reaction kinetics, and thermal effects. Additionally, this report highlights inconsistencies in the existing volume-averaged macroscopic model, particularly in upscaling of reaction kinetics and flux terms, and surface to volume transformations. These inconsistencies suggest that current formulations may not accurately capture the experimentally observed graphite oxidation process, highlighting the need for improved model development. This work advances the development of physics-based computational models for graphite degradation, contributing to improved predictive models for next-generation nuclear reactor designs. Future efforts will focus on refining the infiltration model to address non-physical behaviors and enhance its robustness. Additionally, for oxidation, further studies will employ the principles of volume averaging to rigorously derive the upscaled equations, potentially in collaboration with subject matter experts.

Computational Modeling of Molten Salt Infiltration↗
DOE OSTI2022

Towards physics-inspired data-driven weather forecasting: integrating data assimilation with a deep spatial-transformer-based U-NET in a case study with ERA5

Abstract. There is growing interest in data-driven weather prediction (DDWP), e.g., using convolutional neural networks such as U-NET that are trained on data from models or reanalysis. Here, we propose three components, inspired by physics, to integrate with commonly used DDWP models in order to improve their forecast accuracy. These components are (1) a deep spatial transformer added to the latent space of U-NET to capture rotation and scaling transformation in the latent space for spatiotemporal data, (2) a data-assimilation (DA) algorithm to ingest noisy observations and improve the initial conditions for next forecasts, and (3) a multi-time-step algorithm, which combines forecasts from DDWP models with different time steps through DA, improving the accuracy of forecasts at short intervals. To show the benefit and feasibility of each component, we use geopotential height at 500 hPa (Z500) from ERA5 reanalysis and examine the short-term forecast accuracy of specific setups of the DDWP framework. Results show that the spatial-transformer-based U-NET (U-STN) clearly outperforms the U-NET, e.g., improving the forecast skill by 45 %. Using a sigma-point ensemble Kalman (SPEnKF) algorithm for DA and U-STN as the forward model, we show that stable, accurate DA cycles are achieved even with high observation noise. This DDWP+DA framework substantially benefits from large (O(1000)) ensembles that are inexpensively generated with the data-driven forward model in each DA cycle. The multi-time-step DDWP+DA framework also shows promise; for example, it reduces the average error by factors of 2–3. These results show the benefits and feasibility of these three components, which are flexible and can be used in a variety of DDWP setups. Furthermore, while here we focus on weather forecasting, the three components can be readily adopted for other parts of the Earth system, such as ocean and land, for which there is a rapid growth of data and need for forecast and assimilation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Neutrino Flavor Model Building and the Origins of Flavor and CP Violation

The neutrino sector offers one of the most sensitive probes of new physics beyond the Standard Model of Particle Physics (SM). The mechanism of neutrino mass generation is still unknown. The observed suppression of neutrino masses hints at a large scale, conceivably of the order of the scale of a rand unified theory (GUT), which is a unique feature of neutrinos that is not shared by the charged fermions. The origin of neutrino masses and mixing is part of the outstanding puzzle of fermion masses and mixings, which is not explained ab initio in the SM. Flavor model building for both quark and lepton sectors is important in order to gain a better understanding of the origin of the structure of mass hierarchy and flavor mixing, which constitute the dominant fraction of the SM parameters. Recent activities in neutrino flavor model building based on non-Abelian discrete flavor symmetries and modular flavor symmetries have been shown to be a promising direction to explore. The emerging models provide a framework that has a significantly reduced number of undetermined parameters in the flavor sector. In addition, such a framework affords a novel origin of CP violation from group theory due to the intimate connection between physical CP transformation and group theoretical properties of non-Abelian discrete groups. Model building based on non-Abelian discrete flavor symmetries and their modular variants enables the particle physics community to interpret the current and anticipated upcoming data from neutrino experiments. Non-Abelian discrete flavor symmetries and their modular variants can result from compactification of a higher-dimensional theory. Pursuit of flavor model building based on such frameworks thus also provides the connection to possible UV completions: in particular, to string theory. We emphasize the importance of constructing models in which the uncertainties of theoretical predictions are smaller than, or at most compatible with, the error bars of measurements in neutrino experiments. While there exist proof-of-principle versions of bottom-up models in which the theoretical uncertainties are under control, it is remarkable that the key ingredients of such constructions were discovered first in top-down model building. We outline how a successful unification of bottom-up and top-down ideas and techniques may guide us towards a new era of precision flavor model building in which future experimental results can give us crucial insights into the UV completion of the SM.

Almumin, Yahya (ORCID:0000000196396772)↗
DOE OSTI2026

Differential Privacy in Grid Kitchen: Implementation & Software Documentation

Sharing of power grid feeder models faces significant challenges due to the potential risk of exposing sensitive operational information. Traditional anonymization techniques have shown notable limitations in other sensitive domains, as evidenced by documented re-identification attacks that combine supposedly anonymized datasets with auxiliary information, raising concerns that similar vulnerabilities could affect power grid data. Consequently, there is a pressing need for a more rigorous privacy protection strategy that not only delivers formal mathematical guarantees but also preserves the analytical value of the shared models. To address this challenge, we have enhanced the Grid Kitchen framework by implementing differential privacy mechanisms within the distribution model dehydration pipeline. This implementation carefully calibrates and applies noise to sensitive attributes in feeder models according to configurable privacy levels—low, moderate, and high—each offering different balances between data utility and privacy protection. Our approach uses established noise functions (Gaussian for continuous data and Discrete Laplace for integer values) with parameters carefully calibrated so that the impact of individual data points is effectively masked in the final output. The integration leverages our Noise Catalog, which we developed to categorize feeder model properties by component type, data type, and sensitivity. This catalog guides the application of appropriate noise functions and privacy parameters ($\varepsilon$ and $\delta$) to each attribute, ensuring consistent privacy protection across the model while maintaining its structural integrity and analytical usefulness. This implementation also includes evaluation tools that allow model owners to assess the impact of privacy-preserving transformations before sharing data with external parties. This report provides documentation for the differential privacy capabilities added to the Grid Kitchen project. It includes a primer on differential privacy concepts and their importance in modern data sharing, details the architecture of our implementation, explains the privacy modes and parameter configurations, and offers practical guidance on using the code for applying differential privacy to grid feeder models. Through examples and code snippets, we demonstrate the effective application of these privacy-enhancing technologies, enabling utility operators and researchers to confidently share grid data while protecting sensitive information.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Approaches to genetic tool development for rapid domestication of non-model microorganisms

Abstract Non-model microorganisms often possess complex phenotypes that could be important for the future of biofuel and chemical production. They have received significant interest the last several years, but advancement is still slow due to the lack of a robust genetic toolbox in most organisms. Typically, “domestication” of a new non-model microorganism has been done on an ad hoc basis, and historically, it can take years to develop transformation and basic genetic tools. Here, we review the barriers and solutions to rapid development of genetic transformation tools in new hosts, with a major focus on Restriction-Modification systems, which are a well-known and significant barrier to efficient transformation. We further explore the tools and approaches used for efficient gene deletion, DNA insertion, and heterologous gene expression. Finally, more advanced and high-throughput tools are now being developed in diverse non-model microbes, paving the way for rapid and multiplexed genome engineering for biotechnology.

09 BIOMASS FUELS↗
DOE OSTI2024

Experimental and Numerical Characterization of High Temperature Deformation Behavior of 347H Stainless Steel

This study investigates how temperature affects the plasticity and thermal creep behavior of 347H stainless steel under uniaxial tension. The research combined experimental testing with advanced computational modeling. Two types of experiments were conducted: uniaxial tensile tests at temperatures from 100°C to 750°C using strain rates of ~10⁻⁴ s⁻¹, and creep tests at temperatures between 600°C and 750°C under various stress levels. These experimental results were used to develop and validate a new integrated mechanistic model that can predict material behavior under any loading condition while accounting for both stress and temperature effects. The model was implemented using a polycrystalline microstructure simulation framework based on elasto-viscoplastic Fast Fourier Transform (EVPFFT). It incorporates three key deformation mechanisms: thermally activated dislocation glide, dislocation climb, and vacancy diffusional creep. The model accounts for internal stress distribution within single crystals and considers how precipitates and solute atoms (both interstitial and substitutional) affect dislocation movement. After validation against experimental data, the model was used to generate Ashby-Weertman deformation mechanism maps for 347H steel, providing new insights into how microstructure influences the activation of different creep mechanisms.

Joy, Jobin K.↗
DOE OSTI2022

Farm-level Interactions Study of a Novel Tri-port Soft-switching Medium-Voltage String Inverter (MVSI) based Large-scale PV-Plus-Storage Farms

A medium-voltage photovoltaic power conversion unit supported by battery based on a novel topology of soft switching solid state transformer is presented in this paper. It can reduce the Levelized Cost of Energy (LCOE) of PVS farm in an extended degree. However, its capability to provide grid-support services, such as frequency response, reactive power support, etc, have not been demonstrated. A series/parallel combination of these modules form a large 20 MVA solar farm. To validate them with reduced computation burden, a small-signal model of MVSI is firstly derived and validated in this paper. The model has been validated to match the full-switch model in both steady-state and dynamic conditions. Next, a farm-level simulation based on the proved small-signal model was implemented to demonstrate grid-support services. Real irradiation date from NREL is used to show the operational capabilities of the system. As a result, the overall farm is simulated based on MATLAB/Simulink domain to check the performance for various cases namely: power transfer from dc-ac side, reactive power support during grid voltage sag and utilization of battery during grid voltage disturbance or partial shading over PV panels.

14 SOLAR ENERGY↗
DOE OSTI2021

CyBERT: Cybersecurity Claim Classification by Fine-Tuning the BERT Language Model

We introduce CyBERT, a cybersecurity feature claims classifier based on bidirectional encoder representations from transformers and a key component in our semi-automated cybersecurity vetting for industrial control systems (ICS). To train CyBERT, we created a corpus of labeled sequences from ICS device documentation collected across a wide range of vendors and devices. This corpus provides the foundation for fine-tuning BERT’s language model, including a prediction-guided relabeling process. We propose an approach to obtain optimal hyperparameters, including the learning rate, the number of dense layers, and their configuration, to increase the accuracy of our classifier. Fine-tuning all hyperparameters of the resulting model led to an increase in classification accuracy from 76% obtained with BertForSequenceClassification’s original architecture to 94.4% obtained with CyBERT. Furthermore, we evaluated CyBERT for the impact of randomness in the initialization, training, and data-sampling phases. CyBERT demonstrated a standard deviation of ±0.6% during validation across 100 random seed values. Finally, we also compared the performance of CyBERT to other well-established language models including GPT2, ULMFiT, and ELMo, as well as neural network models such as CNN, LSTM, and BiLSTM. The results showed that CyBERT outperforms these models on the validation accuracy and the F1 score, validating CyBERT’s robustness and accuracy as a cybersecurity feature claims classifier.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Managing time-substitutable electricity usage using dynamic controls

A predictive-control approach allows an electricity provider to monitor and proactively manage peak and off-peak residential intra-day electricity usage in an emerging smart energy grid using time-dependent dynamic pricing incentives. The daily load is modeled as time-shifted, but cost-differentiated and substitutable, copies of the continuously-consumed electricity resource, and a consumer-choice prediction model is constructed to forecast the corresponding intra-day shares of total daily load according to this model. This is embedded within an optimization framework for managing the daily electricity usage. A series of transformations are employed, including the reformulation-linearization technique (RLT) to obtain a Mixed-Integer Programming (MIP) model representation of the resulting nonlinear optimization problem. In addition, various regulatory and pricing constraints are incorporated in conjunction with the specified profit and capacity utilization objectives.

Ghosh, Soumyadip↗
DOE OSTI2021

Data-Driven Learning of Nonautonomous Systems

In this work, we present a numerical framework for recovering unknown nonautonomous dynamical systems with time-dependent inputs. To circumvent the difficulty presented by the nonautonomous nature of the system, our method transforms the solution state into piecewise integration of the system over a discrete set of time instances. The time-dependent inputs are then locally parameterized by using a proper model, for example, polynomial regression, in the pieces determined by the time instances. This transforms the original system into a piecewise parametric system that is locally time invariant. We then design a deep neural network structure to learn the local models. Once the network model is constructed, it can be iteratively used over time to conduct global system prediction. We provide theoretical analysis of our algorithm and present a number of numerical examples to demonstrate the effectiveness of the method.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Data-Driven Learning of Non-Autonomous Systems

We present a numerical framework for recovering unknown non-autonomous dynamical systems with time-dependent inputs. To circumvent the difficulty presented by the non-autonomous nature of the system, our method transforms the solution state into piecewise integration of the system over a discrete set of time instances. The time-dependent inputs are then locally parameterized by using a proper model, for example, polynomial regression, in the pieces determined by the time instances. This transforms the original system into a piecewise parametric system that is locally time invariant. We then design a deep neural network structure to learn the local models. Once the network model is constructed, it can be iteratively used over time to conduct global system prediction. We provide theoretical analysis of our algorithm and present a number of numerical examples to demonstrate the effectiveness of the method.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Multiscale Data-Driven Seismic Full-Waveform Inversion With Field Data Study

Seismic full-waveform inversion (FWI), which uses iterative methods to estimate high-resolution subsurface models from seismograms, is a powerful imaging technique in exploration geophysics. In recent years, the computational cost of FWI has grown exponentially due to the increasing size and resolution of seismic data. Moreover, it is a nonconvex problem and can encounter local minima due to the limited accuracy of the initial velocity models or the absence of low frequencies in the measurements. To overcome these computational issues, we develop a multiscale data-driven FWI method based on fully convolutional networks (FCNs). In preparing the training data, we first develop a real-time style transform method to create a large set of synthetic subsurface velocity models from natural images. We then develop two convolutional neural networks with encoder-decoder structures to reconstruct the low- and high-frequency components of the subsurface velocity models, separately. To validate the performance of our data-driven inversion method and the effectiveness of the synthesized training set, we compare it with conventional physics-based waveform inversion approaches using both synthetic and field data. Finally, these numerical results demonstrate that, once our model is fully trained, it can significantly reduce the computation time and yield more accurate subsurface velocity models in comparison with conventional FWI.

58 GEOSCIENCES↗
DOE OSTI2024

Transforming the bootstrap: using Transformers to compute scattering amplitudes in planar N = 4 Super Yang-Mills theory

Abstract We pursue the use of deep learning methods to improve state-of-the-art computations in theoretical high-energy physics. Planar N = 4 Super Yang-Mills theory is a close cousin to the theory that describes Higgs boson production at the Large Hadron Collider; its scattering amplitudes are large mathematical expressions containing integer coefficients. In this paper, we apply Transformers to predict these coefficients. The problem can be formulated in a language-like representation amenable to standard cross-entropy training objectives. We design two related experiments and show that the model achieves high accuracy (> 98%) on both tasks. Our work shows that Transformers can be applied successfully to problems in theoretical physics that require exact solutions.

Cai, Tianji (ORCID:0000000232359486)↗
DOE OSTI2025

ORBIT-2: Scaling Exascale Vision Foundation Models for Weather and Climate Downscaling

Sparse observations and coarse-resolution climate models limit effective regional decision-making, underscoring the need for robust downscaling. However, existing AI methods struggle with generalization across variables and geographies and are constrained by the quadratic complexity of Vision Transformer (ViT) self-attention. We introduce ORBIT-2, a scalable foundation model for global, hyper-resolution climate downscaling. ORBIT-2 incorporates two key innovations: (1) Residual Slim ViT (Reslim), a lightweight architecture with residual learning and Bayesian regularization for efficient, robust prediction; and (2) TILES, a tile-wise sequence scaling algorithm that reduces self-attention complexity from quadratic to linear, enabling long-sequence processing and massive parallelism. ORBIT-2 scales to 10 billion parameters across 65,536 GPUs, achieving up to 4.1 ExaFLOPS sustained throughput and 74–98% strong scaling efficiency. It supports downscaling to 0.9 km global resolution and processes sequences up to 4.2 billion tokens. On 7 km resolution benchmarks, ORBIT-2 achieves high accuracy with R2 scores in range of 0.98–0.99 against observation data.

Wang, Xiao [ORNL] (ORCID:0000000165451943)↗
DOE OSTI2024

Transforming the Bootstrap: Using Transformers to Compute Scattering Amplitudes in Planar N = 4 Super Yang-Mills Theory

We pursue the use of deep learning methods to improve state-of-the-art computations in theoretical high-energy physics. Planar N = 4 Super Yang-Mills theory is a close cousin to the theory that describes Higgs boson production at the Large Hadron Collider; its scattering amplitudes are large mathematical expressions containing integer coefficients. In this paper, we apply Transformers to predict these coefficients. The problem can be formulated in a language-like representation amenable to standard cross-entropy training objectives. We design two related experiments and show that the model achieves high accuracy (> 98%) on both tasks. Our work shows that Transformers can be applied successfully to problems in theoretical physics that require exact solutions.

Lance, Dixon↗
DOE OSTI2025

Advancing AI-Driven Analysis in X-ray Absorption Spectroscopy: Spectral Domain Mapping and Universal Models

In recent years, rapid progress has been made in developing artificial intelligence (AI) and machine learning (ML) methods for X-ray absorption spectroscopy (XAS) analysis. Compared to traditional XAS analysis methods, AI/ML approaches offer dramatic improvements in efficiency and help eliminate human bias. To advance this field, we advocate an AI-driven XAS analysis pipeline that features several interconnected key building blocks: benchmarks, workflows, databases, and AI/ML models. Specifically, we present two case studies for XAS ML. In the first study, we demonstrate the importance of reconciling the discrepancies between simulation and experiment using spectral domain mapping (SDM). Our ML model, which is trained solely on simulated spectra, predicts an incorrect oxidation state trend for Ti atoms in a combinatorial zinc titanate film. After transforming the experimental spectra into a simulation-like representation using SDM, the same model successfully recovers the correct oxidation state trend. In the second study, we explore the development of universal XAS ML models that are trained on the entire periodic table, which enables them to leverage common trends across elements. Looking ahead, we envision that an AI-driven pipeline can unlock the potential of real-time XAS analysis to accelerate scientific discovery.

36 MATERIALS SCIENCE↗
DOE OSTI2022

A Laplace-Domain Circuit Model for Fault and Stability Analysis Considering Unbalanced Topology

For systems subject to unbalanced faults, analytical model building for stability assessment is a challenging task. This letter presents a straightforward modeling approach. A generalized dynamic circuit representation is achieved by use of the Laplacian transform variable s . Here, we translate the voltage and current relationship at the fault location into the relationship of three subsystems. The final circuit model is an interconnected sequence network with impedances in the Laplace domain. This circuit can be directly converted from a steady-state sequence network. This modeling procedure is illustrated by an example case of an induction motor served by a grid through a series compensated line. Electromagnetic transient simulation results demonstrate that sub-synchronous oscillations can be mitigated when a single-line to ground fault is applied at the motor terminal. Stability analysis results based on the dynamic circuit corroborate the simulation results. What's more, the derived circuit effortlessly reveals why unbalance can enhance stability.

42 ENGINEERING↗