Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

HT Model Dataset

This website contains the dataset that was used for writing the manuscript "HT Model: Using the Molecular Transformer for predicting hydrotreating reactions" (PNNL-SA-186589) The dataset includes a collection of hydrotreating reactions compiled from 41 peer-reviewed literature sources. These sources contain experimental data related to hydrotreating reactions. These reactions involve the reaction of chemical compounds with hydrogen gas in the presence of a catalyst to remove heteroatoms or to convert specific functional groups. The dataset contains reactions both with and without reaction conditions. Reaction conditions refer to the specific parameters under which the reaction takes place, such as temperature and pressure. For each reaction, the dataset includes both SMILES and SELFIES representations. SMILES (Simplified Molecular Input Line Entry System) and SELFIES (SELF-referencIng Embedded Strings) are two popular notations used to represent chemical structures in a compact and standardized format. The dataset was created with the aim of training a predictive model, specifically using the Molecular Transformer architecture.

bioprocessing, hydrotreating, deep learning algori↗
DOE OSTI2020

Analysis and Planning Framework for Nuclear Plant Transformation

Commercial nuclear power in the United States has been an unqualified success by any measure, providing low cost, carbon-free, and safe baseload electricity for decades. The industry today is at the peak of its historical performance in terms of generation output, reliable operations, and demonstrated nuclear safety. However, it is no longer among the lowest-cost electric generation sources with the emergence of subsidized renewables and shale gas generation. The business model that has served the operating nuclear fleet so well over its life is now a drag on cost performance due to its reliance on a large highly skilled labor force. In contrast, digital technology and innovation are enabling dramatic efficiencies in energy production, resulting in fierce competition for a commodity such as electricity. The nuclear power industry has responded to this challenge with many initiatives to improve efficiency and modernize plant equipment, especially where reliability and obsolescence issues are pressing. However, it would be a missed opportunity to merely modernize the plant components and the work processes of an outdated business model that was formulated to manage the technology of the 1960s. Rather, the greater opportunity is to transform that business model to one that fully exploits the capabilities of modern digital technology, resulting in substantially lower production costs and sustainable market viability. One successful example of such transformation is the concept of Integrated Operations, which was introduced into the North Sea oil and gas industry a couple of decades ago when the profitability of operating these fields was severely threatened by low world petroleum prices and the high overhead of operating the offshore oil and gas platforms. This effort resulted in significant changes to how these oil fields were operated and allowed the industry to continue profitable operation of these platforms. This example has some remarkable parallels to the U.S. commercial nuclear industry as described in the next section. This report provides an analysis and planning framework for such a nuclear plant operating model transformation based on the transferable learnings from the North Sea transformation. This framework is termed Integrated Operations for Nuclear or ION. This report describes the key principles and methods of Integrated Operations and how they are being applied in a collaboration between the DOE Light Water Reactor Sustainability Program and Xcel Energy in an initiative to transform its operating model for performance improvement and long-term sustainability. It describes a method to bring the operating costs of a nuclear fleet in line with market-based pricing, transforming work functions to reduce cost with technology innovations. This initiative will continue over the next several years in the detailed development of transformative concepts for nuclear plants, which will be published as follow-up to this initial report on Integrated Operations for Nuclear.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2025

DeepHyper: A Python Package for Massively Parallel Hyperparameter Optimization in Machine Learning

Machine learning models are increasingly applied across scientific disciplines, yet their effectiveness often hinges on heuristic decisions—such as data transformations, training strategies, and model architectures—that are not learned by the models themselves. Automating the selection of these heuristics and analyzing their sensitivity is crucial for building robust and efficient learning workflows. DeepHyper addresses this challenge by democratizing hyperparameter optimization, providing accessible tools to streamline and enhance machine learning workflows from a laptop to the largest supercomputer in the world. Building on top of hyperparameter optimization, it unlocks new capabilities around ensembles of models for improved accuracy and uncertainty quantification. All of these organized around efficient parallel computing.

ensemble↗
DOE OSTI2024

Transformer Neural Networks with Spatiotemporal Attention for Predictive Control and Optimization of Industrial Processes

In the context of real-time optimization and model predictive control of industrial systems, machine learning, and neural networks represent cutting-edge tools that hold promise for enhancing dynamic modeling. This work presents a novel transformer neural network architecture for real-time optimization and model predictive control. This network design includes a modified attention mechanism inspired by positional embedding attention from vision transformers and task-specific modifications to the input-output structure of the transformer’s decoder stack. Experiments were conducted using data from a 450 MW coal-fired power plant to evaluate this approach's effectiveness. The transformer neural network was compared with conventional recurrent models, including GRU and LSTM. The transformer exhibited a 6% increase in the R-squared (R2) value of predictions and an 83% reduction in mean squared error (MSE). Computation time was also reduced by 84% compared to conventional recurrent models.

Gallup, Ethan R.↗
DOE OSTI2021

A novel physics-based and data-supported microstructure model for part-scale simulation of laser powder bed fusion of Ti-6Al-4V

The elasto-plastic material behavior, material strength and failure modes of metals fabricated by additive manufacturing technologies are significantly determined by the underlying process-specific microstructure evolution. In this work a novel physics-based and data-supported phenomenological microstructure model for Ti-6Al-4V is proposed that is suitable for the part-scale simulation of laser powder bed fusion processes. The model predicts spatially homogenized phase fractions of the most relevant microstructural species, namely the stable β-phase, the stable α s -phase as well as the metastable Martensite α m -phase, in a physically consistent manner. In particular, the modeled microstructure evolution, in form of diffusion-based and non-diffusional transformations, is a pure consequence of energy and mobility competitions among the different species, without the need for heuristic transformation criteria as often applied in existing models. The mathematically consistent formulation of the evolution equations in rate form renders the model suitable for the practically relevant scenario of temperature- or time-dependent diffusion coefficients, arbitrary temperature profiles, and multiple coexisting phases. Due to its physically motivated foundation, the proposed model requires only a minimal number of free parameters, which are determined in an inverse identification process considering a broad experimental data basis in form of time-temperature transformation diagrams. Subsequently, the predictive ability of the model is demonstrated by means of continuous cooling transformation diagrams, showing that experimentally observed characteristics such as critical cooling rates emerge naturally from the proposed microstructure model, instead of being enforced as heuristic transformation criteria. Eventually, the proposed model is exploited to predict the microstructure evolution for a realistic selective laser melting application scenario and for the cooling/quenching process of a Ti-6Al-4V cube of practically relevant size. Numerical results confirm experimental observations that Martensite is the dominating microstructure species in regimes of high cooling rates, e.g., due to highly localized heat sources or in near-surface domains, while a proper manipulation of the temperature field, e.g., by preheating the base-plate in selective laser melting, can suppress the formation of this metastable phase.

Inverse parameter identification↗
DOE OSTI2022

C3F: Collaborative Container-based Model Coupling Framework

Solving complex real-world grand challenge problems requires in-depth collaboration of researchers from multiple disciplines. Such collaboration often involves harnessing multiscale and multi-dimensional data and combining models from different fields to simulate systems. However, the progress on this front has been limited mainly due to significant gaps in domain knowledge and tools that are typically employed in silos of the domains. Researchers from different fields face considerable barriers to understanding and reusing each other’s data/models in order to collaborate effectively. For example, in solving the global sustainability problems, researchers from hydrology, climate science, agriculture, and economics need to run their respective models to study different components of the global and local food, energy and water systems while, at the same time, need to interact with other researchers and integrate the results of one model with another. Developing this kind of model coupling workflow calls for (1) a large amount of data being processed and exchanged across domains and organizations, (2) identifying and processing the output of one model to make it ready for integration into another model, (3) controlling the workflow dynamically so that it runs until a certain convergence condition or other criteria is met, and (4) close collaboration among the modelers to explore, tune, and test the configuration and data transformation needed to link the models. We have developed C3F, a flexible collaborative model coupling framework to help researchers accelerate their model integration and linking efforts by leveraging advanced cyberinfrastructure such as high-performance computing and virtual containers. In this paper, we describe our experience and lessons learned in developing this cyberinfrastructure solution to support the linking of Water Balance Model (WBM) and SIMPLE-G agricultural economic model in an NSF funded INFEWS project and a DOE-funded Program on Coupled Human and Earth Systems (PCHES) to study the implications of groundwater scarcity for food-energy-water systems. The C3F model coupling framework can be extended to facilitate other model linkages as well.

containerization↗
DOE OSTI2023

Pilot Business Case Analysis for Digital Infrastructure

The Inflation Reduction Act Has Provided an Opening to Dramatically Change and Improve the Economics of Nuclear Power Until recently, the commercial nuclear sector has faced unprecedented financial challenges driven by low natural gas prices and subsidized renewables, in a marketplace that did not reward carbon-free baseload capacity. The industry is also plagued by labor shortages, increasingly obsolete labor-centric operating models and antiquated analog technology that threaten the viability of the long-term operations of domestic nuclear facilities. In this context, the economic survival of nuclear power plants requires an efficient and technology-centric operating model that harvests the native efficiencies of advanced technology. Such transformations have been made in many other industries – notably, oil and gas. The passage by Congress of the Inflation Reduction Act (IRA) in 2022 provides nuclear operators with unprecedented economic incentives to pursue these upgrades. To realize the benefits of these incentives and achieve the necessary transformations in the nuclear operating model, the nuclear industry must overcome both the continuing unease about licensing pathways and uncertainty related to implementation costs.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Spatio-temporal Fourier Transformer for Long-term Dynamics Prediction (StFT) v1.0

We propose a novel machine learning model spatio-temporal Fourier transformer (StFT) to emulate long-term dynamics of multi-scale and multi-physics systems. Our method StFT overcomes the limitations of rapid error accumulation, particularly in long-term forecasting of systems characterized by complex and coupled dynamics. StFT achieves outstanding accuracy and computational efficiency by effectively capturing multi-scale interactions, and quantify the uncertainties inherent in the predictions. Our model leverages a structured hierarchy of StFT blocks, and explicitly captures dynamics across both macro- and micro- spatial scales. Evaluations conducted on three benchmark datasets (plasma, fluid, and atmospheric dynamics) demonstrate the advantages of our approach over state-of-the-art ML methods.

Bai, Zhe [Lawrence Berkeley National Laboratory (L↗
DOE OSTI2022

Common Information Model for Electromagnetic Transients (CIM for EMT): CRADA 533 [Abstract only}

The control and protection functions of inverter-based resources (IBR) have raised concerns with bulk system reliability. Most of the current interest lies with solar photovoltaic generation but increasing amounts of storage would pose the same risks. Newer North American Electric Reliability Corporation (NERC) guidelines call for electromagnetic transient (EMT) studies of IBR and recommend that transmission operators collect distributed energy resource (DER) data to support such modeling. IEEE Standard P2800.1 is defining tests for model parameterization, so good model data should become available from inverter vendors. (EMT studies also apply to large power transformer reliability, and transformer vendors can provide EMT models.) Utilities don’t currently have the rest of the bulk system represented for EMT studies at large scale. An International Electrotechnical Commission (IEC) standard Common Information Model (CIM) provides a way of supporting these detailed models from physical asset data, e.g., conductors, towers, transformer data sheets, control block diagrams, while avoiding software vendor lock-in. CIM-for-EMT, with proposed schema extensions and open-source converters, provides a way to exchange EMT data between organizations and tools. This project leverages Office of Electricity (OE) funding of CIM-for-EMT code base through the GridAPPS-DTM project, and of GridPACKTM (parallelized transmission solver), for interoperability testing in CIM-for-EMT. The project also leverages partner PGSTech investments in EMTP® interoperability with CIM.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Enhancing the Survivability of Power Systems With Grid-Edge DERs Against DoS Attacks

Power system survivability, defined as the ability of a system to maintain steady-state functionality under varying operational conditions, reflects its resilience against disturbances. While existing research primarily focuses on physical-layer disturbances, the increasing prevalence of grid-edge DERs, which are primarily used for integrating renewable energy, has significantly expanded the cyber attack surface. As a result, operational disruptions caused by cyber threats are posing significant challenges to system survivability and cannot be overlooked. To fill this gap, we redefine system survivability to incorporate the cyber layer’s status and propose a Distributionally Robust Optimization (DRO) approach to enhance power system survivability against potential cyber-physical threats. In this paper, we first analyze the operational guidelines of systems with a high penetration of DERs under various cyber network conditions and redefine survivability in this context. Next, we focus on the most common cyber threat, Denial-of-Service (DoS) attacks, and develop a corresponding attack model. This model allows for the creation of a kernel-based ambiguity set that captures attack uncertainties using historical data. Finally, we transform the proposed DRO model as a tractable optimization problem, with its solution providing an optimal cyber redundancy plan to enhance system survivability in DoS attack scenarios. Simulation results on the IEEE 13-node and 123-node test feeders demonstrate the effectiveness of our proposed model in improving system survivability. This model can also be expanded to include other types of common attacks and serve as a comprehensive planning tool to improve overall cyber physical survival of the system.

cybersecurity↗
DOE OSTI2024

Transforming Agricultural Productivity with AI-Driven Forecasting: Innovations in Food Security and Supply Chain Optimization

Global food security is under significant threat from climate change, population growth, and resource scarcity. This review examines how advanced AI-driven forecasting models, including machine learning (ML), deep learning (DL), and time-series forecasting models like SARIMA/ARIMA, are transforming regional agricultural practices and food supply chains. Through the integration of Internet of Things (IoT), remote sensing, and blockchain technologies, these models facilitate the real-time monitoring of crop growth, resource allocation, and market dynamics, enhancing decision making and sustainability. The study adopts a mixed-methods approach, including systematic literature analysis and regional case studies. Highlights include AI-driven yield forecasting in European hydroponic systems and resource optimization in southeast Asian aquaponics, showcasing localized efficiency gains. Furthermore, AI applications in food processing, such as plasma, ozone and Pulsed Electric Field (PEF) treatments, are shown to improve food preservation and reduce spoilage. Key challenges—such as data quality, model scalability, and prediction accuracy—are discussed, particularly in the context of data-poor environments, limiting broader model applicability. The paper concludes by outlining future directions, emphasizing context-specific AI implementations, the need for public–private collaboration, and policy interventions to enhance scalability and adoption in food security contexts.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2021

SWAT model application for evaluating agricultural conservation practice effectiveness in reducing phosphorous loss from the Western Lake Erie Basin

Lake Erie is threatened by eutrophication and harmful algal blooms due to excess nutrient loading from agricultural sources. Agricultural conservation practices (ACPs) have been developed and implemented to reduce nutrient losses but estimating ACP effectiveness is challenging. The Soil and Water Assessment Tool (SWAT) has been used to investigate ACP effectiveness for water quality improvement. Many SWAT applications have been developed by different investigators to evaluate ACP effectiveness for reducing nutrient, particularly phosphorus (P), loading in the agriculturally-dominated Western Lake Erie Basin (WLEB). Our objective is to establish what has been achieved by past modeling research and make suggestions for future applications and improvements. We synthesized the findings of 28 SWAT modeling studies within the WLEB. Models generally performed satisfactorily against accepted criteria for streamflow and sediment, but performance for P loads, like soluble reactive P, was mostly “unsatisfactory”. The “unsatisfactory” performance maybe due to imperfections and idealizations in model formulations and/or parameterization. Thus, simulations of P transport and transformation processes need improvement. In addition, model parameter selection is the key part of model set-up. Most SWAT modeling studies used default values during initial set-up, then performed calibration and validation. It was found that the calibrated P related parameter values varied widely across different studies, even within the same watershed with some values unrealistic for the study areas. The phenomena of different combinations of model parameters producing similar outputs indicates equifinality. Equifinality in the baseline model may impact results when ACPs are incorporated. Furthermore, the unrealistic values used in ACP assessment undermine the credibility of ACP effectiveness. Future model applications should try to re-examine the calibrated P parameters and make sure they are realistic for the study area as well as reduce equifinality by constraining the model with characterization of watershed conditions, better understanding of hydrologic processes, and parameter values based on real-world observations. As a result, future model applications should focus on improving P transport and transformation processes, using measured watershed characteristics for parameterization, and improving reflections of climate change, which could result in more accurate assessments of ACP effectiveness to meet targeted goals.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Elucidating the temperature dependence of TRIP in Q&P steels using synchrotron X-Ray diffraction, constituent phase properties, and strain-based kinetics models

Understanding the deformation-induced martensitic transformation (DIMT) is critical for interpreting the structure-property relationships that govern the performance of transformation-induced plasticity (TRIP) assisted steels. However, modern TRIP-assisted steels often exhibit DIMT kinetics that are not easily captured by existing empirical models based on bulk tensile strain. We address this challenge by combined bulk uniaxial tensile tests and in-situ high energy synchrotron X-ray diffraction, which resolved the phase volume fractions, stress-strain response, and microstructure evolution of each constituent phase. A modification of the Olson-Cohen model is implemented, which describes the martensitic transformation kinetics as a function of the estimated partitioned strain in austenite, rather than the bulk tensile strain. This DIMT kinetic model is used as a framework to clarify the root cause of an insufficiently understood toughness trough reported for TRIP-assisted steels during deformation at elevated temperatures. Here, the importance of the temperature-dependent toughness is discussed, based on the opportunity to modify deformation processes to tailor the DIMT kinetics and mechanical properties during forming and in service.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Hybrid model to evaluate the frequency-dependent leakage inductance of partially-filled transformers

The leakage inductance of a transformer designed for a power electronic converter can drop significantly as the switching frequency is increased due to skin and proximity effects. Although the magnetic image method-based double-2-D model can predict the low-frequency leakage inductance of a partially-filled transformer with sufficient accuracy, it is inherently a frequency-independent model. While Dowell’s 1-D model uses frequency-dependent relations to account for both skin and proximity effects, its accuracy is severely affected by the assumed winding geometry. In this paper, a semianalytical hybrid model is proposed that uses superposition to combine a modified Dowell’s model with the double-2-D model to predict the true leakage inductance of partially-filled transformers at any given frequency. All three conductor types—round, foil, and litz wire—are modeled and analyzed. The quasi-2-D model is further investigated on a variable inductance transformer (VIT) whose winding geometry can be modified mechanically to vary its leakage inductance. With less than 5 % error throughout, the semi-analytically evaluated leakage inductances are in excellent agreement with the finite element method (FEM) simulated and experimentally measured leakage inductances.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Solitons in lattice field theories via tight-binding supersymmetry

Reflectionless potentials play an important role in constructing exact solutions to classical dynamical systems (such as the Korteweg-de Vries equation), non-perturbative solutions of various large-N field theories (such as the Gross-Neveu model), and closely related solitonic solutions to the Bogoliubov-de Gennes equations in the theory of superconductivity. These solutions rely on the inverse scattering method, which reduces these seemingly unrelated problems to identifying reflectionless potentials of an auxiliary one-dimensional quantum scattering problem. There are several ways of constructing these potentials, one of which is quantum mechanical supersymmetry (SUSY). In this paper, motivated by recent experimental platforms, we generalize this framework to develop a theory of lattice solitons. We first briefly review the classical inverse scattering method in the continuum limit, focusing on the Korteweg-de Vries (KdV) equation and SU(N) Gross-Neveu model in the large N limit. We then generalize this methodology to lattice versions of interacting field theories. Our analysis hinges on the use of trace identities, which are relations connecting the potential of an equation of motion to the scattering data. For a discrete Schrödinger operator, such trace identities had been known as far back as Toda; however, we derive a new set of identities for the discrete Dirac operator. We then use these identities in a lattice Gross-Neveu and chiral Gross-Neveu (Nambu-Jona-Lasinio) model to show that lattice solitons correspond to reflectionless potentials associated with the discrete scattering problem. These models are of significance as they are equivalent to a mean-field theory of a lattice superconductor. To explicitly construct these solitons, we generalize supersymmetric quantum mechanics to tight-binding models. We show that a matrix transformation exists that maps a tight-binding model to an isospectral one which shares the same structure and scattering properties. The corresponding soliton solutions have both modulated hopping and onsite potential, the former of which has no analogue in the continuum limit. We explicitly compute both topological and non-topological soliton solutions as well as bound state spectra in the aforementioned models.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Lattice Design and Advanced Modeling to Guide the Design of High-Performance Lightweight Structural Materials

Lightweight structural materials are required to increase the mobility of fission batteries. The materials must feature a robust combination of mechanical properties to demonstrate structural resilience. The primary objective of this project is to produce lightweight structural materials whose strength-to-weight ratios exceed those of the current widely used structural materials such as 316L stainless steels (316L SS). To achieve this, advanced modeling and simulation tools were employed to design lattice structures with different lattice parameters and different lattice types. A process was successfully developed for transforming lattice-structures models into Multiphysics Object Oriented Simulation Environment (MOOSE) inputs. Finite element modeling (FEM) was used to simulate the uniaxial tensile testing of the lattice-structured parts to investigate the stress distribution at a given displacement. The preliminary results showed that the lattice-structured sample displayed a lower Young’s modulus in comparison with the solid material and that the unit cell size of the lattice had a minimal effect. The novelty here is to apply up-front modeling to determine the best structure for the application before actually producing the sample. The approach of using modeling as a guiding tool for preliminary material design can significantly save time and cost for material development.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Lattice Structured Lightweight Structural Materials

The development of lightweight structural materials is crucial for enhancing the performance and deployment feasibility of fission batteries. This study aims to produce lightweight structural materials whose strength-to-weight ratios exceed those of current widely used structural materials. To achieve this, advanced modeling and simulation tools were employed to design lattice structures with different lattice parameters and different lattice types. A process was successfully developed for transforming lattice-structured models into Multiphysics Object Oriented Simulation Environment (MOOSE) inputs. Finite element modeling (FEM) was used to simulate the uniaxial tensile testing of the lattice-structured parts to investigate the stress distribution at a given displacement. The modeling results showed that the lattice-structured sample displayed a lower Young’s modulus in comparison to the solid material; the increase in solid shell thickness and blend radius enhances the mechanical performance; and the effect of unit cell size on macro scale stress is minimal. Tensile testing was conducted on the solid and lattice-structured materials fabricated by laser powder bed fusion (LPBF) additive manufacturing. The experimental results agreed well with the model prediction. The approach of using modeling as a guiding tool for preliminary material design can significantly save time and cost for new material development.

lightweight material↗
DOE OSTI2022

Inference for Nonparanormal Partial Correlation via Regularized Rank-Based Nodewise Regression

Abstract Partial correlation is a common tool in studying conditional dependence for Gaussian distributed data. However, partial correlation being zero may not be equivalent to conditional independence under non-Gaussian distributions. In this paper, we propose a statistical inference procedure for partial correlations under the high-dimensional nonparanormal (NPN) model where the observed data are normally distributed after certain monotone transformations. The NPN partial correlation is the partial correlation of the normal transformed data under the NPN model, which is a more general measure of conditional dependence. We estimate the NPN partial correlations by regularized nodewise regression based on the empirical ranks of the original data. A multiple testing procedure is proposed to identify the nonzero NPN partial correlations. The proposed method can be carried out by a simple coordinate descent algorithm for lasso optimization. It is easy-to-implement and computationally more efficient compared to the existing methods for estimating NPN graphical models. Theoretical results are developed to show the asymptotic normality of the proposed estimator and to justify the proposed multiple testing procedure. Numerical simulations and a case study on brain imaging data demonstrate the utility of the proposed procedure and evaluate its performance compared to the existing methods. Data used in preparation of this article were obtained from the Alzheimer's Disease Neuroimaging Initiative (ADNI) database.

97 MATHEMATICS AND COMPUTING↗