SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

All sources NASA NTRS DOE OSTI

DOE OSTI2025

DeepHyper: A Python Package for Massively Parallel Hyperparameter Optimization in Machine Learning

Machine learning models are increasingly applied across scientific disciplines, yet their effectiveness often hinges on heuristic decisions—such as data transformations, training strategies, and model architectures—that are not learned by the models themselves. Automating the selection of these heuristics and analyzing their sensitivity is crucial for building robust and efficient learning workflows. DeepHyper addresses this challenge by democratizing hyperparameter optimization, providing accessible tools to streamline and enhance machine learning workflows from a laptop to the largest supercomputer in the world. Building on top of hyperparameter optimization, it unlocks new capabilities around ensembles of models for improved accuracy and uncertainty quantification. All of these organized around efficient parallel computing.

ensemble↗

DOE OSTI2024

Transformer Neural Networks with Spatiotemporal Attention for Predictive Control and Optimization of Industrial Processes

In the context of real-time optimization and model predictive control of industrial systems, machine learning, and neural networks represent cutting-edge tools that hold promise for enhancing dynamic modeling. This work presents a novel transformer neural network architecture for real-time optimization and model predictive control. This network design includes a modified attention mechanism inspired by positional embedding attention from vision transformers and task-specific modifications to the input-output structure of the transformer’s decoder stack. Experiments were conducted using data from a 450 MW coal-fired power plant to evaluate this approach's effectiveness. The transformer neural network was compared with conventional recurrent models, including GRU and LSTM. The transformer exhibited a 6% increase in the R-squared (R2) value of predictions and an 83% reduction in mean squared error (MSE). Computation time was also reduced by 84% compared to conventional recurrent models.

Gallup, Ethan R.↗

DOE OSTI2021

A novel physics-based and data-supported microstructure model for part-scale simulation of laser powder bed fusion of Ti-6Al-4V

The elasto-plastic material behavior, material strength and failure modes of metals fabricated by additive manufacturing technologies are significantly determined by the underlying process-specific microstructure evolution. In this work a novel physics-based and data-supported phenomenological microstructure model for Ti-6Al-4V is proposed that is suitable for the part-scale simulation of laser powder bed fusion processes. The model predicts spatially homogenized phase fractions of the most relevant microstructural species, namely the stable β-phase, the stable α s -phase as well as the metastable Martensite α m -phase, in a physically consistent manner. In particular, the modeled microstructure evolution, in form of diffusion-based and non-diffusional transformations, is a pure consequence of energy and mobility competitions among the different species, without the need for heuristic transformation criteria as often applied in existing models. The mathematically consistent formulation of the evolution equations in rate form renders the model suitable for the practically relevant scenario of temperature- or time-dependent diffusion coefficients, arbitrary temperature profiles, and multiple coexisting phases. Due to its physically motivated foundation, the proposed model requires only a minimal number of free parameters, which are determined in an inverse identification process considering a broad experimental data basis in form of time-temperature transformation diagrams. Subsequently, the predictive ability of the model is demonstrated by means of continuous cooling transformation diagrams, showing that experimentally observed characteristics such as critical cooling rates emerge naturally from the proposed microstructure model, instead of being enforced as heuristic transformation criteria. Eventually, the proposed model is exploited to predict the microstructure evolution for a realistic selective laser melting application scenario and for the cooling/quenching process of a Ti-6Al-4V cube of practically relevant size. Numerical results confirm experimental observations that Martensite is the dominating microstructure species in regimes of high cooling rates, e.g., due to highly localized heat sources or in near-surface domains, while a proper manipulation of the temperature field, e.g., by preheating the base-plate in selective laser melting, can suppress the formation of this metastable phase.

Inverse parameter identification↗

DOE OSTI2022

C3F: Collaborative Container-based Model Coupling Framework

Solving complex real-world grand challenge problems requires in-depth collaboration of researchers from multiple disciplines. Such collaboration often involves harnessing multiscale and multi-dimensional data and combining models from different fields to simulate systems. However, the progress on this front has been limited mainly due to significant gaps in domain knowledge and tools that are typically employed in silos of the domains. Researchers from different fields face considerable barriers to understanding and reusing each other’s data/models in order to collaborate effectively. For example, in solving the global sustainability problems, researchers from hydrology, climate science, agriculture, and economics need to run their respective models to study different components of the global and local food, energy and water systems while, at the same time, need to interact with other researchers and integrate the results of one model with another. Developing this kind of model coupling workflow calls for (1) a large amount of data being processed and exchanged across domains and organizations, (2) identifying and processing the output of one model to make it ready for integration into another model, (3) controlling the workflow dynamically so that it runs until a certain convergence condition or other criteria is met, and (4) close collaboration among the modelers to explore, tune, and test the configuration and data transformation needed to link the models. We have developed C3F, a flexible collaborative model coupling framework to help researchers accelerate their model integration and linking efforts by leveraging advanced cyberinfrastructure such as high-performance computing and virtual containers. In this paper, we describe our experience and lessons learned in developing this cyberinfrastructure solution to support the linking of Water Balance Model (WBM) and SIMPLE-G agricultural economic model in an NSF funded INFEWS project and a DOE-funded Program on Coupled Human and Earth Systems (PCHES) to study the implications of groundwater scarcity for food-energy-water systems. The C3F model coupling framework can be extended to facilitate other model linkages as well.

containerization↗

NASA NTRS1992

Designing for time-dependent material response in spacecraft structures

To study the influence on overall deformations of the time-dependent constitutive properties of fiber-reinforced polymeric matrix composite materials being considered for use in orbiting precision segmented reflectors, simple sandwich beam models are developed. The beam models include layers representing the face sheets, the core, and the adhesive bonding of the face sheets to the core. A three-layer model lumps the adhesive layers with the face sheets or core, while a five-layer model considers the adhesive layers explicitly. The deformation response of the three-layer and five-layer sandwich beam models to a midspan point load is studied. This elementary loading leads to a simple analysis, and it is easy to create this loading in the laboratory. Using the correspondence principle of viscoelasticity, the models representing the elastic behavior of the two beams are transformed into time-dependent models. Representative cases of time-dependent material behavior for the facesheet material, the core material, and the adhesive are used to evaluate the influence of these constituents being time-dependent on the deformations of the beam. As an example of the results presented, if it assumed that, as a worst case, the polymer-dominated shear properties of the core behave as a Maxwell fluid such that under constant shear stress the shear strain increases by a factor of 10 in 20 years, then it is shown that the beam deflection increases by a factor of 1.4 during that time. In addition to quantitative conclusions, several assumptions are discussed which simplify the analyses for use with more complicated material models. Finally, it is shown that the simpler three-layer model suffices in many situations.

Hyer, M. W.↗

DOE OSTI2023

Pilot Business Case Analysis for Digital Infrastructure

The Inflation Reduction Act Has Provided an Opening to Dramatically Change and Improve the Economics of Nuclear Power Until recently, the commercial nuclear sector has faced unprecedented financial challenges driven by low natural gas prices and subsidized renewables, in a marketplace that did not reward carbon-free baseload capacity. The industry is also plagued by labor shortages, increasingly obsolete labor-centric operating models and antiquated analog technology that threaten the viability of the long-term operations of domestic nuclear facilities. In this context, the economic survival of nuclear power plants requires an efficient and technology-centric operating model that harvests the native efficiencies of advanced technology. Such transformations have been made in many other industries – notably, oil and gas. The passage by Congress of the Inflation Reduction Act (IRA) in 2022 provides nuclear operators with unprecedented economic incentives to pursue these upgrades. To realize the benefits of these incentives and achieve the necessary transformations in the nuclear operating model, the nuclear industry must overcome both the continuing unease about licensing pathways and uncertainty related to implementation costs.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2025

Spatio-temporal Fourier Transformer for Long-term Dynamics Prediction (StFT) v1.0

We propose a novel machine learning model spatio-temporal Fourier transformer (StFT) to emulate long-term dynamics of multi-scale and multi-physics systems. Our method StFT overcomes the limitations of rapid error accumulation, particularly in long-term forecasting of systems characterized by complex and coupled dynamics. StFT achieves outstanding accuracy and computational efficiency by effectively capturing multi-scale interactions, and quantify the uncertainties inherent in the predictions. Our model leverages a structured hierarchy of StFT blocks, and explicitly captures dynamics across both macro- and micro- spatial scales. Evaluations conducted on three benchmark datasets (plasma, fluid, and atmospheric dynamics) demonstrate the advantages of our approach over state-of-the-art ML methods.

Bai, Zhe [Lawrence Berkeley National Laboratory (L↗

NASA NTRS1983

Linear approximations of nonlinear systems

A method for designing an automatic flight controller for short and vertical takeoff aircraft is presently being developed at NASA Ames Research Center. This technique involves transformations of nonlinear systems to controllable linear systems and takes into account the nonlinearities of the aircraft. In general, the transformations cannot always be given in closed form. Using partial differential equations, an approximate linear system, called the modified tangent model, was recently introduced. A linear transformation of this tangent model to Brunovsky canonical form can be constructed, and from this an approximation of an exact transformation for the nonlinear system can be found. It is shown that a canonical expansion in Lie brackets about the point x(0) yields the same modified tangent model.

Hunt, L. R.↗

NASA NTRS1992

A second-order structural identification procedure via state space-based system identification

A theory is presented for transforming the system theory-based realization models into the corresponding physical coordinate-based structural models. The theory has been implemented into a computational procedure and applied to several example problems. The results show that the present transformation theory yields an objective model basis possessing a unique set of structural parameters from an infinite set of equivalent system realization models. For proportionally damped systems, the transformation directly and systematically yields the normal modes and modal damping. When non-proportional damping is present, the relative magnitude and phase of the damped model shapes are separately characterized, and a corrective transformation is then employed to capture the undamped normal modes and non-diagonal modal damping matrix.

Alvin, K. F.↗

DOE OSTI2022

Common Information Model for Electromagnetic Transients (CIM for EMT): CRADA 533 [Abstract only}

The control and protection functions of inverter-based resources (IBR) have raised concerns with bulk system reliability. Most of the current interest lies with solar photovoltaic generation but increasing amounts of storage would pose the same risks. Newer North American Electric Reliability Corporation (NERC) guidelines call for electromagnetic transient (EMT) studies of IBR and recommend that transmission operators collect distributed energy resource (DER) data to support such modeling. IEEE Standard P2800.1 is defining tests for model parameterization, so good model data should become available from inverter vendors. (EMT studies also apply to large power transformer reliability, and transformer vendors can provide EMT models.) Utilities don’t currently have the rest of the bulk system represented for EMT studies at large scale. An International Electrotechnical Commission (IEC) standard Common Information Model (CIM) provides a way of supporting these detailed models from physical asset data, e.g., conductors, towers, transformer data sheets, control block diagrams, while avoiding software vendor lock-in. CIM-for-EMT, with proposed schema extensions and open-source converters, provides a way to exchange EMT data between organizations and tools. This project leverages Office of Electricity (OE) funding of CIM-for-EMT code base through the GridAPPS-DTM project, and of GridPACKTM (parallelized transmission solver), for interoperability testing in CIM-for-EMT. The project also leverages partner PGSTech investments in EMTP® interoperability with CIM.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2025

Enhancing the Survivability of Power Systems With Grid-Edge DERs Against DoS Attacks

Power system survivability, defined as the ability of a system to maintain steady-state functionality under varying operational conditions, reflects its resilience against disturbances. While existing research primarily focuses on physical-layer disturbances, the increasing prevalence of grid-edge DERs, which are primarily used for integrating renewable energy, has significantly expanded the cyber attack surface. As a result, operational disruptions caused by cyber threats are posing significant challenges to system survivability and cannot be overlooked. To fill this gap, we redefine system survivability to incorporate the cyber layer’s status and propose a Distributionally Robust Optimization (DRO) approach to enhance power system survivability against potential cyber-physical threats. In this paper, we first analyze the operational guidelines of systems with a high penetration of DERs under various cyber network conditions and redefine survivability in this context. Next, we focus on the most common cyber threat, Denial-of-Service (DoS) attacks, and develop a corresponding attack model. This model allows for the creation of a kernel-based ambiguity set that captures attack uncertainties using historical data. Finally, we transform the proposed DRO model as a tractable optimization problem, with its solution providing an optimal cyber redundancy plan to enhance system survivability in DoS attack scenarios. Simulation results on the IEEE 13-node and 123-node test feeders demonstrate the effectiveness of our proposed model in improving system survivability. This model can also be expanded to include other types of common attacks and serve as a comprehensive planning tool to improve overall cyber physical survival of the system.

cybersecurity↗

NASA NTRS1986

A class of fast staircase algorithms for generalized state-space systems

Several methods are presented for defining generalized state space models (GSSM), with emphasis on 'fast' techniques for transforming GSSMs to condensed state space models (SSM). The fast forms are configured to yield invariant transfer functions. Details of the decomposition process are summarized in terms of the determination of eigenvalues which are separated within staircase matrices. Applications of the techniques are illustrated through implementation of a unimodular transformation which does not affect the finite eigenvalues, by demonstrating deadbeat control of a GSSM, which the definition of a reduced observer of a GSSM, and by embedding a polynomial matrix into a unimodular matrix.

Beelen, T.↗

DOE OSTI2024

Transforming Agricultural Productivity with AI-Driven Forecasting: Innovations in Food Security and Supply Chain Optimization

Global food security is under significant threat from climate change, population growth, and resource scarcity. This review examines how advanced AI-driven forecasting models, including machine learning (ML), deep learning (DL), and time-series forecasting models like SARIMA/ARIMA, are transforming regional agricultural practices and food supply chains. Through the integration of Internet of Things (IoT), remote sensing, and blockchain technologies, these models facilitate the real-time monitoring of crop growth, resource allocation, and market dynamics, enhancing decision making and sustainability. The study adopts a mixed-methods approach, including systematic literature analysis and regional case studies. Highlights include AI-driven yield forecasting in European hydroponic systems and resource optimization in southeast Asian aquaponics, showcasing localized efficiency gains. Furthermore, AI applications in food processing, such as plasma, ozone and Pulsed Electric Field (PEF) treatments, are shown to improve food preservation and reduce spoilage. Key challenges—such as data quality, model scalability, and prediction accuracy—are discussed, particularly in the context of data-poor environments, limiting broader model applicability. The paper concludes by outlining future directions, emphasizing context-specific AI implementations, the need for public–private collaboration, and policy interventions to enhance scalability and adoption in food security contexts.

99 GENERAL AND MISCELLANEOUS↗

NASA NTRS2013

Automating Initial Guess Generation for High Fidelity Trajectory Optimization Tools

Many academic studies in spaceflight dynamics rely on simplified dynamical models, such as restricted three-body models or averaged forms of the equations of motion of an orbiter. In practice, the end result of these preliminary orbit studies needs to be transformed into more realistic models, in particular to generate good initial guesses for high-fidelity trajectory optimization tools like Mystic. This paper reviews and extends some of the approaches used in the literature to perform such a task, and explores the inherent trade-offs of such a transformation with a view toward automating it for the case of ballistic arcs. Sample test cases in the libration point regimes and small body orbiter transfers are presented.

design process↗

DOE OSTI2021

SWAT model application for evaluating agricultural conservation practice effectiveness in reducing phosphorous loss from the Western Lake Erie Basin

Lake Erie is threatened by eutrophication and harmful algal blooms due to excess nutrient loading from agricultural sources. Agricultural conservation practices (ACPs) have been developed and implemented to reduce nutrient losses but estimating ACP effectiveness is challenging. The Soil and Water Assessment Tool (SWAT) has been used to investigate ACP effectiveness for water quality improvement. Many SWAT applications have been developed by different investigators to evaluate ACP effectiveness for reducing nutrient, particularly phosphorus (P), loading in the agriculturally-dominated Western Lake Erie Basin (WLEB). Our objective is to establish what has been achieved by past modeling research and make suggestions for future applications and improvements. We synthesized the findings of 28 SWAT modeling studies within the WLEB. Models generally performed satisfactorily against accepted criteria for streamflow and sediment, but performance for P loads, like soluble reactive P, was mostly “unsatisfactory”. The “unsatisfactory” performance maybe due to imperfections and idealizations in model formulations and/or parameterization. Thus, simulations of P transport and transformation processes need improvement. In addition, model parameter selection is the key part of model set-up. Most SWAT modeling studies used default values during initial set-up, then performed calibration and validation. It was found that the calibrated P related parameter values varied widely across different studies, even within the same watershed with some values unrealistic for the study areas. The phenomena of different combinations of model parameters producing similar outputs indicates equifinality. Equifinality in the baseline model may impact results when ACPs are incorporated. Furthermore, the unrealistic values used in ACP assessment undermine the credibility of ACP effectiveness. Future model applications should try to re-examine the calibrated P parameters and make sure they are realistic for the study area as well as reduce equifinality by constraining the model with characterization of watershed conditions, better understanding of hydrologic processes, and parameter values based on real-world observations. As a result, future model applications should focus on improving P transport and transformation processes, using measured watershed characteristics for parameterization, and improving reflections of climate change, which could result in more accurate assessments of ACP effectiveness to meet targeted goals.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS1981

Daylight imagery of ocean surface waves for wave spectra

A simple second-order model is developed to describe the transformation between surface wave slope and radiance incident on an optical sensor. The model includes azimuth as well as elevation variations in the sky-radiance distribution and their effects on the imaging of wave slopes approaching the sensor azimuth from various directions. The model also includes upwelling radiance. Comparison between the exact transformation from wave slope to radiance and the second-order model suggests that they yield quantitatively similar results. The model is used in physical interpretations of how wave slopes are imaged and in indicating a way of optimizing the linearity and contrast of the transfer function relating wave slope to radiance for various sensor geometries, thereby minimizing sources of error. One-dimensional numerical simulations of the imaging of an analytically generated wave-slope profile demonstrate the utility of this technique in obtaining slope spectra.

Monaldo, F. M.↗

DOE OSTI2022

Elucidating the temperature dependence of TRIP in Q&P steels using synchrotron X-Ray diffraction, constituent phase properties, and strain-based kinetics models

Understanding the deformation-induced martensitic transformation (DIMT) is critical for interpreting the structure-property relationships that govern the performance of transformation-induced plasticity (TRIP) assisted steels. However, modern TRIP-assisted steels often exhibit DIMT kinetics that are not easily captured by existing empirical models based on bulk tensile strain. We address this challenge by combined bulk uniaxial tensile tests and in-situ high energy synchrotron X-ray diffraction, which resolved the phase volume fractions, stress-strain response, and microstructure evolution of each constituent phase. A modification of the Olson-Cohen model is implemented, which describes the martensitic transformation kinetics as a function of the estimated partitioned strain in austenite, rather than the bulk tensile strain. This DIMT kinetic model is used as a framework to clarify the root cause of an insufficiently understood toughness trough reported for TRIP-assisted steels during deformation at elevated temperatures. Here, the importance of the temperature-dependent toughness is discussed, based on the opportunity to modify deformation processes to tailor the DIMT kinetics and mechanical properties during forming and in service.

36 MATERIALS SCIENCE↗

DOE OSTI2023

Hybrid model to evaluate the frequency-dependent leakage inductance of partially-filled transformers

The leakage inductance of a transformer designed for a power electronic converter can drop significantly as the switching frequency is increased due to skin and proximity effects. Although the magnetic image method-based double-2-D model can predict the low-frequency leakage inductance of a partially-filled transformer with sufficient accuracy, it is inherently a frequency-independent model. While Dowell’s 1-D model uses frequency-dependent relations to account for both skin and proximity effects, its accuracy is severely affected by the assumed winding geometry. In this paper, a semianalytical hybrid model is proposed that uses superposition to combine a modified Dowell’s model with the double-2-D model to predict the true leakage inductance of partially-filled transformers at any given frequency. All three conductor types—round, foil, and litz wire—are modeled and analyzed. The quasi-2-D model is further investigated on a variable inductance transformer (VIT) whose winding geometry can be modified mechanically to vary its leakage inductance. With less than 5 % error throughout, the semi-analytically evaluated leakage inductances are in excellent agreement with the finite element method (FEM) simulated and experimentally measured leakage inductances.

24 POWER TRANSMISSION AND DISTRIBUTION↗