Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

A Novel Power-Hardware-in-the-Loop Interface Method for Grid-forming Inverter Systems: Preprint

Power Hardware-in-the-Loop (PHIL) simulation of grid-forming (GFM) inverter systems facilitates the testing of drastic scenarios like on-grid to off-grid transition, islanded microgrid operation without stiff grid etc. To the authors best knowledge, most of studies in literature are focused on PHIL simulation for grid-following inverter systems and only few studies are focused on GFM inverters and those are challenging and problematic especially for high-power applications. In this article, a novel PHIL simulation platform is proposed that enables interfacing of high-power GFM inverter systems. It proposes the concept of a virtual GFM inverter as a part of the proposed PHIL interface for GFM inverter. This addition of virtual GFM inverter in the PHIL interface expands the conventional Ideal Transformer Model (ITM) method and enables it to overcome the issues of instability of existing ITM methods. In the validation stage, a PHIL experiment is conducted on a 3-phase 480 V, 125 kVA GFM inverter system with proposed interfacing method. The results corroborates the fact that the proposed PHIL simulation method performs well and stable for GFM inverter system.

grid-forming inverters↗
DOE OSTI2022

Resilient Inverter-Driven Black Start with Collective Parallel Grid-Forming Operation: Preprint

As the modern power systems are experiencing exceptional changes with increasing penetration of inverter-based resources (IBRs), system restoration using IBRs has received attention. Using local grid-forming (GFM) assets near consumers, engineered to establish grid voltages in the absence of a stiff grid, i.e., bottom-up restoration, a distribution system would obtain high system resilience, by not relying on the bulk power system restoration requiring significant human intervention and procedure to restore. This paper studies the technical feasibility of the novel approach with detailed electromagnetic transient (EMT) simulations. To thoroughly evaluate the potential of GFM inverters and technical challenges in IBR-driven black start, a detailed three-phase inverter model is developed, including negative-sequence control for voltage balance and a phase-by-phase current limiter to sustain momentary overloading during the black start. To examine dynamic aspects of the black start process, the EMT simulation also models transformer and motor dynamics to emulate their inrush and start-up behaviors as well as network dynamics. In addition, active involvement of grid-following distributed energy resources (DER) is also studied to facilitate the black start process. It is shown that, by allowing multiple GFM inverters to collectively black start without master-slave coordination, a system can achieve high resilience even with a fraction of assets lost. Two test cases of inverter-driven black start using two and one GFM inverters, respectively, for a heavily unbalanced 2-MVA distribution feeder are demonstrated. Takeaways for further study and field deployment are provided.

grid-forming inverter↗
DOE OSTI2025

Airport Delay Prediction with Temporal Fusion Transformers

Since flight delay hurts passengers, airlines, and airports, its prediction becomes crucial for the decision-making of all stakeholders in the aviation industry and thus has been attempted by various previous research. However, previous delay predictions are often categorical and at a highly aggregated level. To improve that, this study proposes to apply the novel Temporal Fusion Transformer model and predict numerical airport arrival delays at quarter hour level for U.S. top 30 airports. Inputs to our model include airport demand and capacity forecasts, historic airport operation efficiency information, airport wind and visibility conditions, as well as en-route weather and traffic conditions. The results show that our model achieves satisfactory performance measured by small prediction errors on the test set. In addition, the interpretability analysis of the model outputs identifies the important input factors for delay prediction.

Liu, Ke [University of California Berkeley]↗
DOE OSTI2022

Multimodal imaging of hemorrhagic transformation biomarkers in an ischemic stroke model.

Abstract Hemorrhagic transformation of ischemic stroke has devastating consequences, with high mortality and poor functional outcomes. Animal models of ischemic stroke also demonstrate the potential for hemorrhagic transformation, which complicates biochemical characterization, treatment studies, and hinders poststroke functional outcomes in affected subjects. The incidence of hemorrhagic transformation of ischemic stroke in animal model research is not commonly reported. The postmortem brain of such cases presents a complex milieu of biomarkers due to the presence of healthy cells, regions of varying degrees of ischemia, dead and dying cells, dysregulated metabolites, and blood components (especially reactive Fe species released from lysed erythrocytes). To improve the characterization of hemorrhage biomarkers on an ischemic stroke background, we have employed a combination of histology, X-ray fluorescence imaging (XFI), and Fourier transform infrared (FTIR) spectroscopic imaging to assess 122 photothrombotic (ischemic) stroke brains. Rapid freezing preserves brain biomarkers in situ and minimizes metabolic artifacts due to postmortem ischemia. Analysis revealed that 25% of the photothrombotic models had clear signs of hemorrhagic transformation. The XFI and FTIR metabolites provided a quantitative method to differentiate key metabolic regions in these models. Across all hemorrhage cases, it was possible to consistently differentiate otherwise healthy tissue from other metabolically distinct regions, including the ischemic infarct, the ischemic penumbra, blood vessels, sites of hemorrhage, and a region surrounding the hemorrhage core that contained elevated lipid oxidation. Chemical speciation of deposited Fe demonstrates the presence of heme-Fe and accumulation of ferritin.

Pushie, M. J. (ORCID:0000000174945427)↗
DOE OSTI2026

Summary of Responses to the Request for Information (RFI) on Partnerships for Transformational Artificial Intelligence Models

The Department of Energy (DOE) issued a Request for Information (RFI) in December 2025 inviting public comments regarding partnerships for transformational Artificial Intelligence (AI) models for the Genesis Mission Consortium, a public-private partnership platform. This RFI solicited feedback from industry, nonprofit organizations, universities, independent research organizations and other stakeholders. Specifically, the RFI asked three questions on (1) mobilizing DOE National Laboratories to curate the scientific data in a responsible and privacy-preserving manner, (2) the extent to which existing general-purpose AI models can be leveraged and which scientific disciplines are priorities for such model development, and (3) mechanisms by which these AI models can be provided to scientific communities. This document summarizes the input from 194 unique nonproprietary responses from businesses, universities, nonprofit organizations, research institutes and laboratories as well as a variety of other contributors, including individual contributions.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1983

A numerical method based on the Fourier-Fourier transform approach for modeling 1-D electron plasma evolution

A numerical method is presented for studying one-dimensional electron plasma evolution under typical interplanetary conditions. The method applies the Fourier-Fourier transform approach to a plasma model that is a generalization of the electrostatic Vlasov-Poisson system of equations. Conservation laws that are modified to include the plasma model generalization and also the boundary effects of nonperiodic solutions are given. A new conservation law for entropy in the transformed space is then introduced. These conservation laws are used to verify the numerical solutions. A discretization error analysis is presented. Two numerical instabilities and the methods used for their suppression are treated. It is shown that in interplanetary plasma conditions, the bump-on-tail instability produces significant excitation of plasma oscillations at the Bohm-Gross frequency and its second harmonic. An explanation of the second harmonic excitation is given in terms of wave-wave coupling during the growth phase of the instability.

Klimas, A. J.↗
DOE OSTI2025

SOHIP Abel Transform and Onion Peeling Model Module

This software provides tools for analyzing and modeling physical systems using mathematical transforms and layered models. It includes (1) functions for performing the Abel transform, which is used to relate measurements of bending angles to properties such as refractive index and radius in a medium. The code can compute bending angles from input profiles and also reconstruct these profiles from observed data; (2) the functions for modeling systems with multiple layers using an onion-peeling approach, allowing users to simulate and analyze the behavior of layered materials or structures. These capabilities are useful for researchers and engineers working in fields such as optics, atmospheric science, and materials analysis, enabling them to interpret and model data from experiments or simulations relates to refraction in spherical symmetric medium.

Xu, Shuang [Lawrence Livermore National Laboratory↗
DOE OSTI2020

Residual Stress Modeling with Phase Transformation for Wire Arc Additive Manufacturing of B91 Steel

Wire arc additive manufacturing (WAAM) is an energy-efficient manufacturing technique used for near-net-shape production of functional industrial components. However, heat accumulation during deposition and the associated mechanical and metallurgical changes result in complex residual stress profiles across the cross section of the fabricated components. These residual stresses are detrimental to the service life of the components. In this study, sequentially coupled thermomechanical analysis of WAAM B91 steel is conducted to quantify the residual stress variation across the component. The thermomechanical analysis includes a transient heat transfer model and a static stress model that incorporates the transformation-induced plasticity due to martensitic phase transformation. Here, the experimentally calibrated heat transfer model mirrors the temperature variation of the system during the deposition. The results from the stress model are validated via x-ray diffraction measurements, and the numerical results are in good agreement with the experimental data.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2022

New Hybrid Model for Evaluating the Frequency-Dependent Leakage Inductance of a Variable Inductance Transformer (VIT)

Skin and proximity effects can cause a significant drop in the effective leakage inductance of a transformer when the operating frequency is increased. Although the magnetic image method-based double-2-D model can calculate the lowfrequency leakage inductance with sufficient accuracy, it is inherently a frequency-independent model. While Dowell’s 1-D model uses frequency-dependent relations to account for both skin and proximity effects, its accuracy is severely affected by the assumed winding geometry. In this paper, a hybrid model is proposed that uses superposition to combine a modified Dowell’s model with the double-2-D model. The proposed model is investigated on a variable inductance transformer (VIT)—a partially-filled transformer whose leakage inductance can be varied by moving one of the windings mechanically. The frequency-dependent leakage inductances of the VIT evaluated using the hybrid model are in excellent agreement with the corresponding finite element method (FEM) simulated and experimentally measured values, thereby validating the proposed hybrid model.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRSDate not supplied

Towards an Aviation Large Language Model by Fine-tuning and Evaluating Transformers

In the aviation domain, there are many applications for machine learning and artificial intelligence tools that utilize natural language. For example, there is a desire to know the commonalities in written safety reports such as voluntary post incidents reports or aerial wildfire operations reports to better understand the risks present. Another use-case is the possibility of extracting airspace procedures and constraints currently written in documents such as Letters of Agreement. These applications can benefit from the use of state-of-the-art natural language processing techniques when adapted to the language/phraseology specific to the aviation domain. This paper evaluates the viability of adaptation of NLP tools to the aviation domain by fine-tuning transformer based models using aviation data sets. In 2018, a novel language model based on neural units (also called transformers) was created and became known as “Bidirectional Encoder Representations from Transformers” or BERT. This architecture combined with large amounts of English training data and innovative semi-supervised training tasks set the standard for what would later emerge as Large Language Models. The performance of these models was further improved by hyperparameter tuning and refinement of the semi-supervised training task and resulted in “Robustly Optimized BERT Pre-training Approach through hyperparameter tuning” or RoBERTa models. These pre-trained Large Language Models proved to be useful for a wide variety of natural language processing tasks such as text classification and question answering through a process called fine-tuning. The transformer architecture with pre-trained weights served as the basis with the last few layers replaced with layers fine-tuned to perform a new task e.g., a layer that provides a label for the entire input text. This process of fine-tuning can also be used to adapt the models to new domains; e.g., BioBERT started with the pre-trained BERT model and was completed by additional fine-tuning and training on biomedical documents. Transformer-based architectures can also be used to create rich representations of text called embeddings which can serve as the input to other machine learning models. This allows simpler algorithms such as logistic regression to use context-rich representations of the text while still remaining quick to train and evaluate. In the world of aviation, there is a growing demand for natural language processing and understanding but the domain presents unique challenges. Due to the technical content (and specialized language) of most aviation documents, fine-tuning pre-trained Large Language Models to specific tasks has not met the benchmark on natural language processing tasks set by simpler models trained from scratch on the data. To address this deficiency, this paper evaluates the improvements from fine-tuning a Large Language Model on a large set of aviation documents using the original semi-supervised training tasks before performing specific natural language tasks. In fine-tuning, a domain-specific dataset is used on the original training task but with the pre-trained Large Language Model instead of starting from a random initialization. This approach allows the model to be adapted to the specific domain language without discarding the information gained from training on general English data. This paper utilized two major dataset types to train and assess the RoBERTa fine-tuning performance. The first are 7,057 Letters of Agreement which are Federal Aviation Administration (FAA) documents that formalize airspace operations across the national airspace system. They contain many examples of ‘aviation English’ using domain specific terminology and phrasing which serves as a representative basis to perform the semi-supervised fine-tuning. The second type is the 494 document classification labels to be used for evaluation. This down-stream evaluation aims to show the performance of the fine-tuned model, better understand how much data is needed for an effective fine-tuning, and how fine-tuning can be adapted for different applications in-the domain. After semi-supervised training, evaluation begins by encoding the documents for classification using the fine-tuned RoBERTa model. Then a logistic regression classifier is trained to label the document type and compared against our ground truth labels. This currently leads to a 82.8% accuracy on 10-fold cross validation showing improvement over baseline RoBERTa which achieved 81.0%. We plan to measure the improvements on additional tasks and it is expected that these improvements will lead to more robust models that can tackle the natural language processing challenges present in aviation datasets.

ATM↗
NASA NTRS1998

On the Numerical Formulation of Parametric Linear Fractional Transformation (LFT) Uncertainty Models for Multivariate Matrix Polynomial Problems

Robust control system analysis and design is based on an uncertainty description, called a linear fractional transformation (LFT), which separates the uncertain (or varying) part of the system from the nominal system. These models are also useful in the design of gain-scheduled control systems based on Linear Parameter Varying (LPV) methods. Low-order LFT models are difficult to form for problems involving nonlinear parameter variations. This paper presents a numerical computational method for constructing and LFT model for a given LPV model. The method is developed for multivariate polynomial problems, and uses simple matrix computations to obtain an exact low-order LFT representation of the given LPV system without the use of model reduction. Although the method is developed for multivariate polynomial problems, multivariate rational problems can also be solved using this method by reformulating the rational problem into a polynomial form.

Belcastro, Christine M.↗
DOE OSTI2021

Accelerating Transformer-based Deep Learning Models on FPGAs using Column Balanced Block Pruning

Although Transformer-based language representations achieve state-of-the-art accuracy on various natural language processing (NLP) tasks, the large model size has been challenging the resource constrained computing platforms. Weight pruning, as a popular and effective technique in reducing the number of weight parameters and accelerating the Transformer, has been investigated on GPUs. However, the Transformer acceleration using weight pruning on field-programmable gate array (FPGAs) remains unexplored. This paper investigates the column balanced block-wise pruning on Transformer and designs an FPGA acceleration engine to customize the balanced blockwise matrix multiplication. We implement the Transformer model with proper hardware scheduling, and the experiments show that the Transformer inference on FPGA achieves 10.35 ms latency with the batch size of 32, which is 10.96 × speed up comparing to CPU platform and 2.08 × speed up comparing to GPU platform.

Peng, Hongwu↗
NASA NTRS2012

Getting a Cohesive Answer from a Common Start: Scalable Multidisciplinary Analysis through Transformation of a Systems Model

One of the challenges of systems engineering is in working multidisciplinary problems in a cohesive manner. When planning analysis of these problems, system engineers must trade between time and cost for analysis quality and quantity. The quality often correlates with greater run time in multidisciplinary models and the quantity is associated with the number of alternatives that can be analyzed. The trade-off is due to the resource intensive process of creating a cohesive multidisciplinary systems model and analysis. Furthermore, reuse or extension of the models used in one stage of a product life cycle for another is a major challenge. Recent developments have enabled a much less resource-intensive and more rigorous approach than hand-written translation scripts between multi-disciplinary models and their analyses. The key is to work from a core systems model defined in a MOF-based language such as SysML and in leveraging the emerging tool ecosystem, such as Query/View/Transformation (QVT), from the OMG community. SysML was designed to model multidisciplinary systems. The QVT standard was designed to transform SysML models into other models, including those leveraged by engineering analyses. The Europa Habitability Mission (EHM) team has begun to exploit these capabilities. In one case, a Matlab/Simulink model is generated on the fly from a system description for power analysis written in SysML. In a more general case, symbolic analysis (supported by Wolfram Mathematica) is coordinated by data objects transformed from the systems model, enabling extremely flexible and powerful design exploration and analytical investigations of expected system performance.

Operational QVT↗
DOE OSTI2026

Towards Generalizable and Efficient Circuit Topology Design: A Graph-Transformer-based Surrogate Model with Curriculum Learning

Unlike circuit parameter and sizing optimizations, the automated design of analog circuit topologies poses significant challenges for learning-based approaches. One challenge arises from the combinatorial growth of the topology space with circuit size, which limits the topology optimization efficiency. Moreover, traditional circuit evaluation methods are time-consuming, while the presence of data discontinuity in the topology space makes the accurate prediction of circuit performance exceptionally difficult for unseen topologies. To tackle these challenges, we design a novel Graph-Transformer-based Network (GTN) as the surrogate model for circuit evaluation, offering a substantial acceleration in the speed of circuit topology optimization without sacrificing performance. Our GTN model architecture is designed to embed voltage changes in circuit loops and current flows in connected devices, enabling accurate performance predictions for circuits with unseen topologies. To address the cold start problem when scaling GTN to large-scale circuits, we further introduce a curriculum learning strategy that progressively trains GTN from small-scale to large-scale circuits. This approach enables the model to first learn fundamental physical principles from simpler topologies and gradually adapt to complex configurations, effectively bridging the circuit complexity gap and improving prediction accuracy. Taking the power converter circuit design as an experimental task, our GTN model significantly outperforms an analytical approach and baseline methods directly utilizing graph neural networks. Furthermore, GTN achieves less than 5% relative error and 196× speed-up compared with high-fidelity simulation. Notably, our GTN surrogate model empowers an automatic circuit design framework to discover circuits of comparable quality to those identified through high-fidelity simulation while reducing the time required by up to 98.2%. With curriculum learning, the enhanced GTN achieves a 51% improvement for performance prediction of large-scale circuits compared to the GTN model without this strategy. These advancements establish GTN as a scalable framework for automated analog circuit design across varying circuit complexity levels.

Lu, Haoshu [New Jersey Institute of Technology (NJ↗
DOE OSTI2020

Modeling Failure of Electrical Transformers due to Effects of a HEMP Event

Understanding the effect of a high-altitude electromagnetic pulse (HEMP) on the equipment in the United States electrical power grid is important to national security. A present challenge to this understanding is evaluating the vulnerability of transformers to a HEMP. Evaluating vulnerability by direct testing is cost-prohibitive, due to the wide variation in transformers, their high cost, and the large number of tests required to establish vulnerability with confidence. Alternatively, material and component testing can be performed to quantify a model for transformer failure, and the model can be used to assess vulnerability of a wide variety of transformers. This project develops a model of the probability of equipment failure due to effects of a HEMP. Potential failure modes are cataloged, and a model structure is presented which can be quantified by the results of small-scale coupon tests.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS1988

A kinematic model of ridge-transform geometry evolution

A simple kinematic model is used to study the effects of various parameters on the evolution of zero-offset transforms and very-long-offset transforms. Consideration is given to the effects of initial configuration, degree of asymmetry, and degree of bias in asymmetry on the generation of these ridge transform geometries and on the possible steady-state nature of the transform length spectra. Of the parameters tests, only lack of 'memory' of zero-offset transforms affects transform length distribution.

Stoddard, Paul R.↗
NASA NTRS2010

A Final Approach Trajectory Model for Current Operations

Predicting accurate trajectories with limited intent information is a challenge faced by air traffic management decision support tools in operation today. One such tool is the FAA's Terminal Proximity Alert system which is intended to assist controllers in maintaining safe separation of arrival aircraft during final approach. In an effort to improve the performance of such tools, two final approach trajectory models are proposed; one based on polynomial interpolation, the other on the Fourier transform. These models were tested against actual traffic data and used to study effects of the key final approach trajectory modeling parameters of wind, aircraft type, and weight class, on trajectory prediction accuracy. Using only the limited intent data available to today's ATM system, both the polynomial interpolation and Fourier transform models showed improved trajectory prediction accuracy over a baseline dead reckoning model. Analysis of actual arrival traffic showed that this improved trajectory prediction accuracy leads to improved inter-arrival separation prediction accuracy for longer look ahead times. The difference in mean inter-arrival separation prediction error between the Fourier transform and dead reckoning models was 0.2 nmi for a look ahead time of 120 sec, a 33 percent improvement, with a corresponding 32 percent improvement in standard deviation.

Gong, Chester↗
DOE OSTI2023

Transformer quantum state: A multipurpose model for quantum many-body problems

Here, inspired by the advancements in large language models based on transformers, we introduce the transformer quantum state (TQS): a versatile machine learning model for quantum many-body problems. In sharp contrast to Hamiltonian/task specific models, TQS can generate the entire phase diagram, predict field strengths with experimental measurements, and transfer such a knowledge to new systems it has never been trained on before, all within a single model. With specific tasks, fine-tuning the TQS produces accurate results with small computational cost. Versatile by design, TQS can be easily adapted to new tasks, thereby pointing towards a general-purpose model for various challenging quantum problems.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗