Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,657 records · Page 92
All sourcesNASA NTRSDOE OSTI
NASA NTRS1999

Technical Report Series on Global Modeling and Data Assimilation: Filtering Techniques on a Stretched Grid General Circulation Model - Volume 16

This report documents the techniques used to filter quantities on a stretched grid general circulation model. Standard high-latitude filtering techniques (e.g., using an FFT (Fast Fourier Transformations) to decompose and filter unstable harmonics at selected latitudes) applied on a stretched grid are shown to produce significant distortions of the prognostic state when used to control instabilities near the pole. A new filtering technique is developed which accurately accounts for the non-uniform grid by computing the eigenvectors and eigenfrequencies associated with the stretching. A filter function, constructed to selectively damp those modes whose associated eigenfrequencies exceed some critical value, is used to construct a set of grid-spaced weights which are shown to effectively filter without distortion. Both offline and GCM (General Circulation Model) experiments are shown using the new filtering technique. Finally, a brief examination is also made on the impact of applying the Shapiro filter on the stretched grid.

Takacs, Lawrence L.↗
DOE OSTI2022

IMoFi - Intelligent Model Fidelity: Physics-Based Data-Driven Grid Modeling to Accelerate Accurate PV Integration (Final Report)

This report summarizes the work performed under a project funded by U.S. DOE Solar Energy Technologies Office (SETO) to use grid edge measurements to calibrate distribution system models for improved planning and grid integration of solar PV. Several physics-based data-driven algorithms are developed to identify inaccuracies in models and to bring increased visibility into distribution system planning. This includes phase identification, secondary system topology and parameter estimation, meter-to-transformer pairing, medium-voltage reconfiguration detection, determination of regulator and capacitor settings, PV system detection, PV parameter and setting estimation, PV dynamic models, and improved load modeling. Each of the algorithms is tested using simulation data and demonstrated on real feeders with our utility partners. The final algorithms demonstrate the potential for future planning and operations of the electric power grid to be more automated and data-driven, with more granularity, higher accuracy, and more comprehensive visibility into the system.

14 SOLAR ENERGY↗
DOE OSTI2022

IMoFi (Intelligent Model Fidelity): Physics-Based Data-Driven Grid Modeling to Accelerate Accurate PV Integration Updated Accomplishments

This report summarizes the work performed under a project funded by U.S. DOE Solar Energy Technologies Office (SETO), including some updates from the previous report SAND2022-0215, to use grid edge measurements to calibrate distribution system models for improved planning and grid integration of solar PV. Several physics-based data-driven algorithms are developed to identify inaccuracies in models and to bring increased visibility into distribution system planning. This includes phase identification, secondary system topology and parameter estimation, meter-to-transformer pairing, medium-voltage reconfiguration detection, determination of regulator and capacitor settings, PV system detection, PV parameter and setting estimation, PV dynamic models, and improved load modeling. Each of the algorithms is tested using simulation data and demonstrated on real feeders with our utility partners. The final algorithms demonstrate the potential for future planning and operations of the electric power grid to be more automated and data-driven, with more granularity, higher accuracy, and more comprehensive visibility into the system.

14 SOLAR ENERGY↗
DOE OSTI2024

A Hydropower facility as an Energy Water Signal Processor

In recent times, various efforts have been made to address the challenge of adequately representing hydropower systems in modeling frameworks, accounting for the lack of data to represent the multiple constraints in hydropower operation. This research is a pilot data-driven methodology for characterizing, classifying, and comparing the water-to-energy and energy-to-water signal transformations that hydropower facilities as signal processors accomplish. In this study, a Box Jenkins transfer function/noise model is used to identify the relationship between reservoir inflows and outflows. For examining the feasibility of this methodology, 5-minute fleet data for five storage and five run-of-river facilities was provided by the Tennessee Valley Authority (TVA) and transfer function models are developed. The influence of past inflow and outflow values on the current outflow decisions was investigated and summarized by examining the results of Box Jenkins methodology. Finally, dominance analysis was introduced to add value to the Box Jenkins model results and provide different stakeholders with a set of concepts to convey the functionality of hydropower.

13 HYDRO ENERGY↗
DOE OSTI2026

Least-cost Optimal Distribution Grid Expansion (LODGE): Utility Pilots

The Least-cost Optimal Distribution Grid Expansion (LODGE) model provides the optimal portfolio of distribution system upgrades—e.g., voltage regulators, feeder reconductoring, transformer upgrades and non-wires alternatives (NWA), such as strategic siting of storage and distributed generation—to interconnect distributed energy resources (DERs) and enable load growth. It can be used to assess grid infrastructure costs and explore policy and regulatory solutions for distribution planning and DER valuation.In 2025, Berkeley Lab conducted three pilot analyses to validate LODGE results with empirical utility data before the model’s first release in 2026. The pilots, done with utilities in Washington, Colorado, and New Mexico, provide examples that illustrate how the model works, what it can do, and the value of the analysis.

Heleno, Miguel↗
NASA NTRS2018

Real-Time Parameter Estimation for Flexible Aircraft

A method for estimating aeroelastic stability and control derivatives for flexible aircraft is developed and demonstrated using flight test data for the X-56A subscale demonstrator. The method uses the equation-error approach with frequency-domain data, and can be applied post-flight or in real time during flight. The non-dimensional aeroelastic forces and moments and the explanatory variables (including generalized displacement, rate, and acceleration states for the vibration modes) are estimated using a finite element model and onboard sensor measurements in both a least squares and Kalman filtering framework. The data are then transformed into the frequency domain for parameter estimation using equation error. This method can result in a more efficient analysis than with other iterative methods, and can leverage existing statistical tools for model structure determination, data collinearity detection, combining multiple maneuvers or prior information, and others to improve model quality.

Grauer, Jared A.↗
DOE OSTI2021

Data-driven Optimal Control Strategy for Virtual Synchronous Generator via Deep Reinforcement Learning Approach

This paper aims at developing a data-driven optimal control strategy for virtual synchronous generator (VSG) in the scenario where no expert knowledge or requirement for system model is available. Firstly, the optimal and adaptive control problem for VSG is transformed into a reinforcement learning task. Specifically, the control variables, i.e., virtual inertia and damping factor, are defined as the actions. Meanwhile, the active power output, angular frequency and its derivative are considered as the observations. Moreover, the reward mechanism is designed based on three preset characteristic functions to quantify the control targets: (1) maintaining the deviation of angular frequency within special limits; (2) preserving well-damped oscillations for both the angular frequency and active power output; (3) obtaining slow frequency drop in the transient process. Next, to maximize the cumulative rewards, a decentralized deep policy gradient algorithm, which features model-free and faster convergence, is developed and employed to find the optimal control policy. With this effort, a data-driven adaptive VSG controller can be obtained. By using the proposed controller, the inverter-based distributed generator can adaptively adjust its control variables based on current observations to fulfill the expected targets in model-free fashion. Finally, simulation results validate the feasibility and effectiveness of the proposed approach.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

FFT-based model for irradiated aggregate microstructures in concrete

The concrete biological shield of light water reactors is exposed to neutron and gamma irradiation throughout its lifetime, which results in the long-term degradation of the concrete’s mechanical properties. Under neutron irradiation, the concrete’s aggregates are subjected to radiation-induced volumetric expansion (RIVE), which strongly depends on the mineral content of the aggregate and exhibits the largest expansion in silicate-bearing minerals. In this work, the authors used the fast Fourier transform-based code Microstructure-Oriented Scientific Analysis of Irradiated Concrete (MOSAIC) in 2D to model the expansion of five different aggregates provided by the Japan Concrete Aging Management Program (JCAMP). Comparable rock specimens were irradiated at the JEEP-II test reactor. The model uses realistic aggregate microstructure reconstruction based on high-resolution characterization images. The model accounts for anisotropic RIVE, thermal expansion, and the associated initiation and propagation of damage. The RIVE models are calibrated based on expansion data in the literature. The authors assume that damage occurs exclusively at interfaces between the particles that compose an aggregate and that these interfaces also exhibit swelling. Using a micromechanical model, the evolution of Young’s modulus with RIVE is calculated for each aggregate and compared with Russian irradiation data. The modeled linear expansion agrees well with the experimentally measured expansion. Furthermore, the model also predicts that anisotropic RIVE and thermal expansion result in an earlier onset of damage with neutron fluence than in the isotropic case.

36 MATERIALS SCIENCE↗
DOE OSTI2023

UVCGAN: UNet Vision Transformer cycle-consistent GAN for unpaired image-to-image translation

Unpaired image-to-image translation has broad applications in art, design, and scientific simulations. One early breakthrough was CycleGAN that emphasizes one-to-one mappings between two unpaired image domains via generative-adversarial networks (GAN) coupled with the cycle-consistency constraint, while more recent works promote one-to-many mapping to boost diversity of the translated images. Motivated by scientific simulation and one-to-one needs, this work revisits the classic CycleGAN framework and boosts its performance to outperform more contemporary models without relaxing the cycle-consistency constraint. To achieve this, we equip the generator with a Vision Transformer (ViT) and employ necessary training and regularization techniques. Compared to previous best-performing models, our model performs better and retains a strong correlation between the original and translated image. An accompanying ablation study shows that both the gradient penalty and self-supervised pre-training are crucial to the improvement. To promote reproducibility and open science, the source code, hyperparameter configurations, and pre-trained model are available at https: //github.com/LS4GAN/uvcgan.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Variability and Diversity Load Model Tool [SWR-20-03]

The motivation for the development of this tool and the underlying algorithms and methods was to enable the development of high-temporal resolution, realistic time-series data for quasi-static time-series (QSTS) analysis of distribution systems. Often, aggregated load profile data for a distribution circuit is available (e.g. feeder loading data collected via SCADA at the utility substation) and, while this data is typically accurate it masks the considerable variability of the 100’s or 1000’s of individual loads connected on the circuit. This tool was developed to model both the increased variability expected for these individual loads (e.g. the load of a single distribution transformer connected to 8-12 houses) and the expected diversity between loads on the circuit. It is important to note that the difference in variability and diversity, in the context of this tool, is that variability modeling only adds representative variability due to disaggregated load characteristics (e.g. the presence in the load profile of loads turning off and on like an air conditioner/oven) while the average energy profile remains the same as the user supplied power profile. Diversity modeling generates multiple individual load profiles which, in aggregate, sum to the user supplied power profile. Diversity is effectively variability in the energy usage over longer periods of time than seen in the variability model. Put another way, variability modeling supplies the expected variability due to the operation of various end-use loads and diversity modeling supplies the usage differences due to human behavior, schedules, etc. This load modeling tool was developed for use in generating data for distribution systems. Modeling is summarized by two major functions: 1) taking low resolution load profiles and adding intra-seconds variability onto the profiles, and 2) taking a user supplied load profile and distribution factors and adding both diversity and variability to the user supplied profile.

Zhu, Xiangqi↗
DOE OSTI2025

Intercomparison of Deep Learning Model Architectures for Atmospheric River Prediction

With a rapid surge in the application of machine learning (ML) for a diverse range of tasks in climate science, the present study addresses a challenge for climate scientists when selecting the optimal ML or deep learning (DL) architecture for a given application. In particular, a DL intercomparison study was performed with a focus on forecasting the position of atmospheric rivers (ARs) on short-range time scales (up to 5-day lead times). AR predictions from multiple DL architectures, including various types of convolutional autoencoders and a vision transformer (ViT), were compared against ECMWF ERA5 reanalysis and hindcasts from a global climate model. DL models with similar trainable parameters were trained on ERA5 reanalysis data and AR positions derived from a thresholding algorithm to ensure a fair comparison among the DL models. Each model’s performance and accuracy in forecasting AR location and key input fields within a 5-day window were assessed using metrics of root-mean-square error, anomaly correlation, and mean intersection over union. The ViT architecture outperformed other autoencoder models in most of the metrics. Incorporating additional meteorological fields only yielded slight improvements in forecasting certain fields at longer lead times. The results also suggest that a smaller number of input time steps or smaller number of autoregressive steps can achieve better prediction skills, while also improving the overall computational efficiency. This research offers valuable insights into the strengths and weaknesses of different DL techniques for AR forecasting, hopefully guiding the development of improved models for forecasting this phenomenon.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS1984

Group-kinetic scaling as a basis for modeling large scale turbulence

It was found that new concepts and methods were needed to develop a statistical theory of turbulence. The group kinetic method was devised for the transformation of the dynamical system into a kinetic system as the basis for a statistical treatment of turbulence. Spectral structure is used as a basis for turbulence modeling.

Tchen, C. M.↗
NASA NTRS1985

The role of microstructure and phase distribution in the failure mechanisms and life prediction model for PSZ coatings

Partially Stabilized Zirconia (PSZ) may become widely used for Thermal Barrier Coatings (TBC). Failure of these coatings can occur due to thermal fatigue in oxidizing atmospheres. The failure is due to the strains that develop due to thermal gradients, differences in thermal expansion coefficients, and oxidation of the bond coating. The role of microstructure and the cubic, tetragonal, and monoclinic phase distribution in the strain development and subsequent failure will be discussed. An X-ray diffraction technique for accurate determination of the fraction of each phase in PSZ will be applied to understanding the phase transformations and strain development. These results will be discussed in terms of developing a model for life prediction in PSZ coatings during thermal cycling.

Sisson, R. D., Jr.↗
NASA NTRSDate not supplied

Additional Metadata Guidelines to Improve the Structure and Usability of HDF and NetCDF Files

The Hierarchical Data Format (HDF) and Network Common Data Form (NetCDF) are data file formats created to aid users in the creation or use of scientific data. These file formats are useful for handling large data volumes and hosting extensive metadata as global attributes or variables and are popular with the modeling community. HDF and NetCDF files are largely used with remote sensing data and have been used to support measurements from numerous campaigns, from satellite to aircraft or ground and mobile based measurements. The files from airborne field studies, however, vary greatly in terms of the file structure and the amount and content of metadata. Information relevant to the file that can be useful to the user such as the data producer, location where data was taken, variable descriptions, or information about the instrument might not be included in the file. This metadata might be present in another file in the dataset containing the same data using the International Consortium for Atmospheric Research on Transport and Transformation (ICARTT) format. Recently, the Aerosols, Clouds, and their Interactions for Earth System Models (MACIE) group started a grassroots effort to develop a set of requirements for the HDF and NetCDF files for field studies, aiming to make the data products more interoperable and usable. Particularly, these requirements seek to make the files more compliant to Climate and Forecast (CF) metadata conventions and to standardize the file structure and the global and variable attributes. These requirements would help to ensure that HDF and NetCDF files contain adequate metadata to better support their use for research, e.g., the modeling community, and to enhance the usability and interoperability of data for research communities at large. To be presented are the details of the MACIE requirements as well as examples of the implementation of these requirements for merge files and lidar observation data files.

Sean Leavor↗
DOE OSTI2022

Variational Adiabatic Gauge Transformation on Real Quantum Hardware for Effective Low-Energy Hamiltonians and Accurate Diagonalization

Effective low-energy theories represent powerful theoretical tools to reduce the complexity in modeling interacting quantum many-particle systems. However, common theoretical methods rely on perturbation theory, which limits their applicability to weak interactions. Here we introduce the Variational Adiabatic Gauge Transformation (VAGT), a nonperturbative hybrid quantum algorithm that can use nowadays quantum computers to learn the variational parameters of the unitary circuit that brings the Hamiltonian to either its block-diagonal or full-diagonal form. If a Hamiltonian can be diagonalized via a shallow quantum circuit, then VAGT can learn the optimal parameters using a polynomial number of runs. The accuracy of VAGT is tested through numerical simulations, as well as simulations on Rigetti and IonQ quantum computers.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Distributed Cross-Channel Hierarchical Aggregation for Foundation Models

Vision-based scientific foundation models hold significant promise for advancing scientific discovery and innovation. This potential stems from their ability to aggregate images from diverse sources—such as varying physical groundings or data acquisition systems—and to learn spatio-temporal correlations using transformer architectures. However, tokenizing and aggregating images can be compute-intensive, a challenge not fully addressed by current distributed methods. In this work, we introduce the Distributed Cross-Channel Hierarchical Aggregation (D-CHAG) approach designed for datasets with a large number of channels across image modalities. Our method is compatible with any model-parallel strategy and any type of vision transformer architecture, significantly improving computational efficiency. We evaluated D-CHAG on hyperspectral imaging and weather forecasting tasks. When integrated with tensor parallelism and model sharding, our approach achieved up to a 75% reduction in memory usage and more than doubled sustained throughput on up to 1,024 AMD GPUs on the Frontier Supercomputer.

Tsaris, Aristeidis (aris) [ORNL] (ORCID:0000000277↗
NASA NTRS1998

Wavelet Filtering to Reduce Conservatism in Aeroservoelastic Robust Stability Margins

Wavelet analysis for filtering and system identification was used to improve the estimation of aeroservoelastic stability margins. The conservatism of the robust stability margins was reduced with parametric and nonparametric time-frequency analysis of flight data in the model validation process. Nonparametric wavelet processing of data was used to reduce the effects of external desirableness and unmodeled dynamics. Parametric estimates of modal stability were also extracted using the wavelet transform. Computation of robust stability margins for stability boundary prediction depends on uncertainty descriptions derived from the data for model validation. F-18 high Alpha Research Vehicle aeroservoelastic flight test data demonstrated improved robust stability prediction by extension of the stability boundary beyond the flight regime.

Brenner, Marty↗