Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

FFT-based model for irradiated aggregate microstructures in concrete

The concrete biological shield of light water reactors is exposed to neutron and gamma irradiation throughout its lifetime, which results in the long-term degradation of the concrete’s mechanical properties. Under neutron irradiation, the concrete’s aggregates are subjected to radiation-induced volumetric expansion (RIVE), which strongly depends on the mineral content of the aggregate and exhibits the largest expansion in silicate-bearing minerals. In this work, the authors used the fast Fourier transform-based code Microstructure-Oriented Scientific Analysis of Irradiated Concrete (MOSAIC) in 2D to model the expansion of five different aggregates provided by the Japan Concrete Aging Management Program (JCAMP). Comparable rock specimens were irradiated at the JEEP-II test reactor. The model uses realistic aggregate microstructure reconstruction based on high-resolution characterization images. The model accounts for anisotropic RIVE, thermal expansion, and the associated initiation and propagation of damage. The RIVE models are calibrated based on expansion data in the literature. The authors assume that damage occurs exclusively at interfaces between the particles that compose an aggregate and that these interfaces also exhibit swelling. Using a micromechanical model, the evolution of Young’s modulus with RIVE is calculated for each aggregate and compared with Russian irradiation data. The modeled linear expansion agrees well with the experimentally measured expansion. Furthermore, the model also predicts that anisotropic RIVE and thermal expansion result in an earlier onset of damage with neutron fluence than in the isotropic case.

36 MATERIALS SCIENCE↗
DOE OSTI2023

UVCGAN: UNet Vision Transformer cycle-consistent GAN for unpaired image-to-image translation

Unpaired image-to-image translation has broad applications in art, design, and scientific simulations. One early breakthrough was CycleGAN that emphasizes one-to-one mappings between two unpaired image domains via generative-adversarial networks (GAN) coupled with the cycle-consistency constraint, while more recent works promote one-to-many mapping to boost diversity of the translated images. Motivated by scientific simulation and one-to-one needs, this work revisits the classic CycleGAN framework and boosts its performance to outperform more contemporary models without relaxing the cycle-consistency constraint. To achieve this, we equip the generator with a Vision Transformer (ViT) and employ necessary training and regularization techniques. Compared to previous best-performing models, our model performs better and retains a strong correlation between the original and translated image. An accompanying ablation study shows that both the gradient penalty and self-supervised pre-training are crucial to the improvement. To promote reproducibility and open science, the source code, hyperparameter configurations, and pre-trained model are available at https: //github.com/LS4GAN/uvcgan.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Variability and Diversity Load Model Tool [SWR-20-03]

The motivation for the development of this tool and the underlying algorithms and methods was to enable the development of high-temporal resolution, realistic time-series data for quasi-static time-series (QSTS) analysis of distribution systems. Often, aggregated load profile data for a distribution circuit is available (e.g. feeder loading data collected via SCADA at the utility substation) and, while this data is typically accurate it masks the considerable variability of the 100’s or 1000’s of individual loads connected on the circuit. This tool was developed to model both the increased variability expected for these individual loads (e.g. the load of a single distribution transformer connected to 8-12 houses) and the expected diversity between loads on the circuit. It is important to note that the difference in variability and diversity, in the context of this tool, is that variability modeling only adds representative variability due to disaggregated load characteristics (e.g. the presence in the load profile of loads turning off and on like an air conditioner/oven) while the average energy profile remains the same as the user supplied power profile. Diversity modeling generates multiple individual load profiles which, in aggregate, sum to the user supplied power profile. Diversity is effectively variability in the energy usage over longer periods of time than seen in the variability model. Put another way, variability modeling supplies the expected variability due to the operation of various end-use loads and diversity modeling supplies the usage differences due to human behavior, schedules, etc. This load modeling tool was developed for use in generating data for distribution systems. Modeling is summarized by two major functions: 1) taking low resolution load profiles and adding intra-seconds variability onto the profiles, and 2) taking a user supplied load profile and distribution factors and adding both diversity and variability to the user supplied profile.

Zhu, Xiangqi↗
DOE OSTI2025

Intercomparison of Deep Learning Model Architectures for Atmospheric River Prediction

With a rapid surge in the application of machine learning (ML) for a diverse range of tasks in climate science, the present study addresses a challenge for climate scientists when selecting the optimal ML or deep learning (DL) architecture for a given application. In particular, a DL intercomparison study was performed with a focus on forecasting the position of atmospheric rivers (ARs) on short-range time scales (up to 5-day lead times). AR predictions from multiple DL architectures, including various types of convolutional autoencoders and a vision transformer (ViT), were compared against ECMWF ERA5 reanalysis and hindcasts from a global climate model. DL models with similar trainable parameters were trained on ERA5 reanalysis data and AR positions derived from a thresholding algorithm to ensure a fair comparison among the DL models. Each model’s performance and accuracy in forecasting AR location and key input fields within a 5-day window were assessed using metrics of root-mean-square error, anomaly correlation, and mean intersection over union. The ViT architecture outperformed other autoencoder models in most of the metrics. Incorporating additional meteorological fields only yielded slight improvements in forecasting certain fields at longer lead times. The results also suggest that a smaller number of input time steps or smaller number of autoregressive steps can achieve better prediction skills, while also improving the overall computational efficiency. This research offers valuable insights into the strengths and weaknesses of different DL techniques for AR forecasting, hopefully guiding the development of improved models for forecasting this phenomenon.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Variational Adiabatic Gauge Transformation on Real Quantum Hardware for Effective Low-Energy Hamiltonians and Accurate Diagonalization

Effective low-energy theories represent powerful theoretical tools to reduce the complexity in modeling interacting quantum many-particle systems. However, common theoretical methods rely on perturbation theory, which limits their applicability to weak interactions. Here we introduce the Variational Adiabatic Gauge Transformation (VAGT), a nonperturbative hybrid quantum algorithm that can use nowadays quantum computers to learn the variational parameters of the unitary circuit that brings the Hamiltonian to either its block-diagonal or full-diagonal form. If a Hamiltonian can be diagonalized via a shallow quantum circuit, then VAGT can learn the optimal parameters using a polynomial number of runs. The accuracy of VAGT is tested through numerical simulations, as well as simulations on Rigetti and IonQ quantum computers.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2025

Distributed Cross-Channel Hierarchical Aggregation for Foundation Models

Vision-based scientific foundation models hold significant promise for advancing scientific discovery and innovation. This potential stems from their ability to aggregate images from diverse sources—such as varying physical groundings or data acquisition systems—and to learn spatio-temporal correlations using transformer architectures. However, tokenizing and aggregating images can be compute-intensive, a challenge not fully addressed by current distributed methods. In this work, we introduce the Distributed Cross-Channel Hierarchical Aggregation (D-CHAG) approach designed for datasets with a large number of channels across image modalities. Our method is compatible with any model-parallel strategy and any type of vision transformer architecture, significantly improving computational efficiency. We evaluated D-CHAG on hyperspectral imaging and weather forecasting tasks. When integrated with tensor parallelism and model sharding, our approach achieved up to a 75% reduction in memory usage and more than doubled sustained throughput on up to 1,024 AMD GPUs on the Frontier Supercomputer.

Tsaris, Aristeidis (aris) [ORNL] (ORCID:0000000277↗
DOE OSTI2025

Data and Code for Understanding Generative AI Content with Embedding Models

This repository contains code for the experiments in the paper "Understanding Generative AI Content with Embedding Models". Constructing high-quality features is critical to any quantitative data analysis. While feature engineering was historically addressed by carefully hand-crafting data representations based on domain expertise, deep neural networks (DNNs) now offer a radically different approach. DNNs implicitly engineer features by transforming their input data into hidden feature vectors called embeddings. For embedding vectors produced by foundation models -- which are trained to be useful across many contexts -- we demonstrate that simple and well-studied dimensionality-reduction techniques such as Principal Component Analysis uncover inherent heterogeneity in input data concordant with human-understandable explanations. Of the many applications for this framework, we find empirical evidence that there is intrinsic separability between real samples and those generated by artificial intelligence (AI).

Vargas, Max [Pacific Northwest National Laboratory↗
DOE OSTI2020

Lift & Learn: Physics-informed machine learning for large-scale nonlinear dynamical systems

In this work, we present Lift & Learn, a physics-informed method for learning low-dimensional models for large-scale dynamical systems. The method exploits knowledge of a system’s governing equations to identify a coordinate transformation in which the system dynamics have quadratic structure. This transformation is called a lifting map because it often adds auxiliary variables to the system state. The lifting map is applied to data obtained by evaluating a model for the original nonlinear system. This lifted data is projected onto its leading principal components, and low-dimensional linear and quadratic matrix operators are fit to the lifted reduced data using a least-squares operator inference procedure. Analysis of our method shows that the Lift & Learn models are able to capture the system physics in the lifted coordinates at least as accurately as traditional intrusive model reduction approaches. This preservation of system physics makes the Lift & Learn models robust to changes in inputs. Numerical experiments on the FitzHugh–Nagumo neuron activation model and the compressible Euler equations demonstrate the generalizability of our model.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Transformer Health Monitoring via Synchrophasors

Traditional transformer protection schemes such as differential protection operate once the transformer is in a critical state. Proactive means of transformer health evaluation such as Sweep Frequency Response Analysis, can only be performed periodically with the transformer out-of-service. This work presents a framework where the health and structural integrity of transformers is monitored via synchrophasors in real-time. Synchrophasors and subspace estimation techniques are used to build reduced-order models which are then analyzed across multiple domains. A diagnostics algorithm is developed and tested via Matlab simulations. Test results are promising, opening up new prospects in transformer protection research.

cyber-physical security↗
DOE OSTI2020

Competing Interactions between Mesoscale Length-Scales, Order-Disorder, and Martensitic Transformation in Ferromagnetic Shape Memory Alloys

In the present study, the effects of composition and heat treatments and the resulting microstructural changes on the martensitic transformation and ferromagnetic transition have been investigated in NiCoMnIn metamagnetic shape memory alloys. In this shape memory alloy system, it is observed that upon heat treatments at a wide temperature range, the onset temperature of the martensitic transformation follows a non-monotonic behavior with respect to heat treatment time, and the nature of the non-monotonic behavior is also a function of composition. This behavior cannot be attributed to well-known factors such as precipitation, change in local composition due to the precipitation and/or global degree of order. In this work, a systematic investigation through synthesis, thermal processing and characterization via thermo-physical measurements, transmission electron microscopy and in-situ synchrotron x-ray diffraction experiments has been used to correlate the non-monotonic dependence of the martensitic transformation and ferromagnetic transition temperatures to the evolution of L2 1 domains arising from a order/disorder phase transition. A thermodynamic model for the magneto-structural transition is combined with classical nucleation theory to further ascertain the role of microstructural length-scales on the onset of the martensitic transformation. This work thus provides understanding of the thermodynamic and kinetic factors that can be controlled to tune the coupled magneto-structural transformations in NiCoMnIn metamagnetic shape memory alloys.

36 MATERIALS SCIENCE↗
DOE OSTI2019

Koopman Operators for Generalized Persistence of Excitation Conditions for Nonlinear Systems

It is hard to identify nonlinear biological models strictly from data, with results that are often sensitive to experimental conditions. Automated experimental workflows and liquid handling enables unprecedented throughput, as well as the capacity to generate extremely large datasets. We seek to develop generalized identifiability conditions for informing the design of automated experiments to discover predictive nonlinear biological models. For linear systems, identifiability is characterized by persistence of excitation conditions. For nonlinear systems, no such persistence of excitation conditions exist. We use the input-Koopman operator method to model nonlinear systems and derive identifiability conditions for openloop systems initialized from a single initial condition. We show that nonlinear identifiability is intrinsically tied to the rank of a given dataset’s power spectral density, transformed through the lifted Koopman observable space. We illustrate these identifiability conditions with a simulated synthetic gene circuit model, the repressilator. We illustrate how rank degeneracy in datasets results in overfitted nonlinear models of the repressilator, resulting in poor predictive accuracy. Our findings provide novel experimental design criteria for discovery of globally predictive nonlinear models of biological phenomena.

Boddupalli, Nibodh↗
DOE OSTI2025

Wave-Optics Simulation Framework of Fourier Transform Holography with XMCD

We present a wave-optics simulation framework, implemented within the SRW, for modeling FTH with XMCD contrast. The framework propagates coherent, circularly polarized X-ray wavefronts from an undulator source through the sample and optics to the detector, reconstructing real-space images from the resulting holograms. Three principal extensions are introduced: (i) incorporation of polarization-dependent refractive indices in the sample plane, enabling direct simulation of XMCD contrast, (ii) a memory-efficient wavefront-splitting propagation scheme that treats individual apertures independently and coherently combines their fields at detector, and (iii) an integrated reconstruction module that delivers holographic images within the same framework. Together, these advances establish a versatile tool for quantitative exploration of coherence, aperture geometry, and detector sampling, and for the design and interpretation of XMCD-FTH experiments at synchrotron beamlines.

43 PARTICLE ACCELERATORS↗
DOE OSTI2021

Fault isolation and fault-tolerant control for Takagi-Sugeno fuzzy time-varying delay stochastic distribution systems

A fault isolation, estimation, and fault-tolerant control (FTC) scheme for nonlinear time-varying delay stochastic distribution control systems was presented in this paper. The Takagi-Sugeno fuzzy model was adopted to approach the nonlinear dynamics of time-varying delay systems. According to the output equivalence principle and Laplace transformation, an augmented state vector was given to solve the time-varying delay problem. When multiple actuator faults and interference occur simultaneously, fault detection, isolation and fault estimation was designed to obtained the fault information. To decouple faults and obtain the value and location information of the fault, the system was separated into two parts through the designed multiple conversion matrices, in which one subsystem was only affected by one actuator fault. This has simplified the design of fault isolation and estimation. A adaptive observer for fault estimation was given. Then, fault information such as the time, location, and size was determined. The observer gain matrices were calculated using linear matrix inequality (LMI). When a fault was detected and diagnosed, a FTC algorithm was devised using the proportional-integral control scheme to compensate the fault as much as possible. It has been shown that even if multiple faults actuator occurred simultaneously, the FTC controller still ensured the output probability density function of the system traced the desired probability density function when a fault occurred. Finally, the expected results were obtained through the simulation example, which confirmed the effectiveness of the method.

42 ENGINEERING↗
DOE OSTI2025

Pd–Methyl Bond Energy─Property Correlations, Noncorrelations, Machine Learning Models, and Application to Polymerization Catalysis

Metal–carbon bonds are a key intermediate in a variety of homogeneous organometallic transformations and often determine the critical thermodynamics and kinetics of catalytic processes. Surprisingly, the influence of different ligands on metal–carbon bond strengths has been largely overlooked. Here, in this study, we evaluated nearly 700 experimental Pd–methyl complexes by calculating their bond dissociation energies using density functional theory (DFT) and compared these bond strengths to several fundamental molecular properties, and this revealed several surprising correlations and noncorrelations. Most surprising was that several fundamental properties, such as the bond length, bond force constant, and bond electron density, have no correlation with bond strength, despite these correlations often holding for main-group compounds. We were indeed able to identify key ligand-dependent chemical features/descriptors that provided a highly accurate machine learning model and provided insight into the general factors that control the Pd–carbon bond strength, such as radical delocalization and nucleophilicity. Insights gained from the Pd–Me bond energy analysis were then applied to CO migratory insertion steps that are part of copolymerization reactions.

binding energy↗
DOE OSTI2022

Unveiling the Emergent Traits of Chiral Spin Textures in Magnetic Multilayers

Abstract Magnetic skyrmions are topologically wound nanoscale textures of spins whose ambient stability and electrical manipulation in multilayer films have led to an explosion of research activities. While past efforts focused predominantly on isolated skyrmions, recently ensembles of chiral spin textures, consisting of skyrmions and magnetic stripes, are shown to possess rich interactions with potential for device applications. However, several fundamental aspects of chiral spin texture phenomenology remain to be elucidated, including their domain wall (DW) structure, thermodynamic stability, and morphological transitions. Here the evolution of these textural characteristics are unveiled on a tunable multilayer platform—wherein chiral interactions governing spin texture energetics can be widely varied—using a combination of full‐field electron and soft X‐ray microscopies with numerical simulations. With increasing chiral interactions, the emergence of Néel helicity, followed by a marked reduction in domain compressibility, and finally a transformation in the skyrmion formation mechanism are demonstrated. Together with an analytical model, these experiments establish a comprehensive microscopic framework for investigating and tailoring chiral spin texture character in multilayer films.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Investigating the impacts of solid phase extraction on dissolved organic matter optical signatures and the pairing with high‐resolution mass spectrometry data across a freshwater stream network

Abstract Advancing our understanding of dissolved organic matter (DOM) chemistry in aquatic systems necessitates the integration of data streams from multiple analytical platforms. Some measurements require pretreatment with solid phase extraction (SPE), while others are performed directly on whole water samples. Evidence has suggested that SPE will be biased against select DOM fractions, leading to concerns over the ability to establish data linkages across platforms with variable needs for SPE pretreatment, such as those from optical measurements and those that provide high‐resolution molecular information. Here, we directly addressed this concern by assessing the impact of SPE on DOM optical properties through excitation–emission matrices with parallel factor analysis (PARAFAC) for 47 samples across a stream network within a single watershed reflective of variable DOM sources. PARAFAC data was further paired with molecular information obtained by Fourier transform ion cyclotron resonance mass spectrometry (FTICR‐MS). A comparison of PARAFAC models first revealed no systematic qualitative differences in major components between whole water DOM and DOM isolated by SPE (SPE‐DOM); however, quantitative biases against select components were observed. Further linkages with FTICR‐MS data revealed that the molecular fingerprint associated with each PARAFAC component was consistent between the whole water DOM and SPE‐DOM. Our results suggest that bulk scale linkages across these analytical platforms could be inferred irrespective of the observed quantitative biases resulting from SPE for samples within this example watershed. This work represents a key step toward the systematic evaluation of linkages between optical and high‐resolution mass spectrometry datasets in freshwater lotic environments.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

On amplitudes and field redefinitions

We derive an off-shell recursion relation for correlators that holds at all loop orders. This allows us to prove how generalized amplitudes transform under generic field redefinitions, starting from an assumed behavior of the one-particle-irreducible effective action. The form of the recursion relation resembles the operation of raising the rank of a tensor by acting with a covariant derivative. This inspires a geometric interpretation, whose features and flaws we investigate.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗