Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Data-driven Minimum Entropy Control for Stochastic Nonlinear Systems using the Cumulant-Generating Function

Here, we present a novel minimum entropy control algorithm for a class of stochastic nonlinear systems subjected to non-Gaussian noises. The entropy control can be considered as an optimization problem for the system randomness attenuation, but the mean value has to be considered separately. To overcome this disadvantage, a new representation of the system stochastic properties was given using the cumulant-generating function based on the moment-generating function, in which the mean value and the entropy was reflected by the shape of the cumulant-generating function. Based on the samples of the system output and control input, a time-variant linear model was identified, and the minimum entropy optimization was transformed to system stabilization. Then, an optimal control strategy was developed to achieve the randomness attenuation, and the boundedness of the controlled system output was analyzed. The effectiveness of the presented control algorithm was demonstrated by a numerical example. In this paper, a data-driven minimum entropy design is presented without pre-knowledge of the system model; entropy optimization is achieved by the system stabilization approach in which the stochastic distribution control and minimum entropy are unified using the same identified structure; and a potential framework is obtained since all the existing system stabilization methods can be adopted to achieve the minimum entropy objective.

42 ENGINEERING↗
DOE OSTI2022

ExaCA: A performance portable exascale cellular automata application for alloy solidification modeling

Modeling the as-solidified grain structures that form during alloy processing is a critical component in understanding process-property relationships, particularly for additive manufacturing (AM) where grain structure is very sensitive to processing conditions. While cellular automata (CA)-based models have proven able to predict aspects of microstructure for several alloys and AM process conditions, long run times and large resource sets required limit the utility and the problem size to which existing CA models can be applied. As part of the ExaAM project, an initiative within the Exascale Computing Project (ECP) to develop, test, and optimize an exascale-capable coupled and self-consistent model of AM parts, we developed ExaCA (https://github.com/LLNL/ExaCA) for the liquid–solid phase transformation in the wake of AM melt pools. The CA-based code is parallelized using MPI and the Kokkos programming model, the latter enabling simulation on both CPUs and GPUs within a single-source implementation. Here, we detail the steps taken to transform a baseline, MPI-based CA code into one that is performant on CPUs and GPUs. Performance testing of ExaCA on Summit (a pre-exascale machine at Oak Ridge National Laboratory) was used to quantify CPU–GPU speedup comparing with equal numbers of nodes. Testing showed comparable CPU performance to the MPI-only CA code and a 5-20x speedup when running AM-based test problems using GPUs. The improved performance of CA through GPU utilization and the performance portable nature of ExaCA will enable accurate part-scale modeling by harnessing the power of current and future generations of high performance computing resources. Future work will include improving the strong scaling of ExaCA on GPUs by reducing load imbalance associated with the locality of the problem, and continuing performance optimization across exascale hardware.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Predictive Models of Genetic Redundancy in Arabidopsis thaliana

Abstract Genetic redundancy refers to a situation where an individual with a loss-of-function mutation in one gene (single mutant) does not show an apparent phenotype until one or more paralogs are also knocked out (double/higher-order mutant). Previous studies have identified some characteristics common among redundant gene pairs, but a predictive model of genetic redundancy incorporating a wide variety of features derived from accumulating omics and mutant phenotype data is yet to be established. In addition, the relative importance of these features for genetic redundancy remains largely unclear. Here, we establish machine learning models for predicting whether a gene pair is likely redundant or not in the model plant Arabidopsis thaliana based on six feature categories: functional annotations, evolutionary conservation including duplication patterns and mechanisms, epigenetic marks, protein properties including posttranslational modifications, gene expression, and gene network properties. The definition of redundancy, data transformations, feature subsets, and machine learning algorithms used significantly affected model performance based on holdout, testing phenotype data. Among the most important features in predicting gene pairs as redundant were having a paralog(s) from recent duplication events, annotation as a transcription factor, downregulation during stress conditions, and having similar expression patterns under stress conditions. We also explored the potential reasons underlying mispredictions and limitations of our studies. This genetic redundancy model sheds light on characteristics that may contribute to long-term maintenance of paralogs, and will ultimately allow for more targeted generation of functionally informative double mutants, advancing functional genomic studies.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Vulnerability and resilience of urban energy ecosystems to extreme climate events: A systematic review and perspectives

We reviewed the present studies on the vulnerability and resilience of the energy ecosystem (most parts of the energy ecosystem), considering extreme climate events. This study revealed that the increased interactions formed during the transformation of the energy landscape into an ecosystem could notably increase the vulnerability of the energy infrastructure. Such complex ecosystem cannot be assessed using the present state of the art models used by the energy system modelers. Therefore, this study introduces a novel analogy known as the COVID analogy to understand the propagation of disruption within and beyond the energy ecosystem and organized the present state of the art based on the COVID analogy. The analogy helps to categorize the vulnerability of the energy infrastructure into three stages. The study revealed that although there are many publications covering the vulnerability and resilience of the energy infrastructure, considering extreme climate events, the majority are focused on the direct impact of extreme climate on the energy ecosystem. In addition, most of the studies do not consider the impact of future climate variations during this assessment. The propagation of disruptions was assessed mainly for wildfires and hurricanes. Further, there is a clear research gap in considering vulnerability assessment for interconnected energy infrastructure. Here, the transformation of energy systems into a complex ecosystem notably increases the complexity, making it difficult to assess vulnerability and resilience. A shift from a centralized to decentralized modeling architecture could be beneficial when considering the complexities brought by that transformation. Hybrid models consisting of both physical and data-driven machine learning techniques could also be beneficial in this context.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Derivation of physical equations for high-speed laser welding using large language models

It is challenging to formulate complex physical phenomena that occur in a manufacturing process, particularly when the available data are limited, rendering conventional data-driven approaches ineffective. This study aims to predict humping onset in high-speed laser welding by introducing a novel framework, namely text-to-equations generative pre-trained transformer (T2EGPT). This method leverages the capabilities of large language models (LLMs), in combination with sparse experimental data and enriched literature data, to derive an interpretable and generalizable equation for predicting humping initiation. By capturing key correlations among physical parameters, T2EGPT generates a compact and dimensionless expression that accurately predicts hump formation. The equation reveals that humping arises from the interplay between inertia-driven backward melt flow and capillary-driven surface stabilization, where inertial forces drive molten metal backward and capillary forces resist surface deformation. Furthermore, compared to traditional data-driven models, T2EGPT demonstrates enhanced predictive accuracy and cross-material transferability. More broadly, this study highlights the potential of LLMs to integrate textual information with data-driven discovery, enabling the extraction of physical laws in data-scarce scientific domains.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Neural operator transformers capture bifurcating drift-wave turbulence in fusion plasma simulations

Self-consistent modeling of turbulence-driven transport is critical for optimizing confinement in magnetically confined fusion plasmas, such as tokamaks and stellarators. In particular, capturing the long-term co-evolution of turbulence, flow, and background plasma profiles remains computationally challenging. Direct numerical simulation of these multiscale, highly nonlinear processes is often demanding and impractical for real-time control or design optimization. To address this bottleneck, we investigate transformer-based neural operator partial differential equation surrogates for emulating the dynamics of drift-wave turbulence bifurcation mediated by zonal flows, using the modified Hasegawa–Wakatani (MHW) model as a prototypical system. We find that the finetuned neural operator model has excellent performance in capturing the multi-spatiotemporal-scales of MHW turbulence bifurcation and is robust to testing on rare and out-of-distribution dynamics. Specifically, we demonstrate that a single unified model accurately predicts both quasi-steady-state turbulence and a wide range of dynamical transition processes, such as nonlinear saturation, spontaneous suppression of turbulence, and the emergence of macroscopic zonal flows, over time horizons vastly exceeding the local turbulence correlation time. This computationally efficient approach establishes a strong foundation for fast, AI-based modeling of complex, multiscale phenomena in magnetized fusion plasmas.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Surface energetics of wurtzite and sphalerite polymorphs of zinc sulfide and implications for their formation in nature

Surface energetics of zinc sulfide nanoparticles determines their structure, properties, and occurrence. Using a combination of experimental techniques, we investigated the thermodynamics of the two polymorphs, sphalerite and wurtzite at bulk and nanoscale to understand their occurrence. Calorimetric measurements confirmed that wurtzite has a lower surface energy than sphalerite, which causes a reversal in phase stability at the nanoscale, with wurtzite energetically stable for particle size below 10 nm. Taking these surface energies into account, a simple model of the thermodynamics of the sphalerite - wurtzite transformation as a function of particle size and temperature can explain the occurrence of the zinc sulfide polymorphs in environments as diverse as ore bodies and planetary atmospheres.

58 GEOSCIENCES↗
DOE OSTI2026

Computational simulation of asymmetric phase transformation in cracked Li₇La₃Zr₂O₁₂: Variant selection and chemo-mechanical implications

Coupling between microcracks and phase transformation in ion-conducting ceramics can jointly affect mechanical responses and ion transport. In this work we investigate the cubic-to-tetragonal phase transformation in Li₇La₃Zr₂O₁₂ in the presence of a microcrack under hydrostatic loading and quantify its implications for crack-tip stress concentration and Li-ion transport using phase-field and molecular dynamics simulations. The phase transformation exhibits a strong asymmetry between hydrostatic tension and compression. Under tension, the crack edge nucleates one tetragonal variant that amplifies the crack-tip stress intensity and promotes crack opening. Under compression, the crack tip nucleates a different tetragonal variant that enhances the stress-induced crack-closure tendency. Effective Li diffusivity analysis shows faster transport degradation under compression due to accelerated transformation kinetics, exposing a trade-off between mechanical stability and ionic conductivity. These results highlight the intertwined nature of cracking, phase transformation, and ionic transport in ion-conducting oxides and provide mechanistic insights into chemo-mechanical degradation of solid electrolytes.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Transformer-powered surrogates close the ICF simulation-experiment gap with extremely limited data

Abstract Recent advances in machine learning, specifically transformer architecture, have led to significant advancements in commercial domains. These powerful models have demonstrated superior capability to learn complex relationships and often generalize better to new data and problems. This paper presents a novel transformer-powered approach for enhancing prediction accuracy in multi-modal output scenarios, where sparse experimental data is supplemented with simulation data. The proposed approach integrates transformer-based architecture with a novel graph-based hyper-parameter optimization technique. The resulting system not only effectively reduces simulation bias, but also achieves superior prediction accuracy compared to the prior method. We demonstrate the efficacy of our approach on inertial confinement fusion experiments, where only 10 shots of real-world data are available, as well as synthetic versions of these experiments.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Simulation of Linear Non-Hermitian Boundary-Value Problems with Quantum Singular-Value Transformation

Herein we propose a quantum algorithm for simulating dissipative waves in inhomogeneous linear media as a boundary-value problem. Using the so-called quantum singular value transformation (QSVT), we construct a quantum circuit that models the propagation of electromagnetic waves in a one-dimensional system with outgoing boundary conditions. The corresponding measurement procedure is also discussed. Limitations of the QSVT algorithm are identified in connection with the large condition numbers that the dispersion matrices exhibit at weak dissipation.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Blackstart of Power Grids with Inverter-Based Resources

This paper presents the findings of our investigation into inverter-based resource-(IBR-) driven blackstart of electric grids. Four potential black-start configurations with different setups are presented. To evaluate the technical feasibility of IBR-driven black start in the four configurations, a behavioral model of inverters that mimics current-limited inverter operation is developed using variable resistors in the MATLAB Simulink/Simscape environment. The inverter model is connected to an induction motor through transformers and a transmission line to simulate its startup. Simulation results show that even with the limited current supply capability of inverters because of their physical constraints, IBRs can black-start a motor under certain conditions. Results also show that by using soft-start techniques, such as ramped supply voltage, inrush currents can be reduced, thereby expanding the conditions under which IBRs can provide black-start support. Simulation results with different scenarios lead to discussions and key takeaways that could be valuable for further IBR-driven black-start research.

41 EE - Solar Energy Technologies Office (EE-4S)↗
DOE OSTI2025

Computational Modeling of a 3D Printed Recuperator and Subsequent Experimental Loop for Supercritical Carbon Dioxide Cycles

Oak Ridge National Laboratory (ORNL), in collaboration with mechanical-thermal energy storage (mTES) provider EarthEn, a US Department of Energy (DOE) Lab-Embedded Entrepreneurship Program (LEEP) recipient at ORNL’s Innovation Crossroads 2023, is utilizing a state-of-the-art patented 3D printing technique to design an additively manufactured (AM) supercritical CO2 (sCO2) recuperator (REC) for EarthEn’s charge/discharge cycle. The AM REC will be printed at ORNL’s Manufacturing Demonstration Facility using Inconel Alloy 718 and tested on a closed-loop, ∼100 kW scale experimental facility that is under construction. The testing will compare the printed design against a commercial-off-the-shelf Printed Circuit Heat Exchanger (PCHE) REC. The design of the sCO2 facility is guided by a Modelica-based system model which is primarily dependent on the open-source TRANSFORM library developed at ORNL and uses the open-source CoolProp library for thermophysical properties of sCO2 via the External Media library. It is envisioned that an iterative process will be followed between the physical loop and the system model wherein the initial experimental data will be used to tune the model, which in turn will be used to guide future loop operation. Simultaneously, the AM REC is being designed using computer-aided design models, and it is also being analyzed for hydraulic and thermomechanical response using commercial computational fluid dynamics software, Simcenter STAR-CCM+, on highperformance computing resources.1

See, Nate [ORNL] (ORCID:0000000178581202)↗
DOE OSTI2025

Score-Based Physics-Informed Neural Networks for High-Dimensional Fokker–Planck Equations

The Fokker-Planck (FP) equation is a foundational partial differential equation (PDE) in stochastic processes involving Brownian motions. However, the curse of dimensionality (CoD) poses a formidable challenge when dealing with high-dimensional FP equations. Although Monte Carlo simulation and (vanilla) Physics-Informed Neural Networks (PINNs) have shown the potential to tackle CoD, both methods exhibit significant numerical errors in high dimensions when dealing with the probability density function (PDF) associated with Brownian motion. The point-wise PDF values tend to decrease exponentially as dimensionality increases, surpassing the precision of numerical simulations and resulting in substantial errors. In addition, due to its massive sampling, Monte Carlo fails to offer fast sampling. Modeling the logarithm likelihood (LL) via vanilla PINNs transforms the FP equation into a notoriously difficult Hamilton-Jacobi-Bellman (HJB) equation, which is impractical for PINN learning, whose error grows rapidly with dimension. To this end, we propose a novel approach utilizing a score-based solver to fit the score function in stochastic differential equations (SDEs). The score function, defined as the gradient of the LL, plays a fundamental role in inferring LL and PDF and enables fast SDE sampling, offering an effective means to overcome the CoD. Three fitting methods, Score Matching (SM), Sliced Score Matching (SSM), and Score-PINN, are introduced, each contributing unique advantages in computational complexity, accuracy, and generality. The proposed score-based SDE solver operates in two stages: first, employing score matching or Score-PINN to acquire the score function; and second, solving the LL via an ordinary differential equation (ODE) using the obtained score function. Comparative evaluations across these methods showcase varying trade-offs. The proposed methodology is evaluated across diverse SDEs, including anisotropic Ornstein-Uhlenbeck processes, geometric Brownian motion, and Brownian motion with varying eigenspace. We also test various distributions, including Gaussian, Log-normal, Laplace, and Cauchy distributions. The numerical results demonstrate the score-based SDE solver’s stability, speed, and performance across different experimental settings, solidifying its potential as a solution to CoD for high-dimensional FP equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Memcomputing the Spectrum of Correlated Quantum Systems

The goals and objectives of the grant DE‐SC0020892 were to apply a new computing paradigm, MemComputing, to efficiently simulate properties of correlated quantum systems. The method has been applied to a wide set of problems ranging from quantum state tomography to finding the ground state of correlated systems. In all cases, substantial advantages compared to state-of-the-art approaches have been obtained. The project has also led to the suggestion of the transformer architecture (used nowadays in large-language models) as an efficient quantum state representation, and a better understanding of the role of memory in the generation of long-range order in neural networks. This grant has supported the work of a PhD student, inspired a new class on unconventional computing taught at the University of California, San Diego and has generated several peer-reviewed papers.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Elucidating the mechanical and thermal response of nanotwinned Ni alloys

Transformative advances in nanoscale materials synthesis, characterization and modeling are enabling the synthesis of materials with unprecedent properties and greater understanding of the nanoscale mechanisms that underpin these properties. Nanotwinned Cu alloys have received considerable attention due to their impressive balance of strength and ductility, but they have limited microstructural stability. Recent instantiation of nanotwins in sputter deposited Ni-Mo-W, and several commercial Ni-based superalloys, point to the potential development of a new class of high temperature materials with a very beneficial suite of mechanical and physical properties. The experimental study described in this report was undertaken to elucidate the nanoscale origins of the thermal and mechanical behavior of nanotwinned Ni alloys. Micropillar compression experiments have shown nanotwinned Ni 85 Mo 15-x W x alloys to possess unusually high strengths above 3.5GPa and enhanced microstructural stability. The strength of these nanotwinned alloys is determined by the abrupt formation of shear bands. This study focused on identifying the nanoscale trigger, or triggers, for shear banding in nanotwinned materials using in situ experiments and atomic-scale postmortem analysis. Contrasting and comparing the mechanical response and postmortem nanostructures of “Mo-rich” and “W-rich” nanotwinned specimens elucidated the importance of the lateral motion of easy glide defects that results in erasure of twins and local coarsening of the nanotwinned microstructure. This twin coarsening was then associated with very localized plasticity, strain softening, and shear band formation. The difference between alloys was further studied by considering the role of various material factors: stacking fault energy, coherent twin boundary spacing and flatness, grain size, and the interactions of solute atoms with twin and grain boundaries. The effect of alloy composition and sputtering power and temperature were also investigated and used to control twin spacing and geometry. Combined with atomic-scale simulations, these experiments insights should allow us to identify strategies for achieving concomitant ultrahigh strength and ductility in nanotwinned Ni alloys. In parallel but synergistic studies, our work was buoyed by collaborative state-of-the-art nanoscale orientation and strain mapping at the DOE National Center for Electron Microscopy (NCEM). For example, novel 4D-STEM techniques were used to acquire nanoscale strain maps. The extremely fine twin spacing limited our measurements of atomic-scale thermal expansion within twins, but recent results from NCEM suggest that it will soon be possible to conduct such measurements and to unravel the origins of the novel thermal expansion characteristics of nanotwinned Ni alloys.

36 MATERIALS SCIENCE↗
DOE OSTI2024

NSTX-U National Research Program: White Paper in Response to Call from FESAC Sub-Committee

Both scientific and technical innovation is needed for the realization of an attractive engineering solution for a timely and cost-effective Pilot Plant, the design and construction of which is the overarching recommendation of the FESAC Long Range Plan, and the 2021 NASEM Pilot Plant reports, which underpin the Bold Decadal Vision. The two most significant plasma physics gaps to close for a Compact Pilot Plant (CPP) are core confinement improvement and heat flux mitigation, neither of which have been closed in an integrated fashion for any planned fusion power production device. High core confinement and stability are essential for producing majority self-driven plasmas in CPPs with reduced size and auxiliary heating power requirements, with an improvement in confinement being the major driver for cost reduction of a CPP. The National Spherical Tokamak Experiment - Upgrade (NSTX-U) is a unique low aspect ratio research facility that will address the fundamental challenge of developing the science and technology basis for a CPP design that integrates high core and edge confinement with the ability to mitigate very high incident heat fluxes. NSTX-U capabilities will enable the high performance, already achieved on NSTX, to extend into physics regimes much closer to those anticipated in Spherical Tokamak (ST)-based CPPs. These confinement and stability properties will be assessed by a full complement of diagnostics and analysis tools, which will also aid in the development of the underlying theory and predictive models needed for further optimization. Both conventional and transformative heat flux mitigation methods, such as liquid lithium plasma-facing components, will be developed and tested in-situ in NSTX-U at incident heat fluxes of ~100 MW/m 2 , and will inform plans and reduce risk for a subsequent major upgrade to the device to fully heated, high-Z wall and full liquid lithium divertor capability, a technology that potentially could then be implemented on any magnetic confinement device at any aspect ratio. NSTX-U research is fully complementary to programs performed on other STs, nationally and internationally. Furthermore, NSTX-U research has a direct connection to the private sector by informing design choices for future power production facilities being developed by these companies. The NSTX-U program will operate as a national User Facility, with collaborating researchers, engineers, and graduate students from 19 outside institutions, and open to participation and experiments led by researchers from both public and private entities. The research program will advance workforce development through training of young scientists, engineers, and technicians, and it will also serve for further diagnostic innovation, especially for high heat flux and high-Z wall environments, and implementation of advanced artificial intelligence (AI) for plasma and heat flux control.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Blackstart of Power Grids with Inverter-Based Resources: Preprint

This paper presents the findings of our investigation into inverter-based resource- (IBR-) driven blackstart of electric grids. Four potential black-start configurations with different setups are presented. To evaluate the technical feasibility of IBR-driven black start in the four configurations, a behavioral model of inverters that mimics current-limited inverter operation is developed using variable resistors in the MATLAB Simulink/Simscape environment. The inverter model is connected to an induction motor through transformers and a transmission line to simulate its startup. Simulation results show that even with the limited current supply capability of inverters because of their physical constraints, IBRs can black-start a motor under certain conditions. Results also show that by using soft-start techniques, such as ramped supply voltage, inrush currents can be reduced, thereby expanding the conditions under which IBRs can provide black-start support. Simulation results with different scenarios lead to discussions and key takeaways that could be valuable for further IBR-driven black-start research.

14 SOLAR ENERGY↗
DOE OSTI2021

Domain knowledge-informed, process-mapping AI graph for designing Fe-based alloys

<span style="font-family: Calibri, sans-serif; font-size: 12pt;">Continuous improvement in efficiency of a power plant relies on designing materials for use at increasingly higher temperature and/or pressure, for 100,000s hours of operation. Due to complexity, non-linearity and high-dimensionality of the problem, traditional Machine Learning (ML) approaches require unreasonably large datasets for the data-driven model development. Science-based material and process engineering complements hard data with, sometimes soft and intuitive, empirical domain knowledge. Artificial Intelligence (AI) was used in this study to incorporate such knowledge into computational graph architecture (process-mimicking artificial neuron design, causal layer and graph structures, ensemble modeling of latent states) and learning procedures (variable transformation, fuzzy physics pre-training and freezing of deep layers, virtual microstructure representation, and adversarial multi-objective optimization). The first alloys design pathways suggested by the AI tool (pyroMind) passed a preliminary engineering review on soundness and transparency.</span>

Romanov, Vyacheslav↗