Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Developing Multi-Gene CRISPRa/I Programs to Accelerate DBTL Cycles in ABF Hosts Engineered for Chemical Production (CRADA 468)

Bacterial metabolism is comprised of large and complex gene networks that can produce valuable chemical products. Sophisticated organism engineering efforts are required to optimize production of high-value compounds from these networks. In principle, synthetic multi-gene transcriptional programs could be constructed to reengineer these networks for efficient industrial chemical production. In practice, however, our incomplete ability to understand and model the underlying networks, combined with our limited ability to predictably control the expression of multiple genes makes achieving this goal difficult. To overcome these challenges, we will combine new CRISPR-Cas multi-gene expression programs with computational modeling, machine learning, and multi-omics data to enhance the efficacy of design-build-test-learn (DBTL) cycles. For industrially promising microorganisms in early stages of development, creating technologies for rapidly engineering complex multi-gene programs could be transformative for accelerating data- and model-driven strain design. New CRISPR-Cas tools allow programmable gene activation (CRISPRa) or repression (CRISPRi) at multiple genes simultaneously, using the catalytically inactive Cas9 protein (dCas9) with guide RNAs that recognize DNA targets through predictable Watson-Crick base pairing. To enable accelerated DBTL cycles, we will combine these technologies with advanced Agile BioFoundry (ABF) capabilities for multi-omics data collection and machine learning. We will demonstrate the immediate applicability of these tools by rapidly improving the production of an industrial aromatic in multiple ABF organisms. We recently identified and optimized new transcriptional activators that can be linked to programmable CRISPR-Cas DNA binding domains to activate gene expression in E. coli. We can now use these CRISPRa tools as generalizable trans-acting regulators for combinatorial multi-gene expression tuning that can be easily transferred to new pathways and networks without additional genome engineering. We anticipate these tools will also transfer to new hosts. We have recently found that CRISPRa systems developed in E. coli can be readily ported to Pseudomonas putida, suggesting that multi-gene CRISPRa/i programs for diverse ABF organisms may be within reach.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Low-dimensional physics of clay particle size distribution and layer ordering

Abstract Clays are known for their small particle sizes and complex layer stacking. We show here that the limited dimension of clay particles arises from the lack of long-range order in low-dimensional systems. Because of its weak interlayer interaction, a clay mineral can be treated as two separate low-dimensional systems: a 2D system for individual phyllosilicate layers and a quasi-1D system for layer stacking. The layer stacking or ordering in an interstratified clay can be described by a 1D Ising model while the limited extension of individual phyllosilicate layers can be related to a 2D Berezinskii–Kosterlitz–Thouless transition. This treatment allows for a systematic prediction of clay particle size distributions and layer stacking as controlled by the physical and chemical conditions for mineral growth and transformation. Clay minerals provide a useful model system for studying a transition from a 1D to 3D system in crystal growth and for a nanoscale structural manipulation of a general type of layered materials.

42 ENGINEERING↗
DOE OSTI2025

AI Applications to Physics Experiments at Jefferson Lab

We survey how AI/ML is being deployed across Jefferson Lab's experimental and accelerator programs. In EPSCI, Hydra applies computer vision to automate real-time data-quality monitoring across all four experimental halls, replacing manual inspection of hundreds to thousands of histograms per shift. AIEC (AI Experiment Controls) uses ML to stabilize drift chamber gains and is now part of standard CEBAF production running, while AI Optimized Polarization (AIOP) targets autonomous control of polarized targets and photon beam angular alignment. In CASA, cavity fault classification models identify faulted cavities and trip types from waveform data with ~85% and ~78% agreement to labeled data, respectively, and are deployed in production; a separate effort applies LLMs and hybrid search to make the CEBAF operations logbook AI-ready. QCD-focused work includes transformer- and GAN-based generative models for particle-level event simulation, with distributed GAN training scaling studies on Polaris. Additional efforts span ML-on-FPGA for the EIC and a new Data Science Department coordinating anomaly detection, uncertainty quantification, and HPC-scalable ML lab-wide. Collectively, these projects illustrate AI's growing role in improving efficiency across JLab's nuclear physics mission.

Mei, Xinxin [Thomas Jefferson National Accelerator↗
DOE OSTI2025

Bayesian model mixing with multireference energy density functional

Reliably predicting nuclear properties across the entire chart of isotopes is important for applications ranging from nuclear astrophysics to superheavy science to nuclear technology. To this day, however, all the theoretical models that can scale at the level of the chart of isotopes remain semiphenomenological. Because they are fitted locally, their predictive power can vary significantly; different versions of the same theory provide different predictions. Bayesian model mixing takes advantage of such imperfect models to build a local mixture of a set of models to make improved predictions. Earlier attempts to use Bayesian model mixing for mass table calculations relied on models treated at single-reference energy density functional level, which fail to capture some of the correlations caused by configuration mixing or the restoration of broken symmetries. In this study we have applied Bayesian model mixing techniques within a multireference energy density functional (MR-EDF) framework. We considered predictions of two-particle separation energies from particle number projection or angular momentum projection with four different energy density functionals—a total of eight different MR-EDF models. We used a hierarchical Bayesian stacking framework with a Dirichlet prior distribution over weights together with an inverse log-ratio transform to enable positive correlations between different models. We found that Bayesian model mixing provides significantly improved predictions compared to the participating models. Published by the American Physical Society 2025

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Multifidelity uncertainty quantification with models based on dissimilar parameters

Multifidelity uncertainty quantification (MF UQ) sampling approaches have been shown to significantly reduce the variance of statistical estimators while preserving the bias of the highest-fidelity model, provided that the low-fidelity models are well correlated. However, maintaining a high level of correlation can be challenging, especially when models depend on different input uncertain parameters, which drastically reduces the correlation. Existing MF UQ approaches do not adequately address this issue. In this work, we propose a new sampling strategy that exploits a shared space to improve the correlation among models with dissimilar parameterization. We achieve this by transforming the original coordinates onto an auxiliary manifold using the adaptive basis (AB) method (Tipireddy and Ghanem, 2014). The AB method has two main benefits: (1) it provides an effective tool to identify the low-dimensional manifold on which each model can be represented, and (2) it enables easy transformation of polynomial chaos representations from high- to low-dimensional spaces. This latter feature is used to identify a shared manifold among models without requiring additional evaluations. Here we present two algorithmic flavors of the new estimator to cover different analysis scenarios, including those with legacy and non-legacy high-fidelity (HF) data. We provide numerical results for analytical examples, a direct field acoustic test, and a finite element model of a nuclear fuel assembly. For all examples, we compare the proposed strategy against both single-fidelity and MF estimators based on the original model parameterization.

42 ENGINEERING↗
DOE OSTI2020

Dark C P violation and gauged lepton or baryon number for electroweak baryogenesis

We explore the generation of the baryon asymmetry in an extension of the standard model where the lepton number is promoted to a U(1)ℓ gauge symmetry with an associated Z' gauge boson. This is based on a novel electroweak baryogenesis mechanism first proposed by us in Ref. Extra fermionic degrees of freedom, including a fermionic dark matter χ, are introduced in the dark sector for anomaly cancellation. The lepton number is spontaneously broken at high scale and the effective theory, containing the standard model, the Z', the fermionic dark matter, and an additional complex scalar field S, violates CP in the dark sector. The complex scalar field couples to the Higgs portal and is essential in enabling a strong first order phase transition. Dark CP violation is diffused in front of the bubble walls and creates a chiral asymmetry for χ, which in turn creates a chemical potential for the standard model leptons. Weak sphalerons are then in charge of transforming the net lepton charge asymmetry into net baryon number. We explore the model phenomenology related to the leptophilic Z', the dark matter candidate, the Higgs boson, and the additional scalar, as well as implications for electric dipole moments. We also discuss the case when baryon number U(1)B is promoted to a gauge symmetry, and discuss electroweak baryogenesis and its corresponding phenomenology.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

A parallel and performance portable implementation of a full-field crystal plasticity model

We have developed a parallel implementation of an Elasto-Viscoplastic Fast Fourier Transform-based (EVPFFT) micromechanical solver to enable computationally efficient crystal plasticity modeling for polycrystalline materials. Our primary focus lies in achieving performance portability, allowing a single EVPFFT implementation to run optimally on various homogeneous architectures, including multi-core Central Processing Units (CPUs), as well as on heterogeneous computer architectures comprising multi-core CPUs and Graphics Processing Units (GPUs) from different vendors. To accomplish this goal, we have leveraged MATAR, a C++ software library that simplifies the creation and utilization of multidimensional dense or sparse matrix and array data structures. These data structures are designed to be portable across diverse architectures through the use of Kokkos, a performance-portable library. Additionally, we have employed the Message Passing Interface (MPI) to efficiently distribute the computational workload among processors. The heFFTe (Highly Efficient FFT for Exascale) library is used to facilitate the performance portability of the fast Fourier transforms (FFTs) computation. The computational performance of EVPFFT is evaluated and presented in terms of parallel scalability and simulation runtime on different high-performance computing (HPC) architectures. As a result, the utility of the developed framework to efficiently simulate the micro-mechanical fields in polycrystalline microstructures in engineering applications is discussed.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Comparison of time-frequency-analysis techniques applied in building energy data noise cancellation for building load forecasting: A real-building case study

Time-frequency analysis that disaggregates a signal in both time and frequency domain is an important supporting technique for building energy analysis such as noise cancellation in data-driven building load forecasting. There is a gap in the literature related to comparing various time–frequency-analysis techniques, especially discrete wavelet transform (DWT) and empirical mode decomposition (EMD), to guide the selection and tuning of time–frequency-analysis techniques in data-driven building load forecasting. This article provides a framework to conduct a comprehensive comparison among thirteen DWT/EMD techniques with various parameters in a load forecasting modeling task. A real campus building is used as a case study for illustration. The DWT and EMD techniques are also compared under various data-driven modeling algorithms for building load forecasting. The results in the case study show that the load forecasting models trained with noise-cancelled energy data have increased their accuracy to 9.6% on average tested under unseen data. This study also shows that the effectiveness of DWT/EMD techniques depends on the data-driven algorithms used for load forecasting modeling and the training data. Hence, DWT/EMD-based noise cancellation needs customized selection and tuning to optimize their performance for data-driven building load forecasting modeling.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Sparsifying priors for Bayesian uncertainty quantification in model discovery

We propose a probabilistic model discovery method for identifying ordinary differential equations governing the dynamics of observed multivariate data. Our method is based on the sparse identification of nonlinear dynamics (SINDy) framework, where models are expressed as sparse linear combinations of pre-specified candidate functions. Promoting parsimony through sparsity leads to interpretable models that generalize to unknown data. Instead of targeting point estimates of the SINDy coefficients, we estimate these coefficients via sparse Bayesian inference. The resulting method, uncertainty quantification SINDy (UQ-SINDy), quantifies not only the uncertainty in the values of the SINDy coefficients due to observation errors and limited data, but also the probability of inclusion of each candidate function in the linear combination. UQ-SINDy promotes robustness against observation noise and limited data, interpretability (in terms of model selection and inclusion probabilities) and generalization capacity for out-of-sample forecast. Sparse inference for UQ-SINDy employs Markov chain Monte Carlo, and we explore two sparsifying priors: the spike and slab prior, and the regularized horseshoe prior. UQ-SINDy is shown to discover accurate models in the presence of noise and with orders-of-magnitude less data than current model discovery methods, thus providing a transformative method for real-world applications which have limited data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Theoretical Insights into Reaction-Induced Transformation and Tuning of Catalytic Behavior in Heterogenous Catalysis

Reaction-induced transformations in heterogenous catalysis represent diverse phenomena that challenge traditional views of static catalyst surfaces. From surface adsorbate dynamics, atomic rearrangements, to composition and phase transitions, these processes reveal the profound differences between idealized model systems under ultrahigh vacuum and the complex, evolving interfaces that govern real catalytic behaviors under reaction conditions. Here, this perspective reviews recent theoretical efforts to provide atomic-level mechanistic insights into significant reaction-induced transformations and their impact on catalytic activity and selectivity. It underscores the need for an integrated framework that combines predictive simulations with operando characterization to uncover active sites and mechanisms under realistic operating conditions. Achieving this requires accelerating existing simulations to fully capture diverse reaction-induced surface dynamics, enabling scalable and accurate modeling of catalysts as condition-dependent, dynamically evolving systems. Such approaches are critical to bridge the gap between theory and practice, offering a pathway to more impactful and predictive catalyst design.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Grassmannian diffusion maps based surrogate modeling via geometric harmonics

Abstract A novel surrogate model based on the Grassmannian diffusion maps (GDMaps) and utilizing geometric harmonics (GH) is developed for predicting the response of complex physical phenomena. The method utilizes GDMaps to obtain a low‐dimensional representation of the underlying behavior of physical/mathematical systems with respect to uncertain input parameters. Using this representation, GH, an out‐of‐sample extension technique, is employed to create a global map from the input parameter space to a Grassmannian diffusion manifold. GH is further employed to locally map points on the diffusion manifold onto the tangent space of a Grassmann manifold. The exponential map is then used to project the points in the tangent space onto the Grassmann manifold, where reconstruction of the full solution is performed. The performance of the proposed surrogate model is verified with three examples. The first problem is a toy example used to illustrate the technique. In the second example, errors associated with the various mappings are assessed by studying response predictions of the electric potential of a dielectric cylinder in a homogeneous electric field. The last example applies the method for uncertainty prediction in the strain field evolution in a model amorphous material using the shear transformation zone theory of plasticity.

42 ENGINEERING↗
DOE OSTI2024

Goethite and Hematite Nucleation and Growth from Ferrihydrite: Effects of Oxyanion Surface Complexes

The presence of oxyanions, such as nitrate (NO 3 – ) and phosphate (PO 4 3– ), regulates the nucleation and growth of goethite (Gt) and hematite (Hm) during the transformation of ferrihydrite (Fh). Our previous studies showed that oxyanion surface complexes control the rate and pathway of Fh transformation to Gt and Hm. However, how oxyanion surface complexes control the mechanism of Gt and Hm nucleation and growth during the Fh transformation is still unclear. Here, we used synchrotron scattering methods and cryogenic transmission electron microscopy to investigate the effects of NO 3 – outer-sphere complexes and PO 4 3– inner-sphere complexes on the mechanism of Gt and Hm formation from Fh. Our TEM results indicated that Gt particles form through a two-step model in which Fh particles first transform to Gt nanoparticles and then crystallographically align and grow to larger particles by oriented attachment (OA). In contrast, for the formation of Hm, imaging shows that Fh particles first aggregate and then transform to Hm through interface nucleation. This is consistent with our X-ray scattering results, which demonstrate that NO 3 – outer-sphere and PO 4 3– inner-sphere complexes promote the formation of Gt and Hm, respectively. These results have implications for understanding the coupled interactions of oxyanions and iron oxy-hydroxides in Earth-surface environments.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Parallel three-dimensional simulations of quasi-static elastoplastic solids

Hypo-elastoplasticity is a flexible framework for modeling the mechanics of many hard materials under small elastic deformation and large plastic deformation. Under typical loading rates, most laboratory tests of these materials happen in the quasi-static limit, but there are few existing numerical methods tailor-made for this physical regime. Here, we extend to three dimensions a recent projection method for simulating quasi-static hypo-elastoplastic materials. The method is based on a mathematical correspondence to the incompressible Navier–Stokes equations, where the projection method of Chorin (1968) is an established numerical technique. We develop and utilize a three-dimensional parallel geometric multigrid solver employed to solve a linear system for the quasi-static projection. Our method is tested through simulation of three-dimensional shear band nucleation and growth, a precursor to failure in many materials. As an example system, we employ a physical model of a bulk metallic glass based on the shear transformation zone theory, but the method can be applied to any elastoplasticity model. We consider several examples of three-dimensional shear banding, and examine shear band formation in physically realistic materials with heterogeneous initial conditions under both simple shear deformation and boundary conditions inspired by friction welding.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Mitigating the Impacts of Uncertain Geomagnetic Disturbances on Electric Grids: A Distributionally Robust Optimization Approach

Severe geomagnetic disturbances (GMDs) increase the magnitude of the electric field on the Earth’s surface (E-field) and drive geomagnetically-induced currents (GICs) along the transmission lines in electric grids. These additional currents can pose severe risks, such as current distortions, transformer saturation and increased reactive power losses, each of which can lead to system unreliability. Today several mitigation actions (e.g., changing grid topology) exist that can reduce the harmful GIC effects on the grids. Making such decisions can be challenging, however, because the magnitude and direction of the E-field are uncertain and non-stationary. In this paper, we model uncertain E-fields using the distributionally robust optimization (DRO) approach that determines optimal transmission grid operations such that the worst-case expectation of the system cost is minimized. We also capture the effect of GICs on the nonlinear AC power flow equations. For solution approaches, we develop an accelerated column-and-constraint generation (CCG) algorithm by exploiting a special structure of the support set of uncertain parameters representing the E-field. Extensive numerical experiments based on “epri-21” and “uiuc-150” systems, designed for GMD studies, demonstrate (i) the computational performance of the accelerated CCG algorithm, (ii) the superior performance of distributionally robust grid operations that satisfy nonlinear, nonconvex AC power flow equations and GIC constraints, in comparison with standard stochastic programming-based methods during the out-of-sample testing.

42 ENGINEERING↗
DOE OSTI2023

Quantum simulations in effective model spaces: Hamiltonian-learning variational quantum eigensolver using digital quantum computers and application to the Lipkin-Meshkov-Glick model

Quantum simulations offer the potential to predict the structure and dynamics of nuclear many-body systems that are beyond the capabilities of classical computing. Generally, preparing the ground state of strongly-interacting many-body systems relevant to nuclear physics is however inefficient, even using ideal quantum computers. In addition, currently available NISQ-era quantum devices possess modest numbers of qubits, limiting the size of quantum many-body systems that can be simulated. In this context, a reformulation of the quantum many-body problems using truncated model spaces and Hamiltonians is desirable to make them more amenable to near-term quantum computers. The importance of symmetries in low-energy theories, including effective field theories (EFTs), lattice quantum chromodynamics (QCD), and effective model spaces for nuclear systems, in particular their interplay with the reduction of active Hilbert spaces, is well known. Lesser known is the fact that the non-commutivity of some symmetries and truncations of the model space can be profitably combined with variational calculations to rearrange the entanglement into localized structures and enable more efficient simulations. Here, the goal of the present study is to explore and utilize the non-commutivity of symmetries and model-space truncations of quantum many-body systems important to nuclear physics, particularly in combination with variational algorithms for quantum simulations and effective Hamiltonian learning. We introduce an iterative hybrid classical-quantum algorithm, Hamiltonian learning variational quantum eigensolver (HL-VQE), that simultaneously optimizes an effective Hamiltonian, thereby rearranging entanglement into the effective model space, and the associated ground-state wavefunction. Quantum simulations, using classical computers and IBM's superconducting-qubit quantum computers, are performed to demonstrate the HL-VQE algorithm, in the context of the Lipkin-Meshkov-Glick (LMG) model of interacting fermions, where the Hamiltonian transformation corresponds to an orbital rotation. We use a mapping where the number of qubits scales with the $\log$ of the size of the effective model space, rather than the particle number. HL-VQE is found to provide an exponential improvement in LMG-model calculations of the ground-state energy and wavefunction, compared to naive truncations without Hamiltonian learning, throughout a significant fraction of the Hilbert space. In the context of EFT, this corresponds to counterterms scaling exponentially with the cut-off as opposed to power law. Implementations on IBM's QExperience quantum computers and simulators for 1- and 2-qubit effective model spaces are shown to provide accurate and precise results, reproducing classical predictions. For a range of parameters defining the LMG model, the HL-VQE algorithm is found to have better scaling of quantum resources requirements than previously explored algorithms. In particular, the HL-VQE scales efficiently over a large fraction of the model space, in contrast to VQE alone. This work constitutes a step in the development of entanglement-driven quantum algorithms for descriptions of nuclear many-body systems. This, in part, leverages the potential of noisy intermediate-scale quantum (NISQ) devices. The exponential scaling of counterterms observed in this study suggests the possibility of more general applicability to other non-perturbative EFTs.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

Pixel-Resolved Long-Context Learning for Turbulence at Exascale: Resolving Small-scale Eddies Toward the Viscous Limit

Turbulence plays a crucial role in multiphysics applications, including aerodynamics, fusion, and combustion. Accurately capturing turbulence's multiscale characteristics is essential for reliable predictions of multiphysics interactions, but remains a grand challenge even for exascale supercomputers and advanced deep learning models. The extreme-resolution data required to represent turbulence, ranging from billions to trillions of grid points, pose prohibitive computational costs for models based on architectures like vision transformers. To address this challenge, we introduce a multiscale hierarchical Turbulence Transformer that reduces sequence length from billions to a few millions and a novel RingX sequence parallelism approach that enables scalable long-context learning. We perform scaling and science runs on the Frontier supercomputer. Our approach demonstrates excellent performance up to 1.1 EFLOPS on 32,768 AMD GPUs, with a scaling efficiency of 94\%. To our knowledge, this is the first AI model for turbulence that can capture small-scale eddies down to the dissipative range in three-dimensional turbulence at high Reynolds numbers.

Yin, Junqi [ORNL] (ORCID:0000000338435520)↗
DOE OSTI2023

Relaxing the constant molar overflow assumption in distillation optimization

Abstract The constant molar overflow (CMO) framework, while useful for shortcut distillation models, assumes that all components have the same latent heats of vaporization. A simple transformation, from molar flows to latent‐heat flows, allows shortcut models to retain the mathematical simplicity of the CMO framework while accounting for different latent heats, resulting in the constant heat transport (CHT) framework for adiabatic distillation columns. Although several past works have already proposed this transformation in the literature, it has not been well utilized in recent times. In this article, we show the utility of this transformation in upgrading various applications such as identifying energy‐efficient multicomponent distillation configurations based on heat duty rather than surrogate vapor flow. The method transforms the diagram to a diagram. Furthermore, we derive new and insightful analytical results in distillation, such as cumulative latent‐heat stage fractions having monotonic profiles within a distillation column under the CHT framework.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗