Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 901 records · Page 50
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Accelerating manufacturing for biomass conversion via integrated process and bench digitalization: a perspective

We present a perspective for accelerating biomass manufacturing via digitalization. We summarize the challenges for manufacturing and identify areas where digitalization can help. A profound potential in using lignocellulosic biomass and renewable feedstocks, in general, is to produce new molecules and products with unmatched properties that have no analog in traditional refineries. Discovering such performance-advantaged molecules and the paths and processes to make them rapidly and systematically can transform manufacturing practices. Furthermore, we discuss retrosynthetic approaches, text mining, natural language processing, and modern machine learning methods to enable digitalization. Laboratory and multiscale computation automation via active learning are crucial to complement existing literature and expedite discovery and valuable data collection without a human in the loop. Such data can help process simulation and optimization select the most promising processes and molecules according to economic, environmental, and societal metrics. We propose the close integration between bench and process scale models and data to exploit the low dimensionality of the data and transform the manufacturing for renewable feedstocks.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Neural network surrogate models for equations of state

Equation of state (EOS) data provide necessary information for accurate multiphysics modeling, which is necessary for fields such as inertial confinement fusion. Here, we suggest a neural network surrogate model of energy and entropy and use thermodynamic relationships to derive other necessary thermodynamic EOS quantities. We incorporate phase information into the model by training a phase classifier and using phase-specific regression models, which improves the modal prediction accuracy. Our model predicts energy values to 1% relative error and entropy to 3.5% relative error in a log-transformed space. Although sound speed predictions require further improvement, the derived pressure values are accurate within 10% relative error. Our results suggest that neural network models can effectively model EOS for inertial confinement fusion simulation applications.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Bio-based solutions to transform the climate and clean energy systems [Slides]

The study and use of non-model microbes in both natural and engineered systems, to tackle a range of national and global challenges. Unique combination of expertise from synthetic biology tool development and high throughput screening to aquatic and terrestrial environmental simulation systems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Towards Improved Decision Making for the Smarter Grid: Geomagnetic Disturbance Implicit Modeling Uncertainty Quantification

Geomagnetic disturbances can greatly disrupt the power grid operation. This is caused by low-frequency changes in the earth’s magnetic field, which upon interaction with the deep earth conductivity induces a changing electric field at the earth’s surface. This in turn induces quasi-direct currents which have the potential to cause protective devices to operate, widespread damage to high-voltage transformers, or voltage collapse due to induced reactive power losses. While these impacts have been known since the 1940s, only recently have standards for GMD response been implemented. Because these standards are heavily based on models, it is important to understand when it is appropriate to make certain modeling assumptions. In this work, implicit modeling uncertainty is quantified for the purpose of enhanced decision making for disturbance impact mitigation. Considering electromagnetic models, modeling assumptions of non-uniform grounding, imbalanced operation, transformer saturation, and geomagnetically induced currents, voltage estimation values are analyzed. Statistical characteristics of modeling uncertainty are obtained by analysis of synthetically generated data. Results statistically characterize geomagnetic disturbances modeling uncertainty, providing an important tool for impact mitigation.

grid analysis↗
DOE OSTI2025

Hyperstealth dark matter and long-lived particles

A new dark matter candidate is proposed that arises as the lightest baryon from a confining 𝑆⁢𝑈⁡(𝑁) gauge theory which equilibrates with the Standard Model only through electroweak interactions. Surprisingly, this candidate can be as light as a few GeV. The lower bound arises from the intersection of two competing requirements: (i) the equilibration sector of the model must be sufficiently heavy, at least several TeV, to avoid bounds from colliders, and (ii) the lightest dark meson (that may be the dark 𝜂′, 𝜎, or the lightest glueball) has suppressed interactions with the SM and must decay before big bang nucleosynthesis. The low-energy dark sector consists of one flavor that is electrically neutral and an almost electroweak singlet. The dark matter candidate is the lightest baryon consisting of 𝑁 of these light flavors leading to a highly suppressed elastic scattering rate with the Standard Model (SM). The equilibration sector consists of vectorlike dark quarks that transform under the electroweak group, ensuring that the dark sector can reach thermal equilibrium with the SM in the early Universe. The lightest dark meson lifetimes vary between 10 −3 ≲ 𝑐⁢𝜏 ≲ 10 7 m, providing an outstanding target for LHC production and experimental detection. We delineate the interplay between the lifetime of the light mesons, the suppressed direct detection cross section of the lightest baryon, and the scale of equilibration sector that can be probed at the LHC.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2026

Prong Segmentation using Point Set Transformers in Multiple View Neutrino Detectors

NOvA is a long-baseline neutrino experiment studying neutrino oscillations by detecting neutrinos from the NuMI beam at Fermilab. Its physics analysis relies on accurate prong segmentation, which involves matching each hit to its source particle and identifying the particle type. This task has commonly been addressed using a combination of traditional clustering algorithms and convolutional neural networks (CNNs). However, NOvA’s detector design presents data as two sparse and decoupled 2D images (XZ and YZ views) rather than a native 3D representation, posing a significant challenge for traditional CNN-based models. In this talk, we propose a novel neural network based on the Point Set Transformer. By treating detector hits as sparse point clouds and implementing a cross-view attention mechanism, our model enables efficient information mixing between both views. Evaluated on NOvA simulated data, our model achieves superior accuracy while requiring significantly fewer computational resources compared to other models. Furthermore, the model demonstrates great performance when applied to Liquid Argon Time Projection Chamber (LArTPC) data, which shows its potential as a universal prong segmentation algorithm for multiple view neutrino detectors.

Liu, Jiaxi [UC, Irvine]↗
DOE OSTI2026

Automated ICRF heating surrogate modeling via machine learning

This work introduces automated machine learning workflows that address critical bottlenecks in surrogate model development for Ion Cyclotron Range of Frequencies (ICRF) heating applications. The automated framework includes data analysis tools that transform raw datasets into actionable insights in seconds, replacing weeks of manual exploratory effort and ensuring consistent, reproducible dataset characterization. By integrating advanced hyperparameter optimization (HPO) methods including Bayesian optimization via BoTorch and Tree-structured Parzen Estimators (TPE), the framework significantly reduces model development time from weeks to hours, decreasing computational cost and required expertise, while enabling high-accuracy surrogate models. Compared to traditional hyperparameter scanning (HPS) techniques such as methodical, randomized, and grid searches, HPO methods achieve superior convergence and predictive performance, even when compared to already well-tuned reference models. On NSTX High Harmonic Fast Wave (HHFW) heating datasets, both Random Forest Regressor (RFR) and neural network surrogates demonstrate improved accuracy, achieving R 2 values beyond 0.97 and 0.98, respectively. The results show that while HPO gains are modest for robust architectures like RFR, they become essential for more sensitive models such as neural networks, highlighting the trade-offs across optimization strategies. Through automated workflows that eliminate manual hyperparameter tuning and require minimal ML expertise, this work enables widespread adoption of high-fidelity surrogate models across the fusion community for real-time plasma control, uncertainty quantification, rapid experimental scenario development, and integrated system optimization.

Sanchez-Villar, Alvaro [Princeton Plasma Physics L↗
DOE OSTI2020

Influence of physical and chemical hydrology on bioremediation of a U-contaminated aquifer informed by reactive transport modeling incorporating 238 U/ 235 U ratios

Microbially-catalyzed reductive immobilization of aqueous uranium (U) as a solid phase has been proposed as a U remediation technique. Both laboratory and field experiments have demonstrated that this reduction reaction alters the 238 U/ 235 U ratio, producing a 238 U-enriched U(IV) solid. In contrast, other major U reactive transport processes fractionate these isotopes much less. This suggests the potential to quantify the extent of bioreduction occurring in groundwater containing U using the 238 U/ 235 U ratio, which would substantially improve upon current practices largely relying on U concentration measurements alone. Current reactive transport models for uranium dynamics include only concentration measurements, which are strongly influenced by highly coupled reactive transformation and aqueous transport processes. The complex physical and chemical behavior of U potentially compromises such quantitative analysis of U storage and release. Here we report the first numerical reactive transport model which explicitly incorporates variations in the 238 U/ 235 U ratio of U and demonstrates improved interpretation of the principal chemical reactions and groundwater transport processes affecting the subsurface mobility and distribution of this widespread contaminant. Recent U bioreduction studies performed in a contaminated aquifer in Rifle, Colorado, USA applied Rayleigh distillation models to interpret U stable isotope fractionation observed as a result of acetate amendment. These simplified models were unable to resolve the spatiotemporal pattern of U isotope fractionation recorded in the aqueous solutes. Here, we employ the multi-component, isotope-enabled CrunchTope reactive transport software to interpret these measured U isotope ratios, and demonstrate accurate reproduction of observed trends in both geochemistry and 238 U/ 235 U ratios for two consecutive years of field experiments. Overall, our results indicate that accurately modeling both the U concentration and isotope ratio distributions greatly constrains the parameter space of the model. We find that the transport properties of U in the Rifle aquifer are governed by the presence of low-permeability regions, which the isotopes are uniquely sensitive to. When U reduction is spatially constrained by these low-permeability regions, the shift in the 238 U/ 235 U ratio becomes more muted. Accurate modeling of observed U isotope ratios thus provides a powerful means to better understand bioremediation, and the current study serves to advance the application of this novel method.

238U/235U↗
DOE OSTI2021

Manifold learning for coarse-graining atomistic simulations: Application to amorphous solids

In this work, we introduce a generalized machine learning framework to probabilistically parameterize upper-scale models in the form of nonlinear PDEs consistent with a continuum theory, based on coarse-grained atomistic simulation data of mechanical deformation and flow processes. The proposed framework utilizes a hypothesized coarse-graining methodology with manifold learning and surrogate-based optimization techniques. Coarse-grained high-dimensional data describing quantities of interest of the multiscale models are projected onto a nonlinear manifold whose geometric and topological structure is exploited for measuring behavioral discrepancies in the form of manifold distances. A surrogate model is constructed using Gaussian process regression to identify a mapping between stochastic parameters and distances. Derivative-free optimization is employed to adaptively identify a unique set of parameters of the upper-scale model capable of rapidly reproducing the system's behavior while maintaining consistency with coarse-grained atomic-level simulations. The proposed method is applied to learn the parameters of the shear transformation zone (STZ) theory of plasticity that describes plastic deformation in amorphous solids as well as coarse-graining parameters needed to translate between atomistic and continuum representations. We show that the methodology is able to successfully link coarse-grained microscale simulations to macroscale observables and achieve a high-level of parity between the models across scales.

36 MATERIALS SCIENCE↗
DOE OSTI2025

An Evaluation of Representation Learning Methods in Particle Physics Foundation Models

We present a systematic evaluation of representation learning objectives for particle physics within a unified framework. Our study employs a shared transformer-based particle-cloud encoder with standardized preprocessing, matched sampling, and a consistent evaluation protocol on a jet classification dataset. We compare contrastive (supervised and self-supervised), masked particle modeling, and generative reconstruction objectives under a common training regimen. In addition, we introduce targeted supervised architectural modifications that achieve state-of-the-art performance on benchmark evaluations. This controlled comparison isolates the contributions of the learning objective, highlights their respective strengths and limitations, and provides reproducible baselines. We position this work as a reference point for the future development of foundation models in particle physics, enabling more transparent and robust progress across the community.

Chen, Michael [Caltech]↗
DOE OSTI2020

Advances in Multiscale Modeling of Lignocellulosic Biomass

Applications and associated processing technologies of lignocellulosic biomass are becoming more important as we endeavor to meet societal demand for fuels, chemicals, and materials from renewable resources. Meanwhile, the rapidly expanding availability and capabilities of high-performance computing present an unprecedented opportunity to accelerate development of technologies surrounding lignocellulose utilization. In order to realize this potential, suitable modeling frameworks must be constructed that effectively capture the multiscale complexity and tremendous variety exhibited by lignocellulosic materials. In our assessment of previous endeavors towards this goal, several important shortcomings have been identified: (1) the lack of multiscale integration strategies that capture emergent properties and behaviors spanning different length scales, and (2) the inability of many modeling approaches to effectively capture the variability and diversity of lignocellulose that arise from both natural and process-induced sources. In this perspective, we survey previous modeling approaches for lignocellulose and simulations processes involving its chemical and mechanical transformation, and suggest opportunities for future development to enhance the utility of computational tools to address barriers to widespread adoption of a renewable bioeconomy.

09 BIOMASS FUELS↗
DOE OSTI2024

Excited-State and Nonadiabatic Molecular Dynamics Methods with Broad Applicability

Electronically excited states play a pivotal role in the capture, transformation, storage, and emission of radiative energy. Despite much recent progress for small model systems, excited state properties and dynamics in many larger molecular devices and materials have remained elusive for experiment and theory. Nonadiabatic transitions between electronic states turn radiative energy into heat and are particularly difficult to control. This project aims to develop new electronic structure and nonadiabatic molecular dynamics (NAMD) methods to simulate, predict, and ultimately control the flow and transformation of excitation energy in a wide-range of light-driven molecular devices such as dyes, photocatalysts, and fluorescence emitters.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Multi-scale modeling of wastage layer formation in metallic fuel cladding

Fuel-cladding chemical interaction (FCCI) is a major concern for U-Zr metallic fuels' performance, primarily due to the formation of a brittle layer (wastage) in the cladding. This brittle layer, resulting from intermetallic compounds between cladding constituents Fe, Cr, and lanthanide fission products, significantly impacts the cladding's mechanical integrity. Recent efforts focus on developing a mechanistic modeling framework to understand lanthanide production, transport to the fuel-cladding interface, and phase transformation to intermetallic phases. A multi-scale computational approach has been used to calculate lanthanide transport rates, with atomistic calculations determining Nd diffusivities through the solid fuel matrix and along pore surfaces. These diffusivities inform a mesoscale model to determine an effective diffusion coefficient, accounting for porosity and infiltration with bond sodium. This effective diffusivity is used in engineering-scale simulations via the BISON fuel performance code, which has been validated against EBR-II and FFTF reactor experiments.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2026

Hybrid Oscillator-Qubit Quantum Processors: Instruction Set Architectures, Abstract Machine Models, and Applications

This tutorial offers a pedagogical guide to hybrid quantum processors that integrate discrete-variable (DV) qubits and continuous-variable (CV) oscillators. Aimed at computer scientists, engineers, and physicists, it provides an overview of the experimental, algorithmic, and architectural aspects of this novel and rapidly developing hardware model. Experimental realizations of this model include superconducting, trapped-ion, and neutral-atom platforms. By combining DV and CV components, hybrid oscillator-qubit processors enable a powerful new paradigm that offers complementary strengths for quantum control, error correction, computation, and simulation. Working toward the goal of a full-stack system connecting applications to CV-DV hardware, we define and formulate abstract machine models and instruction set architectures. These essential abstractions enable codesign of hardware and software, and resource estimation for exploring the potential of current and future hardware for computational and simulation tasks. Using these abstractions, we present both new and existing examples that illustrate the benefits of hybrid CV-DV processors relative to traditional DV-only hardware in computation as well as quantum simulation of physical models. Examples include algorithms for transferring states between DV and CV systems, performing the quantum Fourier transform, and simulation of lattice gauge theories. Relative to qubit-only hardware, the bosonic degrees of freedom natively available in hybrid architectures can substantially reduce the circuit complexity of simulations for physical models containing bosons. A key technique is the extension of quantum signal processing ideas to CV-DV systems. This work is intended to serve as a timely and comprehensive guide to this relatively unexplored yet promising approach to quantum computation and to provide a road map to guide future development.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

On the Use of Smart Meter Data to Estimate the Voltage Magnitude on the Primary Side of Distribution Service Transformers

This paper develops a novel method to estimate the voltage magnitude on the primary side of distribution service transformers. The proposed method relies exclusively on smart meters, and therefore it is fully data-driven. This is an important feature because electric utilities have detailed models of only the primary network - that is, the network between the distribution substation and the primary side of service transformers that are installed closer to end-customer sites. The network that connects the secondary side of service transformers to end-customer sites, referred to as the secondary network, is simply represented by a lumped load. For each secondary network, the proposed method uses data acquired from only 2 smart meters: the closest and the farthest-in the sense of electrical distance - from the service transformer. As a reference to this feature, the proposed method is named SM2Vp. To our knowledge, this is the first time a method is shown to provide actionable information for realtime operation and control of power distribution grids using only two smart meters per secondary network. This is important because utilities have experienced barriers in managing and using large data sets for real-time operation and control. SM2Vp is primarily intended to provide pseudo-measurements for distribution system state estimation, but it can also be used directly for voltage control schemes. The performance of SM2Vp is demonstrated by numerical simulations carried out on three secondary network synthetic models and by using field data provided by a utility partner serving customers in southwestern California. A maximum relative error of approximately 3.9% or less is observed for the primary voltage magnitude estimates in all numerical experiments.

distribution service transformer↗
DOE OSTI2023

Number of sites-based solver for determining coverages from steady-state mean-field micro-kinetic models

Kinetic models parameterized by ab-initio calculations have led to significant improvements in understanding chemical reactions in heterogeneous catalysis. These studies have been facilitated by implementations which determine steady-state coverages and rates of mean-field micro-kinetic models. As implemented in the open-source kinetic modeling program, CatMAP, the conventional solution strategy is to use a root-finding algorithm to determine the coverage of all intermediates through the steady-state expressions, constraining all coverages to be non-negative and to properly sum to unity. Though intuitive, this root-finding strategy causes issues with convergence to solution due to these imposed constraints. In this work, we avoid explicitly imposing these constraints, solving the mean-field steady-state micro-kinetic model in the space of number of sites instead of solving it in the space of coverages. Here, we transform the constrained root-finding problem to an unconstrained least-squares minimization problem, leading to significantly improved convergence in solving micro-kinetic models and thus enabling the efficient study of more complex catalytic reactions.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗