Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

tomoCAM : fast model-based iterative reconstruction via GPU acceleration and non-uniform fast Fourier transforms

X-ray-based computed tomography is a well established technique for determining the three-dimensional structure of an object from its two-dimensional projections. In the past few decades, there have been significant advancements in the brightness and detector technology of tomography instruments at synchrotron sources. These advancements have led to the emergence of new observations and discoveries, with improved capabilities such as faster frame rates, larger fields of view, higher resolution and higher dimensionality. These advancements have enabled the material science community to expand the scope of tomographic measurements towards increasingly in situ and in operando measurements. In these new experiments, samples can be rapidly evolving, have complex geometries and restrictions on the field of view, limiting the number of projections that can be collected. In such cases, standard filtered back-projection often results in poor quality reconstructions. Iterative reconstruction algorithms, such as model-based iterative reconstructions (MBIR), have demonstrated considerable success in producing high-quality reconstructions under such restrictions, but typically require high-performance computing resources with hundreds of compute nodes to solve the problem in a reasonable time. Here, tomoCAM , is introduced, a new GPU-accelerated implementation of model-based iterative reconstruction that leverages non-uniform fast Fourier transforms to efficiently compute Radon and back-projection operators and asynchronous memory transfers to maximize the throughput to the GPU memory. The resulting code is significantly faster than traditional MBIR codes and delivers the reconstructive improvement offered by MBIR with affordable computing time and resources. tomoCAM has a Python front-end, allowing access from Jupyter -based frameworks, providing straightforward integration into existing workflows at synchrotron facilities.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Phase Field Modeling of Chemical Reaction Related Damage Evolution in Environmental Barrier Coatings

The advent of next-generation engines necessitates materials capable of withstanding temperatures beyond the reach of current superalloys. SiC-based ceramic matrix composites, augmented with environmental barrier coatings (EBCs), present a promising materials solution. Given the active search for effective and durable EBCs, there is a pressing need for modeling tools to understand and predict damage evolution in these materials to help accelerate their development. This study introduces a phase-field model (PFM) designed to simulate the thermally grown oxides (TGO) and phase transformation in the degradation and failure of EBCs. The model accounts for the severe volume expansion due to oxidation, alongside phase transformations and porosity evolution during thermal cycling, offering a comprehensive view of the damage processes. Simulation results are validated against experimental findings reported in the literature, establishing the model's potential as a significant tool for understanding and improving the resilience of EBCs in cyclic oxidative environments.

fast-diffusion path↗
DOE OSTI2025

RingX: Scalable Parallel Attention for Long-Context Learning on HPC

The attention mechanism has become foundational for remarkable AI breakthroughs since the introduction of the Transformer, driving the demand for increasingly longer context to power frontier models such as large-scale reasoning language models and high-resolution image/video generators. However, its quadratic computational and memory complexities present substantial challenges. Current state-of-the-art parallel attention methods, such as ring attention, are widely adopted for long-context training but utilize a point-to-point communication strategy that fails to fully exploit the capabilities of modern HPC network architectures. In this work, we propose ringX, a scalable family of parallel attention methods optimized explicitly for HPC systems. By enhancing workload partitioning, refining communication patterns, and improving load balancing, ringX achieves up to 3.4 × speedup compared to conventional ring attention on the Frontier supercomputer. Optimized for both bi-directional and causal attention mechanisms, ringX demonstrates its effectiveness through training benchmarks of a Vision Transformer (ViT) on a climate dataset and a Generative Pre-Trained Transformer (GPT) model, Llama3 8B. Our method attains an end-to-end training speedup of approximately 1.5 × in both scenarios. To our knowledge, the achieved 38% model FLOPs utilization (MFU) for training Llama3 8B with a 1M-token sequence length on 4,096 GPUs represents one of the highest training efficiencies reported for long-context learning on HPC systems. Our code implementation is available at https://github.com/jqyin/ringX-attention.

Yin, Junqi [ORNL] (ORCID:0000000338435520)↗
DOE OSTI2021

Towards Neutron Transformation Searches

To probe the origins of the baryon asymmetry, baryon number violation, the last unconfirmed Sakharov condition, must be definitively observed experimentally. Similarly, the nature of dark matter is currently unknown, and calls out for new candidates to be investigated. Each of these issues can be considered through the study of neutron transformations.Some rare baryon number violating processes, such as neutron-antineutron transformations, are expected to probe baryogenesis. Here, I show progress on this discovery target through construction of more accurate Monte Carlo models, the design of future detectors, creation of more complete atmospheric neutrino background simulations, and use of automated analysis techniques within the the NNBAR/HIBEAM experimental program at the European Spallation Source (ESS) and the Deep Underground Neutrino Experiment (DUNE). First simulation-based sensitivities for these experiments will be discussed. Modeling of rare neutron-antineutron transformation and subsequent annihilation will be discussed at length for multiple nuclei useful to these and other collaborations. To go along with this work, more comprehensive lepton-scattering nuclear models must be integrated into neutrino event generators for proper atmospheric neutrino background simulations. I discuss the first furnishing of these backgrounds for DUNE, and I highlight a potential path forward for the community in this vein using precisionmore » electron scattering modeling as a facsimile.Aspects of other potentially related neutron – mirror-neutron oscillations pertinent to dark matter and the neutron lifetime anomaly will also be considered for the ESS HIBEAM experiment. Here, I will present the first experimental sensitivity calculations for a broad range of modular experimental setups which will serve as research and design stepping stones toward NNBAR while producing a multitude of physics results over short time scales.« less

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Chatter detection in simulated machining data: a simple refined approach to vibration data

Vibration monitoring is a critical aspect of assessing the health and performance of machinery and industrial processes. This study explores the application of machine learning techniques, specifically the Random Forest (RF) classification model, to predict and classify chatter—a detrimental self-excited vibration phenomenon—during machining operations. While sophisticated methods have been employed to address chatter, this research investigates the efficacy of a novel approach to an RF model. The study leverages simulated vibration data, bypassing resource-intensive real-world data collection, to develop a versatile chatter detection model applicable across diverse machining configurations. The feature extraction process combines time-series features and Fast Fourier Transform (FFT) data features, streamlining the model while addressing challenges posed by feature selection. By focusing on the RF model’s simplicity and efficiency, this research advances chatter detection techniques, offering a practical tool with improved generalizability, computational efficiency, and ease of interpretation. The study demonstrates that innovation can reside in simplicity, opening avenues for wider applicability and accelerated progress in the machining industry.

42 ENGINEERING↗
DOE OSTI2022

Multi-reward Reinforcement Learning Based Bond-Order Potential to Study Strain-Assisted Phase Transitions in Phosphorene

Here, we introduce a multi-reward reinforcement learning (RL) approach to train a flexible bond-order potential (BOP) for 2D phosphorene based on ab initio training data sets. Our approach is based on a continuous action space Monte Carlo tree search algorithm that is general and scalable and presents an efficient multiobjective optimization scheme for high-dimensional materials design problems. As a proof-of-concept, we deploy this scheme to parametrize multiple structural and dynamical properties of 2D phosphorene polymorphs. Our RL-trained BOP model adequately captures the structure, energetics, transformation barriers, equation of state, elastic constants, and phonon dispersions of various 2D P polymorphs. We use this model to probe the impact of temperature and strain rate on the phase transition from black (α-P) to blue phosphorene (β-P) through molecular dynamics simulations. A decrease in critical strain for this phase transition with increase in temperature is observed, and the underlying atomistic mechanisms are discussed.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Sensitivity of deep convection and cross-tropopause water transport to microphysical parameterizations in WRF

Deep convective storms can overshoot the tropopause, thus altering the composition of the stratosphere by vertically transporting tropospheric air. The transport of water vapor and ice particles into a sub-saturated environment can hydrate the stratosphere, with implications for radiative forcing and ozone chemistry. Cloud-resolved models, if employed at high spatial resolutions, are used to probe process-level questions about cross-tropopause deep convective hydration and its controls. There is considerable diversity in model representations of processes associated with water transport and transformation, and the choice of a microphysics scheme affects model skill in simulating deep convective events. This motivates our evaluation of state-of-the-art, as well as widely used standard schemes, in a high spatial- and temporal-resolution framework. Six bulk microphysics schemes were employed in a WRF-LES setup, initialized with a sounding profile representative of a tropopause-overshooting storm. Here, we used an idealized framework to isolate the effect of microphysics on the dominant processes that control the reach of deep convection and stratospheric hydration. All schemes produced the highest reaching updrafts 8-12 hours into the simulation but the strength and persistence of updrafts varied across the schemes; maximum storm heights ranged 9.1-12.6 km across the schemes. Varying microphysics produced large differences in the vertical extent and horizontal aggregation of convection, and an order of magnitude spread in above-tropopause water vapor concentrations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Control Oriented Models for Co-Design: Technical Overview of MT HVDC, MVDC, and Solid State Transformer Building Blocks

The electric power system is shifting toward a power electronics–enabled grid, where converter based “building blocks” (e.g., high voltage direct current (HVDC) links, multi terminal HVDC (MT HVDC) networks, medium voltage DC (MVDC) links, and solid state transformers (SSTs)) provide fast, precise control of power flows, voltage, and frequency. This report develops and applies publicly shareable electromagnetic transient (EMT) and phasor models to examine how such building blocks can be composed and coordinated to support offshore wind integration, inter area transfers, feeder support, and resilience. Section 2 documents a modular multilevel converter (MMC)–based MT HVDC modeling framework and two use cases: a compact WSCC/IEEE 9 bus test system and a 240 bus “mini WECC” case with five offshore wind plants (OWFs). Phasor to EMT transfer, initialization, and sanity checks are summarized, and neutral demonstrations of normal and contingency operation are reported. Section 3 frames the problem of wind plant inertial frequency response (IFR): shaping energy release and recovery to improve nadir while avoiding aerodynamic stall; representative simulations illustrate the issues without disclosing proprietary control. Section 4 develops MVDC concepts through an IEEE 16 bus loop and an Olympic Peninsula case study that compares AC vs. MVDC corridors and shows how feeder headroom can be pooled via DC couplers. Section 5 surveys SST architectures and identifies a gap: scalable, communication free coordination of multiple SSTs for islanded feeder networks. Across the report, novel methods and configurations under separate publication and IP review are not disclosed; only topic oriented, replicable setups and non proprietary results are shown. These models and use cases are intended as foundations for future publications and co design studies on architecture, control, and coordination of PE enabled grids.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

An analytic hybrid halo + perturbation theory model for small-scale correlators: baryons, halos, and galaxies

In this work, we update Halo Zeldovich Perturbation Theory (HZPT, [1]), an analytic model for the two-point statistics of dark matter, to describe halo and galaxy clustering, and galaxy-matter cross-correlation on nonlinear scales. The model correcting Zeldovich has an analytic Fourier transform, and therefore is valid in both configuration space and Fourier space. The model is accurate at the 2%-level or less for P_mm (k< 1 h/Mpc), P_hm (k< 1 h/Mpc), P_hh (k< 2 h/Mpc), P gm (k< 1 h/Mpc), P gg (k< 1 h/Mpc), ξ mm (r> 1 Mpc/h), ξ hm (r> 2 Mpc/h), ξ hh (r> 2 Mpc/h), ξ gm (r> 1 Mpc/h), ξ gg (r> 2 Mpc/h), for LRG-like mock galaxies. We show that the HZPT model for matter correlators can account for the effects of a wide range of baryonic feedback models and provide two extended dark matter models which are of 1% (3%) accuracy for k < 10 (8) h/Mpc. We explicitly model the non-perturbative features of halo exclusion for the halo-halo and galaxy-galaxy correlators, as well as the presence of satellites for galaxy-matter and galaxy-galaxy correlation functions. We perform density estimation using N-body simulations and a wide range of HOD galaxy mocks to obtain correlations of model parameters with the cosmological parameters Ω m and σ 8 . HZPT can provide a fast, interpretable, and analytic model for combined-probe analyses of redshift surveys using scales well into the non-linear regime.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Inference of strength and phase transition kinetics in dynamically-compressed tin

Dynamic compression experiments in condensed matter are of interest in part because they provide opportunities to examine material response under extreme conditions; however, the inference of material behavior from dynamic experiments is challenging in the presence of phase transitions exhibiting kinetic processes. Demonstrating an approach to quantitative interpretation of such dynamic experiments, here we present a Bayesian model calibration of strength and phase transformation parameters to data drawn from pulsed power and gas gun shot experiments. The posterior predictions of the Bayesian model capture the experimental measurements and account for the various uncertainties in the experimental configurations. This holistic approach to model calibration utilizing multiple types of experimental data identifies important cross correlations among kinetics, strength, and the phase boundary. Improved insight into potential sources of current model form error is provided by comparing the differences between calibrations against different subsets of the experimental data.

42 ENGINEERING↗
DOE OSTI2023

Phase-Field Modeling of Materials Interfaces and Nanostructures

Nanostructured materials offer unique properties for a wide range of energy applications. Among various known processing methods, liquid metal dealloying (LMD)—the selective dissolution of a base alloy element into a metallic melt—has emerged as a powerful technique to produce a new class of nano-/meso-scale open porous and bicontinuous composite structures with ultra-high interfacial area. LMD has recently been complemented by the advent of vapor phase dealloying (VPD), a novel technique that exploits the selective evaporation of one element from a parent alloy containing elements with very different vapor pressures, thereby enabling to fabricate open nanoporous structures of various elements from less-noble metals to inorganic elements regardless of their chemical activity without requiring high LMD temperatures and chemical etching. Together LMD and VPD have greatly expanded the scope of dealloying techniques, limited by traditional electrochemical means to noble metals, and boosted the design of new functional and structural materials that combine a wide variety of elements. Topologically-connected open porous structures with ultra-high surface area allow mass transport within the structure while preserving structural integrity, enabling them to serve as catalysts, fuel cells, supercapacitors, or high-capacity battery materials. Bicontinuous composite structures in turn can display high strength and high ductility or superior radiation-damage resistance due to the ultra-high interface area between interpenetrating solid phases. This research program makes use of state-of-the-art computational methods to understand at a basic level the self-organizing dealloying process with main focus on dealloying kinetics and interfacial pattern formation at the dealloying front controlling initial structure size, topology, and phase compositions. Phase-field simulation studies of LMD focus on solid solutions, line compounds, and intermetallic systems that can form ternary composites by nucleation and growth of a new phase. Studies of VPD employ phase-field modeling and a hybrid method combining a kinetic Monte Carlo (KMC) model of evaporation and surface diffusion with molecular dynamics for vapor-phase transport inside nanopores. Simulations explore mechanisms of interface- and diffusion-controlled dealloying kinetics, both observed in VPD but not fundamentally understood. In addition, our newly developed multi-physics phase-field approach of large-volume-change phase transformations is being used to model novel 3D anode geometries based on dealloyed nanoporous structures including novel sandwiched graphene/Si/silica for highrate Li ion battery. Those studies are aimed at elucidating geometric design principles that improve mechanical stability. We expect this research to enhance the capability to tailor nano-/mesoscale structures for a wide range of energy-related materials applications and to yield further advances in computational methodologies that benefit a broad materials research community.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2021

A holistic framework integrating plant-microbe-mineral regulation of soil bioavailable nitrogen

Abstract Soil organic nitrogen (N) is a critical resource for plants and microbes, but the processes that govern its cycle are not well-described. To promote a holistic understanding of soil N dynamics, we need an integrated model that links soil organic matter (SOM) cycling to bioavailable N in both unmanaged and managed landscapes, including agroecosystems. We present a framework that unifies recent conceptual advances in our understanding of three critical steps in bioavailable N cycling: organic N (ON) depolymerization and solubilization; bioavailable N sorption and desorption on mineral surfaces; and microbial ON turnover including assimilation, mineralization, and the recycling of microbial products. Consideration of the balance between these processes provides insight into the sources, sinks, and flux rates of bioavailable N. By accounting for interactions among the biological, physical, and chemical controls over ON and its availability to plants and microbes, our conceptual model unifies complex mechanisms of ON transformation in a concrete conceptual framework that is amenable to experimental testing and translates into ideas for new management practices. This framework will allow researchers and practitioners to use common measurements of particulate organic matter (POM) and mineral-associated organic matter (MAOM) to design strategic organic N-cycle interventions that optimize ecosystem productivity and minimize environmental N loss.

Environmental Sciences & Ecology↗
DOE OSTI2026

Predicting Atomistic Transitions with Transformers

Accurate knowledge of the atomistic transition pathways in materials and material surfaces is crucial for many material science problems. However, conventional simulation techniques used to find these transitions are extremely computationally intensive. Even with large-scale, accelerated material simulations, the computational cost constrains the applicable domain in practice. Machine learning models, with the potential to learn the complex emergent behaviors governing atomistic transitions as a fast surrogate model, have great promise to predict transitions with a vastly reduced computational cost. Here, we demonstrate how transformers can be trained to predict atomistic transitions in nano-clusters. We show how we evaluate physical validity of the predictions and how a multitude of additional, different microstates can be generated by slightly varying the data provided to the model.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Lignin Utilization

Valorization of lignin has massive potential economic and sustainability benefits for the lignocellulosic biorefinery. However, challenges remain to realize lignin conversion to coproducts, especially related to selective, high yield depolymerization to monomers and quantitative analytics on lignin, the latter of which is critical for accurate process modeling. Towards these goals, the Lignin Utilization project focuses on catalytic lignin deconstruction chemistries for C-O and C-C bond cleavage, development of analytical chemistry techniques to quantitatively characterize lignin, and syntheses of requisite model compounds for understanding lignin transformations. This work is done with and supports multiple BETO projects, including the Biological Lignin Valorization (BLV) project, the Separations Consortium, and others. Outcomes of Lignin Utilization for analytics and synthesis include 1) development of new mass spectrometry (MS) methods to characterize lignin dimers/oligomers in process streams, 2) deployment of a computational-experimental tool (with Biochemical Process Modeling and Simulation) to identify lignin-derived compounds with high fidelity from MS, and 3) delivery of >40 unique compounds. From a catalysis perspective, we have developed new oxidative approaches to cleave C-O and C-C bonds and produce >50% bio-available aromatic monomers for biological funneling for the BLV project and have developed recoverable bases for base-catalyzed deconstruction of lignin.

BIOMASS FUELS↗
DOE OSTI2024

Normalizing flows for domain adaptation when identifying Λ hyperon events

Here this study focuses on the application of a normalizing flow as a method of domain adaptation when classifying physics data. Normalizing flows offer a way to transform data points between two different distributions. The present study investigates a novel method of transforming latent representations of physics data to a normal distribution and then to a physics distribution again. The final distribution models a simulated distribution. After being transformed, the data can be classified by a neural network trained on labeled simulation data. The present study succeeds in training two normalizing flows that can transform between data (or simulation) and a Gaussian distribution.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Exact representations of many-body interactions with restricted-Boltzmann-machine neural networks

Restricted Boltzmann machines (RBMs) are simple statistical models defined on a bipartite graph which have been successfully used in studying more complicated many-body systems, both classical and quantum. In this work, we exploit the representation power of RBMs to provide an exact decomposition of many-body contact interactions into one-body operators coupled to discrete auxiliary fields. This construction generalizes the well known Hirsch's transform used for the Hubbard model to more complicated theories such as pionless effective field theory in nuclear physics, which we analyze in detail. Finally, we also discuss possible applications of our mapping for quantum annealing applications and conclude with some implications for RBM parameter optimization through machine learning.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

HLA-Clus: HLA class I clustering based on 3D structure

In a previous paper, we classified populated HLA class I alleles into supertypes and subtypes based on the similarity of 3D landscape of peptide binding grooves, using newly defined structure distance metric and hierarchical clustering approach. Compared to other approaches, our method achieves higher correlation with peptide binding specificity, intra-cluster similarity (cohesion), and robustness. Here we introduce HLA-Clus, a Python package for clustering HLA Class I alleles using the method we developed recently and describe additional features including a new nearest neighbor clustering method that facilitates clustering based on user-defined criteria. The HLA-Clus pipeline includes three stages: First, HLA Class I structural models are coarse grained and transformed into clouds of labeled points. Second, similarities between alleles are determined using a newly defined structure distance metric that accounts for spatial and physicochemical similarities. Finally, alleles are clustered via hierarchical or nearest-neighbor approaches. We also interfaced HLA-Clus with the peptide:HLA affinity predictor MHCnuggets. By using the nearest neighbor clustering method to select optimal allele-specific deep learning models in MHCnuggets, the average accuracy of peptide binding prediction of rare alleles was improved. The HLA-Clus package offers a solution for characterizing the peptide binding specificities of a large number of HLA alleles. This method can be applied in HLA functional studies, such as the development of peptide affinity predictors, disease association studies, and HLA matching for grafting. HLA-Clus is freely available at our GitHub repository (https://github.com/yshen25/HLA-Clus).

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2022

Aboveground biomass density models for NASA’s Global Ecosystem Dynamics Investigation (GEDI) lidar mission

NASA's Global Ecosystem Dynamics Investigation (GEDI) is collecting spaceborne full waveform lidar data with a primary science goal of producing accurate estimates of forest aboveground biomass density (AGBD). This paper presents the development of the models used to create GEDI's footprint-level (~25 m) AGBD (GEDI04_A) product, including a description of the datasets used and the procedure for final model selection. The data used to fit our models are from a compilation of globally distributed spatially and temporally coincident field and airborne lidar datasets, whereby we simulated GEDI-like waveforms from airborne lidar to build a calibration database. We used this database to expand the geographic extent of past waveform lidar studies, and divided the globe into four broad strata by Plant Functional Type (PFT) and six geographic regions. GEDI's waveform-to-biomass models take the form of parametric Ordinary Least Squares (OLS) models with simulated Relative Height (RH) metrics as predictor variables. From an exhaustive set of candidate models, we selected the best input predictor variables, and data transformations for each geographic stratum in the GEDI domain to produce a set of comprehensive predictive footprint-level models. We found that model selection frequently favored combinations of RH metrics at the 98th, 90th, 50th, and 10th height above ground-level percentiles (RH98, RH90, RH50, and RH10, respectively), but that inclusion of lower RH metrics (e.g. RH10) did not markedly improve model performance. Second, forced inclusion of RH98 in all models was important and did not degrade model performance, and the best performing models were parsimonious, typically having only 1-3 predictors. Third, stratification by geographic domain (PFT, geographic region) improved model performance in comparison to global models without stratification. Fourth, for the vast majority of strata, the best performing models were fit using square root transformation of field AGBD and/or height metrics. There was considerable variability in model performance across geographic strata, and areas with sparse training data and/or high AGBD values had the poorest performance. These models are used to produce global predictions of AGBD, but will be improved in the future as more and better training data become available.

54 ENVIRONMENTAL SCIENCES↗