Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

A crystal-plasticity-informed Gaussian Process Regression model to capture anisotropy in single crystal shape memory alloys

This work presents a machine learning (ML) framework that model the anisotropic actuation responses in a shape memory alloy. A Gaussian Process Regression (GPR) based ML model is trained on a set of different crystal orientations subjected to different actuation conditions. The training employed thermo-mechanical responses from a crystal-plasticity model that captures phase-transformation, stress-induced plasticity, and transformation-induced plasticity. Further, on training the GPR-ML model at fixed stress level for different orientations, it captured the thermo-mechanical responses accounting for the anisotropy, and predicted responses for new orientations with good accuracy. The GPR-ML model is able to capture the transformation temperature variations even when trained using multiple stress levels, and the transformation strain showed significant deviations. The developed GPR-ML model gave reasonable predictions for an unexplored sample set of orientations and loading conditions.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Exploring Functional Materials by Understanding Beam‐Sample Interactions

Abstract Ultra‐low‐dose electron diffraction is performed with a double metal cyanide catalyst (DMC) to understand how electron irradiation stimulates structural alterations in functional materials. The commonly fading diffraction patterns with dose accumulation depend on the irradiated area and the beam current even when below 50 femto Amperes. Heat generation is observed and modeled by statistical, inelastic scattering events to describe how phonon excitations modulate radiation hardness. Specifically, the characteristic 1/e‐decay of Bragg intensities from DMC is delayed from 6 to 30 eÅ −2 at room temperature, which is comparable to the effect of embedding radiation soft matter in ice. DMC's radiation hardness is enhanced by a latency dose that forms during a phase transformation. This unifying model predicts that a critical dose rate exists for any material that varies between 0.1 and 10 4 eÅ −2 s −1 because of a material dependent competition of heat generation and spread. It shows that Brillouin scattering causes time dependent perturbations in electron irradiated solids that trigger time‐temperature‐transformations on a time scale of nanoseconds to microseconds at room temperature, which is not included in traditional models describing the decay of Bragg intensities by radiolysis.

Kisielowski, Christian↗
DOE OSTI2026

Deep learning for time series forecasting: a survey of recent advances

Time series forecasting plays a critical role in numerous real-world applications, such as finance, healthcare, transportation, and scientific computing. In recent years, deep learning has become a powerful tool for modeling complex temporal patterns and improving forecasting accuracy. This survey provides an overview of recent deep learning approaches for time series forecasting, involving various architectures including RNNs, CNNs, GNNs, transformers, large language models, MLP-based models, and diffusion models. We first identify key challenges in the field, such as temporal dependency, efficiency, and cross-variable dependency, which drive the development of forecasting techniques. Then, the general advantages and limitations of each architecture are discussed to contextualize their adaptation in time series forecasting. Furthermore, we highlight promising design trends like multi-scale modeling, decomposition, and frequency-domain techniques, which are shaping the future of the field. This paper serves as a compact reference for researchers and practitioners seeking to understand the current landscape and future trajectory of deep learning in time series forecasting.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Modeling Cosmic Reionization

The transformation of cold neutral intergalactic hydrogen into a highly ionized warm plasma marks the end of the cosmic dark ages and the beginning of the age of galaxies. The details of this process reflect the nature of the early sources of radiation and heat, the statistical characteristics of the large-scale structure of the Universe, the thermodynamics and chemistry of cosmic baryons, and the histories of star formation and black hole accretion. A number of massive data sets from new ground- and space-based instruments and facilities over the next decade are poised to revolutionize our understanding of primeval galaxies, the reionization photon budget, the physics of the intergalactic medium (IGM), and the fine-grained properties of hydrogen gas in the "cosmic web". In this review we survey the physics and key aspects of reionization-era modeling and describe the diverse range of computational techniques and tools currently available in this field.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Inference of bipolar neutrino flavor oscillations near a core-collapse supernova based on multiple measurements at Earth

Neutrinos in compact-object environments, such as core-collapse supernovae, can experience various kinds of collective effects in flavor space, engendered by neutrino-neutrino interactions. These include “bipolar” collective oscillations, which are exhibited by neutrino ensembles where different flavors dominate at different energies. Considering the importance of neutrinos in the dynamics and nucleosynthesis in these environments, it is desirable to ascertain whether an Earth-based detection could contain signatures of bipolar oscillations that occurred within a supernova envelope. To that end, we, in this study, continue examining a cost-function formulation of statistical data assimilation (SDA) to infer solutions to a small-scale model of neutrino flavor transformation. SDA is an inference paradigm designed to optimize a model with sparse data. Our model consists of two monoenergetic neutrino beams with different energies emanating from a source and coherently interacting with each other and with a matter background, with radially varying interaction strengths. We attempt to infer flavor transformation histories of these beams using simulated measurements of the flavor content at locations “in vacuum” (that is, far from the source), which could in principle correspond to Earth-based detectors. Within the scope of this small-scale model, we found that: (i) based on such measurements, the SDA procedure is able to infer whether bipolar oscillations had occurred within the protoneutron star envelope, and (ii) if the measurements sample the full amplitude of the neutrino oscillations in vacuum, then the amplitude of the prior bipolar oscillations is well predicted. This result intimates that the inference paradigm can well complement numerical integration codes, via its ability to infer flavor evolution at physically inaccessible locations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Uncertainty Quantification in High-Low Dynamic System Coupling Using RAVEN and TRANSFORM

This report demonstrates new functionality and applications stemming from the development of high fidelity to low fidelity (high-low) coupling for system simulations and to further explore the capabilities of RAVEN in the performance of uncertainty quantification in this kind of high-low coupled system models. The work builds from previous work on high-low coupling that utilized CTF, the high fidelity subchannel analysis code, with a low fidelity model built in TRANSFORM, the system analysis code, utilizing the Functional Mock-Up Interface (FMI)

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Flow matching meets biology and life science: a survey

Over the past decade, advances in generative modeling, such as generative adversarial networks, masked autoencoders, and diffusion models, have significantly transformed biological research and discovery, enabling breakthroughs in molecule design, protein generation, catalysis discovery, drug discovery, and beyond. At the same time, biological applications have served as valuable testbeds for evaluating the capabilities of generative models. Recently, flow matching has emerged as a powerful and efficient alternative to diffusion-based generative modeling, with growing interest in its application to problems in biology and life sciences. This paper presents the first comprehensive survey of recent developments in flow matching and its applications in biological domains. We begin by systematically reviewing the foundations and variants of flow matching, and then categorize its applications into three major areas: biological sequence modeling, molecule generation and design, and peptide and protein generation. For each, we provide an in-depth review of recent progress. We also summarize commonly used datasets and software tools, and conclude with a discussion of potential future directions.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Understanding Line Losses and Transformer Losses in Rural Isolated Distribution Systems: Preprint

Rural, isolated power systems in the mainland U.S. and in states like Alaska and Hawaii are powered by assets like diesel generators. These rural, isolated power systems also cannot operate at the higher band of medium voltage (like 69kV). They are primarily in the 12 to 14 kV range to keep the cost of the distribution investments lower. Because of this mid-band medium voltage range, the line losses and distribution transformers losses consume significant diesel consumption (almost 10 percent of the peak load). This work considers one such power system powering an isolated system and presents key findings online losses, and transformer losses. Understanding and documenting the impacts is critical for these communities operating their power systems and take actions to reduce expensive diesel consumption. In this paper, we will present one such typical grid and model it in electromagnetic transients (EMT) domain. We used the tower structure, under ground cabling installation to develop high fidelity models of lines. We also used high fidelity models of distribution transformers to present the no-load losses and full load loses. We will also present technical solutions available commercially off-the-shelf to reduce these losses and reduce diesel consumption. This work will be a primer for communities to understand the technical challenges and to understand the possible solution available to solve such challenges for rural, isolated power system operators.

blackstart↗
DOE OSTI2026

pyDiSCaMB : enabling the use of multipolar scattering factors in Phenix

Multipolar scattering models, such as the transferable aspherical atom model, account for atomic chemical interactions and provide a more accurate representation of experimental data. However, the simpler independent atom model (IAM), which assumes non-interacting atoms, is the only model available in the most widely used macromolecular refinement programs. This is primarily because IAM offers a hard-to-beat combination of computational efficiency and modelling power at typical macromolecular resolutions. By contrast, more accurate multipolar modelling has historically been limited due to its computational cost and the absence of an interface between software capable of calculating structure factors and gradients based on multipolar models and software designed for macromolecular refinement. This work introduces pyDiSCaMB , a Python software package designed to integrate between the computational crystallography toolbox ( cctbx ) and the quantum crystallography library DiSCaMB ( Densities in Structural Chemistry and Molecular Biology ), thus enabling multipolar scattering models in Phenix 's toolkit. The implementation, features and capabilities of pyDiSCaMB are presented, the runtimes for the calculation of structure factor and target gradients with respect to atomic parameters are explored, and Fourier images of electrostatic potential, electron density and deformation maps are computed as illustrative examples. The pyDiSCaMB library will make multipolar modelling widely available to the structural biology community, potentially transforming refinement and model-building for both crystallography and cryogenic electron microscopy (cryoEM).

MATTS data bank↗
DOE OSTI2021

Predicting phase transformation kinetics during metal additive manufacturing using non-isothermal Johnson-Mehl-Avrami models: Application to Inconel 718 and Ti-6Al-4V

Here, a computational model was developed to predict solid-state phase transformation kinetics within mechanical parts during metal additive manufacturing processes. This model is a modified version of the Johnson-Mehl-Avrami model for non-isothermal phase transformations that can be applied to various material systems undergoing solid-state phase transformations. Using the thermal history of an additive manufacturing fabricated mechanical part, along with the necessary thermodynamic data and kinetic information as inputs, the model outputs the history of phase fraction evolution during the build process. The model was applied to an Inconel 718 part built by powder bed fusion and a Ti-6Al-4V part built by directed energy deposition. Microstructure characterization and mechanical testing were performed for the validation of the model.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Transforming the study of organisms: Phenomic data models and knowledge bases

The rapidly decreasing cost of gene sequencing has resulted in a deluge of genomic data from across the tree of life; however, outside a few model organism databases, genomic data are limited in their scientific impact because they are not accompanied by computable phenomic data. The majority of phenomic data are contained in countless small, heterogeneous phenotypic data sets that are very difficult or impossible to integrate at scale because of variable formats, lack of digitization, and linguistic problems. One powerful solution is to represent phenotypic data using data models with precise, computable semantics, but adoption of semantic standards for representing phenotypic data has been slow, especially in biodiversity and ecology. Some phenotypic and trait data are available in a semantic language from knowledge bases, but these are often not interoperable. In this review, we will compare and contrast existing ontology and data models, focusing on nonhuman phenotypes and traits. We discuss barriers to integration of phenotypic data and make recommendations for developing an operationally useful, semantically interoperable phenotypic data ecosystem.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Sequence length scaling in vision transformers for scientific images on frontier

Vision Transformers (ViTs) are pivotal for foundational models in scientific imagery, including Earth science applications, due to their capability to process large sequence lengths. While transformers for text have inspired scaling sequence lengths in ViTs, adapting these for ViTs introduces unique challenges. We develop distributed sequence parallelism for ViTs, enabling them to handle up to 1M tokens. Our approach, leveraging DeepSpeed-Ulysses and Long-Sequence-Segmentation with model sharding, is the first to apply sequence parallelism in ViT training, achieving a 94% batch scaling efficiency on 2,048 AMD-MI250X GPUs. Evaluating sequence parallelism in ViTs, particularly in models up to 10B parameters, highlighted substantial bottlenecks. We countered these with hybrid sequence, pipeline, and flash attention strategies, to scale beyond single GPU memory limits. Our method significantly enhances climate modeling accuracy by 20% in temperature predictions, marking the first training of a vision transformer model to convergence with a sequence length of 188K tokens, using full self-attention.

Tsaris, Aristeidis (aris) [ORNL] (ORCID:0000000277↗
DOE OSTI2026

Large language models for transportation research: Methodologies, state of the art, and future opportunities

The rapid rise of large language models (LLMs) is transforming transportation research, with significant advancements emerging between 2023 and 2025, a period marked by the inception and swift growth of adopting and adapting LLMs for various transportation applications. Despite these significant advancements, however, a systematic review and synthesis of the existing literature remains lacking. This paper aims to fill this gap by providing a comprehensive review of the methodologies and applications of LLMs in transportation. We explore key applications, including autonomous driving, travel behavior prediction, and general transportation-related queries, alongside LLM methodologies such as zero- or few-shot learning, prompt engineering, and fine-tuning. From the review, critical research gaps are identified. From the methodological perspective, many of the research limitations can be addressed by integrating LLMs with existing tools and refining LLM architectures. From the application perspective, research opportunities for LLMs to address various transportation challenges are also explored. By synthesizing these findings, this review not only presents the state-of-the-art LLM adoption and adaptation in transportation, but also proposes future research directions as well as insights and recommendations for policymakers and practitioners, paving the way for greater LLM-driven research innovations in transportation in the future.

42 ENGINEERING↗
DOE OSTI2026

Towards Anomaly Detection at the CMS High-Level Trigger System

Traditional trigger strategies in CMS typically rely on model-dependent selections or rigid kinematic cuts, risking the omission of unexpected exotic signatures. To address this, we propose a novel anomaly detection (AD) algorithm for the High-Level Trigger (HLT), designed to serve as a complementary second layer of filtering to the Level-1 AXOL1TL AD algorithm. We employ a transformer-based foundation model trained on a diverse ensemble of Standard Model processes. By combining a joint contrastive and classification objective, and using particle kinematics as inputs, the model learns to map events to a physics-informed latent space where anomalous events are isolated from dominant backgrounds. Preliminary results show that this strategy enhances the signal-to-background ratio across a range of rare SM and BSM scenarios. Furthermore, this work constitutes foundational R&D for the potential implementation of an analogous AD algorithm in the Level-1 trigger system for Phase-2.

Cruz, Roy [U. Wisconsin, Madison (main)] (ORCID:00↗