Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Deep learning for time series forecasting: a survey of recent advances

Time series forecasting plays a critical role in numerous real-world applications, such as finance, healthcare, transportation, and scientific computing. In recent years, deep learning has become a powerful tool for modeling complex temporal patterns and improving forecasting accuracy. This survey provides an overview of recent deep learning approaches for time series forecasting, involving various architectures including RNNs, CNNs, GNNs, transformers, large language models, MLP-based models, and diffusion models. We first identify key challenges in the field, such as temporal dependency, efficiency, and cross-variable dependency, which drive the development of forecasting techniques. Then, the general advantages and limitations of each architecture are discussed to contextualize their adaptation in time series forecasting. Furthermore, we highlight promising design trends like multi-scale modeling, decomposition, and frequency-domain techniques, which are shaping the future of the field. This paper serves as a compact reference for researchers and practitioners seeking to understand the current landscape and future trajectory of deep learning in time series forecasting.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1994

Implications of summertime marine stratocumulus on the North American climate

This study focuses on the effects of summertime stratocumulus over the eastern Pacific. This cloud is linked to the semi-permanent sub-tropical highs that dominate the low-level circulation over the Pacific and Atlantic. Subsidence on the eastern flank of these highs creates an inversion based about 800 m above sea level that caps moist air near the surface. This air overlies cool waters driven by upwelling along the coastal regions of North America. Strong surface north-westerlies mix the boundary layer enough to saturate the air just below the capping inversion. Widespread stratocumulus is thus formed. All calculations were carried out using the GENESIS general circulation model that was run at MSFC. Among the more important properties of the model is that it includes radiative forcing due to absorption of solar radiation and the emission of infrared radiation, interactive clouds (both stratocumulus and cumulus types), exchanges of heat and moisture with the lower boundary. Clouds are interactive in the sense that they impact the circulation by modifying the fields of radiative heating and turbulent fluxes of heat and moisture in the boundary layer. In turn, clouds are modified by the winds through the advection of moisture. In order to isolate the effects of mid- and high-latitude stratocumulus, two runs were made with the model: one with and the other without stratocumulus. The runs were made for a year, but with perpetual July conditions, i.e., solar forcing was fixed. The diurnal solar cycle, however, was allowed for. The sea surface temperature distribution was fixed in both runs to represent climatological July conditions. All dependent variables were represented at 12 surfaces of constant sigma = p/p(sub O), where p is pressure and p(sub O) is surface pressure. To facilitate analysis, model output was transformed to constant pressure surfaces. Structures no smaller in size than 7.5 degrees longitude and 4.5 degrees in latitude were resolved. Smaller features of the circulation were parameterized. The model thus captures synoptic- and planetary-scale circulation features.

Clark, John H. E.↗
NASA NTRS1996

Effect of Pt Doping on Nucleation and Crystallization in Li2O.2SiO2 Glass: Experimental Measurements and Computer Modeling

Heterogeneous nucleation and its effects on the crystallization of lithium disilicate glass containing small amounts of Pt are investigated. Measurements of the nucleation frequencies and induction times with and without Pt are shown to be consistent with predictions based on the classical nucleation theory. A realistic computer model for the transformation is presented. Computed differential thermal analysis data (such as crystallization rates as a function of time and temperature) are shown to be in good agreement with experimental results. This modeling provides a new, more quantitative method for analyzing calorimetric data.

Narayan, K. Lakshmi↗
NASA NTRS1988

A test of alternative Caribbean plate relative motion models

The new NUVEL-1 data set for global relative plate motions is used here to discriminate between the two prevailing models for Caribbean plate motion. One model, by Jordan (1975), assumes that North America-Caribbean motion is reflected by the spreading rate inferred from magnetic anomalies at the Cayman Spreading Center and the azimuths of nearby transforms. The other model, by Sykes et al. (1982), uses rates and azimuths inferred from the geometry of the Lesser Antilles Wadati-Benioff zone. Overall, it is found that the data fit the Jordan geometry better, that the data used in global plate motion models are more suitable than rates and azimuths inferred from the geometry of the Wadati-Benioff zone for determining relative motions, and that incorporation of all relevant plate boundaries is essential.

Stein, Seth↗
NASA NTRS2013

Bulk Insolation Models as Predictors for Locations for High Lunar Hydrogen Concentrations

In this study we consider the bulk effects of surface illumination on topography (insolation) and the possible thermodynamic effects on the Moon's hydrogen budget. Insolation is important as one of the dominant loss processes governing distributions of hydrogen volatiles on the Earth, Mars and most recently Mercury. We evaluated three types of high latitude > 65 deg., illumination models that were derived from the Lunar Observing Laser Altimetry (LOLA) digital elevation models (DEM)'s. These models reflect varying accounts of solar flux interactions with the Moon's near-surface. We correlate these models with orbital collimated epithermal neutron measurements made by the Lunar Exploration Neutron Detector (LEND). LEND's measurements derive the Moon's spatial distributions of hydrogen concentration. To perform this analysis we transformed the topographic model into an insolation model described by two variables as each pixels 1) slope and 2) slope angular orientation with respect to the pole. We then decomposed the illumination models and epithermal maps as a function of the insolation model and correlate the datasets.

Hydrogen Concentrations↗
DOE OSTI2022

Modeling Cosmic Reionization

The transformation of cold neutral intergalactic hydrogen into a highly ionized warm plasma marks the end of the cosmic dark ages and the beginning of the age of galaxies. The details of this process reflect the nature of the early sources of radiation and heat, the statistical characteristics of the large-scale structure of the Universe, the thermodynamics and chemistry of cosmic baryons, and the histories of star formation and black hole accretion. A number of massive data sets from new ground- and space-based instruments and facilities over the next decade are poised to revolutionize our understanding of primeval galaxies, the reionization photon budget, the physics of the intergalactic medium (IGM), and the fine-grained properties of hydrogen gas in the "cosmic web". In this review we survey the physics and key aspects of reionization-era modeling and describe the diverse range of computational techniques and tools currently available in this field.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Inference of bipolar neutrino flavor oscillations near a core-collapse supernova based on multiple measurements at Earth

Neutrinos in compact-object environments, such as core-collapse supernovae, can experience various kinds of collective effects in flavor space, engendered by neutrino-neutrino interactions. These include “bipolar” collective oscillations, which are exhibited by neutrino ensembles where different flavors dominate at different energies. Considering the importance of neutrinos in the dynamics and nucleosynthesis in these environments, it is desirable to ascertain whether an Earth-based detection could contain signatures of bipolar oscillations that occurred within a supernova envelope. To that end, we, in this study, continue examining a cost-function formulation of statistical data assimilation (SDA) to infer solutions to a small-scale model of neutrino flavor transformation. SDA is an inference paradigm designed to optimize a model with sparse data. Our model consists of two monoenergetic neutrino beams with different energies emanating from a source and coherently interacting with each other and with a matter background, with radially varying interaction strengths. We attempt to infer flavor transformation histories of these beams using simulated measurements of the flavor content at locations “in vacuum” (that is, far from the source), which could in principle correspond to Earth-based detectors. Within the scope of this small-scale model, we found that: (i) based on such measurements, the SDA procedure is able to infer whether bipolar oscillations had occurred within the protoneutron star envelope, and (ii) if the measurements sample the full amplitude of the neutrino oscillations in vacuum, then the amplitude of the prior bipolar oscillations is well predicted. This result intimates that the inference paradigm can well complement numerical integration codes, via its ability to infer flavor evolution at physically inaccessible locations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Uncertainty Quantification in High-Low Dynamic System Coupling Using RAVEN and TRANSFORM

This report demonstrates new functionality and applications stemming from the development of high fidelity to low fidelity (high-low) coupling for system simulations and to further explore the capabilities of RAVEN in the performance of uncertainty quantification in this kind of high-low coupled system models. The work builds from previous work on high-low coupling that utilized CTF, the high fidelity subchannel analysis code, with a low fidelity model built in TRANSFORM, the system analysis code, utilizing the Functional Mock-Up Interface (FMI)

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Flow matching meets biology and life science: a survey

Over the past decade, advances in generative modeling, such as generative adversarial networks, masked autoencoders, and diffusion models, have significantly transformed biological research and discovery, enabling breakthroughs in molecule design, protein generation, catalysis discovery, drug discovery, and beyond. At the same time, biological applications have served as valuable testbeds for evaluating the capabilities of generative models. Recently, flow matching has emerged as a powerful and efficient alternative to diffusion-based generative modeling, with growing interest in its application to problems in biology and life sciences. This paper presents the first comprehensive survey of recent developments in flow matching and its applications in biological domains. We begin by systematically reviewing the foundations and variants of flow matching, and then categorize its applications into three major areas: biological sequence modeling, molecule generation and design, and peptide and protein generation. For each, we provide an in-depth review of recent progress. We also summarize commonly used datasets and software tools, and conclude with a discussion of potential future directions.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS1980

An equivalent source model of the satellite-altitude magnetic anomaly field over Australia

The low-amplitude, long-wavelength magnetic anomaly field measured between 400 and 700 km elevation over Australia by the POGO satellites is modeled by means of the equivalent source technique. Magnetic dipole moments are computed for a latitude-longitude array of dipole sources on the earth's surface such that the dipoles collectively give rise to a field which makes a least squares best fit to that observed. The distribution of magnetic moments is converted to a model of apparent magnetization contrast in a layer of constant (40 km) thickness, which contains information equivalent to the lateral variation in the vertical integral of magnetization down to the Curie isotherm and can be transformed to a model of variable thickness magnetization. It is noted that the closest equivalent source spacing giving a stable solution is about 2.5 deg, corresponding to about half the mean data elevation, and that the magnetization distribution correlates well with some of the principle tectonic elements of Australia.

Mayhew, M. A.↗
DOE OSTI2024

Understanding Line Losses and Transformer Losses in Rural Isolated Distribution Systems: Preprint

Rural, isolated power systems in the mainland U.S. and in states like Alaska and Hawaii are powered by assets like diesel generators. These rural, isolated power systems also cannot operate at the higher band of medium voltage (like 69kV). They are primarily in the 12 to 14 kV range to keep the cost of the distribution investments lower. Because of this mid-band medium voltage range, the line losses and distribution transformers losses consume significant diesel consumption (almost 10 percent of the peak load). This work considers one such power system powering an isolated system and presents key findings online losses, and transformer losses. Understanding and documenting the impacts is critical for these communities operating their power systems and take actions to reduce expensive diesel consumption. In this paper, we will present one such typical grid and model it in electromagnetic transients (EMT) domain. We used the tower structure, under ground cabling installation to develop high fidelity models of lines. We also used high fidelity models of distribution transformers to present the no-load losses and full load loses. We will also present technical solutions available commercially off-the-shelf to reduce these losses and reduce diesel consumption. This work will be a primer for communities to understand the technical challenges and to understand the possible solution available to solve such challenges for rural, isolated power system operators.

blackstart↗
DOE OSTI2026

pyDiSCaMB : enabling the use of multipolar scattering factors in Phenix

Multipolar scattering models, such as the transferable aspherical atom model, account for atomic chemical interactions and provide a more accurate representation of experimental data. However, the simpler independent atom model (IAM), which assumes non-interacting atoms, is the only model available in the most widely used macromolecular refinement programs. This is primarily because IAM offers a hard-to-beat combination of computational efficiency and modelling power at typical macromolecular resolutions. By contrast, more accurate multipolar modelling has historically been limited due to its computational cost and the absence of an interface between software capable of calculating structure factors and gradients based on multipolar models and software designed for macromolecular refinement. This work introduces pyDiSCaMB , a Python software package designed to integrate between the computational crystallography toolbox ( cctbx ) and the quantum crystallography library DiSCaMB ( Densities in Structural Chemistry and Molecular Biology ), thus enabling multipolar scattering models in Phenix 's toolkit. The implementation, features and capabilities of pyDiSCaMB are presented, the runtimes for the calculation of structure factor and target gradients with respect to atomic parameters are explored, and Fourier images of electrostatic potential, electron density and deformation maps are computed as illustrative examples. The pyDiSCaMB library will make multipolar modelling widely available to the structural biology community, potentially transforming refinement and model-building for both crystallography and cryogenic electron microscopy (cryoEM).

MATTS data bank↗
NASA NTRSDate not supplied

Exploitation of a Validation Hierarchy for Modeling and Simulation

Across engineering there is an evolving need to increase reliance on physics-based simulation to develop, design and optimize engineering systems. This increased reliance on modeling and simulation has highlighted a growing need to transform the confidence that modeling and simulation analysts have in their results into credibility for systems engineers to design and field systems more quickly and with less physical testing. For isolated components of a complex system, where a single discipline may drive product design, this is less of a concern as the relationship is often straightforward and easy to explain. However, when these isolated components are integrated, and are expected to operate in a multi-disciplinary context in which safety critical systems are involved, new concepts and model assurance standards are required. In this paper we address this challenge by showing how a model validation hierarchy can be exploited to identify those model validation experiments that will contribute most to increasing confidence and credibility of modeling and simulation predictions. The approach that is adopted contains four main steps. The first step is the construction of a model validation hierarchy that links subsystems, assemblies, and components to a hierarchy of physical experiments that can be used support model validation. This hierarchy connects the concerns of systems engineers to those of the modeling and simulation analyst in a clear and logical way. The structure and content of this hierarchy is then used in a second step to establish which physical phenomena have the greatest impact on overall system performance metrics. A gap analysis technique, based upon modeling and simulation concerns, is then used to prioritize the important physical phenomenon. Unfortunately, a common outcome of such gap analyses is the identification of many important gaps and so, in the final step of our process, we advocate the use of a global sensitivity analysis as a means to complete the prioritization.

Verification and Validation↗
NASA NTRSDate not supplied

Exploitation of a Validation Hierarchy for Modeling and Simulation

Across engineering there is an evolving need to increase reliance on physics-based simulation to develop, design and optimize engineering systems. This increased reliance on modeling and simulation has highlighted a growing need to transform the confidence that modeling and simulation analysts have in their results into credibility for systems engineers to design and field systems more quickly and with less physical testing. For isolated components of a complex system, where a single discipline may drive product design, this is less of a concern as the relationship is often straightforward and easy to explain. However, when these isolated components are integrated, and are expected to operate in a multi-disciplinary context in which safety critical systems are involved, new concepts and model assurance standards are required. In this paper we address this challenge by showing how a model validation hierarchy can be exploited to identify those model validation experiments that will contribute most to increasing confidence and credibility of modeling and simulation predictions. The approach that is adopted contains four main steps. The first step is the construction of a model validation hierarchy that links subsystems, assemblies, and components to a hierarchy of physical experiments that can be used support model validation. This hierarchy connects the concerns of systems engineers to those of the modeling and simulation analyst in a clear and logical way. The structure and content of this hierarchy is then used in a second step to establish which physical phenomena have the greatest impact on overall system performance metrics. A gap analysis technique, based upon modeling and simulation concerns, is then used to prioritize the important physical phenomenon. Unfortunately, a common outcome of such gap analyses is the identification of many important gaps and so, in the final step of our process, we advocate the use of a global sensitivity analysis as a means to complete the prioritization.

Verification and Validation↗
DOE OSTI2021

Predicting phase transformation kinetics during metal additive manufacturing using non-isothermal Johnson-Mehl-Avrami models: Application to Inconel 718 and Ti-6Al-4V

Here, a computational model was developed to predict solid-state phase transformation kinetics within mechanical parts during metal additive manufacturing processes. This model is a modified version of the Johnson-Mehl-Avrami model for non-isothermal phase transformations that can be applied to various material systems undergoing solid-state phase transformations. Using the thermal history of an additive manufacturing fabricated mechanical part, along with the necessary thermodynamic data and kinetic information as inputs, the model outputs the history of phase fraction evolution during the build process. The model was applied to an Inconel 718 part built by powder bed fusion and a Ti-6Al-4V part built by directed energy deposition. Microstructure characterization and mechanical testing were performed for the validation of the model.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Transforming the study of organisms: Phenomic data models and knowledge bases

The rapidly decreasing cost of gene sequencing has resulted in a deluge of genomic data from across the tree of life; however, outside a few model organism databases, genomic data are limited in their scientific impact because they are not accompanied by computable phenomic data. The majority of phenomic data are contained in countless small, heterogeneous phenotypic data sets that are very difficult or impossible to integrate at scale because of variable formats, lack of digitization, and linguistic problems. One powerful solution is to represent phenotypic data using data models with precise, computable semantics, but adoption of semantic standards for representing phenotypic data has been slow, especially in biodiversity and ecology. Some phenotypic and trait data are available in a semantic language from knowledge bases, but these are often not interoperable. In this review, we will compare and contrast existing ontology and data models, focusing on nonhuman phenotypes and traits. We discuss barriers to integration of phenotypic data and make recommendations for developing an operationally useful, semantically interoperable phenotypic data ecosystem.

59 BASIC BIOLOGICAL SCIENCES↗