Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Overcoming the disconnect between energy system and climate modeling

Energy system models underpin decisions by energy system planners and operators. Energy system modeling faces a transformation: accounting for changing meteorological conditions imposed by climate change. To enable that transformation, a community of practice in energy-climate modeling has started to form that aims to better integrate energy system models with weather and climate models. In this work, we evaluate the disconnects between the energy system and climate modeling communities, then lay out a research agenda to bridge those disconnects. In the near-term, we propose interdisciplinary activities for expediting uptake of future climate data in energy system modeling. In the long-term, we propose a transdisciplinary approach to enable development of (1) energy-system-tailored climate datasets for historical and future meteorological conditions and (2) energy system models that can effectively leverage those datasets. This agenda increases the odds of meeting ambitious climate mitigation goals by systematically capturing and mitigating climate risk in energy sector decision-making.

17 WIND ENERGY↗
NASA NTRS2008

Optical-Microphysical Cirrus Model

A model is presented that permits the simulation of the optical properties of cirrus clouds as measured with depolarization Raman lidars. It comprises a one-dimensional cirrus model with explicit microphysics and an optical module that transforms the microphysical model output to cloud and particle optical properties. The optical model takes into account scattering by randomly oriented or horizontally aligned planar and columnar monocrystals and polycrystals. Key cloud properties such as the fraction of plate-like particles and the number of basic crystals per polycrystal are parameterized in terms of the ambient temperature, the nucleation temperature, or the mass of the particles. The optical-microphysical model is used to simulate the lidar measurement of a synoptically forced cirrostratus in a first case study. It turns out that a cirrus cloud consisting of only monocrystals in random orientation is too simple a model scenario to explain the observations. However, good agreement between simulation and observation is reached when the formation of polycrystals or the horizontal alignment of monocrystals is permitted. Moreover, the model results show that plate fraction and morphological complexity are best parameterized in terms of particle mass, or ambient temperature which indicates that the ambient conditions affect cirrus optical properties more than those during particle formation. Furthermore, the modeled profiles of particle shape and size are in excellent agreement with in situ and laboratory studies, i.e., (partly oriented) polycrystalline particles with mainly planar basic crystals in the cloud bottom layer, and monocrystals above, with the fraction of columns increasing and the shape and size of the particles changing from large thin plates and long columns to small, more isometric crystals from cloud center to top. The findings of this case study corroborate the microphysical interpretation of cirrus measurements with lidar as suggested previously.

Reichardt, J.↗
DOE OSTI2022

Image transformers for classifying acute lymphoblastic leukemia

Cancer is the leading cause of death by disease in American children. Each year, nearly 16,000 children in the United States and over 300,000 children globally are diagnosed with cancer. Leukemia is a form of blood cancer that originates in the bone marrow and accounts for one-third of pediatric cancers. This disease occurs when the bone marrow contains 20% or more immature white blood cell blasts. Acute lymphoblastic leukemia is the most prevalent leukemia type found in children, with half of all annual cases in the U.S. diagnosed for subjects under 20 years of age. To diagnose acute lymphoblastic leukemia, pathologists often conduct a morphological bone marrow assessment. This assessment determines whether the immature white blood cell blasts in bone marrow display the correct morphological characteristics, such as size and appearance of nuclei. Pathologists also use immunophenotyping via multi-channel flow cytometry to test whether certain antigens are present on the surface of blast cells; the antigens are used to identify the cell lineage of acute lymphoblastic leukemia. These manual processes require well-trained personnel and medical professionals, thus being costly in time and expenses. Computerized decision support via machine learning can accelerate the diagnosis process and reduce the cost. Training a reliable classification model to distinguish between mature and immature white blood cells is essential to the decision support system. Here, we adopted the Vision Transformer model to classify white blood cells. The Vision Transformer achieved superb classification performance compared to state-of-the-art convolutional neural networks while requiring less computational resources for training. Additionally, the latent self-attention architecture provided attention maps for a given image, providing clues as to which portion(s) of the image were significant in decision-making. We applied the Vision Transformer model and a convolutional neural network model to an acute lymphoblastic leukemia classification dataset of 12,528 samples and achieved accuracies of 88.4% and 86.2%.

Cho, Priscilla↗
DOE OSTI2023

A mesoscale crystal plasticity model to predict room-temperature deformation and martensitic transformation of high-strength Quenching and Partitioning (Q&P) Steels and validation with synchrotron X-ray diffraction

Renowned for the superior mechanical properties and adeptness at cold-forming, Quenching and Partitioning (QP) steels have gained prominence as a promising candidate material in fabricating safety-critical components in various industries. The pertinent research on QP steels focus on the martensitic transformation of the Retained Austenite (RA) phase during cold-forming, a crucial mechanism that substantially influences the overall strength and ductility of QP steels. The austenite stability and transformation rate heavily rely on the local strain path and the initial microstructure, which is challenging for analytical prediction. In this paper, a mesoscale model is developed to capture the deformation and transformation kinetics of QP steels inside the microstructure. The model integrates the detailed explicit microstructure, acquired from characterization experiments, into a high-resolution finite element (FE) mesh. It distinctly model the deformation and interaction between the various phases and the effect on the transformation of RA. The model is validated with high energy X-ray diffraction (HEXRD) data, and shows excellent capability in predicting the asymmetric stress-strain behavior under uniaxial tension and compression, as well as the martensitic transformation rate. The model is used to investigate the strain and load partitioning effect of surrounding matrix to the transformation of RA, offering insights into the complex behavior of QP980 and facilitates further material development.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Non-intrusive data-driven model reduction for differential–algebraic equations derived from lifting transformations

In this paper we present a non-intrusive data-driven approach for model reduction of nonlinear systems. The approach considers the particular case of nonlinear partial differential equations (PDEs) that form systems of partial differential–algebraic equations (PDAEs) when lifted to polynomial form. Such systems arise, for example, when the governing equations include Arrhenius reaction terms (e.g., in reacting flow models) and thermodynamic terms (e.g., the Helmholtz free energy terms in a phase-field solidification model). Using the known structured form of the lifted algebraic equations, the approach computes the reduced operators for the algebraic equations explicitly, using straightforward linear algebra operations on the basis matrices. The reduced operators for the differential equations are inferred from lifted snapshot data using operator inference, which solves a linear least squares regression problem. The approach is illustrated for the nonlinear model of solidification of a pure material. The lifting transformations reformulate the solidification PDEs as a system of PDAEs that have cubic structure. The operators of the lifted system for this solidification example have affine dependence on key process parameters, permitting us to learn a parametric reduced model with operator inference. Numerical experiments show the effectiveness of the resulting reduced models in capturing key aspects of the solidification dynamics.

42 ENGINEERING↗
NASA NTRS1990

Shock induced polymorphic transition in quartz, carbon, and boron nitride

The model proposed by Ahrens (1988) to explain the mechanism of the polymorphism in silicates is revised, and the revised model is applied to the quartz/stishovite, graphite/diamond, and graphite-boron nitride (g-BN) phase transformations. In this model, a key assumption is that transformation to a high-density amorphous or possibly liquid phase which rapidly crystallized to the high-pressure phase is triggered by the high temperatures in the shear band and upon crossing the metastable extension of a melting curve. Good agreement between the calcualted results and published data is obtained. The present theory predicts the standard entropy for cubic BN to be 0.4-0.5 J/g K.

Tan, Hua↗
DOE OSTI2023

Near-wall model for compressible turbulent boundary layers based on an inverse velocity transformation

In this work, a near-wall model, which couples the inverse of a recently developed compressible velocity transformation (Griffin et al., Proc. Natl Acad. Sci., vol. 118, 2021, p. 34) and an algebraic temperature–velocity relation, is developed for high-speed turbulent boundary layers. As input, the model requires the mean flow state at one wall-normal height in the inner layer of the boundary layer and at the boundary-layer edge. As output, the model can predict mean temperature and velocity profiles across the entire inner layer, as well as the wall shear stress and heat flux. The model is tested in an a priori sense using a wide database of direct numerical simulation high-Mach-number turbulent channel flows, pipe flows and boundary layers (48 cases, with edge Mach numbers in the range 0.77–11, and semi-local friction Reynolds numbers in the range 170–5700). The present model is significantly more accurate than the classical ordinary differential equation (ODE) model for all cases tested. The model is deployed as a wall model for large-eddy simulations in channel flows with bulk Mach numbers in the range 0.7–4 and friction Reynolds numbers in the range 320–1800. When compared to the classical framework, in the a posteriori sense, the present method greatly improves the predicted heat flux, wall stress, and temperature and velocity profiles, especially in cases with strong heat transfer. In addition, the present model solves one ODE instead of two, and has a computational cost and implementation complexity similar to that of the commonly used ODE model.

42 ENGINEERING↗
DOE OSTI2021

A Novel Data-based Stochastic Distribution Control for Non-Gaussian Stochastic Systems

In this note, we present a novel data-based approach to investigate the non-Gaussian stochastic distribution control problem. As the motivation of this note, the existing methods have been summarised regarding to the drawbacks, for example, neural network weights training for unknown stochastic distribution and so on. To overcome these disadvantages, a new transformation for dynamic probability density function is given by kernel density estimation using interpolation. Based upon this transformation, a representative model has been developed while the stochastic distribution control problem has been transformed into an optimisation problem. Then, data-based direct optimisation and identification-based indirect optimisation have been proposed. In addition, the convergences of the presented algorithms are analysed and the effectiveness of these algorithms has been evaluated by numerical examples. In summary, the contributions of this note are as follows: 1) a new data-based probability density function transformation is given; 2) the optimisation algorithms are given based on the presented model; and 3) a new research framework is demonstrated as the potential extensions to the existing stochastic distribution control.

42 ENGINEERING↗
DOE OSTI2024

De novo atomic protein structure modeling for cryoEM density maps using 3D transformer and HMM

Accurately building 3D atomic structures from cryo-EM density maps is a crucial step in cryo-EM-based protein structure determination. Converting density maps into 3D atomic structures for proteins lacking accurate homologous or predicted structures as templates remains a significant challenge. Here, we introduce Cryo2Struct, a fully automated de novo cryo-EM structure modeling method. Cryo2Struct utilizes a 3D transformer to identify atoms and amino acid types in cryo-EM density maps, followed by an innovative Hidden Markov Model (HMM) to connect predicted atoms and build protein backbone structures. Cryo2Struct produces substantially more accurate and complete protein structural models than the widely used ab initio method Phenix. Additionally, its performance in building atomic structural models is robust against changes in the resolution of density maps and the size of protein structures.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2026

Beyond interpolation: Physics-inspired gating transformers for extrapolating irradiation conditions to novel nuclear fuels

The qualification of advanced nuclear fuels relies on irradiation experiments in test reactors that emulate commercial conditions. Designing these tests requires accurate prediction of key irradiation quantities, particularly heat generation rate and burnup, yet obtaining them typically involves computationally expensive multi-step simulation workflows. We propose a physics-inspired gating transformer (PIGT) that integrates an inverse-square, distance-based attenuation into the encoder representation to bias attention toward physically relevant spatial relationships while retaining data-driven flexibility. Using MiniFuel irradiation data from the High Flux Isotope Reactor at Oak Ridge National Laboratory, we benchmark against ensemble methods, feedforward and recurrent networks, convolutional models, and standard transformers. While baseline models perform well under interpolation, they exhibit a pronounced generalization gap when evaluated on fuels not included in the training set. The proposed model consistently improves extrapolative accuracy and stability, yielding the strongest performance on unseen fuel configurations. These results indicate that a lightweight physics structure embedded within attention mechanisms can substantially improve robustness, enabling more reliable surrogate predictions to accelerate the design of nuclear fuel irradiation experiments.

Fuel qualification↗
DOE OSTI2025

Direct observations of transient weakening during phase transformations in quartz and olivine

Phase transformations are widely invoked as a source of rheological weakening during subduction, continental collision, mantle convection and various other geodynamic phenomena. However, despite more than half a century of research, the likelihood and magnitude of such weakening in nature remain poorly constrained. Here we use experiments performed on a synchrotron beamline to reveal transient weakening of up to three orders of magnitude during the polymorphic quartz to coesite (SiO 2 ) and olivine to ringwoodite (Fe 2 SiO 4 ) phase transitions. Weakening becomes increasingly prominent as the transformation outpaces deformation. We suggest that this behaviour is broadly applicable among silicate minerals undergoing first-order phase transitions and examine the likelihood of weakening due to the olivine-spinel, (Mg,Fe) 2 SiO 4 , transformation during subduction. Modelling suggests that cold, wet slabs are most susceptible to transformational weakening, consistent with geophysical observations of slab stagnation in the mantle transition zone beneath the western Pacific. In conclusion, our study highlights the importance of incorporating transformational weakening into geodynamic simulations and provides a quantitative basis for doing so.

Cross, Andrew J. [Woods Hole Oceanographic Institu↗
NASA NTRS1986

An introduction to three-dimensional climate modeling

The development and use of three-dimensional computer models of the earth's climate are discussed. The processes and interactions of the atmosphere, oceans, and sea ice are examined. The basic theory of climate simulation which includes the fundamental equations, models, and numerical techniques for simulating the atmosphere, oceans, and sea ice is described. Simulated wind, temperature, precipitation, ocean current, and sea ice distribution data are presented and compared to observational data. The responses of the climate to various environmental changes, such as variations in solar output or increases in atmospheric carbon dioxide, are modeled. Future developments in climate modeling are considered. Information is also provided on the derivation of the energy equation, the finite difference barotropic forecast model, the spectral transform technique, and the finite difference shallow water waved equation model.

Washington, W. M.↗
DOE OSTI2020

Steady-State Analysis of Microgrid Distributed Control Under Denial of Service Attacks

Distributed control is an effective method to coordinate multiple components in a microgrid. It usually has mandatory requirements for communication graph topology; changing the topology unexpectedly could disrupt the operation of a distributed controller. A denial of service (DoS) attack blocks communication channels to cause variations in communication graph. Unforeseen topology changes render a controller to receive incomplete system information. Control decisions made based on incorrect information can lead to significantly reduced grid supporting capability. As a result, the vital power balance between generation and load may be challenged. Determining power balance requires steady-state power flow analysis; however, the cyber-induced steady-state analysis is not well studied in the literature. Here, in this article, we show that a cyber–physical microgrid with unknown DoS attacks has augmented uncertain power flow equations, whose special structure creates new computational issue to determining power balance. To deal with the issue, we develop a novel cyber-induced microgrid model: the cyber–physical system is transformed into a pure physical model, where the disruption effects of DoS in the cyber layer are shown to be equivalent to the variations of the physical model parameters. With the transformed system, we develop a sufficient condition to ascertain the system power balance under unknown DoS attacks. In addition, practical insights for resilient distributed control design are shown as well.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2015

Computational Modeling and Experimental Characterization of Martensitic Transformations in Nicoal for Self-Sensing Materials

Fundamental changes to aero-vehicle management require the utilization of automated health monitoring of vehicle structural components. A novel method is the use of self-sensing materials, which contain embedded sensory particles (SP). SPs are micron-sized pieces of shape-memory alloy that undergo transformation when the local strain reaches a prescribed threshold. The transformation is a result of a spontaneous rearrangement of the atoms in the crystal lattice under intensified stress near damaged locations, generating acoustic waves of a specific spectrum that can be detected by a suitably placed sensor. The sensitivity of the method depends on the strength of the emitted signal and its propagation through the material. To study the transition behavior of the sensory particle inside a metal matrix under load, a simulation approach based on a coupled atomistic-continuum model is used. The simulation results indicate a strong dependence of the particle's pseudoelastic response on its crystallographic orientation with respect to the loading direction and suggest possible ways of optimizing particle sensitivity. The technology of embedded sensory particles will serve as the key element in an autonomous structural health monitoring system that will constantly monitor for damage initiation in service, which will enable quick detection of unforeseen damage initiation in real-time and during onground inspections.

Wallace, T. A.↗
DOE OSTI2024

Adaptive Patching for High-resolution Image Segmentation with Transformers

Attention-based models are proliferating in the space of image analytics, including segmentation. The standard method of feeding images to transformer encoders is to divide the images into patches and then feed the patches to the model as a linear sequence of tokens. For high-resolution images, e.g. microscopic pathology images, the quadratic compute and memory cost prohibits the use of an attention-based model, if we are to use smaller patch sizes that are favorable in segmentation. The solution is to either use custom complex multi-resolution models or approximate attention schemes. We take inspiration from Adapative Mesh Refinement (AMR) methods in HPC by adaptively patching the images, as a pre-processing step, based on the image details to reduce the number of patches being fed to the model, by orders of magnitude. This method has a negligible overhead, and works seamlessly with any attention-based model, i.e. it is a pre-processing step that can be adopted by any attention-based model without friction. We demonstrate superior segmentation quality over SoTA segmentation models for realworld pathology datasets while gaining a geomean speedup of 6.9× for resolutions up to 64K2, on up to 2, 048 GPUs.

Zhang, Enzhi↗
DOE OSTI2023

Topological structure of complex predictions

Abstract Current complex prediction models are the result of fitting deep neural networks, graph convolutional networks or transducers to a set of training data. A key challenge with these models is that they are highly parameterized, which makes describing and interpreting the prediction strategies difficult. We use topological data analysis to transform these complex prediction models into a simplified topological view of the prediction landscape. The result is a map of the predictions that enables inspection of the model results with more specificity than dimensionality-reduction methods such as tSNE and UMAP. The methods scale up to large datasets across different domains. We present a case study of a transformer-based model previously designed to predict expression levels of a piece of DNA in thousands of genomic tracks. When the model is used to study mutations in the BRCA1 gene, our topological analysis shows that it is sensitive to the location of a mutation and the exon structure of BRCA1 in ways that cannot be found with tools based on dimensionality reduction. Moreover, the topological framework offers multiple ways to inspect results, including an error estimate that is more accurate than model uncertainty. Further studies show how these ideas produce useful results in graph-based learning and image classification.

Computer Science↗
DOE OSTI2023

Protein model quality assessment using rotation–equivariant transformations on point clouds

Machine learning research concerning protein structure has seen a surge in popularity over the last years with promising advances for basic science and drug discovery. Working with macromolecular structure in a machine learning context requires an adequate numerical representation, and researchers have extensively studied representations such as graphs, discretized 3D grids, and distance maps. As part of CASP14, we explored a new and conceptually simple representation in a blind experiment: atoms as points in 3D, each with associated features. These features—initially just the basic element type of each atom—are updated through a series of neural network layers featuring rotation-equivariant convolutions. Starting from all atoms, we further aggregate information at the level of alpha carbons before making a prediction at the level of the entire protein structure. We find that this approach yields competitive results in protein model quality assessment despite its simplicity and despite the fact that it incorporates minimal prior information and is trained on relatively little data. As a result, its performance and generality are particularly noteworthy in an era where highly complex, customized machine learning methods such as AlphaFold 2 have come to dominate protein structure prediction.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Peer-to-Peer Energy Trading and Grid Impact Studies in Smart Communities

The rise of peer-to-peer (P2P) marketplace paradigms has transformed existing marketplace models, but the extent to which this approach can be applied to the energy marketplace has yet to be considered. In this paper, we examine existing approaches taken in the application of a P2P paradigm to the energy marketplace, further presenting an approach towards facilitating an online P2P energy marketplace, implementing a prototype P2P web application named SolTrade. Furthermore, we submit initial statistics based on simulated transactions facilitated through the platform, which illustrate the physical impact of marketplace transactions on the energy grid. In particular, these results show that, as the number of users rises, the chance of overloading the grid rises, but the chance of the grid being unable to sustain itself without an external source of energy falls.

peer-to-peer economy, web development, energy shar↗