Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Transformer eXplainability and eXploration

The Transformer eXplainability and eXploration library is intended to aid in the explorability and explainability of transformer classification networks, or transformer language models with sequence classification heads. The basic function of this library is to take a trained transformer and test/train dataset and produce an ipywidget dashboard which can be displayed in a jupyter notebook or in jupyter lab.

Martindale, Nathan [Oak Ridge National Lab. (ORNL)↗
DOE OSTI2023

Anomaly Detection and Identification Using a Leave-One-Variable-Out Method

At nuclear power plants (NPPs), anomaly detection and identification (i.e., determining the causes of anomalies) are important tasks for ensuring the safe and efficient operation of NPPs. These tasks are currently labor-intensive and costly, and are made more difficult by the size and complexity of NPP systems. An alternative approach to conducting these tasks is to automate them, such as via the reconstruction-based contribution method, which is a well-researched unsupervised machine learning method that uses a data-driven model of anomaly-free behavior to detect events and then identify each variable’s contributions to those events. The present effort developed a novel contribution approach that utilized a leave-one-variable-out (LOVO) model, with which each variable is predicted using all the other variables. The novelty lay in transforming this model into a reconstruction model and modifying the identification algorithm to work with the new reconstruction model. To evaluate this method in a controlled environment, a synthetic dataset based on spring-mass-damper (SMD) systems (commonly found in mechanical engineering references) was used, with known anomalies introduced into the system. The proposed method successfully detected the anomalies and afforded insights into their causes, thus enabling the appropriate identifications to be made.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2020

A phase-field model for non-isothermal phase transformation and plasticity in polycrystalline yttria-stabilized tetragonal zirconia

Here we propose an elastoplastic phase-field (PF) model to investigate the mechanics of tetragonal-to-monoclinic phase transformation (TMPT) and elastoplastic deformation of polycrystalline yttria-stabilized tetragonal zirconia (YSTZ). A Landau polynomial with non-vanishing chemical energy at the equilibrium temperature is introduced to account for the actual formation energies of the phases. The effects of different grain orientations, latent heat, and temperature on TMPT and deformation mechanisms are considered. The suppressive transformation effects of the grain boundaries (GBs) is modeled using an inhomogeneous kinetic coefficient in the bulk and GBs. The simulation results for single crystals demonstrate the capability of the model to reproduce the orientation-dependent compressive deformation of YSTZ similar to atomistic simulations and micropillar experiments. The single crystal with [100] crystallographic orientation along the loading direction (SC[100]) displays both TMPT and plasticity, SC[101] experiences only phase transformation, while SC[001] undergoes only plastic yielding. The TMPT induced by compressive loading exhibits shape memory effect (SME) below the equilibrium transformation temperature and pseudoelasticity above it, while the critical transformation stress increases with increasing loading temperature. The irrecoverable plastic strain is found to trap a part of the monoclinic phase, which prevents a complete reverse transformation. The polycrystalline cases also display SME and PE at low and high temperatures, respectively. Due to the orientation differences between grains and the stress concentrations at geometric nonlinearities, plastic deformation occurs in polycrystalline YSTZ for an applied load less than the yield stress. The results suggest a possible limitation of plasticity and an improvement of the shape recovery of YSTZ if one can control the orientation of the grains and/or increase the density of stacking faults at the GBs during material processing.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Texture Memory in Hexagonal Metals and Its Mechanism

Texture memory is a phenomenon in which retention of initial textures occurs after a complete cycle of forward and backward transformations, and it occurs in various phase-transforming materials including cubic and hexagonal metals such as steels and Ti and Zr alloys. Texture memory is known to be caused by the phenomena called variant selection, in which some of the allowed child orientations in an orientation relationship between the parent and child phases are preferentially selected. Without such variant selection, the phase transformations would randomize preferred orientations. In this article, the methods of prediction of texture memory and mechanisms of variant selections in hexagonal metals are explored. The prediction method using harmonic expansion of orientation distribution functions with the variant selection in which the Burgers orientation relationship, {110}β//{0001} α-hex <1$\bar{1}$1>β//$\langle2\bar{11}0\rangle$ α-hex , is held with two or more adjacent parent grains at the same time, called “double Burgers orientation relation (DBOR)”, is introduced. This method is shown to be a powerful tool by which to analyze texture memory and ultimately provide predictive capabilities for texture changes during phase transformations. Variation in nucleation and growth rates on special boundaries and an extensive growth of selected variants are also described. Analysis of textures of commercially pure Ti observed in situ by pulsed neutron diffraction reveals that the texture memory in CP-Ti is indeed quite well predicted by consideration of the mechanism of DBOR. The analysis also suggests that the nucleation and growth rates on the special boundary of 90° rotation about $\langle2\bar{11}0\rangle$ α-hex should be about three times larger than those of the other special boundaries, and the selected variants should grow extensively into not only one parent grain but also other grains in α-hex(hexagonal)→β(bcc) transformation. The model calculations of texture development during two consecutive cycles of α-hex→β→α-hex transformation in CP-Ti and Zr are also shown.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Convex Optimization of Integrated Power-Gas Energy Flow Model With Applications to Probabilistic Energy Flow

Energy flow calculation is a fundamental problem of the integrated power and gas system (IPGS) operation and planning. However, the nonlinear gas flow model introduces major challenges to the energy flow calculation. In this paper, we propose a tractably convex optimization model to solve the energy flow problem in IPGSs. It is demonstrated that the proposed optimization model has the same optimal solution as the original nonlinear steady energy flow model. Also, piecewise linearization is adopted to tightly linearize the nonlinear objective function of the model, which transforms the formulated convex optimization into a linear program one. Thus, the computation complexity of the proposed energy flow model is significantly reduced as compared with the existing methods. In addition, the proposed model can be extended to probabilistic energy flow estimation. Extensive case studies are conducted to validate the effectiveness of the proposed energy flow model using three IPGSs.

42 ENGINEERING↗
NASA NTRS2022

Modeling Demographic-Driven Vegetation Dynamics and Ecosystem Biogeochemical Cycling in NASA GISS’s Earth System Model (ModelE-BiomeE v.1.0)

We developed a demographic vegetation model, BiomeE, to improve the modeling of vegetation dynamics and ecosystem biogeochemical cycles in the NASA Goddard Institute of Space Studies' ModelE Earth system model. This model includes the processes of plant growth, mortality, reproduction, vegetation structural dynamics, and soil carbon and nitrogen storage and transformations. The model combines the plant physiological processes of ModelE's original vegetation model, Ent, with the plant demographic and ecosystem nitrogen processes that have been represented in the Geophysical Fluid Dynamics Laboratory's LM3-PPA. We used nine plant functional types to represent global natural vegetation functional diversity, including trees, shrubs, and grasses, and a new phenology model to simulate vegetation seasonal changes with temperature and precipitation fluctuations. Competition for light and soil resources is individual based, which makes the modeling of transient compositional dynamics and vegetation succession possible. Overall, the BiomeE model simulates, with fidelity comparable to other models, the dynamics of vegetation and soil biogeochemistry, including leaf area index, vegetation structure (e.g., height, tree density, size distribution, and crown organization), and ecosystem carbon and nitrogen storage and fluxes. This model allows ModelE to simulate transient and long-term biogeophysical and biogeochemical feedbacks between the climate system and land ecosystems. Furthermore, BiomeE also allows for the eco-evolutionary modeling of community assemblage in response to past and future climate changes with its individual-based competition and demographic processes.

Biogeochemical cycles↗
NASA NTRS1995

Object Detection in Natural Backgrounds Predicted by Discrimination Performance and Models

In object detection, an observer looks for an object class member in a set of backgrounds. In discrimination, an observer tries to distinguish two images. Discrimination models predict the probability that an observer detects a difference between two images. We compare object detection and image discrimination with the same stimuli by: (1) making stimulus pairs of the same background with and without the target object and (2) either giving many consecutive trials with the same background (discrimination) or intermixing the stimuli (object detection). Six images of a vehicle in a natural setting were altered to remove the vehicle and mixed with the original image in various proportions. Detection observers rated the images for vehicle presence. Discrimination observers rated the images for any difference from the background image. Estimated detectabilities of the vehicles were found by maximizing the likelihood of a Thurstone category scaling model. The pattern of estimated detectabilities is similar for discrimination and object detection, and is accurately predicted by a Cortex Transform discrimination model. Predictions of a Contrast- Sensitivity- Function filter model and a Root-Mean-Square difference metric based on the digital image values are less accurate. The discrimination detectabilities averaged about twice those of object detection.

Ahumada, A. J., Jr.↗
DOE OSTI2025

RINO: Renormalization Group Invariance with No Labels

A common challenge with supervised machine learning (ML) in high energy physics (HEP) is the reliance on simulations for labeled data, which can often mismodel the underlying collision or detector response. To help mitigate this problem of domain shift, we propose RINO (Renormalization Group Invariance with No Labels), a self-supervised learning approach that can instead pretrain models directly on collision data, learning embeddings invariant to renormalization group flow scales. In this work, we pretrain a transformer-based model on jets originating from quantum chromodynamic (QCD) interactions from the JetClass dataset, emulating real QCD-dominated experimental data, and then finetune on the JetNet dataset -- emulating simulations -- for the task of identifying jets originating from top quark decays. RINO demonstrates improved generalization from the JetNet training data to JetClass data compared to supervised training on JetNet from scratch, demonstrating the potential for RINO pretraining on real collision data followed by fine-tuning on small, high-quality MC datasets, to improve the robustness of ML models in HEP.

Hao, Zichun [Caltech] (ORCID:0000000256244907)↗
DOE OSTI2022

An Accuracy-Maximization Approach for Claims Classifiers in Document Content Analytics for Cybersecurity

This paper presents our research approach and findings towards maximizing the accuracy of our classifier of feature claims for cybersecurity literature analytics, and introduces the resulting model ClaimsBERT. Its architecture, after extensive evaluations of different approaches, introduces a feature map concatenated with a Bidirectional Encoder Representation from Transformers (BERT) model. We discuss deployment of this new concept and the research insights that resulted in the selection of Convolution Neural Networks for its feature mapping aspects. We also present our results showing ClaimsBERT to outperform all other evaluated approaches. This new claims classifier represents an essential processing stage within our vetting framework aiming to improve the cybersecurity of industrial control systems (ICS). Furthermore, in order to maximize the accuracy of our new ClaimsBERT classifier, we propose an approach for optimal architecture selection and determination of optimized hyperparameters, in particular the best learning rate, number of convolutions, filter sizes, activation function, the number of dense layers, as well as the number of neurons and the drop-out rate for each layer. Fine-tuning these hyperparameters within our model led to an increase in classification accuracy from 76% obtained with BertForSequenceClassification’s original model to a 97% accuracy obtained with ClaimsBERT.

Ameri, Kimia (ORCID:0000000328791871)↗
NASA NTRS1989

Fourier transform methods in local gravity modeling

New algorithms were derived for computing terrain corrections, all components of the attraction of the topography at the topographic surface and the gradients of these attractions. These algoriithms utilize fast Fourier transforms, but, in contrast to methods currently in use, all divergences of the integrals are removed during the analysis. Sequential methods employing a smooth intermediate reference surface were developed to avoid the very large transforms necessary when making computations at high resolution over a wide area. A new method for the numerical solution of Molodensky's problem was developed to mitigate the convergence difficulties that occur at short wavelengths with methods based on a Taylor series expansion. A trial field on a level surface is continued analytically to the topographic surface, and compared with that predicted from gravity observations. The difference is used to compute a correction to the trial field and the process iterated. Special techniques are employed to speed convergence and prevent oscillations. Three different spectral methods for fitting a point-mass set to a gravity field given on a regular grid at constant elevation are described. Two of the methods differ in the way that the spectrum of the point-mass set, which extends to infinite wave number, is matched to that of the gravity field which is band-limited. The third method is essentially a space-domain technique in which Fourier methods are used to solve a set of simultaneous equations.

Harrison, J. C.↗
DOE OSTI2025

Domain-specific text embedding model for accelerator physics

Accelerator physics presents unique challenges for natural language processing (NLP) due to its specialized terminology and complex concepts. A key component in overcoming these challenges is the development of robust text embedding models that transform textual data into dense vector representations, facilitating efficient information retrieval and semantic understanding. In this work, we introduce AccPhysBERT, a sentence embedding model fine-tuned specifically for accelerator physics. Our model demonstrates superior performance across a range of downstream NLP tasks, surpassing existing models in capturing the domain-specific nuances of the field. We further showcase its practical applications, including semantic paper-reviewer matching and integration into retrieval-augmented generation systems, highlighting its potential to enhance information retrieval and knowledge discovery in accelerator physics. Published by the American Physical Society 2025

Hellert, Thorsten (ORCID:0000000227970926)↗
DOE OSTI2021

A constitutive model for glass–ceramic materials

Glass–ceramics have received recent attention for use in glass–ceramic to metal hermetic seals. Due to their heterogeneous microstructure, these materials exhibit a number of advantageous responses over conventional glass based seals. Key amongst them is the possibility of a controllable thermal strain response and apparent coefficient of thermal expansion which may be used to minimize thermally induced residual stresses for aforementioned seals. Furthermore, these behaviors result from an inorganic glass matrix and variety of crystalline ceramic phases including silica polymorph(s) that may undergo reversible solid-to-solid transformations with associated inelastic strain. Correspondingly, these materials exhibit complex thermomechanical responses associated with multiple inelastic mechanisms (viscoelasticity and phase transformation). While modeling these behaviors is essential for developing and analyzing the corresponding applications, no such model exists. Therefore, in this work a three-dimensional continuum constitutive model for glass–ceramic materials combining these various inelastic mechanisms is developed via an internal state variable approach. A corresponding fully implicit three dimensional numerical formulation is also proposed and implemented. The model is used to simulate existing experiments and validate the proposed formalism. As an example, the simple seal problem of a glass–ceramic seal inside a concentric metal shell is explored. Finally, the impact of the cooling rates, viscoelastic shift factors, and inelastic strain on final residual stress state are all investigated and the differing contributions highlighted.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Plant litter traits control microbial decomposition and drive soil carbon stabilization

Efforts to manage soils for carbon (C) sequestration remain limited by our understanding of how differences in plant traits and microbial traits mechanistically drive soil organic C (SOC) storage. Addressing this uncertainty is particularly critical in bioenergy agriculture, due to its potential to enhance soil C and provide a C neutral fuel. As such, we examined differences between two contrasting feedstocks, Zea mays (corn) and Miscanthus x giganteus (miscanthus), in the ability of their litter to form new chemically resistant particulate SOC vs. physically protected mineral associated SOC and used this data to improve the parameterization of a microbial SOC model. We tested a hypothesized conceptual model whereby easy to decompose corn litters drive greater microbial carbon use efficiency (CUE) and the formation of more mineral associated SOC over particulate SOC than more complex miscanthus litters. To do this, we performed a soil microcosm experiment where we added 13 C enriched aboveground and belowground litters to soils and traced the fate of the 13 C into microbial respiration and SOC pools. We found that corn litters promoted higher microbial CUE (0.37) than miscanthus litters (0.24). In turn, corn litter formed approximately 50% more mineral associated SOC than miscanthus litters. Similarly, structurally complex root litters promoted a lower CUE and formed less mineral associated SOC than leaf and shoot litters for both crops. When we used our data to parameterize the SOC model, we found that modelling microbial trait differences uniquely allowed the model to capture the fate of litter C in SOC. Collectively, we found a robust link between litter quality, microbial efficiency, and the formation of SOC. This link bridges the empirical uncertainty in how different crops can form new soil C and provides an empirical basis for modelling SOC transformations.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

MLSPICE: Machine Learning based SPICE Modeling Platform for Power Magnetics

Electrical power converters are critical to a wide range of applications ranging from renewable integration to transportation electrification, and can be a key factor determining the size, weight, and efficiency of energy conversion systems. Magnetic components are typically the largest and least efficient components in power electronics. While there have been major strides in the modeling and analysis of power semiconductor devices and circuit simulations, the necessary advances in the design of power magnetics have lagged. In this project, we have transformed the modeling and design of power magnetics with machine learning enabled methods and catalyze simultaneous disruptive improvements for ML-based power electronics design tools. A fully automated open-source machine learning based magnetics modeling platform – the MagNet project - with innovations in full stack have been developed to greatly accelerate the design process and provide new insights to magnetic material and geometry design. The ARPA-E funded MagNet platform contains three major building blocks: 1) a ML-Integrated Data Acquisition System (MIDAS): a highly automated data acquisition testbed which is capable of measuring a large number of magnetic cores with a wide range of electrical circuit excitations; 2) a ML-integrated Core Loss Model (MICLM): a machine-learning trained modeling method for modeling the core loss and saturation effects of magnetic materials for arbitrary excitation waveforms; 3) ML-guided Magnetics SPICE Simulation Tool (PMSPICE): a fully integrated CAD tool which can simulate the magnetics in SPICE. It can help the designers to quickly model the linear and non-linear characteristics of magnetic components and evaluate their behavior in SPICE simulations. The developed MagNet system has fully demonstrated the proposed performance target and has been open sourced to the entire power electronics community to advance the modeling and design of power magnetics from many different angles.

36 MATERIALS SCIENCE↗