Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Modeling the impact of high thermal conductivity paper on the performance and life of power transformers

Degradation of insulation paper is a key contributor to the failure of power transformers. Insulation degradation accelerates at elevated temperatures, which highlights the potential for better thermal management to prolong life. While several studies have analyzed the benefits of high thermal conductivity oil for reducing temperatures inside a transformer, this study is an initial assessment of the benefits of high thermal conductivity paper on transformer life. Blending particulates with cellulosic fibers offers a pathway for high thermal conductivity paper (with good dielectric properties), which can reduce internal temperatures. Presently, life extensions that can be achieved by the use of such thermally conducting papers were estimated, with the thermal conductivity of the paper being the key parameter under study. The analytical-numerical thermal model used in this study was validated against experimental measurements in a distribution transformer, adding confidence to the utility of the model. This model was then used to provide estimates of hot-spot temperature reduction resulting from the use of papers with higher thermal conductivity than baseline. Transformer life was predicted conventionally by tracking the degree of polymerization of paper over time, based on an Arrhenius model. Results indicate that increasing the thermal conductivity of paper from 0.2 W/mK (baseline) to 1W/mK reduces the hot spot temperature by 10°C. While degradation significantly depends on the moisture and oxygen content, the model shows that such a temperature reduction can increase life for all conditions, by as much as a factor of three.

42 ENGINEERING↗
DOE OSTI2024

An ALEGRA MHD Spark Model

While modeling a generic pulse transformer, we became interested in the possibility of electric sparks between winding layers in a solid encapsulant. We significantly modified a previously developed ALEGRA MHD model of a generic spark in lexan. The cumulative modifications are significant enough to report here. Possibly the most significant modification was a change in how the simulated spark is initiated from a thin initial channel. The change was from imposing an initial hot temperature to imposing a conductivity floor. The reasons and comparisons of results are included. The second significant change was to replace a fixed current rise rate with an external circuit model. We built a model specifically mimicking the distributed inductance and stray capacitance between the coil turns closest to the modeled spark. Excursions from nominal values examine the sensitivity of resulting behaviors to extreme capacitance and inductance values.

42 ENGINEERING↗
DOE OSTI2020

Using metacommunity ecology to understand environmental metabolomes

Abstract Environmental metabolomes are fundamentally coupled to microbially-linked biogeochemical processes within ecosystems. However, significant gaps exist in our understanding of their spatiotemporal organization, limiting our ability to uncover transferrable principles and predict ecosystem function. We propose that a theoretical paradigm, which integrates concepts from metacommunity ecology, is necessary to reveal underlying mechanisms governing metabolomes. We call this synthesis between ecology and metabolomics ‘meta-metabolome ecology’ and demonstrate its utility using a mass spectrometry dataset. We developed three relational metabolite dendrograms using molecular properties and putative biochemical transformations and performed ecological null modeling. Based upon null modeling results, we show that stochastic processes drove molecular properties while biochemical transformations were structured deterministically. We further suggest that potentially biochemically active metabolites were more deterministically assembled than less active metabolites. Understanding variation in the influences of stochasticity and determinism provides a way to focus attention on which meta-metabolomes and which parts of meta-metabolomes are most likely to be important to consider in mechanistic models. We propose that this paradigm will allow researchers to study the connections between ecological systems and their molecular processes in previously inaccessible detail.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

SmileyLlama: modifying large language models for directed chemical space exploration

Here we show that large language models (LLMs) can be transformed via supervised fine-tuning of engineered prompts into SmileyLlama for exploring the chemical space of drug molecules. We benchmark SmileyLlama against pretrained LLMs and chemical language models trained from scratch for generating valid and novel drug-like molecules, and use direct preference optimization to both improve SmileyLlama’s adherence to a prompt and as part of the iMiner reinforcement learning framework to predict molecules with optimized three-dimensional conformations and high binding affinity to drug targets. By training an LLM to speak directly as a chemical language model, while retaining most of its natural language capabilities, we show that SmileyLlama can reliably generate molecules with user-specified properties rather than acting only as a chatbot with knowledge of chemistry or as a virtual assistant. While SmileyLlama is geared toward drug discovery, the supervised fine-tuning/direct preference optimization/LLM framework can be extended to other chemical, biological and materials applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2026

A Data-Driven Method for Modeling Creep-Fatigue Stress- Strain Behavior Using Neural ODEs

In this paper, we introduce a data-driven machine learning approach for modeling one-dimensional stress–strain behavior under cyclic loading, utilizing experimental data from the nickel-based Alloy 617. The study employs uniaxial creep–fatigue test data acquired under various loading histories and compares two distinct neural network-based ODE models. The first model, known as the black-box model, comprehensively describes the strain–stress relationship using a Neural ODE equation. To interpret this black-box model, we apply the Sparse Identification of Nonlinear Dynamical Systems (SINDy) technique, transforming the black-box model into an equation-based model using symbolic regression. The second model, the Neural flow rule model, incorporates Hooke’s Law for the linear elastic component, with the nonlinear part characterized by a Neural ODE. Both models are trained with experimental data to accurately reflect the observed stress–strain behavior. We conduct a detailed comparison with the standard Chaboche model, which includes three back stresses. Our results demonstrate that the neural network-based ODE models precisely capture the experimental creep–fatigue mechanical behavior, exceeding the standard Chaboche model’s accuracy. Furthermore, an interpretable model derived from the black-box neural ODE model through symbolic regression achieves accuracy comparable to the Chaboche model, enhancing its interpretability. The results highlight the potential of neural network-based ODE models to depict complex creep–fatigue behavior, eliminating the necessity for experts to define a specific, material-focused model form.

creep-fatigue↗
DOE OSTI2022

A Methodology to Support the Development of a New State Vision for the U.S. Nuclear Industry

Recent changes in natural gas prices combined with reduced capital costs for solar and wind systems has created challenges for the continued operation of existing nuclear power plants (NPPs) in the United States. A new strategy in the way in which U.S. NPPs are operated, maintained, and supported is needed. One such strategy is to transform the NPP operating model through a business-driven approach that leverages technology to enable new capabilities that improve performance and reduce costs. This paper presents a methodology for developing an achievable yet transformative new state vision that ensures the continued safe and efficient operations of the U.S. NPP fleet. Here this work builds on existing guidance and leverages previous research to comprehensively address both utility needs and high-level human factors engineering design principles when developing a new state vision. The proposed methodology is intended to provide industry-wide guidance for developing a new state vision that leverages both the selected vendor’s capabilities in a way that meets the utility’s modernization goals while ensuring state-of-the-art systems engineering and human factors engineering principles are applied that promote overall plant safety, performance, and efficiency.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2021

An Introduction to Word Embeddings and Language Models

Language models have advanced at a phenomenal pace over the past decade. This document provides a short introduction to terminology, word embeddings (aka low-dimensional representations), and popular large-scale language models (LMs). Word embeddings are used to represent words as numerical vectors and are context-independent, meaning a word can only have a single representation (e.g., club can only be club sandwich, not golf club ). Language models can determine the probability of a given sequence of words occurring in a sentence and can provide context to distinguish between words and phrases that sound similar. LMs are context-dependent (e.g., club can be club sandwich or golf club ) and largely fall in two main classes – autoregressive and autoencoding models. Autoregressive models are pretrained on the classic language modeling task: guess the next token having read all the previous ones. Those models can be fine-tuned and achieve great results on many tasks, the most natural application is text generation. A typical example of such models is GPT, but others include GPT-2, GPT-3, CTLR, TRANSFORMER-XL, REFORMER, XLNET. Autoencoding models are pretrained by corrupting the input tokens in some way and trying to reconstruct the original sentence. They can be fine-tuned and achieve great results on many tasks such as text generation, but their most natural application is sentence classification or token classification. A typical example of such models is BERT, but others include ROBERTA, ALBERT, XML, XML-ROBERTA, FLAUBERT AND LONGFORMER.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A METHODOLOGY TO SUPPORT THE DEVELOPMENT OF A NEW STATE VISION FOR THE UNITED STATES NUCLEAR INDUSTRY

A new strategy in the way in which the United States nuclear power plants (NPPs) are operated, maintained, and supported is needed. One such strategy is to transform the NPP operating model through a business-driven approach that leverages technology to enable new capabilities that improve performance and reduce cost. This paper presents a methodology for developing an achievable yet transformative new state vision that ensures continued safe and efficient operations of the United States NPP fleet. This work builds on existing guidance and leverages previous research to comprehensively address both bottom-up (i.e., utility needs) and top-down (i.e., first principles) considerations important for developing a new state vision. The proposed methodology is intended to provide industry-wide guidance for developing a new state vision that leverages both the selected vendor’s capabilities in a way that meets the utility’s modernization goals while ensuring state-of-the-art systems engineering and human factors engineering principles are applied that promote overall plant safety, performance, and efficiency.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2024

Efficient Distributed Sequence Parallelism for Transformer-Based Image Segmentation

We introduce an efficient distributed sequence parallel approach for training transformer-based deep learning image segmentation models. The neural network models are comprised of a combination of a Vision Transformer encoder with a convolutional decoder to provide image segmentation mappings. The utility of the distributed sequence parallel approach is especially useful in cases where the tokenized embedding representation of image data are too large to fit into standard computing hardware memory. To demonstrate the performance and characteristics of our models trained in sequence parallel fashion compared to standard models, we evaluate our approach using a 3D MRI brain tumor segmentation dataset. We show that training with a sequence parallel approach can match standard sequential model training in terms of convergence. Furthermore, we show that our sequence parallel approach has the capability to support training of models that would not be possible on standard computing resources.

Lyngaas, Isaac↗
DOE OSTI2025

Quantum AI Based Enhanced Detection of Dementia

Quantum computing has the potential to significantly improve the early detection of Alzheimer's Disease and Related Dementias (ADRD). Quantum-enhanced machine learning can be used to perform an early screening of Alzheimer's disease using brain imaging data based on dataset of MRI scans from both healthy individuals and those diagnosed with Alzheimer's. This study aims to demonstrate the potential of quantum transfer learning to enhance the performance of the classical deep learning model for dementia detection. Using the MRI sagittal images available in the OASIS-2 (64 demented and 72 non-demented subjects between 60 and 96 years), we show how quantum techniques can transform a suboptimal classical model into a more effective solution for dementia detection, highlighting their potential impact on advancing healthcare technology. We begin with a simple classical deep learning model with a significantly smaller number of parameters, which gives suboptimal performance on the problem. Then, we apply different configurations of quantum transfer learning based on the pre-trained weak classifier (Figure 1). We fix the weak classifier's initial convolutional layers at their fixed pre-trained parameters and replace the last set of dense layers with a dressed quantum circuit (DQN), which we train to enhance performance. We performed 4-fold cross-validation for both the classical and the hybrid quantum models and trained them using Pennylane's `default.qubit' simulator and IonQ's Aria-1 simulator (noisy simulation). We showed that with significantly fewer parameters, the quantum transfer learning-based hybrid models showed significant performance enhancement over the base weak classical deep learning model for dementia detection. To classify between a demented and non-demented subject, the accuracy of quantum-based AI methods improved by 6 to 14% compared to classical methods. The sensitivity of the models improved by 4 to 17%. This shows that there are fewer chances of misclassifying demented patients. Figure 2 compares the performance of the hybrid quantum models and their base classical model, and Table 1 summarizes the results. We illustrated that with assistance from quantum machine learning, it is possible to enhance detection for dementia based on brain images. This shows the potential for practical utility of quantum computing in ADRD research.

Bhowmik, Sounak [University of Tennessee, Knoxvill↗
DOE OSTI2021

From Machine Learning to Nuclear Digital Twins

This presentation provides multiple case studies of Machine Learning, Uncertainty Quantification, Reduced Order Modeling, to achieve digital transformation and provide the components for nuclear digital twinning. The presentation is for an international virtual event titled: Consultancy Meeting on Applications of AI and Pattern Recognition Techniques for Uncertainty Quantification in Nuclear Power Modelling and Simulation held Oct 21-22, 2021.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2020

Model-less Source Location for Forced Oscillation based on Synchrophasor and Moving Fast Fourier Transformation

Forced oscillations in power systems occur when the grid is driven by an external and periodic force. To quickly detect and locate the source of the forced oscillation is critical in terms of ensuring the reliability of an interconnected power grid. This paper explores the electromechanical wave propagation theory and the Fast Fourier Transformation to analyze the forced oscillations. It proposes a model-less, adaptive, fast, and accurate source location algorithm. The proposed algorithm is extensively evaluated through simulation data from a 70k-bus U.S. Eastern Interconnection test system and field-collected synchrophasor data from the distribution-level wide-area monitoring system, FNET/GridEye. The evaluation results demonstrate the correctness and effectiveness of the proposed model-less forced oscillation source location algorithm.

Wang, Weikang↗
DOE OSTI2022

Supervised Learning for Distribution Secondary Systems Modeling: Improving Solar Interconnection Processes

The current interconnection process and hosting capacity analysis for distributed energy resources (DERs), such as photovoltaics (PV) and battery energy storage systems, are based on analyzing grid network constraints (voltage and thermal) using only medium-voltage distribution network models. This is because most utilities do not have secondary low-voltage system models that connect service transformers and residential customers. This is important because in many cases the main impact of interconnecting DERs could occur on the low-voltage distribution systems. This paper proposes a supervised learning method to approximate local secondary models to improve the interconnection process. The proposed supervised learning method includes a decision tree model that predicts the secondary topology and a logistic regression model that predicts conductor types. The case studies demonstrate the benefits of including secondary low-voltage circuits in the interconnection process. We report the proposed modeling methodology is readily scalable and thus can reduce the cost and effort of PV interconnection for the industry and stakeholders.

14 SOLAR ENERGY↗
DOE OSTI2022

Accelerating Multiscale Materials Modeling with Machine Learning

The focus of this project is to accelerate and transform the workflow of multiscale materials modeling by developing an integrated toolchain seamlessly combining DFT, SNAP, LAMMPS, (shown in Figure 1-1) and a machine-learning (ML) model that will more efficiently extract information from a smaller set of first-principles calculations. Our ML model enables us to accelerate first-principles data generation by interpolating existing high fidelity data, and extend the simulation scale by extrapolating high fidelity data (10 2 atoms) to the mesoscale (10 4 atoms). It encodes the underlying physics of atomic interactions on the microscopic scale by adapting a variety of ML techniques such as deep neural networks (DNNs), and graph neural networks (GNNs). We developed a new surrogate model for density functional theory using deep neural networks. The developed ML surrogate is demonstrated in a workflow to generate accurate band energies, total energies, and density of the 298K and 933K Aluminum systems. Furthermore, the models can be used to predict the quantities of interest for systems with more number of atoms than the training data set. We have demonstrated that the ML model can be used to compute the quantities of interest for systems with 100,000 Al atoms. When compared with 2000 Al system the new surrogate model is as accurate as DFT, but three orders of magnitude faster. We also explored optimal experimental design techniques to choose the training data and novel Graph Neural Networks to train on smaller data sets. These are promising methods that need to be explored in the future.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Time-warping invariant quantum recurrent neural networks via quantum-classical adaptive gating

Adaptive gating plays a key role in temporal data processing via classical recurrent neural networks (RNNs), as it facilitates retention of past information necessary to predict the future, providing a mechanism that preserves invariance to time warping transformations. This paper builds on quantum RNNs (QRNNs), a dynamic model with quantum memory, to introduce a novel class of temporal data processing quantum models that preserve invariance to time-warping transformations of the (classical) input-output sequences. The model, referred to as time warping-invariant QRNN (TWI-QRNN), augments a QRNN with a quantum–classical adaptive gating mechanism that chooses whether to apply a parameterized unitary transformation at each time step as a function of the past samples of the input sequence via a classical recurrent model. The TWI-QRNN model class is derived from first principles, and its capacity to successfully implement time-warping transformations is experimentally demonstrated on examples with classical or quantum dynamics.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Leveraging High-resolution Molecular Composition of Soil Organic Matter to Enhance Carbon Cycling Modeling

Soils store more carbon than the atmosphere and vegetation combined, yet Earth system models still struggle to predict how this vast reservoir will respond to environmental change. A central limitation is that most soil biogeochemical models represent organic matter using bulk conceptual pools or chemically homogeneous fractions, preventing direct use of rapidly expanding molecular-scale datasets. Here we develop and test a new soil decomposition framework that explicitly integrates high-resolution information on organic matter composition. First, we construct a molecularly informed litter decomposition module in which plant inputs are partitioned into five functional compound classes—carbohydrates, proteins, lignin-like aromatics, lipids, and carbonyls—using a molecular mixing model calibrated to solid-state 13 C Nuclear Magnetic Resonance (NMR) spectra. Class-specific kinetics, lignin-dependent physical protection, and substrate-driven microbial carbon use efficiency allow the module to capture metabolic tradeoffs associated with enzyme production and nutrient limitation. We then embed this litter module within a microbially explicit whole-soil model that tracks the transformation of these compound classes through particulate organic matter, dissolved organic matter, mineral-associated organic matter, and microbial biomass. High-resolution Fourier Transform Ion Cyclotron Resonance mass spectrometry (FTICR-MS) data are used to link internal pools to measurable soil organic matter fractions and to constrain key process parameters. Applications at soil-core and ecosystem scales demonstrate that the new model reproduces observed soil respiration dynamics while providing mechanistic attribution of CO 2 fluxes to specific chemical classes and pools. Compared to existing frameworks such as the Community Land Model soil biogeochemistry module and the Millennial model, our approach maintains competitive predictive skill while substantially improving interpretability and opportunities for data–model integration. This work illustrates a viable pathway for leveraging molecular-scale observations to reduce structural uncertainty in soil carbon–climate feedback projections.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Bifunctional nanoprecipitates strengthen and ductilize a medium-entropy alloy

Single-phase high- and medium-entropy alloys with face-centred cubic (fcc) structure can exhibit high tensile ductility and excellent toughness, but their room-temperature strengths are low. Dislocation obstacles such as grain boundaries, twin boundaries, solute atoms and precipitates can increase strength. However, with few exceptions, such obstacles tend to decrease ductility. Interestingly, precipitates can also hinder phase transformations. Here, using a model, precipitate-strengthened, Fe–Ni–Al–Ti medium-entropy alloy, we demonstrate a strategy that combines these dual functions in a single alloy. The nanoprecipitates in our alloy, in addition to providing conventional strengthening of the matrix, also modulate its transformation from fcc-austenite to body-centred cubic (bcc) martensite, constraining it to remain as metastable fcc after quenching through the transformation temperature. During subsequent tensile testing, the matrix progressively transforms to bcc-martensite, enabling substantial increases in strength, work hardening and ductility. This use of nanoprecipitates exploits synergies between precipitation strengthening and transformation-induced plasticity, resulting in simultaneous enhancement of tensile strength and uniform elongation. Further, our findings demonstrate how synergistic deformation mechanisms can be deliberately activated, exactly when needed, by altering precipitate characteristics (such as size, spacing, and so on), along with the chemical driving force for phase transformation, to optimize strength and ductility.

36 MATERIALS SCIENCE↗