Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 577 records · Page 32
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

A Methodology to Support the Development of a New State Vision for the U.S. Nuclear Industry

Recent changes in natural gas prices combined with reduced capital costs for solar and wind systems has created challenges for the continued operation of existing nuclear power plants (NPPs) in the United States. A new strategy in the way in which U.S. NPPs are operated, maintained, and supported is needed. One such strategy is to transform the NPP operating model through a business-driven approach that leverages technology to enable new capabilities that improve performance and reduce costs. This paper presents a methodology for developing an achievable yet transformative new state vision that ensures the continued safe and efficient operations of the U.S. NPP fleet. Here this work builds on existing guidance and leverages previous research to comprehensively address both utility needs and high-level human factors engineering design principles when developing a new state vision. The proposed methodology is intended to provide industry-wide guidance for developing a new state vision that leverages both the selected vendor’s capabilities in a way that meets the utility’s modernization goals while ensuring state-of-the-art systems engineering and human factors engineering principles are applied that promote overall plant safety, performance, and efficiency.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2021

An Introduction to Word Embeddings and Language Models

Language models have advanced at a phenomenal pace over the past decade. This document provides a short introduction to terminology, word embeddings (aka low-dimensional representations), and popular large-scale language models (LMs). Word embeddings are used to represent words as numerical vectors and are context-independent, meaning a word can only have a single representation (e.g., club can only be club sandwich, not golf club ). Language models can determine the probability of a given sequence of words occurring in a sentence and can provide context to distinguish between words and phrases that sound similar. LMs are context-dependent (e.g., club can be club sandwich or golf club ) and largely fall in two main classes – autoregressive and autoencoding models. Autoregressive models are pretrained on the classic language modeling task: guess the next token having read all the previous ones. Those models can be fine-tuned and achieve great results on many tasks, the most natural application is text generation. A typical example of such models is GPT, but others include GPT-2, GPT-3, CTLR, TRANSFORMER-XL, REFORMER, XLNET. Autoencoding models are pretrained by corrupting the input tokens in some way and trying to reconstruct the original sentence. They can be fine-tuned and achieve great results on many tasks such as text generation, but their most natural application is sentence classification or token classification. A typical example of such models is BERT, but others include ROBERTA, ALBERT, XML, XML-ROBERTA, FLAUBERT AND LONGFORMER.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1993

System identification of the Large-Angle Magnetic Suspension Test Facility (LAMSTF)

The Large-Angle Magnetic Suspension Test Facility (LAMSTF), a laboratory-scale research project to demonstrate the magnetic suspension of objects over wide ranges of attitudes, has been developed. This system represents a scaled model of a planned Large-Gap Magnetic Suspension System (LGMSS). The LAMSTF system consists of a planar array of five copper electromagnets which actively suspend a small cylindrical permanent magnet. The cylinder is a rigid body and can be controlled to move in five independent degrees of freedom. Five position variables are sensed indirectly by using infra-red light-emitting diodes and light-receiving phototransistors. The motion of the suspended cylinder is in general nonlinear and hence only the linear, time-invariant perturbed motion about an equilibrium state is considered. One of the main challenges in this project is the control of the suspended element over a wide range of orientations. An accurate dynamic model plans an essential role in controller design. The analytical model of the LAMSTF system includes highly unstable real poles (about 10 Hz) and low-frequency flexible modes (about 0.16 Hz). Projection filters are proposed to identify the state space model from closed-loop test data in time domain. A canonical transformation matrix is also derived to transform the identified state space model into the physical coordinate. The LAMSTF system is stabilized by using a linear quadratic regulator (LQR) feedback controller. The rate information is obtained by calculating the back difference of the sensed position signals. The reference inputs contain five uncorrelated random signals. This control input and the system reponse are recorded as input/output data to identify the system directly from the projection filters. The sampling time is 4 ms and the model is fairly accurate in predicting the step responses for different controllers while the analytical model has a deficiency in the pitch axis.

Huang, Jen-Kuang↗
DOE OSTI2021

A METHODOLOGY TO SUPPORT THE DEVELOPMENT OF A NEW STATE VISION FOR THE UNITED STATES NUCLEAR INDUSTRY

A new strategy in the way in which the United States nuclear power plants (NPPs) are operated, maintained, and supported is needed. One such strategy is to transform the NPP operating model through a business-driven approach that leverages technology to enable new capabilities that improve performance and reduce cost. This paper presents a methodology for developing an achievable yet transformative new state vision that ensures continued safe and efficient operations of the United States NPP fleet. This work builds on existing guidance and leverages previous research to comprehensively address both bottom-up (i.e., utility needs) and top-down (i.e., first principles) considerations important for developing a new state vision. The proposed methodology is intended to provide industry-wide guidance for developing a new state vision that leverages both the selected vendor’s capabilities in a way that meets the utility’s modernization goals while ensuring state-of-the-art systems engineering and human factors engineering principles are applied that promote overall plant safety, performance, and efficiency.

99 GENERAL AND MISCELLANEOUS↗
NASA NTRS1988

A numerical simulation of the full two-dimensional electrothermal de-icer pad

The ability to predict the time-temperature history of electrothermal de-icer pads is important in the subsequent design of improved and more efficient versions. These de-icer pads are installed near the surface of aircraft components, for the specific purpose of removing accreted ice. The proposed numerical model can incorporate the full 2-D geometry through a section of a region (i.e., section of an airfoil), that current 1-D numerical codes are unable to do. Thus, the effects of irregular layers, curvature, etc., can now be accounted for in the thermal transients. Each layer in the actual geometry is mapped via a body-fitted coordinate transformation into uniform, rectangular computational grids. The relevant heat transfer equations are transformed and discretized. To model the phase change that might occur in any accreted ice, in an enthalpy formulation the phase change equations are likewise transformed and discretized. The code developed was tested against numerous classical numerical solutions, as well as against experimental de-icing data on a UH1H rotor blade obtained from the NASA Lewis Research Center. The excellent comparisons obtained show that this code can be a useful tool in predicting the performance of current de-icer models, as well as in the designing of future models.

Masiulaniec, Konstanty C.↗
DOE OSTI2024

Efficient Distributed Sequence Parallelism for Transformer-Based Image Segmentation

We introduce an efficient distributed sequence parallel approach for training transformer-based deep learning image segmentation models. The neural network models are comprised of a combination of a Vision Transformer encoder with a convolutional decoder to provide image segmentation mappings. The utility of the distributed sequence parallel approach is especially useful in cases where the tokenized embedding representation of image data are too large to fit into standard computing hardware memory. To demonstrate the performance and characteristics of our models trained in sequence parallel fashion compared to standard models, we evaluate our approach using a 3D MRI brain tumor segmentation dataset. We show that training with a sequence parallel approach can match standard sequential model training in terms of convergence. Furthermore, we show that our sequence parallel approach has the capability to support training of models that would not be possible on standard computing resources.

Lyngaas, Isaac↗
DOE OSTI2025

Quantum AI Based Enhanced Detection of Dementia

Quantum computing has the potential to significantly improve the early detection of Alzheimer's Disease and Related Dementias (ADRD). Quantum-enhanced machine learning can be used to perform an early screening of Alzheimer's disease using brain imaging data based on dataset of MRI scans from both healthy individuals and those diagnosed with Alzheimer's. This study aims to demonstrate the potential of quantum transfer learning to enhance the performance of the classical deep learning model for dementia detection. Using the MRI sagittal images available in the OASIS-2 (64 demented and 72 non-demented subjects between 60 and 96 years), we show how quantum techniques can transform a suboptimal classical model into a more effective solution for dementia detection, highlighting their potential impact on advancing healthcare technology. We begin with a simple classical deep learning model with a significantly smaller number of parameters, which gives suboptimal performance on the problem. Then, we apply different configurations of quantum transfer learning based on the pre-trained weak classifier (Figure 1). We fix the weak classifier's initial convolutional layers at their fixed pre-trained parameters and replace the last set of dense layers with a dressed quantum circuit (DQN), which we train to enhance performance. We performed 4-fold cross-validation for both the classical and the hybrid quantum models and trained them using Pennylane's `default.qubit' simulator and IonQ's Aria-1 simulator (noisy simulation). We showed that with significantly fewer parameters, the quantum transfer learning-based hybrid models showed significant performance enhancement over the base weak classical deep learning model for dementia detection. To classify between a demented and non-demented subject, the accuracy of quantum-based AI methods improved by 6 to 14% compared to classical methods. The sensitivity of the models improved by 4 to 17%. This shows that there are fewer chances of misclassifying demented patients. Figure 2 compares the performance of the hybrid quantum models and their base classical model, and Table 1 summarizes the results. We illustrated that with assistance from quantum machine learning, it is possible to enhance detection for dementia based on brain images. This shows the potential for practical utility of quantum computing in ADRD research.

Bhowmik, Sounak [University of Tennessee, Knoxvill↗
DOE OSTI2021

From Machine Learning to Nuclear Digital Twins

This presentation provides multiple case studies of Machine Learning, Uncertainty Quantification, Reduced Order Modeling, to achieve digital transformation and provide the components for nuclear digital twinning. The presentation is for an international virtual event titled: Consultancy Meeting on Applications of AI and Pattern Recognition Techniques for Uncertainty Quantification in Nuclear Power Modelling and Simulation held Oct 21-22, 2021.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2020

Model-less Source Location for Forced Oscillation based on Synchrophasor and Moving Fast Fourier Transformation

Forced oscillations in power systems occur when the grid is driven by an external and periodic force. To quickly detect and locate the source of the forced oscillation is critical in terms of ensuring the reliability of an interconnected power grid. This paper explores the electromechanical wave propagation theory and the Fast Fourier Transformation to analyze the forced oscillations. It proposes a model-less, adaptive, fast, and accurate source location algorithm. The proposed algorithm is extensively evaluated through simulation data from a 70k-bus U.S. Eastern Interconnection test system and field-collected synchrophasor data from the distribution-level wide-area monitoring system, FNET/GridEye. The evaluation results demonstrate the correctness and effectiveness of the proposed model-less forced oscillation source location algorithm.

Wang, Weikang↗
NASA NTRS1990

Rotordynamics analysis of a Jeffcott model with deadband

A method is developed for determining the stability margins of a simple Jeffcott model with deadband via analysis of the discrete Fourier transform of the system response. The model in question is of a uniform, unbalanced, flexible shaft that is supported by a bearing as it rotates about its x axis. This model is represented by a system of coupled nonlinear differential equations.

Zalik, R. A.↗
NASA NTRS1993

Fourier transform infrared studies of the interaction of HCl with model polar stratospheric cloud films

Heterogeneous reactions involving hydrochloric acid adsorbed on the surfaces of polar stratospheric clouds (PSCs) are postulated to contribute to polar ozone loss. Using FTIR spectroscopy to probe the condensed phase, we have examined the interaction of HCl with ice and nitric acid trihydrate (NAT) films representative of types II and I PSCs, respectively. For HCl pressures in the range of 10 exp -7 to 10 exp -5 Torr, our FTIR studies show that a small amount of crystalline HCl-6H2O formed on or in ice at 155 K. However, for higher HCl pressures, we observed that the entire film of ice rapidly converted into an amorphous 4:1 H2O:HCl mixture. From HCl-uptake experiments with P(HCl) = 8 x 10 exp -7 Torr, we estimate roughly that the diffusion coefficient of HCl in ice is around 2 x 10 exp -12 sq cm/s at 158 K. For higher temperatures more closely approximating those found in the stratosphere, we were unable to detect bulk HCl uptake by ice. Indirect evidence suggests that HCl adsorption onto the surface of model PSC films inhibited the evaporation of both ice and NAT by 3-5 K.

Koehler, Birgit G.↗
NASA NTRS1985

Experimental determination of systems suitable for study as monotectic binary metallic alloy solidification models

Transparent binary metallic alloy solidification models are important in attempts to understand the processes causing liquid-liquid and solid-liquid phase transformations in metallic alloy systems. These models permit visual observation of the phase transformation and the processes proceding solidification. The number of these transparent monotectic binary models needs to be expanded to distinguish between the unique and general phenomena observed. The expansion of the number of accurately determined monotectic phase diagrams of model systems, and contribution to a data base for eventual use with UNIFAC group contribution methods is examined.

Smith, J. E., Jr.↗
DOE OSTI2022

Supervised Learning for Distribution Secondary Systems Modeling: Improving Solar Interconnection Processes

The current interconnection process and hosting capacity analysis for distributed energy resources (DERs), such as photovoltaics (PV) and battery energy storage systems, are based on analyzing grid network constraints (voltage and thermal) using only medium-voltage distribution network models. This is because most utilities do not have secondary low-voltage system models that connect service transformers and residential customers. This is important because in many cases the main impact of interconnecting DERs could occur on the low-voltage distribution systems. This paper proposes a supervised learning method to approximate local secondary models to improve the interconnection process. The proposed supervised learning method includes a decision tree model that predicts the secondary topology and a logistic regression model that predicts conductor types. The case studies demonstrate the benefits of including secondary low-voltage circuits in the interconnection process. We report the proposed modeling methodology is readily scalable and thus can reduce the cost and effort of PV interconnection for the industry and stakeholders.

14 SOLAR ENERGY↗
NASA NTRS1977

Oblique orthographic projections and contour plots

Oblique orthographic projections allow model to be viewed in any selected orientation specified by Euler-angle transformation. This transformation resolves coordinate system of model to principal plane on which display is to be plotted.

Giles, G. L.↗
DOE OSTI2022

Accelerating Multiscale Materials Modeling with Machine Learning

The focus of this project is to accelerate and transform the workflow of multiscale materials modeling by developing an integrated toolchain seamlessly combining DFT, SNAP, LAMMPS, (shown in Figure 1-1) and a machine-learning (ML) model that will more efficiently extract information from a smaller set of first-principles calculations. Our ML model enables us to accelerate first-principles data generation by interpolating existing high fidelity data, and extend the simulation scale by extrapolating high fidelity data (10 2 atoms) to the mesoscale (10 4 atoms). It encodes the underlying physics of atomic interactions on the microscopic scale by adapting a variety of ML techniques such as deep neural networks (DNNs), and graph neural networks (GNNs). We developed a new surrogate model for density functional theory using deep neural networks. The developed ML surrogate is demonstrated in a workflow to generate accurate band energies, total energies, and density of the 298K and 933K Aluminum systems. Furthermore, the models can be used to predict the quantities of interest for systems with more number of atoms than the training data set. We have demonstrated that the ML model can be used to compute the quantities of interest for systems with 100,000 Al atoms. When compared with 2000 Al system the new surrogate model is as accurate as DFT, but three orders of magnitude faster. We also explored optimal experimental design techniques to choose the training data and novel Graph Neural Networks to train on smaller data sets. These are promising methods that need to be explored in the future.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Time-warping invariant quantum recurrent neural networks via quantum-classical adaptive gating

Adaptive gating plays a key role in temporal data processing via classical recurrent neural networks (RNNs), as it facilitates retention of past information necessary to predict the future, providing a mechanism that preserves invariance to time warping transformations. This paper builds on quantum RNNs (QRNNs), a dynamic model with quantum memory, to introduce a novel class of temporal data processing quantum models that preserve invariance to time-warping transformations of the (classical) input-output sequences. The model, referred to as time warping-invariant QRNN (TWI-QRNN), augments a QRNN with a quantum–classical adaptive gating mechanism that chooses whether to apply a parameterized unitary transformation at each time step as a function of the past samples of the input sequence via a classical recurrent model. The TWI-QRNN model class is derived from first principles, and its capacity to successfully implement time-warping transformations is experimentally demonstrated on examples with classical or quantum dynamics.

97 MATHEMATICS AND COMPUTING↗