SEARCH · Engineering Papers
Results for “Transformer model”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
CryoTransformer: a transformer model for picking protein particles from Cryo-EM micrographs
Cryo-electron microscopy (cryo-EM) is a powerful technique for determining the structures of large protein complexes. Picking single protein particles from cryo-EM micrographs (images) is a crucial step in reconstructing protein structures from them. However, the widely used template-based particle picking process requires some manual particle picking and is labor-intensive and time-consuming. Though machine learning and artificial intelligence (AI) can potentially automate particle picking, the current AI methods pick particles with low precision or low recall. The erroneously picked particles can severely reduce the quality of reconstructed protein structures, especially for the micrographs with low signal-to-noise-ratio (SNR).
Model transformations for state-space self-tuning control of multivariable stochastic systems
The design of self-tuning controllers for multivariable stochastic systems is considered analytically. A long-division technique for finding the similarity transformation matrix and transforming the estimated left MFD to the right MFD is developed; the derivation is given in detail, and the procedures involved are briefly characterized.
Model Transformation for a System of Systems Dependability Safety Case
Software plays an increasingly larger role in all aspects of NASA's science missions. This has been extended to the identification, management and control of faults which affect safety-critical functions and by default, the overall success of the mission. Traditionally, the analysis of fault identification, management and control are hardware based. Due to the increasing complexity of system, there has been a corresponding increase in the complexity in fault management software. The NASA Independent Validation & Verification (IV&V) program is creating processes and procedures to identify, and incorporate safety-critical software requirements along with corresponding software faults so that potential hazards may be mitigated. This Specific to Generic ... A Case for Reuse paper describes the phases of a dependability and safety study which identifies a new, process to create a foundation for reusable assets. These assets support the identification and management of specific software faults and, their transformation from specific to generic software faults. This approach also has applications to other systems outside of the NASA environment. This paper addresses how a mission specific dependability and safety case is being transformed to a generic dependability and safety case which can be reused for any type of space mission with an emphasis on software fault conditions.
EUROPA Mission: Multidisciplinary Model Transformation Through Simplified Intermediate Representations
No abstract available
Will AI Foundation Models Transform Scientific Research?
Explore the source record for details and available documents.
Frequency Response Analysis of the Molten Salt Demonstration Reactor Dynamic Simulation Model in Transient Simulation Framework of Reconfigurable Models (TRANSFORM)
Explore the source record for details and available documents.
Towards an astronomical foundation model for stars with a transformer-based model
ABSTRACT Rapid strides are currently being made in the field of artificial intelligence using transformer-based models like Large Language Models (LLMs). The potential of these methods for creating a single, large, versatile model in astronomy has not yet been explored. In this work, we propose a framework for data-driven astronomy that uses the same core techniques and architecture as used by LLMs. Using a variety of observations and labels of stars as an example, we build a transformer-based model and train it in a self-supervised manner with cross-survey data sets to perform a variety of inference tasks. In particular, we demonstrate that a single model can perform both discriminative and generative tasks even if the model was not trained or fine-tuned to do any specific task. For example, on the discriminative task of deriving stellar parameters from Gaia XP spectra, we achieve an accuracy of 47 K in Teff, 0.11 dex in log g, and 0.07 dex in [M/H], outperforming an expert XGBoost model in the same setting. But the same model can also generate XP spectra from stellar parameters, inpaint unobserved spectral regions, extract empirical stellar loci, and even determine the interstellar extinction curve. Our framework demonstrates that building and training a single foundation model without fine-tuning using data and parameters from multiple surveys to predict unmeasured observations and parameters is well within reach. Such ‘Large Astronomy Models’ trained on large quantities of observational data will play a large role in the analysis of current and future large surveys.
Crystal plasticity modeling of strain-induced martensitic transformations to predict strain rate and temperature sensitive behavior of 304 L steels: Applications to tension, compression, torsion, and impact
This paper advances crystallographically-based Olson-Cohen (direct γ → α’) and deformation mechanism (indirect γ→ε→α’) phase transformation models for predicting strain-induced austenite to martensite transformation. Here, the advanced transformation models enable predictions of not only strain-path sensitive, but also of strain-rate and temperature sensitive deformation of polycrystalline stainless steels (SSs). The deformation of constituent grains in SSs is modeled as a combination of anisotropic elasticity, crystallographic slip, and phase transformation, while the hardening is based on the evolution of dislocation density and explicit shifts in phase fractions. Such grain-scale deformation is implemented within the meso-scale elasto-plastic self-consistent (EPSC) homogenization model, which is coupled with the implicit finite element (FE) method to provide a constitutive response at each FE integration point for solving boundary value problems at the macro-scale. Parameters pertaining to the hardening and transformation models within FEEPSC are calibrated and validated on a suite of data including flow curves and phase fractions for monotonic compression, tension, and torsion as a function of strain-rate and temperature for wrought and additively manufactured (AM) SS304L. To illustrate the potential and accuracy of the integrated multi-level FE-EPSC simulation framework, geometry, mechanical response, phase fractions, and texture evolution are simulated during gas-gun impact deformation of a cylinder and quasi-static tension of a notched specimen made of AM SS304L. Details of the simulation framework, comparison between experimental and simulation results, and insights from the results are presented and discussed.
Phase transformation kinetics model for metals
We develop a new model for phase transformation kinetics in metals by generalizing the Levitas–Preston (LP) phase field model of martensite phase transformations (see Levitas and Preston (2002a,b) and Levitas et al. (2003)) to arbitrary pressure. Furthermore, we account for and track: the interface speed of the pressure-driven phase transformation, properties of critical nuclei, as well as nucleation at grain sites and on dislocations and homogeneous nucleation. The volume fraction evolution of each phase is described by employing KJMA (Kolmogorov, 1937; Johnson and Mehl, 1939; Avrami, 1939, 1940, 1941) kinetic theory. We then test our new model for iron under ramp loading conditions and compare our predictions for the α → ϵ iron phase transition to experimental data of Smith et al. (2013). In conclusion, more than one combination of material and model parameters (such as dislocation density and interface speed) led to good agreement of our simulations to the experimental data, thus highlighting the importance of having accurate microstructure data for the sample under consideration.
labquake_future_prediction
The labquake_future_prediction code is a collection of python modules and scripts that serves as supporting information for the article “Predicting future laboratory fault friction through deep learning” for publication in the journal of “Geophysical Research Letters”. It is designed to predict laboratory fault slips in the immediate future by scanning continuous acoustic emission (AE) waveforms recorded in laboratory biaxial shear experiments. The predictions are made with a deep learning model based on convolutional encoder-decoder (CED) models and the Transformer model primarily developed for Natural Language Processing (NLP). The deep learning model is trained with the tensorflow package using publicly available laboratory data sets in standard binary file format in numpy. The utility functions for reading data files, configuring model hyperparameters, constructing the CED and Transformer models, training and testing of the models are defined in python module files. The workflow of training the models for labquake future predictions and the multiple GPU’s rapid model hyperparameter optimization as described in the journal article, are demonstrated in accompanying python script files and Jupyter notebooks.
Path-BigBird: An AI-Driven Transformer Approach to Classification of Cancer Pathology Reports
PURPOSE Surgical pathology reports are critical for cancer diagnosis and management. To accurately extract information about tumor characteristics from pathology reports in near real time, we explore the impact of using domain-specific transformer models that understand cancer pathology reports. METHODS We built a pathology transformer model, Path-BigBird, by using 2.7 million pathology reports from six SEER cancer registries. We then compare different variations of Path-BigBird with two less computationally intensive methods: Hierarchical Self-Attention Network (HiSAN) classification model and an offthe-shelf clinical transformer model (Clinical BigBird). We use five pathology information extraction tasks for evaluation: site, subsite, laterality, histology, and behavior. Model performance is evaluated by using macro and micro F 1 scores. RESULTS We found that Path-BigBird and Clinical BigBird outperformed the HiSAN in all tasks. Clinical BigBird performed better on the site and laterality tasks. Versions of the Path-BigBird model performed best on the two most difficult tasks: subsite (micro F 1 score of 72.53, macro F 1 score of 35.76) and histology (micro F 1 score of 80.96, macro F 1 score of 37.94). The largest performance gains over the HiSAN model were for histology, for which a Path-BigBird model increased the micro F 1 score by 1.44 points and the macro F 1 score by 3.55 points. Overall, the results suggest that a Path-BigBird model with a vocabulary derived from wellcurated and deidentified data is the best-performing model. CONCLUSION The Path-BigBird pathology transformer model improves automated information extraction from pathology reports. Although Path-BigBird outperforms Clinical BigBird and HiSAN, these less computationally expensive models still have utility when resources are constrained.
Dynamic-Phasor Model of Transformer Inrush Simulation for Unbalanced Distribution System
Abstract— Power system stability is a critical concern, with respect to the increasing distributed energy resources, microgrids, and emphasis on enhancing resiliency. For stability analysis and evaluation, modeling and simulation capabilities are important. In this paper, the authors’ previous work on modeling transformer inrush dynamics is improved with a new formulation of the key equation that converts the dynamic-phasor into real-time value. Simulation results in an electromechanical time-scale are compared with PSCAD results in an electromagnetic time-scale to demonstrate the improvement. The impact of simulation step size is discussed. A bug in PSCAD transformer setting that leads to incorrect magnetizing flux and current under Delta winding type is identified and reported. In addition, explanations and analysis are provided on two modeling techniques utilized since the previous work. This work is being incorporated into the next release of GridLAB-D. The improved transformer inrush calculation enables more accurate predictions of current amplitudes for analyzing the dynamics of distribution operation and control.
Image Discrimination Models Predict Object Detection in Natural Backgrounds
Object detection involves looking for one of a large set of object sub-images in a large set of background images. Image discrimination models only predict the probability that an observer will detect a difference between two images. In a recent study based on only six different images, we found that discrimination models can predict the relative detectability of objects in those images, suggesting that these simpler models may be useful in some object detection applications. Here we replicate this result using a new, larger set of images. Fifteen images of a vehicle in an other-wise natural setting were altered to remove the vehicle and mixed with the original image in a proportion chosen to make the target neither perfectly recognizable nor unrecognizable. The target was also rotated about a vertical axis through its center and mixed with the background. Sixteen observers rated these 30 target images and the 15 background-only images for the presence of a vehicle. The likelihoods of the observer responses were computed from a Thurstone scaling model with the assumption that the detectabilities are proportional to the predictions of an image discrimination model. Three image discrimination models were used: a cortex transform model, a single channel model with a contrast sensitivity function filter, and the Root-Mean-Square (RMS) difference of the digital target and background-only images. As in the previous study, the cortex transform model performed best; the RMS difference predictor was second best; and last, but still a reasonable predictor, was the single channel model. Image discrimination models can predict the relative detectabilities of objects in natural backgrounds.
Geomagnetically Induced Current Field Test on Large Grid-Connected Power Transformers: Analysis, Model Development, and Simulations
Geomagnetic-induced current (GIC) flow in power grids can cause undesirable effects such as transformer overheating, harmonics, higher reactive power demand, etc. Many simulation models have been developed to study these effects, but real-world verification on modern transformer designs is rare. Here, this paper presents the first long-duration GIC field test in the U.S. performed on high-voltage, grid-connected transformers featuring winding clamps and tie rods instead of conventional tie bars. Field measurements were taken to evaluate GIC effects. These measurements also aided in developing and validating thermal and electromagnetic transient (EMT) models of the transformers. During the test, significant current and voltage distortions were observed along with considerable transformer reactive power losses. Analysis of the field measurements showed that the transformers’ hottest spot was at the inner windings, and their k-factors were close to factory test and software default values. Thermal simulations indicated that the transformers would not violate their thermal limits even for a GIC waveform that peaks at about 200 A/phase. EMT simulations revealed that increased transformer loading may reduce GIC-induced reactive power demand and harmonics in certain scenarios. The study also highlighted potential inaccuracies in using the k-factor method to calculate transformer reactive power losses.
2022 Spring Internship Exit Presentation
As efforts of the National Aeronautics and Space Administration (NASA) and the Federal Aviation Administration (FAA) continue to digitize the air traffic management (ATM) domain, there is countless times of need for downstream natural language processing (NLP) tasks such as named entity recognition, text summarization, classification, and more. Although there are a plethora of open-sourced pre-trained transformer models in the NLP field such as BERT, RoBERTa, XLNet, and GPT-3, these models are trained on general corpora and perform poorly on domain-specific terminology and phraseology seen in ATM documents such as Notice to Airmen (NOTAMs) and Letters of Agreement (LoA). Our proposed research objective will be to first gather a large corpus of air traffic management related documents, orders, notices, books, technical papers, conference papers, articles, and other miscellaneous sources of text data from the FAA, NASA, and accredited conference and publication societies. After gathering this data, many steps will have to be taken to collate and preprocess the data into a format understandable by our test transformer models. Thirdly, we will set up training pipelines to train the RoBERTa model on its unsupervised training task masked language modelling (MLM) using resources provided by the NASA Advanced Supercomputing (NAS) facilities. Finally, these fine-tuned transformer models will be evaluated on their performance on down-stream NLP tasks as mentioned above, to show whether they will be effective when working with ATM related data or not. Once complete, this model could be made open-sourced on the HuggingFace website, where the rest of the ATM community can access and utilize this tool.
Attention for Causal Relationship Discovery from Biological Neural Dynamics
This paper explores the potential of the transformer models for learning Granger causality in networks with complex nonlinear dynamics at every node, as in neurobiological and biophysical networks. Our study primarily focuses on a proof-of-concept investigation based on simulated neural dynamics, for which the ground-truth causality is known through the underlying connectivity matrix. For transformer models trained to forecast neuronal population dynamics, we show that the cross-attention module effectively captures the causal relationship among neurons, with an accuracy equal to or superior to that of the most popular Granger causality discovery method. While we acknowledge that real-world neurobiology data will bring further challenges, including dynamic connectivity and unobserved variability, this research offers an encouraging preliminary glimpse into the utility of the transformer model for causal representation learning in neuroscience.
scPlantAnnotate: an accurate and robust transformer-based model for plant cell type annotation
Accurate cell type annotation remains a major bottleneck in plant single-cell RNA sequencing (scRNA-seq), where existing tools are often adapted from animal studies and perform sub-optimally on plant data. The lack of plant-specific computational frameworks limits the construction of plant cell atlases and downstream biological discovery. We develop and evaluate scPlantAnnotate, a Transformer-based reference annotation framework tailored for plant scRNA-seq data, and benchmark it against state-of-the-art deep learning and conventional methods across multiple plant species. Species-specific scPlantAnnotate models were trained using curated datasets from Arabidopsis thaliana, Zea mays, Oryza sativa, and Glycine max. We compared scPlantAnnotate with leading baselines under both standard random-split evaluation and a more stringent leave-one-dataset-out setting, which tests robustness to completely unseen datasets and tissue types. scPlantAnnotate consistently outperforms existing approaches across all four species under random-split evaluation. In the leave-one-dataset-out setting for A. thaliana, where performance drops markedly for all methods due to strong batch effects and dataset heterogeneity, scPlantAnnotate nonetheless achieves the highest Accuracy, Macro-F1, Balanced Accuracy, and Macro-AUROC on average and ranks first on most held-out datasets. These results demonstrate improved robustness to dataset shifts, a critical yet underexplored challenge in plant scRNA-seq analysis. A freely accessible web server enables users to annotate their own datasets using pretrained models. scPlantAnnotate provides a plant-specific, Transformer-based framework for single-cell annotation that delivers state-of-the-art performance and enhanced robustness to unseen datasets. By addressing limitations of existing tools and enabling scalable reference-based annotation, scPlantAnnotate supports the development of comprehensive plant cell atlases and facilitates broader use of single-cell genomics in plant biology.