SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

Semantic Search with Sentence-BERT for Design Information Retrieval

Managing and referencing design knowledge is a critical activity in the design process. However, reliably retrieving useful knowledge can be a frustrating experience for users of knowledge management systems due to inherent limitations of standard keyword-based searches. In this research, we consider the task of retrieving relevant lessons learned from the NASA Lessons Learned Information System (LLIS). To this end, we apply a state-of-the-art natural language processing (NLP) technique for information retrieval: semantic search with sentence-BERT, which is a modification of a Bidirectional Encoder Representations from Transformers (BERT) model that uses siamese and triplet network architectures to obtain semantically meaningful sentence embeddings. While the pretrained sBERT model shows excellent out-of-the-box performance, we further fine-tune the model on data from the LLIS so that it learns on design engineering-relevant vocabulary. We quantify the improvement in query results using both standard sBERT and fine-tuned sBERT over the LLIS’s built-in keyword search. Additionally, we demonstrate a use case for the query system by searching for lessons learned relevant to specific requirements from a NASA project as part of a broader knowledge management and retrieval system. Results indicate that applying state-of-the-art natural language processing techniques, especially when fine-tuned using engineering data, to design information retrieval tasks shows significant promise in modernizing design knowledge management systems.

Hannah S. Walsh↗

NASA NTRSDate not supplied

Classification of Notices to Airmen using Natural Language Processing

This paper establishes the feasibility of using Natural Language Processing (NLP) to classify NOTAMs or Notices to Airmen – a pilot messaging framework to gather real-time situational awareness. Present day air mobility operations heavily rely on NOTAMs. However, pilots often have difficulty interpreting NOTAMs due to the sheer volume of inapplicable messages and unclear abbreviations. Using NLP, the presented study analyzes the accuracy of classifying NOTAMs and, thereby, the efficiency of generating actionable interpretations in real time. To this effect, efficacies of four NLP neural network architectures were analyzed, including three Recurrent Neural Networks (RNNs) with GloVe, Word2Vec, and FastText word embeddings, and one trained Bi-Directional Encoder Representations from Transformers (BERT) model. The four neural networks were trained and evaluated on three open-source datasets of varying text lengths, vocabularies, and grammars, taken from e-commerce product descriptions, social media tweets, and unstructured descriptions for data and analytics services on open data marketplaces such as NASA’s Data and Reasoning Fabric (DRF) platform. This provided cross-analysis of each neural network architecture’s performance per text type. The best performing architecture, BERT, was then fine-tuned on a collection of open-source NOTAM data. Post-training, a real-time NOTAM classification service was implemented to draw inference on new NOTAMs using the trained model, which demonstrated close to 99% accuracy in classification. This modular classification service is envisioned to be integrated with a data and analytics delivery platform, such as the DRF, thus availing real-time contextualization of NOTAMs to air mobility clients, humans, and machines for enhanced decision making.

Aiden C. Szeto↗

NASA NTRSDate not supplied

Classification of Notices to Airmen using Natural Language Processing

Aiden Szeto↗

DOE OSTI2023

Computed axial lithography optimization system

A system for determining a light intensity field for use in manufacturing a 3D object from a volume of material. The system receives a 3D specification of a 3D geometry for the 3D object that specifies voxels within the volume that contain material that is to be part of the 3D object. The system employs a cost function for effectiveness of a light intensity field in manufacturing the 3D object. The cost function may be an adjoint of an Attenuated Radon Transform that models an energy dose that each voxel would receive during manufacture of the 3D object using the light intensity field. The system applies an optimization technique that employs the cost function to generate a measure of the effectiveness of possible light intensity fields and outputs an indication of a light intensity field that will be effective in manufacturing the 3D object.

Shusteff, Maxim↗

DOE OSTI2022

Simple Autoignition Model for the Derived Cetane Number of Oxygenated Compounds and Fuel Blends

A four-step autoignition model was used to derive an expression for the ignition delay and derived cetane number (DCN) measured by ignition quality testers (IQT) for oxygenated compounds and blends as a function of each compound’s global initiation and net chain branching rate constants. Expressions for these values were determined assuming they could be related to the functional groups present in a compound. Measurements for 124 compounds and 94 binary and ternary blends, gathered from the literature and performed at UW-Madison, were used to obtain the dependence of the measured ignition delay on each functional group. The new model was able to describe the ignition behavior of both the compounds and blends, with an average DCN error of 5.0 (22%) and 3.5 (15%), respectively. Additionally, the blend model was transformed into a mixing rule by incorporating existing IQT ignition delay data for each compound. Use of the mixing rule was found to offer improvements over the full prediction, with an average DCN error of 3.2 (14%).

Witkowski, Dustin↗

NASA NTRS1991

A nested grid, nonhydrostatic, elastic model using a terrain-following coordinate transformation - The radiative-nesting boundary conditions

This study presents a nested-grid nonhydrostatic and elastic model using a terrain-following coordinate transformation as well as a unique application of grid-nesting techniques to the time-splitting elastic model. A simulation of the 10-m-high Witch of Agnesi Mountain provides the control to test this new model. The results show that the model produces the same solution as that derived from a simple linear analytic model. It is demonstrated that the new nested-grid model improves model resolution without resorting to the costly method of placing a fine-resolution grid over the entire domain. Since the wave reflection from the boundaries of the fine-grid model is well controlled, the boundary of the nested fine-grid model can be located even at the wave-active region. The model can be used to simulate various weather systems in which scale interactions are important.

Chen, Chaing↗

NASA NTRS1991

Transform methods for precision continuum and control models of flexible space structures

An open loop optimal control algorithm is developed for general flexible structures, based on Laplace transform methods. A distributed parameter model of the structure is first presented, followed by a derivation of the optimal control algorithm. The control inputs are expressed in terms of their Fourier series expansions, so that a numerical solution can be easily obtained. The algorithm deals directly with the transcendental transfer functions from control inputs to outputs of interest, and structural deformation penalties, as well as penalties on control effort, are included in the formulation. The algorithm is applied to several structures of increasing complexity to show its generality.

Lupi, Victor D.↗

DOE OSTI2022

Learning to simulate high energy particle collisions from unlabeled data

In many scientific fields which rely on statistical inference, simulations are often used to map from theoretical models to experimental data, allowing scientists to test model predictions against experimental results. Experimental data is often reconstructed from indirect measurements causing the aggregate transformation from theoretical models to experimental data to be poorly-described analytically. Instead, numerical simulations are used at great computational cost. We introduce Optimal-Transport-based Unfolding and Simulation (OTUS), a fast simulator based on unsupervised machine-learning that is capable of predicting experimental data from theoretical models. Without the aid of current simulation information, OTUS trains a probabilistic autoencoder to transform directly between theoretical models and experimental data. Identifying the probabilistic autoencoder’s latent space with the space of theoretical models causes the decoder network to become a fast, predictive simulator with the potential to replace current, computationally-costly simulators. Here, we provide proof-of-principle results on two particle physics examples, Z-boson and top-quark decays, but stress that OTUS can be widely applied to other fields.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2022

Dynamic mass accountancy modeling of a molten salt reactor using equilibrium thermodynamics

A mechanistic-based mass accountancy model in the context of liquid-fueled molten salt reactors was implemented in the dynamic systems modeling software library TRANSFORM by way of coupling with the equilibrium thermodynamics code Thermochimica. Liquid-fueled molten salt reactors present new challenges for mass accountancy because of the dissolution of fuel and evolved fission products, which may be soluble in the salt, off-gas, or precipitate. Two cases of mass loss from the molten salt were addressed: off-gassing and precipitation. The software implementation was tested through a series of increasingly complex demonstration problems, culminating in a model of the primary fuel and primary coolant loops of the molten salt demonstration reactor. Analysis shows that negligible mass was lost from the salt under normal operating conditions, but an overheating event caused by partial loss of fuel loop cooling resulted in release of measurable amounts of uranium (among other elements) via off-gassing. The tools developed here are primarily aimed at capability development but are readily available for use in further modeling of molten salt reactor concepts. As a result, these tools have not yet been validated, and future experimental work to perform this validation is recommended.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

NASA NTRS2010

Experiment and Modeling of Simultaneous Creep, Plasticity and Transformation of High Temperature Shape Memory Alloys During Cyclic Actuation

The present work is focused on studying the cycling actuation behavior of HTSMAs undergoing simultaneous creep and transformation. For the thermomechanical testing, a high temperature test setup was assembled on a MTS frame with the capability to test up to temperatures of 600 C. Constant stress thermal cycling tests were conducted to establish the actuation characteristics and the phase diagram for the chosen HTSMA. Additionally, creep tests were conducted at constant stress levels at different test temperatures to characterize the creep behavior of the alloy over the operational range. A thermodynamic constitutive model is developed and extended to take into account a) the effect of multiple thermal cycling on the generation of plastic strains due to transformation (TRIP strains) and b) both primary and secondary creep effects. The model calibration is based on the test results. The creep tests and the uniaxial tests are used to identify the viscoplastic behavior of the material. The parameters for the SMA properties, regarding the transformation and transformation induced plastic strain evolutions, are obtained from the material phase diagram and the thermomechanical tests. The model is validated by predicting the material behavior at different thermomechanical test conditions.

Kumar, Parikshith K.↗

NASA NTRS1981

Sample data effects of high-pass filters

Four mathematical models of linear first and second order high pass washout filters were analyzed. These models were Euler's Integration, Zero-Order Hold, Bilinear Transformation, and Second-Order Adams-Bashforth Integration. Bode responses for each model at various sample rates were compared to the continuous filter response. Higher sample rates produced Bode responses approaching the continuous response and the Bilinear Transformation model produced the best responses over the frequency spectrum and sample rates. Pole location analysis of each model in the z-plane showed the Bilinear Transformation and Zero-Order Hold models gave stable poles regardless of time step size, whereas the other models did not display stable poles. A near constant gain error over the entire frequency spectrum was discovered in the Zero-Order Hold cases and a correction gain was calculated for the first-order high-pass filter case.

Chin, D. O.↗

DOE OSTI2024

Improving streamflow predictions across CONUS by integrating advanced machine learning models and diverse data

Accurate streamflow prediction is crucial to understand climate impacts on water resources and develop effective adaption strategies. A global long short-term memory (LSTM) model, using data from multiple basins, can enhance streamflow prediction, yet acquiring detailed basin attributes remains a challenge. To overcome this, we introduce the Geo-vision transformer (ViT)-LSTM model, a novel approach that enriches LSTM predictions by integrating basin attributes derived from remote sensing with a ViT architecture. Applied to 531 basins across the Contiguous United States, our method demonstrated superior prediction accuracy in both temporal and spatiotemporal extrapolation scenarios. Geo-ViT-LSTM marks a significant advancement in land surface modeling, providing a more comprehensive and effective tool for better understanding the environment responses to climate change.

Tayal, Kshitij↗

DOE OSTI2025

Deformable phrase level attention: A flexible approach for improving AI based medical coding

Objective: Improving the AI-driven automated medical encoding of clinical text plays a vital role in gathering information on the occurrence of diseases to improve population-level health. This work presents a novel attention mechanism designed to enhance text classification models and ensure appropriate classification of medical concepts in unstructured electronic health records. Materials and Methods: We developed a deformable, phrase-level attention mechanism to identify important lexical word-level and contextual phrase-level information from clinical text documents. We evaluated conventional and transformer-based deep learning models that we extended with our attention mechanism on the extraction of critical cancer information (e.g., site, subsite, laterality, histology, behavior) from 629,908 electronic pathology reports and on the automated medical encoding of 52,722 hospital discharge summaries. Results: Transformer-based models with the deformable, phrase-level attention mechanism achieved the best performance on the extraction of critical cancer information from pathology reports. Conventional- and transformer-based models show similar or better performance than their baseline counterparts on the automated medical encoding of clinical documents. Discussion: The addition of phrase-level information allowed models extended with our proposed method to outperform standard word-level attention. Our method showed favorable properties for the real-world application in terms of model robustness and phenotyping. These results indicate that our method is promising for automated data harmonization for common data models. Conclusion: This work proposes a novel deformable, phrase-level attention mechanism that enhances text classification models in the extraction of medical concepts from clinical text documents. We demonstrate strong performances on two clinical text datasets and showcase real-world deployability of our method.

Automated medical encoding↗

DOE OSTI2021

Cross-national analysis of food security drivers: comparing results based on the Food Insecurity Experience Scale and Global Food Security Index

Abstract The second UN Sustainable Development Goal establishes food security as a priority for governments, multilateral organizations, and NGOs. These institutions track national-level food security performance with an array of metrics and weigh intervention options considering the leverage of many possible drivers. We studied the relationships between several candidate drivers and two response variables based on prominent measures of national food security: the 2019 Global Food Security Index (GFSI) and the Food Insecurity Experience Scale’s (FIES) estimate of the percentage of a nation’s population experiencing food security or mild food insecurity (FI ). We compared the contributions of explanatory variables in regressions predicting both response variables, and we further tested the stability of our results to changes in explanatory variable selection and in the countries included in regression model training and testing. At the cross-national level, the quantity and quality of a nation’s agricultural land were not predictive of either food security metric. We found mixed evidence that per-capita cereal production, per-hectare cereal yield, an aggregate governance metric, logistics performance, and extent of paid employment work were predictive of national food security. Household spending as measured by per-capita final consumption expenditure (HFCE) was consistently the strongest driver among those studied, alone explaining a median of 92% and 70% of variation (based on out-of-sample R 2 ) in GFSI and FI , respectively. The relative strength of HFCE as a predictor was observed for both response variables and was independent of the countries used for model training, the transformations applied to the explanatory variables prior to model training, and the variable selection technique used to specify multivariate regressions. The results of this cross-national analysis reinforce previous research supportive of a causal mechanism where, in the absence of exceptional local factors, an increase in income drives increase in food security. However, the strength of this effect varies depending on the countries included in regression model fitting. We demonstrate that using multiple response metrics, repeated random sampling of input data, and iterative variable selection facilitates a convergence of evidence approach to analyzing food security drivers.

42 ENGINEERING↗

NASA NTRS2010

ICER-3D Hyperspectral Image Compression Software

Software has been developed to implement the ICER-3D algorithm. ICER-3D effects progressive, three-dimensional (3D), wavelet-based compression of hyperspectral images. If a compressed data stream is truncated, the progressive nature of the algorithm enables reconstruction of hyperspectral data at fidelity commensurate with the given data volume. The ICER-3D software is capable of providing either lossless or lossy compression, and incorporates an error-containment scheme to limit the effects of data loss during transmission. The compression algorithm, which was derived from the ICER image compression algorithm, includes wavelet-transform, context-modeling, and entropy coding subalgorithms. The 3D wavelet decomposition structure used by ICER-3D exploits correlations in all three dimensions of sets of hyperspectral image data, while facilitating elimination of spectral ringing artifacts, using a technique summarized in "Improving 3D Wavelet-Based Compression of Spectral Images" (NPO-41381), NASA Tech Briefs, Vol. 33, No. 3 (March 2009), page 7a. Correlation is further exploited by a context-modeling subalgorithm, which exploits spectral dependencies in the wavelet-transformed hyperspectral data, using an algorithm that is summarized in "Context Modeler for Wavelet Compression of Hyperspectral Images" (NPO-43239), which follows this article. An important feature of ICER-3D is a scheme for limiting the adverse effects of loss of data during transmission. In this scheme, as in the similar scheme used by ICER, the spatial-frequency domain is partitioned into rectangular error-containment regions. In ICER-3D, the partitions extend through all the wavelength bands. The data in each partition are compressed independently of those in the other partitions, so that loss or corruption of data from any partition does not affect the other partitions. Furthermore, because compression is progressive within each partition, when data are lost, any data from that partition received prior to the loss can be used to reconstruct that partition at lower fidelity. By virtue of the compression improvement it achieves relative to previous means of onboard data compression, this software enables (1) increased return of hyperspectral scientific data in the presence of limits on the rates of transmission of data from spacecraft to Earth via radio communication links and/or (2) reduction in spacecraft radio-communication power and/or cost through reduction in the amounts of data required to be downlinked and stored onboard prior to downlink. The software is also suitable for compressing hyperspectral images for ground storage or archival purposes.

Xie, Hua↗

NASA NTRS2010

Automata learning algorithms and processes for providing more complete systems requirements specification by scenario generation, CSP-based syntax-oriented model construction, and R2D2C system requirements transformation

Systems, methods and apparatus are provided through which in some embodiments, automata learning algorithms and techniques are implemented to generate a more complete set of scenarios for requirements based programming. More specifically, a CSP-based, syntax-oriented model construction, which requires the support of a theorem prover, is complemented by model extrapolation, via automata learning. This may support the systematic completion of the requirements, the nature of the requirement being partial, which provides focus on the most prominent scenarios. This may generalize requirement skeletons by extrapolation and may indicate by way of automatically generated traces where the requirement specification is too loose and additional information is required.

Hinchey, Michael G.↗

DOE OSTI2024

Understanding Line Losses and Transformer Losses in Rural Isolated Distribution Systems

Rural, isolated power systems in the mainland U.S. and in states like Alaska and Hawaii are powered by assets like diesel generators. These rural, isolated power systems also cannot operate at the higher band of medium voltage (like 69kV). They are primarily in the 12 to 14 kV range to keep the cost of the distribution investments lower. Because of this mid-band medium voltage range, the line losses and distribution transformers losses consume significant diesel consumption (almost 10 percent of the peak load). This work considers one such power system powering an isolated system and presents key findings online losses, and transformer losses. Understanding and documenting the impacts is critical for these communities operating their power systems and take actions to reduce expensive diesel consumption. In this paper, we will present one such typical grid and model it in electromagnetic transients (EMT) domain. We used the tower structure, under ground cabling installation to develop high fidelity models of lines. We also used high fidelity models of distribution transformers to present the no-load losses and full load loses. We will also present technical solutions available commercially off-the-shelf to reduce these losses and reduce diesel consumption. This work will be a primer for communities to understand the technical challenges and to understand the possible solution available to solve such challenges for rural, isolated power system operators.

blackstart↗

DOE OSTI2025

Generalizable machine learning potentials for quantum-accurate predictions of non-equilibrium behavior in 2D materials

Machine learning interatomic potentials (ML-IAPs) are emerging as transformative tools in materials modeling, promising quantum-level accuracy at a fraction of the computational cost. However, their ability to generalize beyond equilibrium configurations and to reliably capture defect- and temperature-driven behavior remains underexplored. Here, we develop and benchmark two state-of-the-art ML-IAPs, Spectral Neighbor Analysis Potential (SNAP) and Allegro, on a comprehensive dataset for monolayer MoSe₂. Using density functional theory (DFT) as the reference, we evaluate their performance in capturing stress–strain behavior, phase transition energetics, defect evolution, edge stability, and fracture toughness. Allegro, a deep equivariant neural network potential, surpasses both SNAP and the classical Tersoff potential in accuracy, efficiency, and transferability. Importantly, both ML potentials accurately reproduce experimental fracture measurements and ab initio predictions of inversion domain formation—phenomena well beyond their training sets. Our findings establish ML-IAPs as viable replacements for traditional force fields in the study of non-equilibrium mechanical phenomena, enabling large-scale, high-fidelity simulations in 2D materials and beyond. In conclusion, this work provides a broadly applicable framework for data-driven modeling of structural and functional transformations under extreme conditions.

2D materials↗