Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Transformer model”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRS1981

Photolysis of CH4-NH3 mixtures and PH3 as models for the photochemical transformations on the primitive earth and Jupiter

Methane, ammonia and phosphine are some of the possible constituents of the atmospheres of the Jovian planets and their satellites. Photolysis of NH3 in the presence of CH4 at 185 nm in the temperature range of 25 C to -100 C results in the decomposition of CH4. The reaction is inhibited by added H2 or SF6. These findings are consistent with the reaction of hot hydrogen atoms with CH4 to give the CH3 radical. P2H4 is the initial product formed by the photolysis of PH3 at 206 nm. Kinetic studies established that it is the intermediate in the formation of P4 from PH3. The potential significance of these reactions to the atmospheric photochemistry of the Jovian planets and moons is discussed.

Ferris, J. P.↗
NASA NTRS1990

Model checking for linear temporal logic: An efficient implementation

This report provides evidence to support the claim that model checking for linear temporal logic (LTL) is practically efficient. Two implementations of a linear temporal logic model checker is described. One is based on transforming the model checking problem into a satisfiability problem; the other checks an LTL formula for a finite model by computing the cross-product of the finite state transition graph of the program with a structure containing all possible models for the property. An experiment was done with a set of mutual exclusion algorithms and tested safety and liveness under fairness for these algorithms.

Sherman, Rivi↗
NASA NTRS1994

Turbulence models for compressible boundary layers

It is shown that to satisfy the general accepted compressible law of the wall derived from the Van Driest transformation, turbulence modeling coefficients must actually be functions of density gradients. The transformed velocity profiles obtained by using standard turbulence model constants have too small a value of the effective von Karman constant kappa in the log-law region (inner layer). Thus, if the model is otherwise accurate, the wake component is overpredicted and the predicted skin friction is lower than the expected value.

Huang, P. G.↗
NASA NTRSDate not supplied

Language Model For Earth Science: Exploring Potential Downstream Applications As Well As Current Challenges

The use of deep learning techniques to build transformer language models such as SciBERT and GPT3 have transformed the natural language technology (NLT) landscape. These new NLTs are being used in speech to text and vice versa, auto-mated text classification, sentiment analysis, topic modeling, text summarization, and cognitive assistants. While Earth science has no shortage of unstructured data such as journal and conference papers, little efforts have focused on harnessing NLTs for knowledge extraction and supporting the scientific process. This paper surveys the use of language models in different science. BERT-E, a new Earth science-specific language model, is presented. BERT-E is generated using a transfer learning solution. A language model that has already been trained for general Science (SciBERT) is fine-tuned using abstracts and full text extracted from various Earth science-related articles. A downstream keywords classification application is used for evaluation, and the use of BERT-E shows improved performance. The need to develop a robust set of benchmarks in evaluating the language model such as BERT-E is discussed. Finally, example applications are presented to inspire additional ideas for applications using domain-specific language models.

R Ramachandran↗
NASA NTRS1984

Inversion of vegetation canopy reflectance models for estimating agronomic variables. II - Use of angle transforms and error analysis as illustrated by Suits' model

The technique for inverting a vegetation canopy reflectance model described earlier (Goel and Strebel, 1983) is investigated further. The novel concept of an 'angle transform' is introduced. This concept allows the formation of functions of reflectances at different view zenith and azimuth angles, which are either sensitive or insensitive to a certain agronomic parameter. A proper combination of these functions can allow determination of all the important agronomic and spectral parameters from measured canopy reflectance data. The technique is demonstrated using Suits' (1972) model for homogeneous canopies. It is shown that leaf area index, leaf reflectance and transmittance, and average leaf angle all can be determined from the canopy reflectance at a set of selected view zenith and azimuth angles. A sensitivity analysis of the calculated values to the errors in the data is also carried out. Guidelines are formulated for the number and types of observations required to obtain the values of a particular canopy variable to within a given degree of accuracy for a given level of error in the measurement of canopy reflectance.

Goel, N. S.↗
NASA NTRS2023

SafeAeroBERT: Towards a Safety-Informed Aerospace-Specific Language Model

As aviation systems continue to operate with high traffic, large amounts of documents containing safety-relevant data continue to be generated via reporting systems such as the ASRS. Advanced natural language processing techniques, specifically pre-trained language models, have shown great success in domain-specific applications; however, the text in aviation safety reports is inundated with jargon and thus not fully utilized by general pre-trained models. In this research, we work towards developing a safety-informed aerospace-specific language model by pre-training a Bidirectional Encoder Representations from Transformer (BERT) model on reports from the Aviation Safety Reporting System and the National Transportation Safety Board. The resulting model, called SafeAeroBERT, is fine-tuned for the specific task of document classification, and can be further tuned for named-entity recognition, relation detection, information retrieval, and summarization. Results from the classification task are compared between SafeAeroBERT, the base BERT, and SciBERT models and show SafeAeroBERT outperforms the general BERT and SciBERT on classifying reports about human factors, aircraft, and procedure. SafeAeroBERT can be used on custom tasks, not limited to document classification, and is intended to aid an intelligent knowledge manager for safety report repositories.

Aviation↗
NASA NTRSDate not supplied

SafeAeroBERT: Towards a Safety-Informed Aerospace-Specific Language Model

As aviation systems continue to operate with high traffic, large amounts of documents containing safety-relevant data continue to be generated via reporting systems such as the Aviation Safety Reporting System (ASRS). Advanced natural language processing techniques, specifically pre-trained language models, have shown great success in domain-specific applications; however, the text in aviation safety reports is inundated with jargon and thus not fully utilized by general pre-trained models. In this research, we work towards developing a safety-informed aerospace-specific language model by pre-training a Bidirectional Encoder Representations from Transformer (BERT) model on reports from the Aviation Safety Reporting System and the National Transportation Safety Board. The resulting model, called SafeAeroBERT, is fine-tuned for the specific task of document classification, and can be further tuned for named-entity recognition, relation detection, information retrieval, and summarization. Results from the classification task are compared between SafeAeroBERT, the base BERT, and SciBERT models and show SafeAeroBERT outperforms the general BERT and SciBERT on classifying reports about weather and procedure. SafeAeroBERT can be used on custom tasks, not limited to document classification, and is intended to aid an intelligent knowledge manager for safety report repositories.

Aviation↗
NASA NTRS2020

Assimilation of SMAP and ASCAT Soil Moisture Retrievals into the JULES Land Surface Model Using the Local Ensemble Transform Kalman Filter

A land data assimilation system is developed to merge satellite soil moisture retrievals into the Joint U.K. Land Environment Simulator (JULES) land surface model (LSM) using the Local Ensemble Transform Kalman Filter (LETKF). The system assimilates microwave soil moisture retrievals from the Soil Moisture Active Passive (SMAP) radiometer and the Advanced Scatterometer (ASCAT) after bias correction based on cumulative distribution function fitting. The soil moisture assimilation estimates are evaluated with ground-based soil moisture measurements over the continental U.S. for five consecutive warm seasons (May–September of 2015–2019). The result shows that both SMAP and ASCAT retrievals improve the accuracy of soil moisture estimates. Especially, the SMAP single-sensor assimilation experiment shows the best performance with the increase of temporal anomaly correlation by ΔR ~ 0.05 for surface soil moisture and ΔR ~ 0.03 for root-zone soil moisture compared with the LSM simulation without satellite data assimilation. SMAP assimilation is more skillful than ASCAT assimilation primarily because of the greater skill of the assimilated SMAP retrievals compared to the ASCAT retrievals. The skill improvement also depends significantly on the region; the higher skill improvement in the western U.S. compared to the eastern U.S. is explained by the Kalman gain in the two experiments. Additionally, the regional skill differences in the single-sensor assimilation experiments are attributed to the number of assimilated observations. Finally, the soil moisture assimilation estimates provide more realistic land surface information than model-only simulations for the 2015 and the 2016 western U.S. droughts, suggesting the advantage of using satellite soil moisture retrievals in the current drought monitoring system.

SMAP↗
NASA NTRS1975

On Similarity Transformation and Geodetic Network Distortions Based on Doppler Satellite Observations

Models used in geodesy to transform two sets of coordinates are studied and distortions in geodetic networks are investigated. Commonly used transformation models are first reviewed and most of them are interpreted. Differences between various models are discussed. Pitfalls in partial solutions are then considered. It is shown that only as many chords and/or directional elements can be used in the computation as are needed to completely determine the size or shape of the polyhedron implied in the set of Cartesian coordinates. Each additional element causes the normal matrix to be singular provided that all correlations between the chords are used. A number of tables and maps indicating distortions in the NAD 27, Precise Traverse M-R '72, AUS, and SAD 69 geodetic datums are also included. The residuals of the coordinates are scanned for systematic patterns after transforming each geodetic system to the NWL9D Doppler system. Also, an attempt is made to show scale distortions in the NAD 27.

Leick, Alfred↗
NASA NTRS1979

Synthesis of a control model for a liquid nitrogen cooled, closed circuit, cryogenic nitrogen wind tunnel and its validation

The details of the efforts to synthesize a control-compatible multivariable model of a liquid nitrogen cooled, gaseous nitrogen operated, closed circuit, cryogenic pressure tunnel are presented. The synthesized model was transformed into a real-time cryogenic tunnel simulator, and this model is validated by comparing the model responses to the actual tunnel responses of the 0.3 m transonic cryogenic tunnel, using the quasi-steady-state and the transient responses of the model and the tunnel. The global nature of the simple, explicit, lumped multivariable model of a closed circuit cryogenic tunnel is demonstrated.

Balakrishna, S.↗
NASA NTRSDate not supplied

What Went Wrong: A Survey of Wildfire UAS Mishaps through Named Entity Recognition

Increasingly, unmanned aircraft systems (UAS) are being applied to wildfire incidents for tasks such as mapping, aerial ignition, and delivery. As a result, aviation incident reporting systems for wildfires are beginning to accumulate data related to UAS mishaps in wildfire response. In this research, we apply state-of-the-art natural language processing (NLP) techniques to develop a custom Named Entity Recognition (NER) model which extracts entities relevant to safety analysts. The custom NER model is built by fine-tuning an existing Bidirectional Encoder Representations from Transformers (BERT) model, resulting in a generalizable NER model that can extract engineering relevant entities including failure modes, causes, effects, control processes, and recommendations from failure-relevant text. This model performs passably, with a weighted average f1 score of 0.33 across entity types, indicating more labeled training data is needed. Extracted entities are used to form a Failure Modes and Effects Analysis (FMEA)-style survey of wildfire UAS mishaps reported using the SAFECOM system. Similar mishaps are manually clustered and reported as single rows within an FMEA. Foreach cluster, we compute frequency, severity, and overall riskin accordance with FAA standards. This methodology can beapplied as part of a broader safety management system totrack trends in mishaps (e.g., likelihood, severity) and discoverknowledge (e.g., causes, effects) that can be utilized to improvesafety outcomes and system performance.

Machine Learning↗
NASA NTRSDate not supplied

Towards an Aviation Large Language Model by Fine-tuning and Evaluating Transformers

In the aviation domain, there are many applications for machine learning and artificial intelligence tools that utilize natural language. For example, there is a desire to know the commonalities in written safety reports such as voluntary post incidents reports or create more accurate transcripts of air traffic management conversations. Another use-case is the possibility of extracting airspace procedures and constraints currently written in documents such as Letters of Agreement (LOA) which is used as the evaluation case in this paper. These applications can benefit from the use of state-of-the-art Natural Language Processing (NLP) techniques when adapted to the language/phraseology specific to the aviation domain. This paper evaluates the viability of transferring pre-trained large language models to the aviation domain by adapting transformer based models using aviation datasets. This paper utilized two datasets to adapt a ‘Robustly Optimized Bidirectional Encoder Representations from Transformers Approach’ (RoBERTa) model and two down-stream classification tasks to assess its performance. These datasets are all built upon Letters of Agreement which are Federal Aviation Administration (FAA) documents that formalize airspace operations across the national airspace system. The first two datasets are used for the adaptation of RoBERTa to the aviation domain and were of different sizes to assess the number of documents needed to adapt to the aviation domain. They contain many examples of ‘aviation English’ using domain specific terminology and phrasing which serves as a representative basis to perform the unsupervised adaptation. The second dataset is a separate set of LOA documents with two sets of classification labels to be used for evaluation; one at the document level and one at the line level. These down-stream evaluations allowed the measurement of improvement by adapting RoBERTa. The accuracy increased by 4-6% on both tasks and the F1 score on the class of interest increased by 4-8% from the adaptation.

Air Traffic Management↗
NASA NTRSDate not supplied

Towards an Aviation Large Language Model by Fine-tuning and Evaluating Transformers

In the aviation domain, there are many applications for machine learning and artificial intelligence tools that utilize natural language. For example, there is a desire to know the commonalities in written safety reports such as voluntary post incidents reports or create more accurate transcripts of air traffic management conversations. Another use-case is the possibility of extracting airspace procedures and constraints currently written in documents such as Letters of Agreement (LOA) which is used as the evaluation case in this paper. These applications can benefit from the use of state-of-the-art Natural Language Processing (NLP) techniques when adapted to the language/phraseology specific to the aviation domain. This paper evaluates the viability of transferring pre-trained large language models to the aviation domain by adapting transformer based models using aviation datasets. This paper utilized two datasets to adapt a ‘Robustly Optimized Bidirectional Encoder Representations from Transformers Approach’ (RoBERTa) model and two down-stream classification tasks to assess its performance. These datasets are all built upon Letters of Agreement which are Federal Aviation Administration (FAA) documents that formalize airspace operations across the national airspace system. The first two datasets are used for the adaptation of RoBERTa to the aviation domain and were of different sizes to assess the number of documents needed to adapt to the aviation domain. They contain many examples of ‘aviation English’ using domain specific terminology and phrasing which serves as a representative basis to perform the unsupervised adaptation. The second dataset is a separate set of LOA documents with two sets of classification labels to be used for evaluation; one at the document level and one at the line level. These down-stream evaluations allowed the measurement of improvement by adapting RoBERTa. The accuracy increased by 4-6% on both tasks and the F1 score on the class of interest increased by 4-8% from the adaptation.

Air Traffic Management↗
NASA NTRS2022

Modeling Demographic-Driven Vegetation Dynamics and Ecosystem Biogeochemical Cycling in NASA GISS’s Earth System Model (ModelE-BiomeE v.1.0)

We developed a demographic vegetation model, BiomeE, to improve the modeling of vegetation dynamics and ecosystem biogeochemical cycles in the NASA Goddard Institute of Space Studies' ModelE Earth system model. This model includes the processes of plant growth, mortality, reproduction, vegetation structural dynamics, and soil carbon and nitrogen storage and transformations. The model combines the plant physiological processes of ModelE's original vegetation model, Ent, with the plant demographic and ecosystem nitrogen processes that have been represented in the Geophysical Fluid Dynamics Laboratory's LM3-PPA. We used nine plant functional types to represent global natural vegetation functional diversity, including trees, shrubs, and grasses, and a new phenology model to simulate vegetation seasonal changes with temperature and precipitation fluctuations. Competition for light and soil resources is individual based, which makes the modeling of transient compositional dynamics and vegetation succession possible. Overall, the BiomeE model simulates, with fidelity comparable to other models, the dynamics of vegetation and soil biogeochemistry, including leaf area index, vegetation structure (e.g., height, tree density, size distribution, and crown organization), and ecosystem carbon and nitrogen storage and fluxes. This model allows ModelE to simulate transient and long-term biogeophysical and biogeochemical feedbacks between the climate system and land ecosystems. Furthermore, BiomeE also allows for the eco-evolutionary modeling of community assemblage in response to past and future climate changes with its individual-based competition and demographic processes.

Biogeochemical cycles↗