Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “optimal start”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 937 records · Page 52
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Mitigation of Acute Ammonia Emissions With Biochar During Swine Manure Agitation Before Pump-Out: Proof-of-the-Concept

Ammonia (NH 3 ) emissions from animal agriculture can cause eutrophication of water ecosystems and are precursors to secondary particulate matter (PM 2 .5 ). NH 3 emissions from stored swine manure represent nutrient loss affecting the fertilizing value of manure. The short-term emission bursts occur when farmers agitate manure before emptying storage and fertilizing fields. There is no proven technology to mitigate gaseous emissions during agitation, while the hazards of acute releases (e.g., H 2 S) are well-known. Biochar mitigates NH 3 emissions from manure over the long-term. The objective of this research was to evaluate the mitigation of acute NH 3 emissions during/after agitation. Two biochars, highly alkaline and porous (HAP from corn stover) and red oak (RO), were tested. The 6 and 12 mm-thick layers of biochar powder were surficial applied followed by 3 min agitation. NH 3 concentrations were measured before/during/after agitation. Mitigation was assessed by comparing: (i) the maximum (peak) flux, (ii) total emission (from agitation start till NH 3 concentration returned to the before-agitation), and (iii) the total emissions during agitation. The 12 mm HAP significantly ( p < 0.05) reduced (i–iii) by 63, 70, and 85%, respectively. The 6 mm HAP significantly reduced (i–iii) by 76, 75, and 78%, respectively. The 12 mm RO significantly reduced (i–iii) by 9, 53, and 57%, respectively. The 6 mm RO significantly reduced (i–iii) by 61, 86, and 63%, respectively. The NH 3 emission kinetics model confirmed that a 6 mm dose was just as effective as the larger dose. More research is needed to optimize and scale-up mitigating emissions and retention of nutrients in manure with biochar.

Chen, Baitong↗
NASA NTRS1976

A general unified approach to modelling switching-converter power stages

A method for modeling switching-converter power stages is developed, whose starting point is the unified state-space representation of the switched networks and whose end result is either a complete state-space description or its equivalent small-signal low-frequency linear circuit model. A new canonical circuit model is proposed, whose fixed topology contains all the essential input-output and control properties of any dc-to-dc switching converter, regardless of its detailed configuration, and by which different converters can be characterized in the form of a table conveniently stored in a computer data bank to provide a useful tool for computer aided design and optimization. The new canonical circuit model predicts that, in general, switching action introduces both zeros and poles into the duty ratio to output transfer function in addition to those from the effective filter network.

Middlebrook, R. D.↗
NASA NTRS1999

A complex ligase ribozyme evolved in vitro from a group I ribozyme domain

Like most proteins, complex RNA molecules often are modular objects made up of distinct structural and functional domains. The component domains of a protein can associate in alternative combinations to form molecules with different functions. These observations raise the possibility that complex RNAs also can be assembled from preexisting structural and functional domains. To test this hypothesis, an in vitro evolution procedure was used to isolate a previously undescribed class of complex ligase ribozymes, starting from a pool of 10(16) different RNA molecules that contained a constant region derived from a large structural domain that occurs within self-splicing group I ribozymes. Attached to this constant region were three hypervariable regions, totaling 85 nucleotides, that gave rise to the catalytic motif within the evolved catalysts. The ligase ribozymes catalyze formation of a 3',5'-phosphodiester linkage between adjacent template-bound oligonucleotides, one bearing a 3' hydroxyl and the other a 5' triphosphate. Ligation occurs in the context of a Watson-Crick duplex, with a catalytic rate of 0.26 min(-1) under optimal conditions. The constant region is essential for catalytic activity and appears to retain the tertiary structure of the group I ribozyme. This work demonstrates that complex RNA molecules, like their protein counterparts, can share common structural domains while exhibiting distinct catalytic functions.

NASA Discipline Exobiology↗
DOE OSTI2022

Manufacturing Science

The Manufacturing Science area of leadership spans the application of fundamental science research and development activities to understand the critical steps in manufacturing processes for the purposes of control and optimization. Although manufacturing is an applied area of research, fundamental science underpins our knowledge of both traditional and transformational manufacturing processes. For example, casting, welding, and many modern additive manufacturing processes that see wide use are broadly classified as solidification processes, with material transitioning from liquid to solid phases upon cooling from higher temperature. However, our understanding of the detailed physics involved in such processes is incomplete, and solidification remains a grand challenge for materials science. To enable adoption of transformative fabrication processes or to accelerate development of traditional ones, it is necessary to develop a detailed mechanistic understanding spanning multiple scales, starting with atomic building blocks (defects, interfaces, composition) and extending all the way up to macroscale performance in moderate and/or extreme environments.

36 MATERIALS SCIENCE↗
NASA NTRS2019

CHANGO: A Software Tool for Boost Stage Guidance of the Space Launch System Exploration Mission 1

The Day of Launch Initiation Load Update (DOLILU) System is the means by which the Space Launch System (SLS) Vehicle trajectory is designed, verified, and uploaded on the Day of Launch (DOL) in order to ensure a safe flight. Launch vehicles are designed to fly down a narrow angle of attack and sideslip angle corridor in order to keep them within structural load limits. The angle of attack and sideslip angle response to the launch vehicle experiences can vary significantly based upon the winds experienced on the DOL. SLS Boost Stage flight employs an open-loop guidance scheme through Solid Rocket Booster (SRB) separation. In the SLS open-loop scheme, the vehicle will fly a prescribed set of attitudes as a function of the change in altitude since launch. This set of reference attitude values and corresponding altitude reference independent values are designed with ground software using winds measured on the DOL with the goal of minimizing angle of attack and sideslip angle, thereby minimizing related ascent integrated vehicle structural loads. The table of Boost Stage attitude commands as a function of altitude gained since launch is called the chi table. A software tool called CHANGO (Chi Angle Optimizer) designs the Boost Stage chi table which is uploaded to the vehicle’s flight computer and used during ascent by the flight software (FSW). The wind and atmospheric conditions are measured prior to launch and pre-processed to become input to the CHANGO software along with a set of parameters developed in advance of the DOL. CHANGO’s target set consists of the heading and altitude rate at SRB separation determined well before launch by the Program to Optimize Simulated Trajectories (POST). CHANGO consists of a simplified three degree-of-freedom (3-DOF) simulation representing the SLS launch configuration. In general, the launch azimuth is strongly correlated with the heading at SRB separation, and the initial pitchover rate is strongly correlated with the altitude rate at SRB separation. CHANGO uses an adaptation of Powell’s method to vary the initial pitchover rate and launch azimuth to solve a 2-dimentional minimization problem. CHANGO’s trajectory simulation is phase-based, with flight events separating the phases. Each flight phase has different attitude alignment logic. CHANGO’s 3-DOF simulation starts when the vehicle’s thrust-to-weight ratio equals one, and ends at a pre-calculated SRB separation time.

Ahmad, Naeem↗
DOE OSTI2022

Toward fully automated UED operation using two-stage machine learning model

To demonstrate the feasibility of automating UED operation and diagnosing the machine performance in real time, a two-stage machine learning (ML) model based on self-consistent start-to-end simulations has been implemented. This model will not only provide the machine parameters with adequate precision, toward the full automation of the UED instrument, but also make real-time electron beam information available as single-shot nondestructive diagnostics. Furthermore, based on a deep understanding of the root connection between the electron beam properties and the features of Bragg-diffraction patterns, we have applied the hidden symmetry as model constraints, successfully improving the accuracy of energy spread prediction by a factor of five and making the beam divergence prediction two times faster. The capability enabled by the global optimization via ML provides us with better opportunities for discoveries using near-parallel, bright, and ultrafast electron beams for single-shot imaging. It also enables directly visualizing the dynamics of defects and nanostructured materials, which is impossible using present electron-beam technologies.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Regal Beloit Final Technical Report

The original project proposal submitted from NovaTorque Inc (NovaTorque) in 2016 was to improve the existing motor with 95% efficiency by reducing the losses by 21% to achieve 96%. A few months after the proposal was submitted, NovaTorque lost funding and went out of business. The assets of NovaTorque were then acquired by Regal Beloit Corporation (“Regal Beloit” or “Regal”). When the NovaTorque proposal was selected, the project was transferred to Regal Beloit. Once we had production samples from the new Regal production line, they were tested at the Regal Beloit test lab in Wausau, WI. The original NovaTorque motor had an efficiency of 95%, but when the technology was transferred to Regal Beloit, there were multiple manufacturing improvements made, even though the basic electromagnetic design did not change. The test results in Wausau showed that the motors made at the Regal Beloit plant had an efficiency of 96%. Since the goal of the project was to reduce the losses by 21%, a new efficiency target of 96.8% became the project objective. In the first budget period, motors were tested to get the baseline performance. We then used FEA modeling with ANSYS Maxwell to model the existing motor to get correlation between the FEA simulations and the actual test results. Once the model was validated, we evaluated some changes that could be made to the stator to reduce the losses and improve efficiency without changing the rotor or stator housing, keeping the modifications easy to implement. The changes were primarily in the area of making the stator axially shorter and adding Soft Magnetic Composite (SMC) tooth tips. In the second budget period, we proceeded to design and build the new stator that was identified above and identified additional improvements in the process that included a stator machining modification and the use of rectangular wire. When we actually built the motor, the choice of rectangular wire turned out to be a problem. The wire was made by squishing round wire to get the rectangular shape, which caused work hardening, making the wire too stiff to make the desired coils. We shifted to annealed square wire which was better, but we still could not maintain the proper coil envelope. The result was that we had reduced cross section area for stator laminations. We also had to have a radial offset resulting in a radial misalignment between the stator and rotor because of the oversized coils. After the motor was completed, it was shipped to Texas A&M University for testing. With the loss of flux from these issues, the measured motor efficiency was only 96.3%. The primary focus of this motor was to make sure our FEA simulation model predicted the measured losses and overall efficiency, this we moved on to the FEA simulation. The FEA simulations of the motor “as built” with misalignments had good correlation with the test results, so the next step was to use that model to optimize the design of the motor for a final build. This time we considered changes to the stator and rotor and also minor changes to the housing diameter. In the third budget period, we did the detailed design and construction of the final prototypes. The final prototypes had a slight increase in the stator diameter to fit in a standard Regal Beloit housing. We shortened the stator and use more layers of wire in the coils. We also increased the cone angle of the rotor and stator from the original 110 degrees to 130 degrees to get some additional efficiency and optimized the stator cross section. The predicted efficiency from the FEA simulations was 96.9%. When completed, the motors were tested. We were a little short of reaching our target efficiency goal of 96.8%. We were only able to get to 96.7% efficiency. While it may be possible with additional iterations in designs and future builds to gain that additional 0.1%, we believe that we are close to the best we can achieve from a practical viewpoint, and additional iterations would be more work than the potential gains would be worth. No other motor in this class can even reach the 96% that we started with.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

X-ray Multilayers for Next-Generation Astrophysics Missions

In this work, we proposed to demonstrate the feasibility of high-reflectivity multilayer coatings that will enable next-generation astrophysics missions in the x-ray region (2 – 7 nm wavelengths) where currently high-performance coatings largely do not exist. This paucity is due to multilayer interfacial effects that severely reduce the coating reflectance; our approach was based on understanding these interfacial effects and on tailored process modifications to control and minimize them. The experimental validation of our approach was made difficult by our multilayer deposition tool being down for a large portion of the project. The required repairs used significant amount of funding and time, which forced to reduce the scope of the project. Our experimental study focused on Cr/C multilayers and their optimization using N2 reactive sputtering. We showed that the highest reflectance was achieved when depositing the multilayers with 35% N2, and with a larger Cr thickness ratio than what was initially predicted by simulations. This result consists of a 13% relative increase in reflectivity compared to our previous work. It will benefit upcoming NASA proposals and LLNL’s leadership in this area, as well as topics highly relevant to LLNL/DOE’s priority areas, such as x-ray laser science and semiconductor manufacturing. This work will be further investigated as part of a Laboratory Directed Research and Development (LDRD) Exploratory Research (ER) project starting in Fiscal Year (FY) 2024.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Assessment of Carbonated Brine Injection as Low-risk Strategy for Geologic Carbon Storage

The main objective of this early-phase research was to evaluate the techno-economic feasibility and risk associated with combined brine and CO 2 storage in SWD wells using brine dissolution in the North Dakota portion of the Williston Basin. Three simulation studies were conducted to investigate: (1) CO 2 phases at different conditions, (2) wellbore compatibility, and (3) long-term storage fate in reservoir. (1) A simple reservoir model and injection simulations were created using data to represent the BEST (brine extraction and storage test) site, an operational SWD facility located near Watford City, North Dakota. The pressure evolution caused by CO 2 comingled in produced water injectate in a layer cake reservoir was then modeled while tracking aqueous CO 2 throughout the project. The salinity of the injection water, the salinity of the reservoir brine, and the amount of dissolved CO 2 comingled in the injection water were varied. (2) A wellbore corrosion model was performed using the CO 2 concentrations selected based on the reservoir modeling to examine the carbonated produced water impact on wellbore. (3) Reactive transport modeling was conducted with the optimal CO 2 concentration for this injection site to study the rock-fluid interactions and CO 2 fate in the reservoir. Results suggest that CO 2 dissolved in produced water can be injected without appreciably increasing subsurface pressure or leakage risks. Pressure buildup was found to vary with salinity but not with CO 2 mass fraction. Simulation results show that lower CO 2 percent mass fraction leads to a higher amount of CO 2 that can be dissolved at a higher injection salinity. Furthermore, the long-term goal of dissolution trapping in a traditional carbon storage project is accomplished from the start, mitigating risks associated with potential migration of buoyant CO 2 , so long as the reservoir pressure and temperature are used to determine the maximum mass fraction of the dissolved CO 2 .

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

An optimized approach and inflation media for obtaining complimentary mass spectrometry-based omics data from human lung tissue

Human disease states are biomolecularly multifaceted and can span across phenotypic states, therefore it is important to understand diseases on all levels, across cell types, and within and across microanatomical tissue compartments. To obtain an accurate and representative view of the molecular landscape within human lungs, this fragile tissue must be inflated and embedded to maintain spatial fidelity of the location of molecules and minimize molecular degradation for molecular imaging experiments. Here, we evaluated agarose inflation and carboxymethyl cellulose embedding media and determined effective tissue preparation protocols for performing bulk and spatial mass spectrometry-based omics measurements. Mass spectrometry imaging methods were optimized to boost the number of annotatable molecules in agarose inflated lung samples. This optimized protocol permitted the observation of unique lipid distributions within several airway regions in the lung tissue block. Laser capture microdissection of these airway regions followed by high-resolution proteomic analysis allowed us to begin linking the lipidome with the proteome in a spatially resolved manner, where we observed proteins with high abundance specifically localized to the airway regions. We also compared our mass spectrometry results to lung tissue samples preserved using two other inflation/embedding media, but we identified several pitfalls with the sample preparation steps using this preservation method. Overall, we demonstrated the versatility of the inflation method, and we can start to reveal how the metabolome, lipidome, and proteome are connected spatially in human lungs and across disease states through a variety of different experiments.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRS2005

Training Software in Artificial-Intelligence Computing Techniques

The Artificial Intelligence (AI) Toolkit is a computer program for training scientists, engineers, and university students in three soft-computing techniques (fuzzy logic, neural networks, and genetic algorithms) used in artificial-intelligence applications. The program promotes an easily understandable tutorial interface, including an interactive graphical component through which the user can gain hands-on experience in soft-computing techniques applied to realistic example problems. The tutorial provides step-by-step instructions on the workings of soft-computing technology, whereas the hands-on examples allow interaction and reinforcement of the techniques explained throughout the tutorial. In the fuzzy-logic example, a user can interact with a robot and an obstacle course to verify how fuzzy logic is used to command a rover traverse from an arbitrary start to the goal location. For the genetic algorithm example, the problem is to determine the minimum-length path for visiting a user-chosen set of planets in the solar system. For the neural-network example, the problem is to decide, on the basis of input data on physical characteristics, whether a person is a man, woman, or child. The AI Toolkit is compatible with the Windows 95,98, ME, NT 4.0, 2000, and XP operating systems. A computer having a processor speed of at least 300 MHz, and random-access memory of at least 56MB is recommended for optimal performance. The program can be run on a slower computer having less memory, but some functions may not be executed properly.

Howard, Ayanna↗
DOE OSTI2023

A Guideline to Mitigate Interfacial Degradation Processes in Solid‐State Batteries Caused by Cross Diffusion

Abstract Diffusion of transition metals across the cathode–electrolyte interface is identified as a key challenge for the practical realization of solid‐state batteries. This is related to the formation of highly resistive interphases impeding the charge transport across the materials. Herein, the hypothesis that formation of interphases is associated with the incorporation of Co into the Li 7 La 3 Zr 2 O 12 lattice representing the starting point of a cascade of degradation processes is investigated. It is shown that Co incorporates into the garnet structure preferably four‐fold coordinated as Co 2+ or Co 3+ depending on oxygen fugacity. The solubility limit of Co is determined to be around 0.16 per formula unit, whereby concentrations beyond this limit causes a cubic‐to‐tetragonal phase transition. Moreover, the temperature‐dependent Co diffusion coefficient is determined, for example, D 700 °C = 9.46 × 10 −14 cm 2 s −1 and an activation energy E a = 1.65 eV, suggesting that detrimental cross diffusion will take place at any relevant process condition. Additionally, the optimal protective Al 2 O 3 coating thickness for relevant temperatures is studied, which allows to create a process diagram to mitigate any degradation with a minimum compromise on electrochemical performance. This study provides a tool to optimize processing conditions toward developing high energy density solid‐state batteries.

Chemistry↗
DOE OSTI2023

Toward perturbation theory methods on a quantum computer

Perturbation theory, used in a wide range of fields, is a powerful tool for approximate solutions to complex problems, starting from the exact solution of a related, simpler problem. Advances in quantum computing, especially over the past several years, provide opportunities for alternatives to classical methods. Here, we present a general quantum circuit estimating both the energy and eigenstates corrections that is far superior to the classical version when estimating second-order energy corrections. We demonstrate our approach as applied to the two-site extended Hubbard model. In addition to numerical simulations based on qiskit, results on IBM’s quantum hardware are also presented. Our work offers a general approach to studying complex systems with quantum devices, with no training or optimization process needed to obtain the perturbative terms, which can be generalized to other Hamiltonian systems both in chemistry and physics.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2003

Computer Simulation of a Digital Global Velocimeter

The objective is to continue the development of a Monte-Carlo simulation of a Doppler Global Velocimetry (DGV) optical system starting with the Mie scattering from the particles in the fluid to the collection optics, through the beamsplitter, Iodine absorption cell and finally to the CCD sensor. The simulation program will read in a configuration file that completely describes the attributes of a DGV system (optical, physical layout, location of detectors, laser properties, CCD sensor noise characteristics...). The simulation program will output a set of images that simulate the data collected from a DGV system. NASA GRC has the DGV software developed at NASA Langley and this simulation program will prove invaluable for determining the accuracy of the DGV technique and provide insight into optimizing the data reduction techniques. The DGV simulation software will also prove valuable in a DDF funded projected entitled Planar Particle Imaging Doppler Velocimetry (PPIDV), which was recently awarded in March 2002.

Cheung, H. Mike↗
NASA NTRS2008

Quiet Cruise Efficient Short Take-off and Landing Subsonic Transport System

This NASA funded study conceived a revolutionary airplane concept to enable future traffic growth by using regional air space. This requires a very quiet airplane with STOL capability. Starting with a Blended Wing Body that is cruise efficient with inherent low noise characteristics from forward noise shielding and void of aft downward noise reflections, integration of embedded distributed propulsion enables incorporation of the revolutionary concept for jet noise shielding. Embedded distributed propulsion also enables incorporation of a fan bleed internally blown flap for quiet powered lift. The powered lift provides STOL capability for operation at regional airports with rapid take-off and descent to further reduce flyover noise. This study focused on configuring the total engine noise shielding STOL concept with a BWB airplane using the Boeing Phantom Works WingMOD multidisciplinary optimization code to define a planform that is pitch controllable. The configuration was then sized and mission data developed to enable NASA to assess the flyover and sideline noise. The foundational technologies needed are identified including military dual use benefits.

Kawai, Ron↗
NASA NTRS1994

A mathematical model for simulating noise suppression of lined ejectors

A mathematical model containing the essential features embodied in the noise suppression of lined ejectors is presented. Although some simplification of the physics is necessary to render the model mathematically tractable, the current model is the most versatile and technologically advanced at the current time. A system of linearized equations and the boundary conditions governing the sound field are derived starting from the equations of fluid dynamics. A nonreflecting boundary condition is developed. In view of the complex nature of the equations, a parametric study requires the use of numerical techniques and modern computers. A finite element algorithm that solves the differential equations coupled with the boundary condition is then introduced. The numerical method results in a matrix equation with several hundred thousand degrees of freedom that is solved efficiently on a supercomputer. The model is validated by comparing results either with exact solutions or with approximate solutions from other works. In each case, excellent correlations are obtained. The usefulness of the model as an optimization tool and the importance of variable impedance liners as a mechanism for achieving broadband suppression within a lined ejector are demonstrated.

Watson, Willie R.↗
NASA NTRS2005

Thermal Protection Materials Technology for NASA's Exploration Systems Mission Directorate

To fulfill the President s Vision for Space Exploration - successful human and robotic missions between the Earth and other solar system bodies in order to explore their atmospheres and surfaces - NASA must reduce trip time, cost, and vehicle weight so that payload and scientific experiment capabilities are maximized. As a collaboration among NASA Centers, this project will generate products that will enable greater fidelity in mission/vehicle design trade studies, support risk reduction for material selections, assist in optimization of vehicle weights, and provide the material and process templates for development of human-rated qualification and certification Thermal Protection System (TPS) plans. Missions performing aerocapture, aerobraking, or direct aeroentry rely on technologies that reduce vehicle weight by minimizing the need for propellant. These missions use the destination planet s atmosphere to slow the spacecraft. Such mission profiles induce heating environments on the spacecraft that demand thermal protection heatshields. This program offers NASA essential advanced thermal management technologies needed to develop new lightweight nonmetallic TPS materials for critical thermal protection heatshields for future spacecraft. Discussion of this new program (a December 2004 new start) will include both initial progress made and a presentation of the work to be preformed over the four-year life of the program. Additionally, the relevant missions and environments expected for Exploration Systems vehicles will be presented, along with discussion of the candidate materials to be considered and of the types of testing to be performed (material property tests, space environmental effects tests, and Earth and Mars gases arc jet tests).

Valentine, Peter G.↗
NASA NTRSDate not supplied

Towards an Aviation Large Language Model by Fine-tuning and Evaluating Transformers

In the aviation domain, there are many applications for machine learning and artificial intelligence tools that utilize natural language. For example, there is a desire to know the commonalities in written safety reports such as voluntary post incidents reports or aerial wildfire operations reports to better understand the risks present. Another use-case is the possibility of extracting airspace procedures and constraints currently written in documents such as Letters of Agreement. These applications can benefit from the use of state-of-the-art natural language processing techniques when adapted to the language/phraseology specific to the aviation domain. This paper evaluates the viability of adaptation of NLP tools to the aviation domain by fine-tuning transformer based models using aviation data sets. In 2018, a novel language model based on neural units (also called transformers) was created and became known as “Bidirectional Encoder Representations from Transformers” or BERT. This architecture combined with large amounts of English training data and innovative semi-supervised training tasks set the standard for what would later emerge as Large Language Models. The performance of these models was further improved by hyperparameter tuning and refinement of the semi-supervised training task and resulted in “Robustly Optimized BERT Pre-training Approach through hyperparameter tuning” or RoBERTa models. These pre-trained Large Language Models proved to be useful for a wide variety of natural language processing tasks such as text classification and question answering through a process called fine-tuning. The transformer architecture with pre-trained weights served as the basis with the last few layers replaced with layers fine-tuned to perform a new task e.g., a layer that provides a label for the entire input text. This process of fine-tuning can also be used to adapt the models to new domains; e.g., BioBERT started with the pre-trained BERT model and was completed by additional fine-tuning and training on biomedical documents. Transformer-based architectures can also be used to create rich representations of text called embeddings which can serve as the input to other machine learning models. This allows simpler algorithms such as logistic regression to use context-rich representations of the text while still remaining quick to train and evaluate. In the world of aviation, there is a growing demand for natural language processing and understanding but the domain presents unique challenges. Due to the technical content (and specialized language) of most aviation documents, fine-tuning pre-trained Large Language Models to specific tasks has not met the benchmark on natural language processing tasks set by simpler models trained from scratch on the data. To address this deficiency, this paper evaluates the improvements from fine-tuning a Large Language Model on a large set of aviation documents using the original semi-supervised training tasks before performing specific natural language tasks. In fine-tuning, a domain-specific dataset is used on the original training task but with the pre-trained Large Language Model instead of starting from a random initialization. This approach allows the model to be adapted to the specific domain language without discarding the information gained from training on general English data. This paper utilized two major dataset types to train and assess the RoBERTa fine-tuning performance. The first are 7,057 Letters of Agreement which are Federal Aviation Administration (FAA) documents that formalize airspace operations across the national airspace system. They contain many examples of ‘aviation English’ using domain specific terminology and phrasing which serves as a representative basis to perform the semi-supervised fine-tuning. The second type is the 494 document classification labels to be used for evaluation. This down-stream evaluation aims to show the performance of the fine-tuned model, better understand how much data is needed for an effective fine-tuning, and how fine-tuning can be adapted for different applications in-the domain. After semi-supervised training, evaluation begins by encoding the documents for classification using the fine-tuned RoBERTa model. Then a logistic regression classifier is trained to label the document type and compared against our ground truth labels. This currently leads to a 82.8% accuracy on 10-fold cross validation showing improvement over baseline RoBERTa which achieved 81.0%. We plan to measure the improvements on additional tasks and it is expected that these improvements will lead to more robust models that can tackle the natural language processing challenges present in aviation datasets.

ATM↗