Engineering PapersSearch

SEARCH · Engineering Papers

Results for “Retrieval Augmented Generation System”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

A hypertext system that learns from user feedback

Retrieving specific information from large amounts of documentation is not an easy task. It could be facilitated if information relevant in the current problem solving context could be automatically supplied to the user. As a first step towards this goal, we have developed an intelligent hypertext system called CID (Computer Integrated Documentation). Besides providing an hypertext interface for browsing large documents, the CID system automatically acquires and reuses the context in which previous searches were appropriate. This mechanism utilizes on-line user information requirements and relevance feedback either to reinforce current indexing in case of success or to generate new knowledge in case of failure. Thus, the user continually augments and refines the intelligence of the retrieval system. This allows the CID system to provide helpful responses, based on previous usage of the documentation, and to improve its performance over time. We successfully tested the CID system with users of the Space Station Freedom requirements documents. We are currently extending CID to other application domains (Space Shuttle operations documents, airplane maintenance manuals, and on-line training). We are also exploring the potential commercialization of this technique.

Mathe, Nathalie

Requirement Discovery Using Embedded Knowledge Graph with ChatGPT

The field of Advanced Air Mobility (AAM) is witnessing a transformation with innovations such as electric aircraft and increasingly automated airspace operations. Within AAM, the Urban Air Mobility (UAM) concept focuses on providing air-taxi services in densely populated urban areas. This research introduces the utilization of Large Language Models (LLMs), such as OpenAI's GPT-4, to enhance the UAM Requirement discovery process. This study explores two distinct approaches to leverage LLMs in the context of UAM Requirement discovery. The first approach evaluates the LLM's ability to provide responses without relying on additional outside systems, such as a relational or graph database. Instead, a vector store provides relevant information to the LLM based on the user’s question, a process known as Retrieval Augmented Generation (RAG). The second approach integrates the LLM with a graph database. The LLM acts as an intermediary between the user and the graph database, translating user questions into cypher queries for the database and database responses into human-readable answers for the user. Our team implemented and tested both solutions to analyze requirements within a UAM dataset. This paper will talk about our approaches, implementations, and findings related to both approaches.

systems engineering

Requirement Discovery Using Embedded Knowledge Graph With ChatGPT

The field of Advanced Air Mobility (AAM) is witnessing a transformation with innovations such as electric aircraft and increasingly automated airspace operations. Within AAM, the Urban Air Mobility (UAM) con-cept focuses on providing air-taxi services in densely populated urban areas. This research introduces the utilization of Large Language Models (LLMs), such as OpenAI's GPT-4, to enhance the UAM Requirement discovery process. This study explores two distinct approaches to leverage LLMs in the context of UAM Requirement discovery. The first approach evaluates the LLM's ability to provide responses without relying on additional outside systems, such as a relational or graph database. Instead, a vector store provides relevant information to the LLM based on the user’s question, a process known as Retrieval Augmented Generation (RAG). The second approach integrates the LLM with a graph database. The LLM acts as an intermediary between the user and the graph database, translating user questions into cypher queries for the database and database responses into human-readable answers for the user. Our team implemented and tested both solutions to analyze require-ments within a UAM dataset. This paper will talk about our approaches, implementations, and findings related to both approaches.

systems engineering

Requirement Discovery Using Embedded Knowledge Graph With ChatGPT - Poster

The field of Advanced Air Mobility (AAM) is witnessing a transformation with innovations such as electric aircraft and increasingly automated airspace operations. Within AAM, the Urban Air Mobility (UAM) con-cept focuses on providing air-taxi services in densely populated urban areas. This research introduces the utilization of Large Language Models (LLMs), such as OpenAI's GPT-4, to enhance the UAM Requirement discovery process. This study explores two distinct approaches to leverage LLMs in the context of UAM Requirement discovery. The first approach evaluates the LLM's ability to provide responses without relying on additional outside systems, such as a relational or graph database. Instead, a vector store provides relevant information to the LLM based on the user’s question, a process known as Retrieval Augmented Generation (RAG). The second approach integrates the LLM with a graph database. The LLM acts as an intermediary between the user and the graph database, translating user questions into cypher queries for the database and database responses into human-readable answers for the user. Our team implemented and tested both solutions to analyze require-ments within a UAM dataset. This paper will talk about our approaches, implementations, and findings related to both approaches.

systems engineering

Developing Concepts of Operations Using Multi-Step Tool Techniques With Large Language Models

The National Aeronautics and Space Administration (NASA) Air Mobility Pathfinders (AMP) project is developing and evaluating concepts of operations (ConOps) for safe, secure, and scalable Urban Air Mobility (UAM) operations. The AMP project’s Operational Concepts, Architecture, and Requirements Integration (OCARI) Team is using a Model Based System Engineering (MBSE) approach for integration, interoperability, and traceability of Advanced Air Mobility (AAM) ecosystems centered around urban air taxi services. The team’s goal is to define structures and behaviors needed for system feasibility, readiness, and interoperability, establish a UAM knowledge base, and trace and validate assumptions and requirements relevant to AAM. NASA Langley Research Center (LaRC) is spearheading an innovative digital engineering approach to integrate, communicate, and facilitate the research of multi-modal transportation systems. The Knowledge-based Digital Platform (KbDP) is a concept being developed that ties the workflows of Project Managers (PM), Principal Investigators (PI), and System Engineers together across organizational boundaries. It does so through the management of an information database defined by mathematical, data science, and system engineering principles. Machine Learning (ML) algorithms play a key role in this concept by extracting meaningful knowledge from relational and graph databases, document repositories, and system artifacts, which the human user leverages to greatly improve the efficiency and effectiveness of their research. Recent advancements in the field of Large Language Models (LLMs), specifically models trained for tool use, such as Command-R , now allow for the reliable implementation of single-step and multi-step tool-centric systems. These techniques provide the LLM with a set of tools, in our case Python functions, that can be called on to answer a much wider range of questions compared to LLMs implemented using a traditional single-source or Retrieval Augmented Generation (RAG) approach. Through this method, the LLM can pull information from multiple data sources, such as relational or graph databases, document repositories, application programming interfaces (APIs), and SysML artifacts depending on the user’s question. The LLM can also output the information in a variety of different formats, using output generation tools, such as CSV, UML, or SysML artifacts. Additionally, tools can be assigned roles and can work together to provide answers to queries in an “agent” like approach, similar to that implemented by Microsoft’s AutoGen framework where different agents can converse with each other to accomplish tasks. Previously, our team developed a chatbot system with “agent like” functionality in the form of different “modes” the user could select from a user interface (UI), this architecture can be seen on the left in figure 1. Three different modes were implemented, the first mode allowed the LLM to utilize the structures and algorithms within a graph database to trace UAM requirements. The second mode gave the LLM access to a vector search capable of providing relevant information from thousands of document pages related to UAM ConOps and requirements. The third mode served as a general assistant where users could enter open-ended questions and custom prompts to utilize the LLM for different use-cases. This system improved the process surrounding generating and analyzing information related to UAM requirements, however, the implementation provided a clunky user experience. Users were required to know what mode to select within the UI in advance before entering their question to the selected tool. Moreover, the different tools were isolated from each other, they lacked bidirectional links that would allow for tools to collaborate to generate better responses. Our team is working on a new architecture, seen on the right in the below figure, with the goal to address many of the UX shortcomings of our original system while improving the accuracy and depth of responses from the LLM. This new system will automatically select the appropriate tool to use based off the user’s question. Each tool will be capable of calling on any of the other tools available to the LLM, resulting in a collaborative pipeline where tools can pass data between other tools until enough data is received to generate an answer to the user’s question. Using a locally deployed, open-source, LLM, the NASA OCARI team, in collaboration with Collins Aerospace, will implement a prototype application that will bridge knowledge across multiple sources to assist System Engineers (SEs) with requirements discovery and tracing, research question and use case identification, and assumption validation. Such a system will also allow SEs to more easily, and intuitively, explore the AAM ecosystem, ultimately improving the efficiency and effectiveness of the SE's research and decision-making processes surrounding ConOps development and validation. In this session, our team will provide a video demonstration of our new prototype architecture in action. We will also present an overview of our prototype system architecture and talk about its advantages over traditional LLM deployments along with how those advantages can provide additional value to the field of System Engineering.

systems engineering

Management of the orbital environment

Data regarding orbital debris are presented to shed light on the requirements of environmental management in space, and strategies are given for active intervention and operational strategies. Debris are generated by inadvertent explosions of upper stages, intentional military explosions, and collisional breakups. Design and operation practices are set forth for minimizing debris generation and removing useless debris from orbit in the low-earth and geosynchronous orbits. Self-disposal options include propulsive maneuvers, drag-augmentation devices, and tether systems, and the drag devices are described as simple and passive. Active retrieval and disposition are considered, and the difficulty is examined of removing small debris. Active intervention techniques are required since pollution prevention is more effective than remediation for the problems of both earth and space.

Loftus, Joseph P., Jr.

Global Assimilation of Multi-Sensor Snow Observations for Improved Characterization of Snow Processes

Snow conditions on the land surface are recognized to be key components of the global hydrological cycle as they play a critical role in the determination of local and regional climate. In many mid-latitude and high-latitude regions, the seasonal water storage and associated spring snowmelt dominate the local hydrology. The contribution to the runoff and moisture conditions from snow is vital in supporting agriculture and in determining water resources management practices. Consequently, accurate characterization of snow properties becomes important for both end-use applications and weather and climate research. Recently a joint effort between the u.S. Air Force and NASA has enabled a blended, multi-sensor snow product known as the AFWA NASA Snow Algorithm (ANSA). This global snow dataset has been generated by utilizing the Earth Observation System (EOS) Moderate Resolution Imaging Spectroradiometer (MODIS) and Advanced Microwave Scanning Radiometer for EOS (AMSR-E) datasets. ANSA product includes estimates of snow cover extent, snow water equivalent (SWE) and SWE-derived snow depth fields. The MODIS-based products enable snow cover mappings under cloud-free conditions whereas the passive microwave data from AMSR-E provides measurements under cloudy conditions. These remotely-sensed snow observations are further augmented with the information from ground-based snow measurements through data fusion techniques. The resulting ANSA products are employed in the NASA Land Information System (LIS) data assimilation framework, which provides a comprehensive environment for integrating community land surface models, ground and satellite-based observations, and ensemble-based data assimilation tools. LIS incorporates the multisensor ANSA snow retrievals with the land surface model estimates to generate spatially and temporally continuous estimates of snow states, through data assimilation. A suite of experiments to assimilate ANSA snow cover, SWE and snow depth estimates with different land surface models in LIS are conducted and the resulting estimates of snow conditions are evaluated against a number of in-situ observational datasets, over several regions of the world. These evaluations are used to compare and contrast the advantages and disadvantages of these multi-sensor snow observations.

Kumar, Sujay

Real-Time Multimission Event Notification System for Mars Relay

As the Mars Relay Network is in constant flux (missions and teams going through their daily workflow), it is imperative that users are aware of such state changes. For example, a change by an orbiter team can affect operations on a lander team. This software provides an ambient view of the real-time status of the Mars network. The Mars Relay Operations Service (MaROS) comprises a number of tools to coordinate, plan, and visualize various aspects of the Mars Relay Network. As part of MaROS, a feature set was developed that operates on several levels of the software architecture. These levels include a Web-based user interface, a back-end "ReSTlet" built in Java, and databases that store the data as it is received from the network. The result is a real-time event notification and management system, so mission teams can track and act upon events on a moment-by-moment basis. This software retrieves events from MaROS and displays them to the end user. Updates happen in real time, i.e., messages are pushed to the user while logged into the system, and queued when the user is not online for later viewing. The software does not do away with the email notifications, but augments them with in-line notifications. Further, this software expands the events that can generate a notification, and allows user-generated notifications. Existing software sends a smaller subset of mission-generated notifications via email. A common complaint of users was that the system-generated e-mails often "get lost" with other e-mail that comes in. This software allows for an expanded set (including user-generated) of notifications displayed in-line of the program. By separating notifications, this can improve a user's workflow.

Wallick, Michael N.

Generating Land Surface Reflectance for the New Generation of Geostationary Satellite Sensors with the MAIAC Algorithm

The latest generation of geostationary satellite sensors, including the GOES-16/ABI and the Himawari 8/AHI, provide exciting capability to monitor land surface at very high temporal resolutions (5-15 minute intervals) and with spatial and spectral characteristics that mimic the Earth Observing System flagship MODIS. However, geostationary data feature changing sun angles at constant view geometry, which is almost reciprocal to sun-synchronous observations. Such a challenge needs to be carefully addressed before one can exploit the full potential of the new sources of data. Here we take on this challenge with Multi-Angle Implementation of Atmospheric Correction (MAIAC) algorithm, recently developed for accurate and globally robust applications like the MODIS Collection 6 re-processing. MAIAC first grids the top-of- atmosphere measurements to a fixed grid so that the spectral and physical signatures of each grid cell are stacked (“remembered”) over time and used to dramatically improve cloud/shadow/snow detection, which is by far the dominant error source in the remote sensing. It also exploits the changing sun-view geometry of the geostationary sensor to characterize surface BRDF with augmented angular resolution for accurate aerosol retrievals and atmospheric correction. The high temporal resolutions of the geostationary data indeed make the BRDF retrieval much simpler and more robust as compared with sun-synchronous sensors such as MODIS. As a prototype test for the geostationary-data processing pipeline on NASA Earth Exchange (GEONEX), we apply MAIAC to process 18 months of data from Himawari 8/AHI over Australia. We generate a suite of test results, including the input TOA reflectance and the output cloud mask, aerosol optical depth (AOD), and the atmospherically-corrected surface reflectance for a variety of geographic locations, terrain, and land cover types. Comparison with MODIS data indicates a general agreement between the retrieved surface reflectance products. Furthermore, the geostationary results satisfactorily capture the movement of clouds and variations in atmospheric dust/aerosol concentrations, suggesting that high quality land surface and vegetation datasets from the advanced geostationary sensors can help complement and improve the corresponding EOS products.

geostationary satellite sensors

NASA Pilot-Engaged Expert Response Using IBM Watson Technology: Prototype Evaluation of Knowledge Retrieval System

NASA Langley Research Center and IBM have been investigating the use of IBM Watson technology in aerospace research and development. One application of Watson technology is the Pilot-Engaged Expert Response (PEER) use case. The PEER system is envisioned as an in-cockpit advisor that will act as a source of situationally-relevant information for pilots and other flight crew members to assist in decision making about real-time events and situations that arise in the course of aircraft operations. PEER will make available vast stores of knowledge and information quickly and directly, putting important informational resources where they are needed most. IBM has worked with NASA to develop an architecture and articulate a roadmap for the development of the PEER system. That vision is built around Watson Discovery Advisor (WDA) software solution, derived from IBM's Jeopardy!-winning automatic question answering system. PEER makes use of WDA's sophisticated question-answering capabilities as its core, adding important User Interface components and other customizations for the cockpit environment, including communication with flight systems and other external data sources. The development plan for PEER includes four development stages, with the current project constituting the first phase. In this project, a prototype instance of PEER was successfully adapted to the aviation domain, enabling users to ask questions about aviation topics and receive useful and accurate answers to these questions. Major tasks accomplished include the development of procedures for domain adaptation through automatic lexicon extraction from domain glossaries; generation of question-answer training data which was used to train the system; and assessment of the effectiveness of domain adaptation, which showed a dramatic improvement in the ability of the PEER system to answer domain-relevant questions. In addition, the vision for the PEER system was pushed forward by the articulation of a plan for the automatic enhancement of question-answering with contextual information. This initial phase focused on two main goals: 1) the targeted domain adaptation of the underlying WDA system to the aviation domain; and, 2) the design of the software systems needed to leverage flight-contextual data. Domain adaptation of the WDA system proceeds via three main activities: Domain data ingestion, lexical customization and model training. A textual corpus consisting of 1,147 individual documents with more than 7.5 million words of text was ingested into the system and this served as the basis of all further development. A domain lexicon of over 3,500 aviation-domain terms was semi-automatically generated from domain documents and used to train the system. In addition, a set of over 500 question-answer (QA) pairs relevant to the PEER use case was developed; these were used to train and assess the system. These important first steps established the basis for the PEER system. In addition, steps were taken towards the integration of the PEER system into the cockpit environment with the development of a functional design for the Contextual Data Augmentation (CDA) subsystem. This subsystem brings to bear contextual data to improve system responses. It has three main submodules: the Contextual Data Collection module, the Contextual Data Selection module, and the Contextual QA Augmentation module. These modules form a processing pipeline that addresses the problems associated with automatically integrating information from external resources into the knowledge-retrieval mechanism.

Machine learning

Biogeochemical Response to Mesoscale Physical Forcing in the California Current System

In the first part of the project, we investigated the local response of the coastal ocean ecosystems (changes in chlorophyll, concentration and chlorophyll, fluorescence quantum yield) to physical forcing by developing and deploying Autonomous Drifting Ocean Stations (ADOS) within several mesoscale features along the U.S. west coast. Also, we compared the temporal and spatial variability registered by sensors mounted in the drifters to that registered by the sensors mounted in the satellites in order to assess the scales of variability that are not resolved by the ocean color satellite. The second part of the project used the existing WOCE SVP Surface Lagrangian drifters to track individual water parcels through time. The individual drifter tracks were used to generate multivariate time series by interpolating/extracting the biological and physical data fields retrieved by remote sensors (ocean color, SST, wind speed and direction, wind stress curl, and sea level topography). The individual time series of the physical data (AVHRR, TOPEX, NCEP) were analyzed against the ocean color (SeaWiFS) time-series to determine the time scale of biological response to the physical forcing. The results from this part of the research is being used to compare the decorrelation scales of chlorophyll from a Lagrangian and Eulerian framework. The results from both parts of this research augmented the necessary time series data needed to investigate the interactions between the ocean mesoscale features, wind, and the biogeochemical processes. Using the historical Lagrangian data sets, we have completed a comparison of the decorrelation scales in both the Eulerian and Lagrangian reference frame for the SeaWiFS data set. We are continuing to investigate how these results might be used in objective mapping efforts.

Niiler, Pearn P.

GEONEX: Land Monitoring From a New Generation of Geostationary Satellite Sensors

The latest generation of geostationary satellites carry sensors such as ABI (Advanced Baseline Imager on GOES-16) and the AHI (Advanced Himawari Imager on Himawari) that closely mimic the spatial and spectral characteristics of Earth Observing System flagship MODIS for monitoring land surface conditions. More importantly they provide observations at 5-15 minute intervals. Such high frequency data offer exciting possibilities for producing robust estimates of land surface conditions by overcoming cloud cover, enabling studies of diurnally varying local-to-regional biosphere-atmosphere interactions, and operational decision-making in agriculture, forestry and disaster management. But the data come with challenges that need special attention. For instance, geostationary data feature changing sun angle at constant view for each pixel, which is reciprocal to sun-synchronous observations, and thus require careful adaptation of EOS algorithms. Our goal is to produce a set of land surface products from geostationary sensors by leveraging NASA's investments in EOS algorithms and in the data/compute facility NEX. The land surface variables of interest include atmospherically corrected surface reflectances, snow cover, vegetation indices and leaf area index (LAI)/fraction of photosynthetically absorbed radiation (FPAR), as well as land surface temperature and fires. In order to get ready to produce operational products over the US from GOES-16 starting 2018, we have utilized 18 months of data from Himawari AHI over Australia to test the production pipeline and the performance of various algorithms for our initial tests. The end-to-end processing pipeline consists of a suite of modules to (a) perform calibration and automatic georeference correction of the AHI L1b data, (b) adopt the Multi-Angle Implementation of Atmospheric Correction (MAIAC) algorithm to produce surface spectral reflectances along with compositing schemes and QA, and (c) modify relevant EOS retrieval algorithms (e.g., LAI and FPAR, GPP, etc.) for subsequent science product generation. Initial evaluation of Himawari AHI products against standard MODIS products indicate general agreement, suggesting that data from geostationary sensors can augment low earth orbit (LEO) satellite observations.

geostationary

Understanding and Utilizing PBL Height Data from Multiple Observing Systems in the GEOS System

The accuracy of PBL height simulation is a key issue in many applications including forecasting near surface meteorology and air quality, however, it is a very challenging problem due to the lack of not only comprehensive, global Planetary Boundary Layer (PBL) observations but also a strategy and infrastructure to utilize PBL height data from a variety of sensors. Following the designation of PBL as an incubation class observable in the 2017 Decadal Survey, the PBL Incubation Study Team Report [14] made clear that “a future global PBL observing system requires modeling and data assimilation as essential components.” There is an urgent need for global modeling development in order to utilize Program of Record (POR) observations, assess their impacts, and identify gaps to be filled by future PBL missions. Our overall objective is to develop PBL data assimilation capabilities in the NASA Global Earth Observing System (GEOS), focusing on PBL height from multiple observing systems, to support the assessment and use of future PBL observations. The NASA GEOS system is composed of the GEOS global atmospheric general circulation model (AGCM) and the atmospheric data assimilation system (ADAS). The PBL parameterizations include the “Lock” K-profile scheme driven by surface and cloud-top buoyancy fluxes ([4]), and the “Louis” local scheme for stable conditions based on the Richardson number ([5]). Above the mixed layer defined by the Lock surface plume, shallow cumulus convection is represented by the mass flux scheme of [9]. Additional parameterizations are summarized in [1]. The ADAS employs the hybrid 4D Ensemble- Variational (EnVar) configuration ([15]), with the ensemble providing flow-dependent background error covariance information. The resultant analysis increments are fed back to the forecast model through the 4D incremental analysis update (IAU) approach ([11]). In this study, PBL height data are being or have been generated from radiosondes, GNSS RO, satellite (CATS, CALIPSO and ICESat-2) and ground-based (MPLNET) lidars, and wind profiler. Investigations have been conducted to specify quality marks for PBL height retrievals for the data assimilation purpose. These PBL height data have different strengths and weaknesses ([2], [3], [6], [7], [8], [10]), and the satellite PBL height data provide better global coverage and complement in-situ PBL height data. Radiosondes offer high accuracy and in situ measurement of temperature and humidity profiles, but with poor spatio-temporal sampling. The in-situ observing systems like MPLNET and wind profiler provide long history of PBL height records at each station. The GNSS RO based PBL height is retrieved based on the sharp gradients in refractivity profile that represent the fine vertical structure of temperature and moisture changes above the PBL. However, not all RO refractivity profiles reach the surface depending on location and regime, and RO refractivity retrievals can be negatively biased below 2km. The PBL height data from satellite lidars provide high resolution along track PBL height retrievals, but over land they are affected by previous day convective PBL aerosol and strongly associated with mixing layer and retrievals cannot be made below thick, attenuating clouds. A successful assimilation of PBL height data requires a thorough understanding of the observing method and the retrieval algorithm for each observing system in order to use the PBL height data from multiple observing systems properly. Due to the sensitivity of PBL height data to the observing method and choice of algorithm, it is important to use a model definition appropriate for each observation type to compute differences between PBL height data and model PBL height (OmFs). The GEOS model currently includes two PBL height definitions suitable for direct comparison with observed PBL height, and additional definitions are being added in this study. Evaluation of different model PBL height definitions is underway. Meanwhile, efforts have been made in the GEOS data assimilation system to develop PBL height data assimilation capability. PBL height data can be assimilated using two different approaches. The traditional approach is to construct an observation operator and its tangent linear and adjoint, which link control variables to PBL height data from each observing system. This observation operator can be very complicated, e.g., the lidar-based PBL height observation operator includes the backscatter lidar forward observation operator, the algorithm to derive PBL height from attenuated total backscatter, interpolation, and calculations handling the mismatch between observed and model scales. The other approach is to augment PBL height to the control variable vector, and it is adopted in this study. The latter approach was also used in previous studies, e.g., the assimilation of PBL height data from radiosonde and aircraft in the Real Time Mesoscale Analysis (RTMA) system for a dispersion modelling study ([13]); the PBL height assimilation study using lidar PBL height data at Greensburg, Kansas for a field campaign ([12]). The PBL height assimilation from multiple observing systems in this study allows us to take advantage of the diverse PBL height data that provide much better global coverage collectively under different meteorological conditions and with different temporal and spatial scales. As all the PBL heights are tightly coupled with the PBL thermodynamic variables, the strong correlations, which are provided by the 4D ensemble forecast, enable PBL height data from various sources to interact and combine coherently and provide additional information for PBL temperature and moisture fields. The results of comparisons among PBL height data from different sources and the evaluation of the model PBL height definitions with the PBL height data will be presented, and the PBL height data synergy strategies and preliminary results will also be discussed at the conference.

Y. Zhu