Results for “natural language processing (NLP)” — Page 2

NASA NTRSDate not supplied

Towards an Aviation Large Language Model by Fine-tuning and Evaluating Transformers

In the aviation domain, there are many applications for machine learning and artificial intelligence tools that utilize natural language. For example, there is a desire to know the commonalities in written safety reports such as voluntary post incidents reports or create more accurate transcripts of air traffic management conversations. Another use-case is the possibility of extracting airspace procedures and constraints currently written in documents such as Letters of Agreement (LOA) which is used as the evaluation case in this paper. These applications can benefit from the use of state-of-the-art Natural Language Processing (NLP) techniques when adapted to the language/phraseology specific to the aviation domain. This paper evaluates the viability of transferring pre-trained large language models to the aviation domain by adapting transformer based models using aviation datasets. This paper utilized two datasets to adapt a ‘Robustly Optimized Bidirectional Encoder Representations from Transformers Approach’ (RoBERTa) model and two down-stream classification tasks to assess its performance. These datasets are all built upon Letters of Agreement which are Federal Aviation Administration (FAA) documents that formalize airspace operations across the national airspace system. The first two datasets are used for the adaptation of RoBERTa to the aviation domain and were of different sizes to assess the number of documents needed to adapt to the aviation domain. They contain many examples of ‘aviation English’ using domain specific terminology and phrasing which serves as a representative basis to perform the unsupervised adaptation. The second dataset is a separate set of LOA documents with two sets of classification labels to be used for evaluation; one at the document level and one at the line level. These down-stream evaluations allowed the measurement of improvement by adapting RoBERTa. The accuracy increased by 4-6% on both tasks and the F1 score on the class of interest increased by 4-8% from the adaptation.

Air Traffic Management↗

NASA NTRSDate not supplied

Transforming Science Prioritization Processes Using Artificial Intelligence

Artificial Intelligence (AI) and Machine Learning (ML) have potential to augment significantly the current labor-intensive processes of science prioritization, specifically by the National Academies’ Decadal Survey on behalf of NASA and NSF. Here we summarize what we believe to be the first exploratory demonstration-of-concept results from an application of AI/ML to Survey science prioritization. Specifically, we applied Latent Dirichlet Allocation (LDA) and Natural Language Processing (NLP) to reveal trends in published astrophysics research that may indicate science priorities and which could be applied to strategic planning. For the purpose of the work that we summarize here, AI/ML is able to analyze – that is, to “understand,” in a manner of speaking – a vast amount of text to reveal complex relationships among research topics, including the growth or decline of science community activities in those topics over time. We trained ourselves and AI/ML algorithms by using ~400,000 abstracts in the period 1998 to 2010 to “forecast” the Academies’ Astro2010 recommendations and compare with the solicited white papers. Comparing our results with actual Astro2010 recommendations allowed us to identify candidate metrics that better predicted the actual results of the Survey. We found, for example, that Compound Annual Growth Rate (CAGR) of papers published in a topic area is a good proxy measure for importance of this topic area of research. With this training complete, we identified candidate astrophysics astrophysics science priorities for the 2021+ period using the research during 2007 - 2019 . We conclude that appropriate application of AI can potentially significantly reduce the current workload of the Decadal Survey processes and reveal otherwise unrecognized characteristics in the body of astronomical research. We emphasize throughout the exploratory nature of our work, encouraging colleagues to pursue promising results further. Our most critical governing assumption was that increased (or decreased) research activity can be used to identify scientific or technology topic areas worthy of increased (or decreased) future emphasis. We discuss advantages, limitations, and recognize the “black box” nature of our technique. We note ethics issues associated, for example, with using AI/ML to reveal “hidden” meanings and biases in published work. Furthermore, inevitable improvements in AI may soon enable widespread and welcome identification of and advocacy for science and technology priorities by disparate and diverse groups and organizations. Consequently, we continue to urge a near-term, in-depth evaluation of appropriate applications of AI, including implications and consequences, as well as support for multiple follow-on assessments, of which ours is only a beginning.

Artificial Intelligence↗

NASA NTRS2019

Reusing Data and Metadata to Create New Metadata Through Machine-Learning & Other Programmatic Methods

Recent improvements in natural language processing (NLP) enable metadata to be created programmatically from reused original metadata or even the dataset itself. Transfer-learning applied to NLP has greatly improved performance and reduced training data requirements. In this talk, we’ll compare machine-generated metadata to human-generated metadata and discuss characteristics of metadata and data archives that affect suitability for machine-learning reuse of metadata. Where as human-generated metadata is often populated once, populated from the perspective of data supplier, populated by many individuals with different words for the same thing, and limited in length, machine-generated metadata can be updated any number of times, generated from the perspective of any user, constrained to a standardized set of terms that can be evolved over time, and be any length required. Machine-learning generated metadata offers benefits but also additional needs in terms of version control, process transparency, human-computer interaction, and IT requirements. As a successful example, we’ll discuss how a dataset of abstracts and associated human-tagged keywords from a standardized list of several thousand keywords were used to create a machine-learning model that predicted keyword metadata for open-source code projects on code.nasa.gov. We’ll also discuss a less successful example from data.nasa.gov to show how data archive architecture and characteristics of initial metadata can be strong controls on how easy it is to leverage programmatic methods to reuse metadata to create additional metadata.

Gosses, Justin↗

NASA NTRSDate not supplied

A Hybrid Approach to Labeling Datasets in Earth Science Publications

NASA Data Centers provide the public with thousands of datasets that result in published papers, reports, and conference proceedings. Collecting accurate metrics on usage of these datasets is key to connecting different areas of knowledge and evaluating the datasets’ impact. While most of the datasets have Digital Object Identifiers (DOIs) assigned, most publications do not cite them hampering the automated search of these publications. Instead, articles mention attributes like organization, instrument, mission, variable, or a publication describing the dataset. Often only domain experts can deduce the dataset that was used in the publication text. The lack of a citation slows the spread of information and reduces the research’s impact. With thousands of papers produced each year, an automated means of labeling datasets is critical. This paper explores a hybrid approach of heuristics and a Natural Language Processing (NLP) Named Entity Recognition (NER) model to find and label the datasets used within Earth Science papers. Heuristics are used to produce the labelled sentences and any potential dataset candidates that can be derived from a sentence. The heuristic labels the sentences with the names of mission, instrument, re-analysis models, and science keywords taken from the Global Change Master Directory (GCMD) ontology. Additionally, it uses those labels to generate the dataset citation candidates. If the mission, instrument, and variable are sufficient to create the citation for the dataset the citation and the label the domain expert reviews the output without going through the NLP model. If the extracted label is not sufficient to label the dataset on its own, the sentence and its associated dataset labels will be inputted into the NER model. The model outputs the labeled sentence and the potential dataset candidates with their associated probabilities. The domain expert then reviews the NER model’s output and the correct labels are determined. The newly labelled papers can then be used as additional training data. This creates an iterative process for the approach to continuously improve. Because all the possible mentions are gathered by the model, the domain expert can quickly and easily label the papers resulting in large time savings.

Jacob Atkins↗

NASA NTRSDate not supplied

Knowledge Discovery for Early Failure Assessment of Complex Engineered Systems Using Natural Language Processing

Emerging complex engineered systems may have unexpected safety issues due to novel operational environments, increasing autonomy, human-machine interaction, and other factors. To prevent failures in operation or testing that necessitate costly redesign, it is desirable to predict likely failure modes early in the design process. Information about past engineering failures in natural language format presents one possible solution by enabling the retrieval of information that can inform new designs. However, identifying documents containing usable information and extracting the required information can be prohibitively time-consuming when implemented at scale. In this research, an automated natural language processing (NLP) framework is proposed to discover relevant knowledge from documents containing failure-related design information. The framework is applied to NASA’s Lessons Learned Information System (LLIS),which is publicly available. Documents containing usable information are filtered using two different NLP-based models. Next, from the identified usable documents, a failure taxonomy is extracted using a partitioned hierarchical topic modeling approach. Partitions of the document describe different sections of the failure taxonomy – i.e., failure, cause of failure, and recommendations – as indicated by the structure of the original document. The extracted failure taxonomy can be leveraged in early design failure assessment methods. Moreover, the framework can be used to identify documents containing usable failure-related design information from other databases and extract relevant information from these documents.

Documentation and Information Science↗

NASA NTRSDate not supplied

Knowledge Discovery for Early Failure Assessment of Complex Engineered Systems Using Natural Language Processing

Emerging complex engineered systems may have unexpected safety issues due to novel operational environments, increasing autonomy, human-machine interaction, and other factors. To prevent failures in operation or testing that necessitate costly redesign, it is desirable to predict likely failure modes early in the design process. Information about past engineering failures in natural language format presents one possible solution by enabling the retrieval of information that can inform new designs. However, identifying documents containing usable information and extracting the required information can be prohibitively time-consuming when implemented at scale. In this research, an automated natural language processing (NLP) framework is proposed to discover relevant knowledge from documents containing failure-related design information. The framework is applied to NASA’s Lessons Learned Information System (LLIS),which is publicly available. Documents containing usable information are filtered using two different NLP-based models. Next, from the identified usable documents, a failure taxonomy is extracted using a partitioned hierarchical topic modeling approach. Partitions of the document describe different sections of the failure taxonomy – i.e., failure, cause of failure, and recommendations – as indicated by the structure of the original document. The extracted failure taxonomy can be leveraged in early design failure assessment methods. Moreover, the framework can be used to identify documents containing usable failure-related design information from other databases and extract relevant information from these documents.

Documentation and Information Science↗

NASA NTRSDate not supplied

Search Enhancements using Natural Language Processing Techniques

NASA Goddard Earth Sciences Data and Information Services Center (GESDISC) is one of the 12 NASA Science Mission Directorate Data Centers. The main goal of GESDISC is to provide earth science data, information, and services to the earth science data community. Consequently, data discovery is at the center of our mission and our search engine is the primary tool for our users to interact, find, and access our data. Existing search approaches are largely focused on hard-matching of keywords in the search query with dataset metadata. Here we propose to expand the search by introducing a complementary natural language processing (NLP) search. At the heart of our proposed NLP search, we trained a joint embedding using scientific text corpus and a curated set of dataset metadata. The embedding learns the association between words in our dataset metadata and those of the scientific text corpus. This enables us to go beyond simple hard-matching of a query and data set metadata and have a notion of “similarity” between the search query and the datasets. We further integrated our NLP search into the Elastic Search (ES) framework leveraging similarity search capabilities offered through the “dense_vector” field type. Our preliminary evaluations show that our proposed NLP search has the potential to be utilized to complement the existing search engine and serve as a base for a dataset recommendation system.

Armin Mehrabian↗

NASA NTRSDate not supplied

A Brief Introduction to AI/ML Applications of Air Traffic Management Data at NASA Ames

This presentation will give a brief overview of several AI/ML This presentation will give a brief overview of several AI/ML projects that NASA Ames interns are exploring in partnership with NASA Aeronautic Research Institute (NARI) and the FAA. NASA is interested in Natural Language Processing (NLP) of various legacy text and speech data within air traffic management e.g., Notices To Airmen (NOTAMs), Letters of Agreement (LoAs), Standard Operating Procedures (SOPs), and Air Traffic Control Center audio briefings. Since our focus is on applying state of the art AI/ML tools to legacy air traffic management data, we first showcase the different data sources of interest followed by a brief introduction to the techniques and language models used. We present some exciting preliminary results on each topic including both unsupervised learning techniques (e.g., clustering) and other modern language models (e.g., BERT) that help extract useful information from these data sources that are interpretable by both man and machine.

Air Traffic Management↗

NASA NTRSDate not supplied

Wildfire Emergency Response Hazard Extraction and Analysis of Trends (HEAT) through Natural Language Processing and Time Series

Emerging wildfire operations aim to improve safety and performance through the integration of technologies including UAS and UTM. Recent advances in natural language processing (NLP) techniques, as well as the availability of wildfire incident reports, has made possible a large-scale analysis of wildfire hazards and trends. Identifying longitudinal trends will help us target risk mitigation and safety management activities. Note: This presentation does not include sound please disregard icon.

Sequoia R. Andrade↗

NASA NTRSDate not supplied

Verb Sense Disambiguation for Densifying Knowledge Graphs in Earth Science

We begin with an ambitious goal: to create a knowledge graph that spans the entire discipline of Earth science. In order to achieve this, we need to apply Natural Language Processing (NLP) techniques on Earth science journal articles to extract their semantic components for the graph. When sentences from Earth science journal articles are broken down into their semantic components and loaded onto a graph, the relationships among these semantic components are represented by the verbs in the sentences. However, since there are multiple verbs in English that can be used to denote the same meaning, the knowledge graph can become sparse and so can the results when we query the graph. In order to ensure quality results, it would be desirable to consolidate similar verbs into a single "class". So, this is the problem at hand: how do we make sure that multiple verbs that mean the same thing are represented as a single class of verb in the knowledge graph? Or in other words, how do we distinguish which meaning a particular verb takes given a particular sentence? In this poster, we demonstrate a potential technique to solve this problem.

Ashish Acharya↗

NASA NTRSDate not supplied

Integrating Human System Information with the Systems Platform for Aggregating and Relating Capabilities (SPARC)

Within Human Health and Performance, there exists a wealth of human system information that’s used a regular basis in support of NASA human exploration objectives, but the challenge is that all of this information was stored in multiple different locations and organized for specific uses, limiting its effectiveness and straining communication across multiple groups. To address this challenge, our project, the Systems Platform for Aggregating and Relating Capabilities (SPARC) was tasked with developing a new NASA internal web application that aggregates and relates multiple programs’ human system products, such as technical standards, program requirements and verifications, human system risks, research and evidence, and exploration capabilities, into one centralized platform that addresses the needs of human health and performance from multiple different perspectives. Using agile development methodologies, user-experience (UX) driven design principles, data visualization, and a strong emphasis on continuous improvement though consistent stakeholder engagement, the SPARC project released a beta version in less than 4 months, broadly launched version 1.0.0 Agency-wide three months after the beta, and has over 180 users in the first year of development. Our second year of development will see us moving from our initial capabilities to increasingly robust and complex integrations and visualizations, including Directed Acyclical Graphs (DAGs), natural language processing (NLP) for dynamic generation of relationships between the sources of truth, and an expansion into hierarchical levels of system design in support of the human exploration programs.

Data science↗

NASA NTRSDate not supplied

Transcribing Air Traffic Control System Command Center Planning Telecons Using Cloud-Based Automatic Speech Recognition

This paper addresses the challenge of using Automatic Speech Recognition (ASR) technology to transcribe regular teleconferences that happen between FAA Air Traffic Control System Command Center (ATCSCC) planners, stakeholders and air users. These planning teleconferences (aka telecons or planning webinars) are an integral part of managing air traffic in the U.S. National Airspace System (NAS). In particular, the meetings facilitate the creation and modification of various traffic management initiatives (TMIs), that are used to regulate the flow of air traffic. This is typically a human intensive process, requiring specialists to listen to the entire meeting audio (10-20 minutes duration) and inferring the state of the NAS (e.g., weather phenomenon) that was discussed. It would be advantageous to have digital transcripts of the audio and have useful information (e.g., related to TMIs) automatically extracted from the transcripts. In this regard, we are exploring the adoption of state-of-the-art speech to text and Natural Language Processing (NLP) tools that will achieve our objective of digitizing the webinar audio. Unfortunately, the highly technical phraseology present in the audio and limited data availability for model building make ASR difficult. To overcome this challenge, we have taken the critical first step in creating a human transcription dataset from ~20 hours of speech in the ATCSCC audio with the help of subject matter experts. A novelty of our work is the creation of a ground truth transcription dataset for ATCSCC teleconference webinars, which is particularly important for Aviation domain-specific NLP tasks. Using Microsoft Speech Studio, a cloud-based ASR platform, we have fine-tuned the English pre-trained ASR models (available in speech studio) and achieved an average word error rate (WER) of 6.81%. The baseline ASR also provides a digital version of each planning webinar, making it accessible and text-searchable for future references. Additionally, the transcriptions can serve as a bridge between raw audio data and a range of text-based NLP tasks, such as named entity recognition (NER) and intent classification, potentially enhancing the digital footprint of the webinars and other connected data sources. Our work has several potential applications. Firstly, the transcriptions can be analyzed to understand the complex decision process of creating, implementing and modifying TMIs and may also contribute to TMI prediction services. Secondly, our dataset and model can be used to develop more accurate ASR systems for aviation-specific language, which can bring about digital communication in the aviation industry (and aid current “voice only” communications, which are inherently error-prone). Lastly, the transcriptions themselves can be used as a valuable resource for training other NLP models.

Stephen S. B. Clarke↗

NASA NTRSDate not supplied

Classification of Notices to Airmen using Natural Language Processing

This paper establishes the feasibility of using Natural Language Processing (NLP) to classify NOTAMs or Notices to Airmen – a pilot messaging framework to gather real-time situational awareness. Present day air mobility operations heavily rely on NOTAMs. However, pilots often have difficulty interpreting NOTAMs due to the sheer volume of inapplicable messages and unclear abbreviations. Using NLP, the presented study analyzes the accuracy of classifying NOTAMs and, thereby, the efficiency of generating actionable interpretations in real time. To this effect, efficacies of four NLP neural network architectures were analyzed, including three Recurrent Neural Networks (RNNs) with GloVe, Word2Vec, and FastText word embeddings, and one trained Bi-Directional Encoder Representations from Transformers (BERT) model. The four neural networks were trained and evaluated on three open-source datasets of varying text lengths, vocabularies, and grammars, taken from e-commerce product descriptions, social media tweets, and unstructured descriptions for data and analytics services on open data marketplaces such as NASA’s Data and Reasoning Fabric (DRF) platform. This provided cross-analysis of each neural network architecture’s performance per text type. The best performing architecture, BERT, was then fine-tuned on a collection of open-source NOTAM data. Post-training, a real-time NOTAM classification service was implemented to draw inference on new NOTAMs using the trained model, which demonstrated close to 99% accuracy in classification. This modular classification service is envisioned to be integrated with a data and analytics delivery platform, such as the DRF, thus availing real-time contextualization of NOTAMs to air mobility clients, humans, and machines for enhanced decision making.

Aiden C. Szeto↗

NASA NTRSDate not supplied

Classification of Notices to Airmen using Natural Language Processing

This paper establishes the feasibility of using Natural Language Processing (NLP) to classify NOTAMs or Notices to Airmen – a pilot messaging framework to gather real-time situational awareness. Present day air mobility operations heavily rely on NOTAMs. However, pilots often have difficulty interpreting NOTAMs due to the sheer volume of inapplicable messages and unclear abbreviations. Using NLP, the presented study analyzes the accuracy of classifying NOTAMs and, thereby, the efficiency of generating actionable interpretations in real time. To this effect, efficacies of four NLP neural network architectures were analyzed, including three Recurrent Neural Networks (RNNs) with GloVe, Word2Vec, and FastText word embeddings, and one trained Bi-Directional Encoder Representations from Transformers (BERT) model. The four neural networks were trained and evaluated on three open-source datasets of varying text lengths, vocabularies, and grammars, taken from e-commerce product descriptions, social media tweets, and unstructured descriptions for data and analytics services on open data marketplaces such as NASA’s Data and Reasoning Fabric (DRF) platform. This provided cross-analysis of each neural network architecture’s performance per text type. The best performing architecture, BERT, was then fine-tuned on a collection of open-source NOTAM data. Post-training, a real-time NOTAM classification service was implemented to draw inference on new NOTAMs using the trained model, which demonstrated close to 99% accuracy in classification. This modular classification service is envisioned to be integrated with a data and analytics delivery platform, such as the DRF, thus availing real-time contextualization of NOTAMs to air mobility clients, humans, and machines for enhanced decision making.

Aiden Szeto↗

NASA NTRSDate not supplied

The Development and Deployment of Machine Learning Models for Aircraft Engine Concept Assessment

In today's competitive landscape, the effective development and utilization of machine-learning (ML) applications have become imperative across various sectors. This study presents an outline of the procedure involved in creating and implementing ML models for conceptualizing and evaluating aircraft engines. These models leverage supervised deep-learning algorithms to analyze patterns within an open-source repository containing data on both production and research conventional turbofan engines. The main areas of focus encompass crucial engine parameters like thrust-specific fuel consumption (TSFC), engine weight, engine diameter, and turbomachinery stage counts. While the creation of ML models is fundamental for their utilization, ensuring their seamless deployment holds equal significance. To address this aspect, a conversational AI chatbot is constructed, utilizing natural language processing (NLP) techniques, to facilitate the deployment of these ML models. The comprehensive workflow encompasses several key stages: gathering and enhancing engine data, training and cross validating the ML models, testing and evaluating their performance, and finally, deploying, monitoring, and updating the ML models. By following this systematic approach, the aim is to streamline the development and deployment process of ML models tailored for aircraft engine assessment.

Development↗

NASA NTRSDate not supplied

The Development and Deployment of Machine Learning Models for Aircraft Engine Concept Assessment

In today's competitive landscape, the effective development and utilization of machine-learning (ML) applications have become imperative across various sectors. This study presents an outline of the procedure involved in creating and implementing ML models for conceptualizing and evaluating aircraft engines. These models leverage supervised deep-learning algorithms to analyze patterns within an open-source repository containing data on both production and research conventional turbofan engines. The main areas of focus encompass crucial engine parameters like thrust-specific fuel consumption (TSFC), engine weight, engine diameter, and turbomachinery stage counts. While the creation of ML models is fundamental for their utilization, ensuring their seamless deployment holds equal significance. To address this aspect, a conversational AI chatbot is constructed, utilizing natural language processing (NLP) techniques, to facilitate the deployment of these ML models. The comprehensive workflow encompasses several key stages: gathering and enhancing engine data, training and cross validating the ML models, testing and evaluating their performance, and finally, deploying, monitoring, and updating the ML models. By following this systematic approach, the aim is to streamline the development and deployment process of ML models tailored for aircraft engine assessment.

Development↗

NASA NTRSDate not supplied

Machine Learning Application in Aircraft Engine Conceptual Design

In the current competitive environment, the successful creation and application of machine learning (ML) technologies have become crucial across multiple industries. This study outlines the process of creating and implementing ML models for conceptualizing and evaluating aircraft engines. These models use supervised deep-learning algorithms to analyze patterns within an open-source repository containing data on both production and research conventional turbofan engines. Key focus areas include crucial engine parameters such as thrust-specific fuel consumption (TSFC), engine weight, engine diameter, and turbomachinery stage counts. While developing ML models is fundamental, ensuring their seamless deployment is equally important. To address this, a conversational AI chatbot is constructed using natural language processing (NLP) techniques to facilitate the deployment of these ML models. The comprehensive workflow includes several key stages: gathering and enhancing engine data, training and cross validating the ML models, testing and evaluating their performance, and finally, deploying, monitoring, and updating the ML models. By following this systematic approach, the aim is to streamline the development and deployment process of ML models tailored for aircraft engine conceptual design.

Aircraft Engine↗

NASA NTRS2025

Navigating Team Dynamics: Automated Detection of Micro-Behaviors Between Team Members Through Longitudinal Interaction Data

The success in future long term space exploration missions will depend on the cooperation, coordination, and mutual understanding among the crew members. Micro-behaviors are momentary, subtle linguistic and paralinguistic indicators of thinking and feeling toward another member of the team (Cortina et al., 2001; Smith & Griffiths, 2022) that can significantly impact team dynamics and influence the overall team performance (Paromita & Chaspari, 2024). Due to their interactive nature, micro-behaviors have a sender (i.e., the team member expressing the micro-behavior) and a target (the team member impacted by the micro-behavior). Detection of these behaviors can assist in avoiding possible conflict among crew members and promoting the overall team success. Our prior research focused on an initial proof of concept of machine learning (ML) models and natural language processing (NLP) techniques that were used for automatically detect micro-behaviors among crew members of the US National Aeronautics and Space Administration’s (NASA) Human Exploration Research Analog (HERA) Campaigns 4 and 5 missions (Paromita et al., 2023). Results underscored the importance of incorporating contextual information in the ML models in the form of sentiment analysis, type of task, and dyadic interaction among team members. Here, we expand the scope of our prior work in two ways. First, we assess ML/NLP methods on new behavioral annotations coded using an adapted version of Smith & Griffins (2022) theoretical framework in terms of Violation (i.e., presence of valenced behavior, uplifting/positive or discouraging/negative), Intensity (i.e., force of behavior in terms of how uplifting or discouraging is the behavior), and Intent (i.e., motive of the behavior in terms of whether it was deliberate or unintentional). Second, we expand the design of the ML model to preserve information about the role of each team member within the occurrence of the micro-behavior (in contrast to the previous model that only considered the sender and the target without determining the team member role). This allows to consider all team members' contributions in the conversation and model long-term dependencies in the dialogue. Our experiments for this study are conducted on data from 5 teams of the NASA HERA C4 (NASA grant NNX16AQ48G (PI: Bell)). Conversations were extracted from the 1.5 hour Team Interaction Battery (TIB) task that occurred 5 times in-mission per crew. This resulted in a total of 13,058 conversational turns (i.e., 17.8% uplifting, 3.3% discouraging, 75.76% neutral, 3.14% nulls). Our findings with the revised behavioral coding and ML/NLP models indicate a 43.66% macro F1-score (i.e., 38.29% precision (P), 50.8% recall (R)) for a dialog state-tracking model that includes information from the sender only, and a 40.9% F1-score (i.e., 38.7% P, 43.36% R) for the same model that includes information from both the sender and the target of the micro-behavior. These are significantly higher compared to simple random forest models that classify behaviors strictly based on speech content and do not consider iterative team dynamics, achieving a 36.07% F1-score (i.e., 39.04% R, 33.53% P). Our findings demonstrate potential ways to leverage large conversational datasets to better capture complex team dynamics. We will discuss future directions including proposed models that can incorporate additional mission days and tasks beyond the TIB for objectively quantifying team behavior at high temporal resolution in space exploration missions.

Projna Paromita↗