Casey L Smith | Engineering Papers

NASA NTRS2021

A Cognitive Walkthrough of Multiple Drone Delivery Operations

Advances of early twenty-first century aviation and transportation technologies provide opportunities for enhanced aerial projects, and the overall integration of unmanned aircraft systems (UAS) into the National Airspace System (NAS) has applications across a wide range of operations. Through these, remote operators have learned to manage several UAS at the same time in a variety of operational environments. The present work details a component piece of an ongoing body of research into multi-UAS operations. Beginning in early 2020, NASA has collaborated with Uber Technologies to design and develop concepts of operations, roles and responsibilities, and ground control station (GCS) concepts to enable food delivery operations via multiple, small UAS (sUAS). A cognitive walkthrough was chosen as the method for data collection. This allowed information to be gathered from UAS subject matter experts (SMEs) that could further mature designs for future human-in-the-loop (HITL) simulations; in addition, it allowed information to be collected remotely during the stringent restrictions of the COVID-19 pandemic. Consequently, the described cognitive walkthrough activity utilized remote data collection protocols mediated through the usage of programs designed for presentation and telecommunications. Scenarios were designed, complete with airspace, contingencies, and remedial actions, to be presented to the SMEs. Information was collected using a combination of rating scales and open-ended questions. Results received from the SMEs revealed expected hazards, workloads, and information concerns inherent in the contingency scenarios. SMEs also provided insight into the design of GCS tools and displays as well as the duties and relationships of human operators (i.e., monitors) and automation (i.e., informers and flight managers). Implications of these findings are discussed.

unmanned aircraft systems↗

NASA NTRS2021

UAS Integration in the NAS Flight Test 6: Full Mission Results

Recent standards development efforts for the integration of Unmanned Aircraft Systems (UAS) into the National Airspace System (NAS) such as those in RTCA Inc. Special Committee 228 (SC-228) have focused on relatively large UAS transitioning to and from Class A airspace. In an effort to expand the range of vehicle classes that can access the NAS, the NASA UAS Integration in the NAS project has investigated Low Size, Weight, and Power (Low SWaP) technologies that would allow smaller UAS to detect-and-avoid (DAA) traffic. Through batch and human in the loop (HITL) simulation studies, the UAS Integration in the NAS DAA subproject have identified candidate performance standards that would contribute to enabling extended Low SWaP, UAS operations under 10,000 feet. These candidate performance standards include minimum field of regard (FOR) values for Low SWaP air surveillance sensors as well as a DAA well-clear (DWC) definition which can be applied to non-cooperative traffic to reduce the required maneuver initiation range. To test the assumptions of the project’s simulation studies and validate the candidate performance standards, a live flight research event was executed at NASA Armstrong Flight Research Center. The UAS Integration in the NAS Project Flight Test 6 Full Mission sought to characterize UAS pilot responses to traffic conflicts using a representative Low SWAP DAA system in an operational NAS environment. To achieve this, live, virtual and constructive distributed environment (LVC-DE) elements were combined to simulate a sector of Oakland center airspace and induce encounters with a live, manned aircraft. A Navmar Applied Sciences Tigershark XP was used as the UAS ownship and was integrated into the test architecture to enable it to be controlled from a Vigilant Spirit Control Station (VSCS) research ground control station. Qualified UAS pilots were recruited to act as subject pilots under test (SPUT) to control the Tigershark XP in a simulated mission while coordinating with a participating air traffic controller in simulated airspace. The intruder speed, intruder equipage and encounter geometry were varied between six scripted encounters per SPUT. Various metrics were collected including pilot reaction time from the onset of DAA alert, ATC coordination rate, probability and severity of losses of DAA well clear, and subjective ratings of system acceptability. The implications of these results on the development of standards for Low SWAP DAA systems will be discussed.

UAV↗

NASA NTRS2020

An Evaluation of UAS Pilot Workload and Acceptability Ratings with Four Simulated Radar Declaration Ranges

Currently, minimum operational performance standards (MOPS) are being developed for a broader range of unmanned aircraft system (UAS) platforms, including smaller UAS that will feature onboard sensors that are low in size, weight, and power, otherwise known as low SWaP. The low SWaP sensors used to detect non-cooperative traffic will have limited declaration ranges compared to those designed for medium-to-large UAS. A human-in-the-loop (HITL) study was conducted examining four possible radar declaration ranges (i.e., 1.5 NM, 2 NM, 2.5 NM, and 3 NM) for a potential low SWaP sensor with a detect and avoid (DAA) system encountering various non-cooperative encounters in Oakland Center airspace. Participants had lower workload, particularly workload associated with temporal demand and effort, in scenarios that featured larger declaration ranges. Furthermore, participants reported better ability to remain DAA well clear within the larger declaration range conditions, specifically with the 2.5 NM and 3 NM conditions.

UAS↗

NASA NTRSDate not supplied

Display and Automation Considerations for the Airborne Collision Avoidance System Xu

In this paper we examine several display and automation considerations of a collision avoidance system that is currently under development: the Airborne Collision Avoidance System (ACAS) Xu. This study builds on previous work conducted as part of NASA’s Unmanned Aircraft Systems (UAS) Integration into the National Airspace System (NAS) project. ACAS Xu represents the next-generation successor to the Traffic Alert and Collision Avoidance System (TCAS II), wherein the Xu variant is intended for UAS applications. Whereas TCAS II exclusively issues RAs in the vertical dimension, a major distinction between ACAS Xu and previous collision avoidance (CA) systems is the introduction of horizontal and “blended” RAs (i.e., RAs with both horizontal and vertical components). This present work was conducted as an engineering analysis involving two parts. In Part 1, a two-by-two, within-subjects study was performed that manipulated how RAs were presented to a pilot situated at a UAS ground control station. Five participants experienced four experimental trials in which text and aural alerting characteristics were manipulated. In Part 2, another five participants experienced four trials in which the levels of automation were manipulated with regard to the CA and return-to-course (RTC) tasks. The results for Part 1 found no effect of display or alerting configuration on pilot performance. However, it was discovered that pilot response time to RAs greatly depended on the RA type. In particular, pilots were quicker to respond to vertical RAs (M = 4.52 seconds) than horizontal (M = 7.42 seconds) and blended (M = 9.68 seconds) RAs in which both dimensions were issued simultaneously. For Part 2 of the study, pilots found both auto-CA and auto-RTC functions equally useful. Most pilots were comfortable with the automation, however responses were mixed. Three of five participants indicated high levels of comfort with the auto-CA function, while two rated their comfort as low. Pilots’ comfort for the auto-RTC functionality was slightly higher: four out of five pilots gave high ratings, while one pilot gave a low rating. Overall, pilots ordinally ranked their preference for automated functions as auto-CA together with auto-RTC (when an aural alert announces a change between CA and RTC states), auto-CA, and auto-CA and RTC (without the aural state-change announcement). Recommendations for improving the display of automation are also discussed.

collision avoidance↗

NASA NTRSDate not supplied

An Evaluation of UAS Pilot Workload and Acceptability Ratings with Four Simulated Radar Declaration Ranges

Currently, minimum operational performance standards (MOPS) are being developed for a broader range of unmanned aircraft system (UAS) platforms, including smaller UAS that will feature onboard sensors that are low in size, weight, and power, otherwise known as low SWaP. The low SWaP sensors used to detect non-cooperative traffic will have limited declaration ranges compared to those designed for medium-to-large UAS. A human-in-the-loop (HITL) study was conducted examining four possible radar declaration ranges (i.e., 1.5 NM, 2 NM, 2.5 NM, and 3 NM) for a potential low SWaP sensor with a detect and avoid (DAA) system encountering various non-cooperative encounters in Oakland Center airspace. Participants had lower workload, particularly workload associated with temporal demand and effort, in scenarios that featured larger declaration ranges. Furthermore, participants reported better ability to remain DAA well clear within the larger declaration range conditions, specifically with the 2.5 NM and 3 NM conditions.

UAS↗

NASA NTRSDate not supplied

UAS Pilot Assessments of Display and Alerting for the Next Generation Airborne Collision Avoidance System Xu

Unmanned aircraft systems (UAS) must comply with specific standards to operate in the National Airspace System (NAS). Among the requirements are the detect and avoid (DAA) capabilities, which include display, alerting, and guidance specifications. Previous studies have queried pilots for their subjective feedback on these display elements on earlier systems; the present study sought pilot evaluations with an initial iteration of the unmanned variant of a Next Generation Airborne Collision Avoidance System (ACAS XU). Sixteen participants piloted simulated aircraft with both standalone and integrated DAA displays. Their opinions were gathered using post-block and post-simulation questionnaires as well as guided debriefs. The data showed pilots had better understanding and comfort with the system when using an integrated display. Pilots also rated ACAS Xu alerting and guidance as generally acceptable and effective. Implications for further development of ACAS XU and DAA displays are discussed.

unmanned aircraft systems↗

NASA NTRSDate not supplied

A Cognitive Walkthrough of Multiple Drone Delivery Operations

Advances of early twenty-first century aviation and transportation technologies provide opportunities for enhanced aerial projects, and the overall integration of unmanned aircraft systems (UAS) into the National Airspace System (NAS) has applications across a wide range of operations. Through these, remote operators have learned to manage several UAS at the same time in a variety of operational environments. The present work details a component piece of an ongoing body of research into multi-UAS operations. Beginning in early 2020, NASA has collaborated with Uber Technologies to design and develop concepts of operations, roles and responsibilities, and ground control station (GCS) concepts to enable food delivery operations via multiple, small UAS (sUAS). A cognitive walkthrough was chosen as the method for data collection. This allowed information to be gathered from UAS subject matter experts (SMEs) that could further mature designs for future human-in-the-loop (HITL) simulations; in addition, it allowed information to be collected remotely during the stringent restrictions of the COVID-19 pandemic. Consequently, the described cognitive walkthrough activity utilized remote data collection protocols mediated through the usage of programs designed for presentation and telecommunications. Scenarios were designed, complete with airspace, contingencies, and remedial actions, to be presented to the SMEs. Information was collected using a combination of rating scales and open-ended questions. Results received from the SMEs revealed expected hazards, workloads, and information concerns inherent in the contingency scenarios. SMEs also provided insight into the design of GCS tools and displays as well as the duties and relationships of human operators (i.e., monitors) and automation (i.e., informers and flight managers). Implications of these findings are discussed.

unmanned aircraft systems↗

NASA NTRSDate not supplied

ACAS Xr Part Task Sim, Preliminary Experiment Design

In early 2022, the Human Autonomy Teaming Lab (NASA Ames Research Center) will conduct a manned, human-in-the-loop (HITL) simulation. This part task HITL will begin the lab’s Hazard Perception and Avoidance (HPA) technical work under NASA’s Advanced Air Mobility (AAM), Automated Flight and Contingency Management (AFCM) Sub-Project. The goals will be to assess levels of automation for manned, electric vertical takeoff and landing (eVTOL) aircraft. This simulation will test manual and automated Resolution Advisory (RA) responses and return-to-course (RTC) maneuvers for the first version of the Airborne Collision Avoidance System’s (ACAS) rotary-wing (Xr) variant. This will be conducted on a fixed-based simulator designed to fly eVTOL aircraft while maneuvering for intruding traffic. Variables for this study include levels of autonomy (i.e., manual and automated) as well as the types of alerts at the onset of conflicts (i.e., Corrective and RA). The data collected will include response times, losses of well clear, and maneuver sizes and durations as well as subjective ratings like acceptability, perceived workload, and meaningful human control. Additional details and future anticipations are also discussed.

air taxis↗

NASA NTRSDate not supplied

Streamlining Tactical Operator Handoffs During Multi-Vehicle Applications

Increased automation has shifted the operator control paradigm from a single operator controlling a single vehicle, to multiple operators collaborating to control multiple vehicles; this paradigm is known as m:N. Many questions remain unanswered in this new operational paradigm about the division of assets as workload for individual operators varies overtime. This paper explores the management of workload by enabling operators to temporarily handoff vehicles among each other. A study was conducted to explore both a manual and assisted method for performing handoffs during manipulated contingency scenarios. The assisted handoff method allowed subjects to easily choose and group nominal and/or contingency vehicles. The number of contingencies was also manipulated to determine the effect workload had on how pilots utilized the ability to handoff vehicles. Results show subjects performed handoffs more often when there were more contingencies and when the assisted handoff tool was available. In addition, the assisted tool make subjects feel more comfortable, enabling them to feel like they could take longer to resolve contingency situations. Lastly, even during contingencies, subjects were able to successfully complete secondary tasks.

M:N operations↗

NASA NTRSDate not supplied

A Remote, Human-in-the-Loop Evaluation of a Multiple-Drone Delivery Operation

Over time, advances in unmanned aircraft systems (UAS) have enabled a shift in the operational paradigm from one operator managing one aircraft to that of multiple operators working together to manage multiple aircraft. This shift has highlighted the need for effective human-autonomy teaming methods to maintain manageable workload levels for operators as well as high standards of system performance and safety. This paper presents a study aimed at evaluating whether automation can help operators manage workload during small UAS (sUAS) package delivery scenarios featuring contingency situations. These contingency situations, resulting from unplanned UAS Volume Reservations (UVRs), required flight path reroutes for multiple aircraft simultaneously. The study manipulated the number of aircraft affected by the UVRs and the level of automation support. The presence of terrain conflicts was also controlled within each scenario. Due to the COVID-19 pandemic, subjects were not able to gain direct access to the Ground Control System (GCS). Therefore, the study was conducted using a subject-surrogate paradigm that required subjects to relay commands through a verbal protocol from remote locations outside of the lab to a researcher surrogate who had direct control of the GCS interfaces at the lab location. Results show that the automated support condition was associated with faster reroute response times, more efficient reroute maneuvers, and significantly lower levels of perceived workload than the manual reroute condition. However, the automation support level did not significantly impact pilots’ ability to avoid the UVR successfully; pilots were overwhelmingly capable of avoiding the UVR in all conditions. The presence of terrain conflicts primarily impacted pilot performance by leading to multiple uploads per vehicle, which was not typically required when pilots only needed to maneuver laterally. Although subjects did not have direct control over the GCS, subjective ratings indicate that the displays under test provided them with sufficient information to manage their aircraft and promptly respond to the unplanned UVRs. Overall, the objective and subjective data strongly suggest that the verbal protocol and subject-surrogate paradigm were effective methods for collecting data remotely amid the COVID-19 pandemic.

multi-UAS↗

NASA NTRSDate not supplied

Hazard Perception & Avoidance (HPA): Part Task 1 - Results Outbrief

n April 2022, the Human Autonomy Teaming Lab (NASA Ames Research Center) conducted a manned, human-in-the-loop (HITL) simulation. This part task HITL began the lab’s Hazard Perception and Avoidance (HPA) technical work under NASA’s Advanced Air Mobility (AAM), Automated Flight and Contingency Management (AFCM) Sub-Project. The goals were to assess levels of automation for manned, electric vertical takeoff and landing (eVTOL) aircraft. This simulation tested manual and automated Resolution Advisory (RA) responses and return-to-course (RTC) maneuvers for the first version of the Airborne Collision Avoidance System’s (ACAS) rotary-wing (Xr) variant. This was conducted on a fixed-based simulator designed to fly eVTOL aircraft while maneuvering for intruding traffic. Variables for this study included levels of autonomy (i.e., manual and automated) as well as the types of alerts at the onset of conflicts (i.e., Corrective and RA). The data collected included response times, losses of well clear, and maneuver sizes and durations as well as subjective ratings like acceptability, and perceived workload. Additional details and future anticipations are also discussed.

air taxis↗

NASA NTRSDate not supplied

Assured Vehicle Automation 1 Sim - Results Outbrief

In early 2022, the Human Autonomy Teaming Lab (NASA Ames Research Center) conducted a manned, human-in-the-loop (HITL) simulation. This part-task HITL began the lab’s Hazard Perception and Avoidance (HPA) technical work under NASA’s Advanced Air Mobility (AAM), Automated Flight and Contingency Management (AFCM) Sub-Project. The results of that sim guided the objectives of the current study, which were to examine pilots’ use of the Airborne Collision Avoidance System (ACAS) rotorcraft variant (Xr) v2 in multiple phases of flight with two separate Xr Modes, fully leverage Xr v2 features (e.g., use radar altimeter data to inform low altitude Resolution Advisory [RA] behavior, utilize the ability to designate “terminal-area intruders,” and display airspeed-based Detect and Avoid [DAA] guidance), emulate a “Traffic Advisory” (TA), and present Xr in a higher-fidelity environment. Therefore, this study was conducted in the Vertical Motion Simulator, and the variables included Phase of Flight (En-route, Hover, and Approach) as well as ACAS Xr Mode (TA/RA and DAA). The data collected included response times, losses of well clear, and maneuver sizes and durations as well as subjective ratings like acceptability and usability. Additional details and future anticipations are also discussed.

air taxis↗

NASA NTRSDate not supplied

Hazard Perception & Avoidance (HPA), Assured Vehicle Automation 1 Simulation (AVA-1h) Results Outbrief

In late 2022, the Human Autonomy Teaming Lab (NASA Ames Research Center) conducted a manned, human-in-the-loop (HITL) simulation. This HITL was the lab’s second Hazard Perception and Avoidance (HPA) technical work under NASA’s Advanced Air Mobility (AAM), Automated Flight and Contingency Management (AFCM) Sub-Project. The goals were to assess detect and avoid technology for manned, electric vertical takeoff and landing (eVTOL) aircraft. This simulation tested Resolution Advisory (RA) responses and return-to-course (RTC) maneuvers for the second version of the Airborne Collision Avoidance System’s (ACAS) rotary-wing (Xr) variant. This was conducted at the center's Vertical Motion Simulator (VMS), on a motion-based platform, and was configured to fly eVTOL aircraft while maneuvering for intruding traffic. Variables for this study included ACAS Xr modes (i.e., TA/RA and DAA) and phases of flight (i.e., Cruise, Hover, and Approach). The data collected included response times, losses of well clear, and pilots' noncompliances to alerts and guidance as well as subjective ratings like acceptability and perceived workload. Additional details and future anticipations are also discussed.

ACAS Xr↗

NASA NTRSDate not supplied

Assessing Helicopter Pilots’ Detect and Avoid and Collision Avoidance Performance with ACAS Xr

The latest variant of the Federal Aviation Administration’s Airborne Collision Avoidance System (ACAS X) is being designed for both crewed and uncrewed rotorcraft. Referred to as ACAS Xr, the system joins a suite of other ACAS X variants poised to replace the Traffic Alert and Collision Avoidance System (TCAS II). ACAS Xr is tuned to support current-day helicopter platforms as well as electric Vertical Takeoff and Landing (eVTOL) vehicles that are still under development. Given this flexibility, ACAS Xr may be used by helicopter crews currently in operation or by remotely-operated eVTOL aircraft in the emerging Advanced Air Mobility (AAM) market. To cover the range of potential uses, two distinct configurations are being proposed for ACAS Xr: Collision Avoidance System (CAS) and Detect and Avoid (DAA). Under the CAS configuration, ACAS Xr provides minimal caution-level alerting but issues directive warning-level alerting and guidance. The DAA configuration, by contrast, provides caution-level alerting and guidance, in addition to the warning-level alerting and guidance. The current study was performed as part of the National Aeronautics and Space Administration’s AAM project. Six helicopter pilots were recruited to fly a variety of scripted traffic scenarios in a full-motion, crewed eVTOL simulator. Participants flew 60 encounters over two days, reacting to pre-recorded intruder aircraft that were scripted to fly into the participant’s aircraft from different approach angles, relative altitudes, and during different phases of flight. The pilots flew half of the encounters with the CAS configuration and half with the DAA configuration. Within each block of 30 encounters, pilots experienced 10 conflicts while in cruise, 10 in hover, and 10 while on approach to a heliport. Results showed that pilot response times were consistently under 5 seconds for RAs and under 10 seconds for DAA alerts, when present, during all three phases of flight. Unsurprisingly, the DAA configuration was associated with lower rates of en-route and high-severity losses of DAA well clear compared to the CAS configuration in all phases of flight except for the terminal area. Rates of losses of DAA well clear were found to be substantially higher in the Hover scenario, compared to Cruise. Pilots failed to fully comply with RAs at a rate of 0.10-0.18 in all conditions except for the DAA configuration in the Hover scenario, which was associated with a higher non-compliance rate of 0.4 due to Descend RAs issued at low altitudes. The implications of these results with regards to the ongoing development of ACAS Xr is discussed.

air taxis↗

NASA NTRSDate not supplied

Helicopter Pilot Evaluations of the Airborne Collision Avoidance System Xr in a High-Fidelity Motion Simulation

New aircraft and aerial Urban Air Mobility operations require updated technologies to maintain vehicle separation during flight. Specifically, higher-density airspace will need more than traditional air traffic control to keep vehicles well clear of each other and avoid Near Midair Collisions. In response, the Federal Aviation Administration is currently developing the Airborne Collision Avoidance System X (ACAS X) for next-generation air traffic deconfliction, which provides caution-level and warning-level alerts in response to multiple aircraft types. This study recruited six helicopter pilots to fly an electric vertical takeoff and landing vehicle model in simulated operations under Visual Flight Rules (VFR). Flights were accomplished using the high-fidelity Vertical Motion Simulator at the NASA Ames Research Center. Participants controlled the vehicle using two side inceptors and foot rudders. The rotorcraft variant of ACAS X (ACAS Xr) was provided for alerting and guidance during traffic conflicts. Pilots used this system while giving feedback to the researchers through questionnaires, debriefs, and other discussions. Variables of interest to the study were phases of flight (i.e., Cruise, Hover, and Approach) and ACAS Xr configurations: The Collision Avoidance System configuration behaves similarly to current commercial traffic systems used for tactical deconfliction in crewed vehicles, and the Detect and Avoid configuration was developed to provide extra, corrective-level guidance for unmanned aircraft systems. Results showed that pilots found the alerting and guidance from ACAS Xr useful, effective, and acceptable for VFR operations. Certain elements, like speed guidance and text banners, were found to be of no use to the pilots. Hover and Approach scenarios were considered the most difficult for ACAS Xr alerting. Reasons for this difficulty were partially due to learning interference (i.e., overcoming previously learned behavior) and partially due to the vehicle model (i.e., NASA’s Lift Plus Cruise design). Still, alerting-based confounds reveal the need for more development for ACAS Xr during these Hover and Approach flight phases. Study caveats and future projects are discussed.

air taxis↗