Engineering topics

Lee, Seung Man

Publications and source records attributed to Lee, Seung Man.

13 records

NASA NTRS2017

Ensuring Interoperability between UAS Detect-and-Avoid and Manned Aircraft Collision Avoidance

The UAS community in the United States has identified the need for a collision avoidance region in which UAS Detect-and-Avoid (DAA) vertical guidance is restricted to preclude interoperability issues with manned aircraft collision avoidance system vertical resolution advisories (RAs). This paper documents the process by which the collision avoidance region was defined. Three candidate definitions were evaluated on 1.3 million simulated pairwise encounters between UAS and manned aircraft covering a wide range of horizontal and vertical closure rates, angles, and miss distances. They were evaluated with regard to UAS DAA interoperability with manned aircraft collision avoidance systems in terms of: 1) the primary objective of restricting DAA vertical guidance before RAs when the aircraft are close, and 2) the secondary objective of avoiding unnecessary restrictions of DAA vertical guidance at a DAA alert when the aircraft are further apart. The collision avoidance region definition that fully achieves the primary objective and best achieves the secondary objective was recommended to and accepted by the UAS community in the United States. By this definition, UAS and manned aircraft are in the collision avoidance region--during which DAA vertical guidance is restricted--when the time to closest point of approach is less than 50 seconds and either the time to co-altitude is less than 50 seconds or the current vertical separation is less than 800 feet.

collision avoidance↗

NASA NTRS2017

UAS Well Clear Recovery Against Non-Cooperative Intruders Using Vertical Maneuvers

This paper documents a study that drove the development of a mathematical expression in the minimum operational performance standards (MOPS) of detect-and-avoid (DAA) systems for unmanned aircraft systems (UAS). This equation describes the conditions under which vertical maneuver guidance could be provided during recovery of well clear separation with a non-cooperative VFR aircraft in addition to horizontal maneuver guidance. Although suppressing vertical maneuver guidance in these situations increased the minimum horizontal separation from 500 to 800 feet, the maximum severity of loss of well clear increased in about 35 of the encounters compared to when a vertical maneuver was preferred and allowed. Additionally, analysis of individual cases led to the identification of a class of encounter where vertical rate error had a large effect on horizontal maneuvers due to the difficulty of making the correct left-right turn decision: crossing conflict with intruder changing altitude. These results supported allowing vertical maneuvers when UAS vertical performance exceeds the relative vertical position and velocity accuracy of the DAA tracker given the current velocity of the UAS and the relative vertical position and velocity estimated by the DAA tracker. Looking ahead, these results indicate a need to improve guidance algorithms by utilizing maneuver stability and near mid-air collision risk when determining maneuver guidance to regain well clear separation.

detect and avoid↗

NASA NTRS2017

UAS Well Clear Recovery Against Non-Cooperative Intruders Using Vertical Maneuvers

This paper documents a study that drove the development of a mathematical expression in the detect-and-avoid (DAA) minimum operational performance standards (MOPS) for unmanned aircraft systems (UAS). This equation describes the conditions under which vertical maneuver guidance should be provided during recovery of DAA well clear separation with a non-cooperative VFR aircraft. Although the original hypothesis was that vertical maneuvers for DAA well clear recovery should only be offered when sensor vertical rate errors are small, this paper suggests that UAS climb and descent performance should be considered-in addition to sensor errors for vertical position and vertical rate-when determining whether to offer vertical guidance. A fast-time simulation study involving 108,000 encounters between a UAS and a non-cooperative visual-flight-rules aircraft was conducted. Results are presented showing that, when vertical maneuver guidance for DAA well clear recovery was suppressed, the minimum vertical separation increased by roughly 50 feet (or horizontal separation by 500 to 800 feet). However, the percentage of encounters that had a risk of collision when performing vertical well clear recovery maneuvers was reduced as UAS vertical rate performance increased and sensor vertical rate errors decreased. A class of encounter is identified for which vertical-rate error had a large effect on the efficacy of horizontal maneuvers due to the difficulty of making the correct left/right turn decision: crossing conflict with intruder changing altitude. Overall, these results support logic that would allow vertical maneuvers when UAS vertical performance is sufficient to avoid the intruder, based on the intruder's estimated vertical position and vertical rate, as well as the vertical rate error of the UAS' sensor.

UAS in the NAS↗

NASA NTRS2016

NAS-Wide Fast-Time Simulation Study for Evaluating Performance of UAS Detect-and-Avoid Alerting and Guidance Systems

This presentation contains the analysis results of NAS-wide fast-time simulations with UAS and VFR traffic for a single day for evaluating the performance of Detect-and-Avoid (DAA) alerting and guidance systems. This purpose of this study was to help refine and validate MOPS alerting and guidance requirements. In this study, we generated plots of all performance metrics that are specified by RTCA SC-228 Minimum Operational Performance Standards (MOPS): 1) to evaluate the sensitivity of alerting parameters on the performance metrics of each DAA alert type: Preventive, Corrective, and Warning alerts and 2) to evaluate the effect of sensor uncertainty on DAA alerting and guidance performance.

DAA Alerting Performance↗

NASA NTRS2016

Effect of Vertical Rate Error on Recovery from Loss of Well Clear Between UAS and Non-Cooperative Intruders

When an Unmanned Aircraft System (UAS) encounters an intruder and is unable to maintain required temporal and spatial separation between the two vehicles, it is referred to as a loss of well-clear. In this state, the UAS must make its best attempt to regain separation while maximizing the minimum separation between itself and the intruder. When encountering a non-cooperative intruder (an aircraft operating under visual flight rules without ADS-B or an active transponder) the UAS must rely on the radar system to provide the intruders location, velocity, and heading information. As many UAS have limited climb and descent performance, vertical position andor vertical rate errors make it difficult to determine whether an intruder will pass above or below them. To account for that, there is a proposal by RTCA Special Committee 228 to prohibit guidance systems from providing vertical guidance to regain well-clear to UAS in an encounter with a non-cooperative intruder unless their radar system has vertical position error below 175 feet (95) and vertical velocity errors below 200 fpm (95). Two sets of fast-time parametric studies was conducted, each with 54000 pairwise encounters between a UAS and non-cooperative intruder to determine the suitability of offering vertical guidance to regain well clear to a UAS in the presence of radar sensor noise. The UAS was not allowed to maneuver until it received well-clear recovery guidance. The maximum severity of the loss of well-clear was logged and used as the primary indicator of the separation achieved by the UAS. One set of 54000 encounters allowed the UAS to maneuver either vertically or horizontally, while the second permitted horizontal maneuvers, only. Comparing the two data sets allowed researchers to see the effect of allowing vertical guidance to a UAS for a particular encounter and vertical rate error. Study results show there is a small reduction in the average severity of a loss of well-clear when vertical maneuvers are suppressed, for all vertical error rate thresholds examined. However, results also show that in roughly 35 of the encounters where a vertical maneuver was selected, forcing the UAS to do a horizontal maneuver instead increased the severity of the loss of well-clear for that encounter. Finally, results showed a small reduction in the number of severe losses of well-clear when the high performance UAS (2000 fpm climb and descent rate) was allowed to maneuver vertically, and the vertical rate error was below 500 fpm. Overall, the results show that using a single vertical rate threshold is not advisable, and that limiting a UAS to horizontal maneuvers when vertical rate errors are above 175 fpm can make a UAS less safe about a third of the time. It is suggested that the hard limit be removed, and system manufacturers instructed to account for their own UAS performance, as well as vertical rate error and encounter geometry, when determining whether or not to provide vertical guidance to regain well-clear.

detect and avoid↗

NASA NTRS2016

Defining the Collision Avoidance Region for DAA Systems

Unmanned aircraft systems (UAS) will be required to equip with a detect-‐and-‐avoid (DAA) system in order to satisfy the federal aviation regulations to maintain well clear of other aircraft, some of which may be equipped with a Traffic Collision Avoidance System (TCAS) to mitigate the possibility of mid-‐air collisions. As such, the minimum operational performance standards (MOPS) for UAS DAA systems are being designed with TCAS interoperability in mind by a group of industry, government, and academic institutions named RTCA Special Committee-228 (SC-228). This document will discuss the development of the spatial-‐temporal volume known as the collision avoidance region in which the DAA system is not allowed to provide vertical guidance to maintain or regain DAA well clear that could conflict with resolution advisories (RAs) issued by the intruder aircraft's TCAS system. Three collision avoidance region definition candidates were developed based on the existing TCAS RA and DAA alerting definitions. They were evaluated against each other in terms of their interoperability with TCAS RAs and DAA alerts in an unmitigated factorial encounter analysis of 1.3 million simulated pairs.

collision avoidance region↗

NASA NTRS2016

ACES M and S: Unmitigated Factorial Encounter Study on DAA/TCAS Interoperability

Realization of the expected proliferation of Unmanned Aircraft System (UAS) operations in the National Airspace System (NAS) depends on the development and validation of standards for UAS Detect and Avoid (DAA) Systems. The RTCA Special Committee 228 is charged with leading the development of draft Minimum Operational Performance Standards (MOPS) for UAS DAA Systems. NASA, as a participating member of RTCA SC-228 is committed to supporting the development and validation of draft requirements for DAA alerting and guidance systems. This presentation contains the results of two combinatorial encounter analysis studies using NASA's SAA Control fast-time simulation capability for this purpose. In these studies, encounters between two aircraft were simulated one at a time for the full factorial combination of encounter geometries (e.g., encounter angle, CPA offset) and aircraft performance (e.g., ownership and intruder ground speeds and vertical rates). The first study analyzes the relationships (e.g., timeline) between the different alerting-safety regions in the SC-228 MOPS (in order of increasing severity): 1) DAA warning alert, 2) well clear recovery (WCR) guidance, 3) DAA-Collision Avoidance (CA), and 4) TCAS RA. This study will focus primarily on encounter situations in which TCAS RA occurs prior to any of the other alerting-safety boundaries. In particular, this study will investigate whether using vertical distance or vertical distance at closest point of approach (i.e., vertical miss distance or VMD) is more appropriate for the definition of the DAA-CA region. In addition, cases where transitions between different regions skip an intermediate region will be analyzed. The second study in this presentation explores a proposal to use an altitude rate error threshold to determine if vertical maneuvers are acceptable for DAA WCR guidance against non-cooperative intruders. This study incorporates the radar from the Honeywell sensor model and examines a series of pairwise encounters between a non-cooperative intruder and a UAS ownship, with different combinations of intruder states and ownship performance levels. The study uses SAA Control as a simulation platform and pilot model, and Omnibands to provide DWC recovery guidance. Two simulation sets, one that allows vertical DWC recovery guidance and one that does not, are compared to determine if encounters with altitude rate errors above 250 feet-per-minute are more likely to have more severe losses of well clear, as determined by the Loss of Well-Clear Severity metric.

SaaControl↗

NASA NTRS2016

Evaluating Alerting and Guidance Performance of a UAS Detect-And-Avoid System

A key challenge to the routine, safe operation of unmanned aircraft systems (UAS) is the development of detect-and-avoid (DAA) systems to aid the UAS pilot in remaining "well clear" of nearby aircraft. The goal of this study is to investigate the effect of alerting criteria and pilot response delay on the safety and performance of UAS DAA systems in the context of routine civil UAS operations in the National Airspace System (NAS). A NAS-wide fast-time simulation study was conducted to assess UAS DAA system performance with a large number of encounters and a broad set of DAA alerting and guidance system parameters. Three attributes of the DAA system were controlled as independent variables in the study to conduct trade-off analyses: UAS trajectory prediction method (dead-reckoning vs. intent-based), alerting time threshold (related to predicted time to LoWC), and alerting distance threshold (related to predicted Horizontal Miss Distance, or HMD). A set of metrics, such as the percentage of true positive, false positive, and missed alerts, based on signal detection theory and analysis methods utilizing the Receiver Operating Characteristic (ROC) curves were proposed to evaluate the safety and performance of DAA alerting and guidance systems and aid development of DAA system performance standards. The effect of pilot response delay on the performance of DAA systems was evaluated using a DAA alerting and guidance model and a pilot model developed to support this study. A total of 18 fast-time simulations were conducted with nine different DAA alerting threshold settings and two different trajectory prediction methods, using recorded radar traffic from current Visual Flight Rules (VFR) operations, and supplemented with DAA-equipped UAS traffic based on mission profiles modeling future UAS operations. Results indicate DAA alerting distance threshold has a greater effect on DAA system performance than DAA alerting time threshold or ownship trajectory prediction method. Further analysis on the alert lead time (time in advance of predicted loss of well clear at which a DAA alert is first issued) indicated a strong positive correlation between alert lead time and DAA system performance (i.e. the ability of the UAS pilot to maneuver the unmanned aircraft to remain well clear). While bigger distance thresholds had beneficial effects on alert lead time and missed alert rate, it also generated a higher rate of false alerts. In the design and development of DAA alerting and guidance systems, therefore, the positive and negative effects of false alerts and missed alerts should be carefully considered to achieve acceptable alerting system performance by balancing false and missed alerts. The results and methodology presented in this study are expected to help stakeholders, policymakers and standards committees define the appropriate setting of DAA system parameter thresholds for UAS that ensure safety while minimizing operational impacts to the NAS and equipage requirements for its users before DAA operational performance standards can be finalized.

UAS Detect-and-Avoid (DAA) System↗

NASA NTRS2015

Analysis of UAS DAA Alerting in Fast-Time Simulations without DAA Mitigation

Realization of the expected proliferation of Unmanned Aircraft System (UAS) operations in the National Airspace System (NAS) depends on the development and validation of performance standards for UAS Detect and Avoid (DAA) Systems. The RTCA Special Committee 228 is charged with leading the development of draft Minimum Operational Performance Standards (MOPS) for UAS DAA Systems. NASA, as a participating member of RTCA SC-228 is committed to supporting the development and validation of draft requirements for DAA alerting system performance. A recent study conducted using NASA's ACES (Airspace Concept Evaluation System) simulation capability begins to address questions surrounding the development of draft MOPS for DAA alerting systems. ACES simulations were conducted to study the performance of alerting systems proposed by the SC-228 DAA Alerting sub-group. Analysis included but was not limited to: 1) correct alert (and timeliness), 2) false alert (and severity and duration), 3) missed alert, and 4) probability of an alert type at the time of loss of well clear. The performance of DAA alerting systems when using intent vs. dead-reckoning for UAS ownship trajectories was also compared. The results will be used by SC-228 to inform decisions about the surveillance standards of UAS DAA systems and future requirements development and validation efforts.

alerting↗

NASA NTRS2015

Analysis of UAS DAA Surveillance in Fast-Time Simulations without DAA Mitigation

Realization of the expected proliferation of Unmanned Aircraft System (UAS) operations in the National Airspace System (NAS) depends on the development and validation of performance standards for UAS Detect and Avoid (DAA) Systems. The RTCA Special Committee 228 is charged with leading the development of draft Minimum Operational Performance Standards (MOPS) for UAS DAA Systems. NASA, as a participating member of RTCA SC-228 is committed to supporting the development and validation of draft requirements for DAA surveillance system performance. A recent study conducted using NASA's ACES (Airspace Concept Evaluation System) simulation capability begins to address questions surrounding the development of draft MOPS for DAA surveillance systems. ACES simulations were conducted to study the performance of sensor systems proposed by the SC-228 DAA Surveillance sub-group. Analysis included but was not limited to: 1) number of intruders (both IFR and VFR) detected by all sensors as a function of UAS flight time, 2) number of intruders (both IFR and VFR) detected by radar alone as a function of UAS flight time, and 3) number of VFR intruders detected by all sensors as a function of UAS flight time. The results will be used by SC-228 to inform decisions about the surveillance standards of UAS DAA systems and future requirements development and validation efforts.

unmanned aircraft systems↗

NASA NTRS2015

Evaluating Performance of UAS Detect-And-Avoid System Using a Fast-Time Simulation Tool

Most unmanned aircraft systems will be required to be equipped with a Detect-and-Avoid (DAA) system. The surveillance performance of the DAA system to detect and track intruder aircraft will depend on the encounter geometries that unmanned aircraft are expected to have with other aircraft in the airspace. The performance of DAA alerting and avoidance system is also dependent on the timeliness of alerting for UAS pilots to give a sufficient time to determine and command a resolution maneuver to avoid well clear separation violations. This presentation introduces general background of UAS DAA systems and concept of well clear separation standard to satisfy see and avoid regulations. The presentation shows the several UAS mission profiles and analysis of the encounter geometries that were simulated using historical VFR traffic data and some proposed UAS missions. This presentation introduces several potential metrics for evaluating the performance of a DAA system and shows the results that measured through fast-time simulation with traffic scenarios that include NAS-wide VFR manned aircraft and IFR UAS flights. At the end, some research areas will be briefly discussed.

UAS↗

NASA NTRS2014

Investigating Detect-and-Avoid Surveillance Performance for Unmanned Aircraft Systems

Most unmanned aircraft systems will be required to be equipped with a Detect-and-Avoid (DAA) system with a surveillance component. The surveillance performance requirements of the DAA system to detect and track intruder aircraft will depend on the encounter geometries that unmanned aircraft are expected to have with other aircraft in the airspace. This presentation shows the analysis of the encounter geometries that were simulated using historical low-altitude traffic data and some proposed UAS missions. This analysis suggests how the overall safety and performance of a surveillance system may relate to surveillance parameters such as surveillance range, horizontal and vertical fields or regard. This study proposed and investigated potential safety and performance metrics for evaluating the performance of a surveillance system, such as the ratio of undetected and late-detected separation violations, and the time to violation at first detection for given sets of surveillance parameters.

UAS↗

NASA NTRS2014

Investigating Surveillance Performance for UAS Detect-and-Avoid Systems

Most unmanned aircraft systems will be required to be equipped with a detect-and-avoid system that is capable of maintaining appropriate separation from other aircraft. One of the critical components of detect-and-avoid systems is a surveillance system that identifies potential threat aircraft in real time and tracks these aircraft so that their future trajectories may be used to predict conflicts. The performance of the detect-and-avoid system generally depends on technical parameters of the surveillance system, such as the surveillance range. The quantitative requirements for detect-and-avoid systems will be determined to meet safety metrics for the operation of unmanned aircraft systems in the National Airspace System. This study employs a sensor model comprised of the surveillance range, and horizontal and vertical fields of regard that mainly characterize the overall performance of a surveillance system. In this study, potential metrics for evaluating the performance of a surveillance system were investigated through fast-time simulation with a traffic scenario that included both proposed unmanned aircraft flights and historical visual flight rule aircraft tracks. Using the simulation results, an overall analysis of encounter geometry highlights the encounter characteristics that relate surveillance parameters to safety metrics and detect-and-avoid system performance. Then, given several candidate surveillance volumes, performance and safety metrics are derived; these metrics include the ratio of undetected and late-detected violations and the time to violation at first detection. These example metrics demonstrate the utility of the database of encounters created in this work, a database which will be useful in the derivation of required detect-and-avoid surveillance system requirements.

Lee, Seung Man↗