Engineering Papers⌕ Search

Engineering topics

Santiago, Confesor

Publications and source records attributed to Santiago, Confesor.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS2016

Flight Test 4 Preliminary Results: NASA Ames SSI

Realization of the expected proliferation of Unmanned Aircraft System (UAS) operations in the National Airspace System (NAS) depends on the development and validation of performance standards for UAS Detect and Avoid (DAA) Systems. The RTCA Special Committee 228 is charged with leading the development of draft Minimum Operational Performance Standards (MOPS) for UAS DAA Systems. NASA, as a participating member of RTCA SC-228 is committed to supporting the development and validation of draft requirements as well as the safety substantiation and end-to-end assessment of DAA system performance. The Unmanned Aircraft System (UAS) Integration into the National Airspace System (NAS) Project conducted flight test program, referred to as Flight Test 4, at Armstrong Flight Research Center from April -June 2016. Part of the test flights were dedicated to the NASA Ames-developed Detect and Avoid (DAA) System referred to as JADEM (Java Architecture for DAA Extensibility and Modeling). The encounter scenarios, which involved NASA's Ikhana UAS and a manned intruder aircraft, were designed to collect data on DAA system performance in real-world conditions and uncertainties with four different surveillance sensor systems. Flight test 4 has four objectives: (1) validate DAA requirements in stressing cases that drive MOPS requirements, including: high-speed cooperative intruder, low-speed non-cooperative intruder, high vertical closure rate encounter, and Mode CS-only intruder (i.e. without ADS-B), (2) validate TCASDAA alerting and guidance interoperability concept in the presence of realistic sensor, tracking and navigational errors and in multiple-intruder encounters against both cooperative and non-cooperative intruders, (3) validate Well Clear Recovery guidance in the presence of realistic sensor, tracking and navigational errors, and (4) validate DAA alerting and guidance requirements in the presence of realistic sensor, tracking and navigational errors. The results will be presented at RTCA Special Committee 228 in support of final verification and validation of the DAA MOPS.

Detect-And-Avoid (DAA)↗
NASA NTRS2016

Simulation and Flight Test Data Collection to Support Phase 1 Detect-and-Avoid MOPS

The UAS Integration into the NAS (UASNAS) project is studying the minimum operational performance standards for unmanned aerial system (UAS) detect-and-avoid (DAA) systems in order to operate in the National Airspace System (NAS). The project is executing a series of fast-time simulation, human-in-the-loop experiments, and flight tests in support of this effort. Armstrong Flight Research Center is hosting a VIP Day in conjunction with the UAS-NAS Project's Flight Test 4 activity, and the Ames Project Engineer (or designate) will be presenting a briefing. This briefing will review the data collection and analysis activities for Flight Test 4 within the Separation assurance-Sense-and-avoid Interoperability (SSI) team.

data analysis↗
NASA NTRS2016

Simulation and Flight Test Data Collection Review for Supporting Phase 1 Detect and Avoid MOPS

RTCA Special Committee 228 is a consortium of government, industry, and academic organizations tasked to develop minimum operational performance standards for UAS detect and avoid systems. The UAS in the NAS (National Airspace System) project is studying the minimum operational performance standards for unmanned aerial systems (UAS's) detect-and-avoid (DAA) system in order to operate in the National Airspace System. Over the past 3 years, the project has executed a series of fast-time simulation, human-in-the-loop experiments, and flight tests in support of this effort. The purpose of this briefing is to summarize the models developed and data collected to overcome UAS integration barriers, so UAS can remain well clear of all traffic.

Santiago, Confesor↗
NASA NTRS2016

Piloted Well Clear Performance Evaluation of Detect and Avoid Systems with Suggestive Guidance

Regulations to establish operational and performance requirements for unmanned aircraft systems (UAS) are being developed by a consortium of government, industry and academic institutions (RTCA, 2013). Those requirements will apply to the new detect-and-avoid (DAA) systems and other equipment necessary to integrate UAS with the United States (U.S) National Airspace System (NAS) and will be determined according to their contribution to the overall safety case. That safety case requires demonstration that DAA-equipped UAS collectively operating in the NAS meet an airspace safety threshold (AST). Several key gaps must be closed in order to link equipment requirements to an airspace safety case. Foremost among these is calculation of the systems risk ratio, the degree to which a particular system mitigates violation of an aircraft separation standard (FAA, 2013). The risk ratio of a DAA system, in combination with risk ratios of other collision mitigation mechanisms, will determine the overall safety of the airspace measured in terms of the number of collisions per flight hour. It is not known what the effectiveness is of a pilot-in-the-loop DAA system or even what parameters of the DAA system most improve the pilots ability to maintain separation. The relationship between the DAA system design and the overall effectiveness of the DAA system that includes the pilot, expressed as a risk ratio, must be determined before DAA operational and performance requirements can be finalized. Much research has been devoted to integrating UAS into non-segregated airspace (Dalamagkidis, 2009, Ostwald, 2007, Gillian, 2012, Hesselink, 2011, Santiago, 2015, Rorie 2015 and 2016). Several traffic displays intended for use as part of a DAA system have gone through human-in-the-loop simulation and flight-testing. Most of these evaluations were part of development programs to produce a deployable system, so it is unclear how to generalize particular aspects of those designs to general requirements for future traffic displays (Calhoun, 2014). Other displays have undergone testing to collect data that may generalize to new displays, but have not been evaluated in the context of the development of an overall safety case for UAS equipped with DAA systems in the NAS (Bell, 2012). Other research efforts focus on DAA surveillance performance and separation standards. Together with this work, they are expected to facilitate validation of the airspace safety case (Park, 2014 and Johnson, 2015). The contribution of the present work is to quantify the effectiveness of the pilot-automation system to remain well clear as a function of display features and surveillance sensor error. This quantification will help enable selection of a minimum set of DAA design features that meets the AST, a set that may not be unique for all UAS platforms. A second objective is to collect and analyze pilot performance parameters that will improve the modeling of overall DAA system performance in non-human-in-the-loop simulations. Simulating the DAA-equipped UAS in such batch experiments will allow investigation of a much larger number of encounters than is possible in human simulations. This capability is necessary to demonstrate that a particular set of DAA requirements meets the AST under all foreseeable operational conditions.

detect and avoid↗
NASA NTRS2016

UAS Integration in the NAS Project: DAA-TCAS Interoperability "mini" HITL Primary Results

At the May 2015 SC-228 meeting, requirements for TCAS II interoperability became elevated in priority. A TCAS interoperability workgroup was formed to identify and address key issues/questions. The TCAS workgroup came up with an initial list of questions and a plan to address those questions. As part of that plan, NASA proposed to run a mini HITL to address display, alerting and guidance issues. A TCAS Interoperability Workshop was held to determine potential display/alerting/guidance issues that could be explored in future NASA mini HITLS. Consensus on main functionality of DAA guidance when TCAS II RA occurs. Prioritized list of independent variables for experimental design. Set of use cases to stress TCAS Interoperability.

human systems integration↗
NASA NTRS2016

Evaluating Alerting and Guidance Performance of a UAS Detect-And-Avoid System

A key challenge to the routine, safe operation of unmanned aircraft systems (UAS) is the development of detect-and-avoid (DAA) systems to aid the UAS pilot in remaining "well clear" of nearby aircraft. The goal of this study is to investigate the effect of alerting criteria and pilot response delay on the safety and performance of UAS DAA systems in the context of routine civil UAS operations in the National Airspace System (NAS). A NAS-wide fast-time simulation study was conducted to assess UAS DAA system performance with a large number of encounters and a broad set of DAA alerting and guidance system parameters. Three attributes of the DAA system were controlled as independent variables in the study to conduct trade-off analyses: UAS trajectory prediction method (dead-reckoning vs. intent-based), alerting time threshold (related to predicted time to LoWC), and alerting distance threshold (related to predicted Horizontal Miss Distance, or HMD). A set of metrics, such as the percentage of true positive, false positive, and missed alerts, based on signal detection theory and analysis methods utilizing the Receiver Operating Characteristic (ROC) curves were proposed to evaluate the safety and performance of DAA alerting and guidance systems and aid development of DAA system performance standards. The effect of pilot response delay on the performance of DAA systems was evaluated using a DAA alerting and guidance model and a pilot model developed to support this study. A total of 18 fast-time simulations were conducted with nine different DAA alerting threshold settings and two different trajectory prediction methods, using recorded radar traffic from current Visual Flight Rules (VFR) operations, and supplemented with DAA-equipped UAS traffic based on mission profiles modeling future UAS operations. Results indicate DAA alerting distance threshold has a greater effect on DAA system performance than DAA alerting time threshold or ownship trajectory prediction method. Further analysis on the alert lead time (time in advance of predicted loss of well clear at which a DAA alert is first issued) indicated a strong positive correlation between alert lead time and DAA system performance (i.e. the ability of the UAS pilot to maneuver the unmanned aircraft to remain well clear). While bigger distance thresholds had beneficial effects on alert lead time and missed alert rate, it also generated a higher rate of false alerts. In the design and development of DAA alerting and guidance systems, therefore, the positive and negative effects of false alerts and missed alerts should be carefully considered to achieve acceptable alerting system performance by balancing false and missed alerts. The results and methodology presented in this study are expected to help stakeholders, policymakers and standards committees define the appropriate setting of DAA system parameter thresholds for UAS that ensure safety while minimizing operational impacts to the NAS and equipage requirements for its users before DAA operational performance standards can be finalized.

UAS Detect-and-Avoid (DAA) System↗
NASA NTRS2016

Overview of Sense and Avoid/Separation Assurance Interoperability (SSI)

The UAS in the NAS project is studying the minimum operational performance standards for unmanned aerial systems (UAS's) detect-and-avoid (DAA) system in order to operate in the National Airspace System. The project is executing a series of fast-time simulation, human-in-the-loop experiments, and flight test in support of this effort. Officials of the Swiss Government (Peter Winter, Director of Aeronautical Systems, and Michel Godinat, Head of the Certification Office) are visiting NASA to become more familiar with the advances being made to integrate UAS into civilian airspace. This briefing will review the activities and results over the past 3 years within the Separation AssuranceSense-and-Avoid Interoperability team.

Santiago, Confesor↗
NASA NTRS2015

Proposal for a Sample Pilot Response Model

Regulations to establish operational and performance requirements for unmanned aircraft systems (UAS) are being developed by a consortium of government, industry and academic institutions. Those requirements will apply to the new detect and avoid (DAA) systems and other equipment necessary to integrate UAS with the National Airspace System (NAS) and are determined according to their contribution to the overall safety case for such an integration. In order to perform end-to-end verification and validation of DAA requirements, a committee accepted pilot response model is needed to emulate UAS pilots in closed-loop Monte Carlo fast-time simulations. This briefing focuses on background of the pilot model used in previous Airspace Concept Evaluation System studies, identification of strengths and weaknesses in that model, and proposals on model improvements. The briefing also looks to build consensus around a common architecture and assumptions for modeling the pilot for UAS DAA systems.

Santiago, Confesor↗
NASA NTRS2015

NASA ACES V&V Alignment Briefing

Regulations to establish operational and performance requirements for unmanned aircraft systems (UAS) are being developed by a consortium of government, industry and academic institutions. Those requirements will apply to the new detect and avoid (DAA) systems and other equipment necessary to integrate UAS with the National Airspace System (NAS) and are determined according to their contribution to the overall safety case for such an integration. This briefing focuses on providing an overview of the Airspace Concept Evaluation System (ACES) platform, review of detect-and-avoid models incorprated in ACES, sumamry of two planned ACES studies, and a way forward to impact the SC-228 VV plan.

Santiago, Confesor↗
NASA NTRS2015

Analysis of UAS DAA Alerting in Fast-Time Simulations without DAA Mitigation

Realization of the expected proliferation of Unmanned Aircraft System (UAS) operations in the National Airspace System (NAS) depends on the development and validation of performance standards for UAS Detect and Avoid (DAA) Systems. The RTCA Special Committee 228 is charged with leading the development of draft Minimum Operational Performance Standards (MOPS) for UAS DAA Systems. NASA, as a participating member of RTCA SC-228 is committed to supporting the development and validation of draft requirements for DAA alerting system performance. A recent study conducted using NASA's ACES (Airspace Concept Evaluation System) simulation capability begins to address questions surrounding the development of draft MOPS for DAA alerting systems. ACES simulations were conducted to study the performance of alerting systems proposed by the SC-228 DAA Alerting sub-group. Analysis included but was not limited to: 1) correct alert (and timeliness), 2) false alert (and severity and duration), 3) missed alert, and 4) probability of an alert type at the time of loss of well clear. The performance of DAA alerting systems when using intent vs. dead-reckoning for UAS ownship trajectories was also compared. The results will be used by SC-228 to inform decisions about the surveillance standards of UAS DAA systems and future requirements development and validation efforts.

alerting↗
NASA NTRS2015

Analysis of UAS DAA Surveillance in Fast-Time Simulations without DAA Mitigation

Realization of the expected proliferation of Unmanned Aircraft System (UAS) operations in the National Airspace System (NAS) depends on the development and validation of performance standards for UAS Detect and Avoid (DAA) Systems. The RTCA Special Committee 228 is charged with leading the development of draft Minimum Operational Performance Standards (MOPS) for UAS DAA Systems. NASA, as a participating member of RTCA SC-228 is committed to supporting the development and validation of draft requirements for DAA surveillance system performance. A recent study conducted using NASA's ACES (Airspace Concept Evaluation System) simulation capability begins to address questions surrounding the development of draft MOPS for DAA surveillance systems. ACES simulations were conducted to study the performance of sensor systems proposed by the SC-228 DAA Surveillance sub-group. Analysis included but was not limited to: 1) number of intruders (both IFR and VFR) detected by all sensors as a function of UAS flight time, 2) number of intruders (both IFR and VFR) detected by radar alone as a function of UAS flight time, and 3) number of VFR intruders detected by all sensors as a function of UAS flight time. The results will be used by SC-228 to inform decisions about the surveillance standards of UAS DAA systems and future requirements development and validation efforts.

unmanned aircraft systems↗
NASA NTRS2015

Java Architecture for Detect and Avoid Extensibility and Modeling

Unmanned aircraft will equip with a detect-and-avoid (DAA) system that enables them to comply with the requirement to "see and avoid" other aircraft, an important layer in the overall set of procedural, strategic and tactical separation methods designed to prevent mid-air collisions. This paper describes a capability called Java Architecture for Detect and Avoid Extensibility and Modeling (JADEM), developed to prototype and help evaluate various DAA technological requirements by providing a flexible and extensible software platform that models all major detect-and-avoid functions. Figure 1 illustrates JADEM's architecture. The surveillance module can be actual equipment on the unmanned aircraft or simulators that model the process by which sensors on-board detect other aircraft and provide track data to the traffic display. The track evaluation function evaluates each detected aircraft and decides whether to provide an alert to the pilot and its severity. Guidance is a combination of intruder track information, alerting, and avoidance/advisory algorithms behind the tools shown on the traffic display to aid the pilot in determining a maneuver to avoid a loss of well clear. All these functions are designed with a common interface and configurable implementation, which is critical in exploring DAA requirements. To date, JADEM has been utilized in three computer simulations of the National Airspace System, three pilot-in-the-loop experiments using a total of 37 professional UAS pilots, and two flight tests using NASA's Predator-B unmanned aircraft, named Ikhana. The data collected has directly informed the quantitative separation standard for "well clear", safety case, requirements development, and the operational environment for the DAA minimum operational performance standards. This work was performed by the Separation Assurance/Sense and Avoid Interoperability team under NASA's UAS Integration in the NAS project.

unmanned aircraft systems↗
NASA NTRS2015

Pilot Evaluation of a UAS Detect-and-Avoid System's Effectiveness in Remaining Well Clear

Unmanned aircraft will equip with a detect-and-avoid (DAA) system that allows them to comply with the requirement to see and avoid other aircraft, an important layer in the overall set of procedural, strategic and tactical separation methods designed to prevent mid-air collisions. Although the effectiveness of the DAA system will be set to a minimum threshold by regulators, different combinations of algorithms, displays and procedures could be used to meet that minimum. The research presented in this paper indicates the effectiveness of the combined pilot-DAA system as a function of the DAA design requirements and provides data that may be used to model the behavior of pilots when employing such systems. Over the course of two simulations 21 professional UAS pilots evaluated eight different DAA system designs and metrics were collected on their ability to maintain the well clear separation standard. The independent variables were the time horizon at which pilots were alerted to potential losses of well clear, the location of the traffic display, and the tools and informational elements available on the display to aid the pilot in detecting and resolving those potential losses. In the second experiment the UAS encountered two categories of aircraft: those equipped with simulated transponders that could be seen dozens of miles away and those without that were only detectable by a simulated radar within a range of six nautical miles. The data indicate that integrating the traffic display with the primary mission map directly in front of the pilot reduced the frequency of losses of well clear. Improved detection and resolution tools, including explicit maneuver guidance and a trial planning capability, had less of an effect in reducing the frequency of losses but significantly reduced the time in loss when they occurred. The amount of warning time provided to the pilot had a strong effect on their ability to remain well clear: when alerts were first presented with less than about 15 seconds to a predicted loss of well clear pilots were able to maneuver successfully in only 26 percent of encounters, whereas they were about 83 percent successful when they had more than 15 seconds. Pilots' ability to separate from the two categories of aircraft was nearly the same after accounting for the amount of alert time provided in each encounter, although the limited surveillance volume for the non-transponder equipped aircraft meant alerts tended to occur later and therefore were more difficult to resolve.

loss of well clear↗
NASA NTRS2015

Characteristics of a Well Clear Definition and Alerting Criteria for Encounters Between UAS and Manned Aircraft in Class E Airspace

Unmanned aircraft systems (UAS) will be required to equip with a detect-and-avoid (DAA) system in order to satisfy the federal aviation regulations to remain well clear of other aircraft. For a DAA system to satisfy the requirement to stay well clear of other airborne traffic, a quantitative definition of well clear needs to be defined and evaluated. This study investigates the implications of UAS using proposed well clear definitions as a separation standard for conducting operations in the National Airspace System (NAS). The first analysis considers three well clear definitions and presents the relative state conditions of intruder aircraft as they encroach upon the well clear boundary. The second analysis focuses on the definition of the alerting criteria needed to inform the UAS operator of a potential loss of well clear. All analyses are conducted in a NAS-wide fast-time simulation environment using UAS aircraft models, proposed UAS missions, and historical air defense radar data to populate the background traffic operating under visual flight rules. The results presented in this study inform the safety case, requirements development, and the operational environment for DAA minimum operational performance standards.

Safety↗
NASA NTRS2015

Airspace Safety Threshold Study: NAS-Wide Encounter Rate Evaluation Using Historical Radar Data and ACES

Realization of the expected proliferation of Unmanned Aircraft System (UAS) operations in the National Airspace System (NAS) depends on the development and validation of performance standards for UAS Detect and Avoid (DAA) Systems. The RTCA Special Committee 228 (SC-228) is charged with leading the development of draft Minimum Operational Performance Standards (MOPS) for UAS DAA Systems. NASA, as a participating member of RTCA SC-228 is committed to supporting the development and validation of draft requirements as well as the safety substantiation and end-to-end assessment of DAA system performance. With regard to the safety aspect being studied by the SC-228 DAA Safety sub-group, NASA has conducted a study using the ACES (Airspace Concept Evaluation System) simulation capability to determine: 1) the rate at which IFR aircraft encounter other IFR and VFR aircraft, and 2) the rate at which UAS aircraft encounter VFR aircraft as well as the corresponding encounter geometries. Five different separation thresholds were used (two for encounter and one each for well-clear, near mid-air collision, and closest point of approach). The results will be used by the SC-228 DAA Safety sub-group to inform decisions about the safety aspect of UAS DAA systems and future requirements development and validation efforts.

encounter geometry↗
NASA NTRS2014

Pilot Evaluation of a UAS Detect-and-Avoid System's Effectiveness in Remaining Well Clear

Unmanned aircraft will equip with a detect-and-avoid (DAA) system that enables them to comply with the requirement to "see and avoid" other aircraft, an important layer in the overall set of procedural, strategic and tactical separation methods designed to prevent mid-air collisions. Regulators will establish minimum operating standards for DAA effectiveness, but different combinations of algorithms, displays and procedures could be used to meet those standards. The research presented in this paper indicates the effectiveness of the combined pilot-DAA system as a function of the DAA design requirements and provides data that may be used to model the behavior of pilots when employing such systems. Two simulations involving 21 professional unmanned aircraft system (UAS) pilots evaluated eight different DAA system designs in order to assess their ability to maintain the "well clear" separation standard, i.e., the state of maintaining a safe distance from other aircraft that would not normally cause the initiation of a collision avoidance maneuver by either aircraft. When the traffic display was integrated with the primary mission map directly in front of the pilot, there were fewer losses of well clear. Greater warning time provided to the pilot was strongly correlated with success in remaining well clear. Pilots' ability to separate from aircraft with cooperative and non-cooperative surveillance systems was nearly the same after accounting for the amount of alert time provided in each encounter, although the limited surveillance volume for the airborne-equipped aircraft meant alerts tended to occur later and therefore were more difficult to resolve.

detect-and-avoid↗
NASA NTRS2014

Investigating the Impacts of a Separation Standard for UAS Operations in Enroute and Transition Airspace

Unmanned aircraft systems will be required to equip with a detect and avoid system in order to satisfy the federal aviation regulations to remain well clear of other aircraft. To comply with regulations in today’s operations manned aircraft must “see and avoid” other aircraft and use subjective judgment to determine whether those aircraft are well clear. For a detect-and- avoid (DAA) system to satisfy the requirement to stay well clear, a quantitative definition of well clear needs to be defined and evaluated. Definitions for the boundary of well clear have been proposed by the Unmanned Aircraft System (UAS) Executive Committee Science and Research Panel (SaRP) and the Radio Technical Commission for Aeronautics (RTCA) Special Committee 228 on Detect and Avoid Systems. This study investigates the interoperability implications of UAS using proposed well clear definitions as a separation standard for conducting operations in the national airspace system. The first analysis in the study focuses on the effect of variations in well clear definition parameters on the rate of losses of well clear per flight hour. The second analysis considers three well clear definitions and presents the relative state conditions of intruder aircraft as they encroach upon the well clear boundary. The third analysis focuses on the definition of the alerting criteria needed to inform the UAS operator of a potential loss of well clear. All three analyses are conducted in a NAS-wide fast-time simulation environment using UAS aircraft models, proposed UAS missions, and historical air defense radar data to populate the background traffic operating under visual flight rules. The results from the three analyses presented in this study inform the safety case, requirements development, and the operational environment for the DAA minimum operational performance standards.

UAS Seperation Standard↗