Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Audit”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 505 records · Page 28

Early Operations Flight Correlation of the Lunar Laser Communications Demonstration (LLCD) on the Lunar Atmosphere and Dust Environment Explorer (LADEE)

The Lunar Atmosphere and Dust Environment Explorer (LADEE) mission launched on September 7, 2013 with a one month cruise before lunar insertion. The LADEE spacecraft is a power limited, octagonal, composite bus structure with solar panels on all eight sides with four vertical segments per side and 2 panels dedicated to instruments. One of these panels has the Lunar Laser Communications Demonstration (LLCD), which represents a furthering of the laser communications technology demonstration proved out by the Lunar Reconnaissance Orbiter (LRO). LLCD increases the bandwidth of communication to and from the moon with less mass and power than LROs technology demonstrator. The LLCD Modem and Controller boxes are mounted to an internal cruciform composite panel and have no dedicated radiator. The thermal design relies on power cycling of the boxes and radiation of waste heat to the inside of the panels, which then reject the heat when facing cold space. The LADEE mission includes a slow roll and numerous attitudes to accommodate the challenging thermal requirements for all the instruments on board. During the cruise phase, the internal Modem and Controller avionics for LLCD were warmer than predicted by more than modeling uncertainty would suggest. This caused concern that if the boxes were considerably warmer than expected while off, they would also be warmer when operating and could limit the operational time when in lunar orbit. The thermal group at Goddard Space Flight Center evaluated the models and design for these critical avionics for LLCD. Upon receipt of the spacecraft models and audit was performed and data was collected from the flight telemetry to perform a sanity check of the models and to correlate to flight where possible. This paper describes the efforts to correlate the model to flight data and to predict the thermal performance when in lunar orbit and presents some lessons learned.

LADEE↗

Radio Frequency Identification for Space Habitat Inventory and Stowage Allocation Management

To date, the most extensive space-based inventory management operation has been the International Space Station (ISS). Approximately 20,000 items are tracked with the Inventory Management System (IMS) software application that requires both flight and ground crews to update the database daily. This audit process is manually intensive and laborious, requiring the crew to open cargo transfer bags (CTBs), then Ziplock bags therein, to retrieve individual items. This inventory process contributes greatly to the time allocated for general crew tasks.

Wagner, Carole Y.↗

The Use of UML for Software Requirements Expression and Management

It is common practice to write English-language "shall" statements to embody detailed software requirements in aerospace software applications. This paper explores the use of the UML language as a replacement for the English language for this purpose. Among the advantages offered by the Unified Modeling Language (UML) is a high degree of clarity and precision in the expression of domain concepts as well as architecture and design. Can this quality of UML be exploited for the definition of software requirements? While expressing logical behavior, interface characteristics, timeliness constraints, and other constraints on software using UML is commonly done and relatively straight-forward, achieving the additional aspects of the expression and management of software requirements that stakeholders expect, especially traceability, is far less so. These other characteristics, concerned with auditing and quality control, include the ability to trace a requirement to a parent requirement (which may well be an English "shall" statement), to trace a requirement to verification activities or scenarios which verify that requirement, and to trace a requirement to elements of the software design which implement that requirement. UML Use Cases, designed for capturing requirements, have not always been satisfactory. Some applications of them simply use the Use Case model element as a repository for English requirement statements. Other applications of Use Cases, in which Use Cases are incorporated into behavioral diagrams that successfully communicate the behaviors and constraints required of the software, do indeed take advantage of UML's clarity, but not in ways that support the traceability features mentioned above. Our approach uses the Stereotype construct of UML to precisely identify elements of UML constructs, especially behaviors such as State Machines and Activities, as requirements, and also to achieve the necessary mapping capabilities. We describe this approach in the context of a space-based software application currently under development at the Jet Propulsion Laboratory.

model-based engineering↗

NASA Human Spaceflight Scenarios - Do All Our Models Still Say No?

Historically, NASA human spaceflight planning has included healthy doses of life cycle cost analysis. Planners put projects and their cost estimates in a budget context. Estimated costs became expected budgets. Regardless, real budgets rarely matched expectations. So plans would come and go as NASA canceled projects. New projects would arise and the cycle would begin again. Repeatedly, NASA schedule and performance ambitions come up against costs growing at double-digit rates while budgets barely rise a couple of percent a year. Significant skepticism greets proposed NASA programs at birth, as cost estimates for new projects are traditionally very high, and worse, far off the mark for those carried forward. In this environment the current "capability driven framework" for NASA human spaceflight evolved, where long term life cycle cost analysis are even viewed as possibly counter-productive. Here, a space exploration project, for example the Space Launch System, focuses on immediate goals. A life cycle is that of a project, not a program, and for only that span of time to a near term milestone like a first test launch. Unfortunately, attempting to avoid some pitfalls in long-term life cycle cost analysis breeds others. Government audits have noted that limiting the scope of cost analysis "does not provide the transparency necessary to assess long-term affordability" making it difficult to understand if NASA "is progressing in a cost-effective and affordable manner." Even in this short-term framework, NASA realizes the importance of long-term considerations, that it must "maximize the efficiency and sustainability of the Exploration Systems development programs", that this is "critical to free resources for re-investment...such as other required deep space exploration capabilities." Assuming the value of long-term life cycle cost analysis, where due diligence meets reconnaissance, and accepting past shortcomings, the work here approaches life cycle cost analysis for human spaceflight differently. 1) If costs have traditionally been so high that adding them up is discouraging, are there any new facts on the ground offering paths to significantly lower costs? 2) If NASA's spaceflight budget and process is an over-arching constraint, with its planning limitations favoring short-term outlooks, is there a way to step outside the budget box? 3) If life cycle answers have historically been too uncertain to be useful, is there a process where stakeholders gain valuable insights merely from emphasizing a common understanding around questions? We analyze the potential life cycle cost of assorted NASA human spaceflight architectures - an architecture as a sum of individual systems, working together. With the prior questions of high costs, limited budgets and uncertainties in mind, public private partnerships are central in these architectures. The cost data for current commercial public private partnerships is encouraging, as are cost estimates for future partnership approaches beyond low Earth orbit. Private capital, directly or indirectly, an ingredient of public private partnerships, may be a significant factor in finding a path around the limits of the NASA spaceflight budget. Also, understanding and reviewing the pros, cons and uncertainties of assorted architectures can assist in developing a common understanding around key questions as important if not more so than the numbers and answers. Lastly, a scenario planning technique is briefly explored that can mature a common understanding about the agencies situation at hand and how diverse stakeholders can go forward together. Scenario planning, rather than focusing on answers, places emphasis on stakeholders developing a common understanding about the future. Putting aside costs, this is especially true of questions about sustainability and growth, results, benefits and expectations. While efficiency exercises or analysis look to reduce resources in one place to apply them elsewhere, moving around slices in a pie, scenario planning can get at the heart of the matter, growing the pie, transforming it, and making the pieces relevant. Especially important is the question of sustainability for different scenarios in the broad sense of the word - not just the narrow ability to survive or continue, but also the ability to adapt, prosper and grow.

space systems life cycle costs↗

Commercial Off-The-Shelf (COTS) Parts Risk and Reliability User and Application Guide

All COTS parts are not created equal. Because they are not created equal, the notion that one can force the commercial industry to follow a set of military specifications and standards, along with the certifications, audits and qualification commitments that go with them, is unrealistic for the sale of a few parts. The part technologies that are Defense Logistics Agency (DLA) certified or Military Specification (MS) qualified, are several generations behind the state-of-the-art high-performance parts that are required for the compact, higher performing systems for the next generation of spacecraft and instruments. The majority of the part suppliers are focused on the portion of the market that is producing high-tech commercial products and systems. To that end, in order to compete in the high performance and leading edge advanced technological systems, an alternative approach to risk assessment and reliability prediction must be considered.

White, Mark↗

Agency Agreements Process Champion Support Intern

This document will provide information on the 2018 Spring semester NIFS Intern who represented the Office of Chief Financial Officer (OCFO) as a Reimbursable Accountant at Kennedy Space Center (KSC). This intern supported the Agency Agreements Process Champions and Team Lead, Susan Kroskey, Sandy Massey and Mecca Murphy, with major initiatives to advance the KSC OCFO's vision of creating and innovating healthy financial management practices that maximize the value of resources entrusted to NASA. These initiatives include, but are not limited to: updating the Agency Guidance and NASA Procedural Guidance 9090.1 Agreements, implementing a new budget structure to be utilized across all centers, submitting a Call Request (CRQ) to enhance non-federal customer reporting, initiating a discussion to incorporate a 3-year funding program for NASA agreements, and undertaking the Office of Inspector General (OIG) Audit. In support of these initiatives, this intern identified technical methods to enhance and reduce the workload of financial processes for reimbursable and non-reimbursable agreements, prepared reports in support of accounting functions, and performed administrative work and miscellaneous technical tasks in support of the OCFO as requested. In conclusion of the internship, the intern will become knowledgeable on reimbursable accounting, reimbursable policy, types of reimbursable agreements, the agreements process, estimated pricing reports, and the roles and responsibilities of the Financial Accounting and Financial Services offices.

Process Champion↗

Quality Assurance Surveillance Tool Designed for SLS EM-1 Certificate of Flight Readiness

Exploration Mission-1 (EM-1) is an uncrewed mission that is launching on the Space Launch System (SLS) Block 1 vehicle. This is a critical flight test for the agency's human deep space exploration goals. Exploration Ground Systems (EGS) Quality (SA-F2) is responsible for providing Quality Assurance Surveillance Plans (QASPs) for NASA contracts and facilities. A QASP assures products and services are provided to EGS in accordance with the contract requirements. In order to encapsulate the multiple KSC Safety and Mission Assurance surveillance and audit activities that need to be accomplished specifically for EM-1 Certificate of Flight Reediness (CoFR), a QASP tool needed to be implemented. This tool would be able to intuitively track Work Authorization Documents (WADs) under development by Test Operations Support Contract (TOSC), reviewed and accepted WADs, Government Mandatory Inspection Points (GMIPs), Nonconformances (NCs), Material Review Board (MRB) Status, Corrective Actions, Alterations, Deviations, and Waivers for flight hardware and Ground Support Equipment (GSE). This would help ensure that all SLS EM-1 flight hardware and related GSE surveillances and requirements are being recognized and completed with zero constraints identified for the mission.

Morris, Patrick↗

Architecture Modeling on the Europa Project

In 2015 NASA chartered a partnership between the Jet Propulsion Laboratory (JPL) and the Johns Hopkins Applied Physics Laboratory (APL) to begin planning a mission to study the Jovian moon Europa. The project has adopted a Model-Based Systems Engineering (MBSE) approach to its architecting process since its early formulation, developing certain modeling practices and tools as needed, with the expectation that this process would result in a more consistent and verifiable architecture than with a more traditional document-based approach. A sound architecture is essential to provide the rationale for requirements on the system design, and to define the trade space of acceptable design points within which technical and programmatic concerns as well as project objectives can be addressed. This paper provides an overview of the framework used by the Europa project to describe the mission architecture and discusses how a system model was instrumental in providing a single-source-of-truth for this description. Several key modeling patterns to represent the architecture are presented, along with audit methods to ensure the consistency and the correctness of the model. Finally, the benefits and challenges of using a model-based approach to generate traditional requirements documents and other gate products are assessed.

Dubos, Gregory F.↗

Exposing Hidden Parts of the SE Process: MBSE Patterns and Tools for Tracking and Traceability

An interesting benefit of applying Model-Based Systems Engineering (MBSE) is that the rigor and coordination intrinsic to MBSE forces us to apply Systems Engineering to our own traditional activities, processes, and products, which results in richer, more expressive models, more powerful reasoning, and a clearer and more effective Systems Engineering (SE) process. Our MBSE frameworks and languages contain semantic richness sufficient to describe our systems at any particular point in time, often with an emphasis on the description of the system at major milestones. This is unarguably a real asset. However, when we apply MBSE in service of missions that are in development, rapidly evolving, of a larger scale, and where interpersonal communication is a critical part of the design process, we discover that our frameworks and languages are still not quite rich enough to enable us to ask the kinds of questions and get the kinds of answers we want in order to address the concerns of day to day work. This paper will discuss some patterns and tools we have developed to help address some of the not-always-explicit SE concerns that we have identified through our MBSE work. Particularly, this paper will discuss flexible yet practical methods for defining and capturing maturity, workflow, and agreement traceability within our system models, extensible ways to perform and track model audits, and ways to report and interact with this knowledge in the context of MBSE applied to support NASA’s Europa Project.

Jackson, Maddalena↗

Exposing Hidden Parts of the SE Process: MBSE Patterns and Tools for Tracking and Traceability

An interesting benefit of applying Model-Based Systems Engineering (MBSE) is that the rigor and coordination intrinsic to MBSE forces us to apply Systems Engineering to our own traditional activities, processes, and products, which results in richer, more expressive models, more powerful reasoning, and a clearer and more effective Systems Engineering (SE) process. Our MBSE frameworks and languages contain semantic richness sufficient to describe our systems at any particular point in time, often with an emphasis on the description of the system at major milestones. This is unarguably a real asset. However, when we apply MBSE in service of missions that are in development, rapidly evolving, of a larger scale, and where interpersonal communication is a critical part of the design process, we discover that our frameworks and languages are still not quite rich enough to enable us to ask the kinds of questions and get the kinds of answers we want in order to address the concerns of day to day work. This paper will discuss some patterns and tools we have developed to help address some of the not-always-explicit SE concerns that we have identified through our MBSE work. Particularly, this paper will discuss flexible yet practical methods for defining and capturing maturity, workflow, and agreement traceability within our system models, extensible ways to perform and track model audits, and ways to report and interact with this knowledge in the context of MBSE applied to support NASA’s Europa Project

Jackson, Maddalena↗

SAGE III ISS Contamination Monitoring Package: Observations in Orbit

The Stratospheric Aerosol and Gas Experiment III (SAGE III) telescope and instrument assembly employ the methods of solar occultation and lunar occultation to retrieve near-global vertical profiles of atmospheric ozone, water vapor, nitrogen dioxide, aerosol extinctions, and other gaseous species and atmospheric state parameters. The SAGE III grating spectrometer measures light within the spectral range of 280 nm to 1037 nm at approximately 1 nm resolution, but retrievals in the Ultraviolet (UV) are particularly sensitive to contamination of the optical train. Therefore, a contamination door that contains a quartz optical window can be closed over the telescope aperture during periods of enhanced external contaminant flux. This optically transparent window permits continued science event acquisition at an acceptably diminished signal-to-noise ratio, which is expected to decline with ongoing accretion of contaminant material. To date, this impact has been short term, and science quality through the window returns to baseline performance after a contamination source is removed and spontaneous desorption removes material from the low-affinity quartz surface. Two Contamination Monitoring Packages (CMPs) consisting of eight Thermoelectric Quartz Crystal Microbalances (TQCMs) from QCM Research provide characterization and redundant monitoring of contaminant deposition from the 2π steradian solid angle on the payload side of the Expedite the Processing of Experiments to the Space Station (ExPRESS) Payload Adapter. CMP data are closely examined by the SAGE III team to determine when the contamination door should remain closed during science events and in what direction the instrument assembly scan head should stow when not acquiring science measurements. Additionally, should the CMPs indicate an unacceptable accretion rate, the ight computer will close the contamination door as part of the automatic fault detection system. Along with spectrometer measurements of the quartz window's transmission, the payload CMPs enable auditing of the mission contamination budget. The process of Thermogravimetric Analysis (TGA) can be used to help identify chemical constituents accreted on the CMP sensors. To be presented here along with an explanation of the CMP systems are the first two and a half years of observations of the contaminant deposition environment around the payload in quiescence and during special events like docking vehicles.

Tyler T Dawson↗

SAGE III/ISS Contamination Monitoring Package: Observations in Orbit

The Stratospheric Aerosol and Gas Experiment III (SAGE III) telescope and instrument assembly employ the methods of solar occultation and lunar occultation to retrieve near-global vertical profiles of atmospheric ozone, water vapor, nitrogen dioxide, aerosol extinctions, and other gaseous species and atmospheric state parameters. The SAGE III grating spectrometer measures light within the spectral range of 280 nm to 1037 nm at approximately 1 nm resolution, but retrievals in the Ultraviolet (UV) are particularly sensitive to contamination of the optical train. Therefore, a contamination door that contains a quartz optical window can be closed over the telescope aperture during periods of enhanced external contaminant flux. This optically transparent window permits continued science event acquisition at an acceptably diminished signal-to-noise ratio, which is expected to decline with ongoing accretion of contaminant material. To date, this impact has been short term, and science quality through the window returns to baseline performance after a contamination source is removed and spontaneous desorption removes material from the low-affinity quartz surface. Two Contamination Monitoring Packages (CMPs) consisting of eight Thermoelectric Quartz Crystal Microbalances (TQCMs) from QCM Research provide characterization and redundant monitoring of contaminant deposition from the 2pi steradian solid angle on the payload side of the Expedite the Processing of Experiments to the Space Station (ExPRESS) Payload Adapter. CMP data are closely examined by the SAGE III team to determine when the contamination door should remain closed during science events and in what direction the instrument assembly scan head should stow when not acquiring science measurements. Additionally, should the CMPs indicate an unacceptable accretion rate, the flight computer will close the contamination door as part of the automatic fault detection system. Along with spectrometer measurements of the quartz window's transmission, the payload CMPs enable auditing of the mission contamination budget. The process of Thermogravimetric Analysis (TGA) can be used to help identify chemical constituents accreted on the CMP sensors. To be presented here along with an explanation of the CMP systems are the first two and a half years of observations of the contaminant deposition environment around the payload in quiescence and during special events like docking vehicles.

Tyler T Dawson↗

HAT m:N Cognitive Task Analysis (CTA)

This Cognitive Task Analysis (CTA) study was designed to understand the capability of the m:N Tactical Operator (TO) interfaces developed by the Human-Autonomy Teaming Laboratory at NASA Ames to support operators responsible for simplified pilot operations of 100 independently operated small UAS (sUAS) in a constrained geographic area. The m:N sUAS TO interface includes a central Tactical Situation Display (TSD) digital map with moving icons reflecting the sUAS location and planned flight route. The interface also has two side panels. The left panel includes a tabular list of UAS assets and mission tasking, a list of recently viewed assets, and a list of events and alerts. The right panel includes a tabular list of UAS assets and their associated telemetry, text-based chat communication window, and a tabbed checklist window. This CTA was adapted from the incident-based applied cognitive task analysis (Militello & Hutton, 1998) and included demographics questions, scenario-based simulations, a task diagram and knowledge audit methods. In addition to examining the support provided by this m:N sUAS TO interface, this CTA study, conducted with aviation subject matter experts in analogous roles to the future tactical operator, was designed to illuminate and project likely cognitive requirements of the tactical operator. Interviewees participated in two scenario-based simulations using the m:N sUAS TO interfaces. In the first simulation, the interviewees supervised 12 sUAS operating in downtown San Diego, California transiting to and from a central sUAS Hive, restaurant locations, and customer drop off locations. Interviewees were asked to react to a UAS Volume Reservation (UVR) event with a two-phase impact on food delivery operations. In the second scenario, the interviewees supervised 100 sUAS operating in the same airspace and with the same mission. Interviewees used the interfaces to recognize and react to two sUAS air vehicle problems. After each scenario, we asked the interviewees a semi-structured list of questions to elicit their reflections about using the interfaces. Interviewees were confident in their ability to respond to two off-nominal situations in each simulation. Interviewees felt that, given high levels of automation on the sUAS, they would be able to manage the events without requiring additional support or handing off the sUAS to a colleague or supervisor in both the n=12 or n=100 sUAS settings. In the n=12 sUAS condition, interviewees used the center map to understand the asset location and progress along with mission tasking. An additional display window, Asset Telemetry, helped interviewees understand battery state and sUAS altitude. When the number of sUAS increased, interviewees altered their behavior. Rather than maintaining awareness of individual assets, interviewees appeared to become more reactive, managing exceptions. Interviewees reported that they spent less time looking at the nominal aircraft, and focused their attention primarily on the off-nominal aircraft. In addition, in the second simulation with n=100 sUAS, interviewees reported that they relied more on the side panels (Mission Timeline and Asset Telemetry) to gather information.

human-autonomy teaming↗

Modernizing NASA’s Space Flight Safety and Mission Success (S&MS) Assurance Framework In Line With Evolving Acquisition Strategies and Systems Engineering Practices

This paper presents the objectives-driven, case-based safety and mission success (S&MS) assurance framework being developed by the NASA Office of Safety and Mission Assurance (OSMA), including its motivations and its implementation via a S&MS Assurance Standard that is under development, supplemented by supporting standards including an S&MS Analysis Management Standard that is also under development. A need to evolve NASA’s S&MS assurance framework has emerged in recent years, resulting from the need to accommodate new acquisition models; the need to accommodate evolving systems engineering (SE) practices; the need to stipulate acceptable levels of S&MS risk; the need for improved integration of S&MS into SE; and the need for clearer risk acceptance accountability. The objectives-driven, case-based S&MS assurance framework proposed here is responsive to that need. Its key features include: • The establishment, by NASA Acquirers, of fundamental S&MS performance objectives that define limits of acceptability for the likelihoods that mission technical objectives will be accomplished and that people, assets, and environments put at risk by the mission will not be adversely affected; • The development and approval of Providers’ S&MS plans for meeting Acquirers’ S&MS performance objectives, including commitments to support Acquirer audit, investigation, and reporting needs; • The development, by Providers, of S&MS assurance cases that argue, supported by evidence, that the Provider has met, or is on track to meeting, the fundamental S&MS objectives; • The evaluation, throughout the program/project life cycle, of Provider S&MS assurance cases as the primary S&MS-related technical basis for Acquirer risk acceptance and the granting to the Provider of authority to proceed through the program/project life cycle. This proposed S&MS assurance framework is notable for its lack of prescription of traditional S&MS requirements and strategies such as defined failure tolerances, margins, or analysis requirements. Instead, Providers are given latitude to propose their own strategies for meeting the fundamental S&MS performance objectives, subject to independent review and Acquirer approval. The result is a framework for S&MS assurance that is at once both rigorous and flexible.

Assurance Case↗

Assessing Several Non-Traditional Data Sources for Value in Aviation Safety

The NASA System-Wide Safety (SWS) project and its predecessor projects have been developing Machine Learning (ML) algorithms for commercial aviation safety for many years. These algorithms have been applied to Flight Operations Quality Assurance (FOQA); radar track data (e.g., Threaded Track); and safety reports, including Aviation Safety Reporting System (ASRS) and Aviation Safety Action Plan (ASAP). SWS is working with partners to get access to other data that air carriers provide, such as maintenance data, and has been assisting carriers in working with other data, such as Line Operations Safety Audit (LOSA) data, using manual methods. However, the project has discussed whether there are other data that are not traditionally used in aviation safety analysis that may be useful. This paper discusses four sets of data and models that are not traditionally used in aviation safety but that have shown promise for such use. In the future, we plan to incorporate such data into ML algorithms to use with data that we have used before and determine the additional benefit that is actually achieved under different contexts from the inclusion of these non-traditional data sources.

Nikunj C. Oza↗

An Approach to Identifying Aspects of Positive Pilot Behavior within the Aviation Safety Reporting System

The National Airspace System (NAS) is constantly evolving as air traffic continues to ramp up to pre-pandemic numbers and projected to grow to unprecedented levels in the coming years. As well as increasing demand to the current system, emerging operations such as Unmanned Autonomous Systems are also expected to add to complexity in the airspace. To address these issues, the industry and government agencies supporting the NAS will need to rely upon additional automation and new technologies to address future operational requirements, while continuing to be a world-leading safe transportation system. As these new technologies are implemented, the system continues to rely on human pilots and controllers in the loop to monitor the system and intervene in situations the automation cannot handle. The goal of proactively addressing safety is of foremost concern to ensure passenger confidence. The industry has implemented various Safety Monitoring Systems to identify safety risks and proactively address them before they result in a serious incident or accident. One such program is the Aviation Safety Reporting System (ASRS). ASRS is a long-established system where pilots and controllers voluntarily and anonymously report safety incidents they experienced and observed during line operations by providing rich text narratives describing the events, the environment, and conditions leading to the safety event of concern. These narratives provide insight and context around events of interest and can be used to identify emerging problems. They can trigger investigations within Flight Operational Quality Assurance or Flight Data Monitoring programs. However, this process typically focuses on the adverse events and the unsafe aspects of the operations surrounding the reported or detected events. This perspective of investigating factors that went wrong around an adverse event is commonly referred to as Safety I. Alternatively, characterizing successful actions that operators perform every day under varying conditions that keep the system within safe operating bounds is a concept referred to as Safety II. The benefit of the Safety II view is that the scope is much larger than that of Safety I since a vast majority of the operations result in successful flights. Many of the successful techniques used to manage operational threats are not documented in standard operating procedures or taught during training. They are typically acquired over time by working with experienced pilots during line operations or in many cases after experiencing a problem for the first time and reacting to it in situ, drawing from years of experience to manage the threat. In an attempt to quantify these positive actions, we are proposing an approach to extracting key behaviors within ASRS reports that can support the Safety II concept. Our analysis assumes that ASRS reports contain some descriptions of corrective actions that operators performed to prevent a situation from leading to an accident. Leveraging recent advances in Natural Language Process modeling, we have developed an approach to extract positive sentiment from reports, embed these positive statements in a vector space where they can be numerically analyzed, and clustering these statements into similar contextual categories. From these contextualized categories we can attempt to summarized and distilled aspects of the positive behavior. The goal is to identify categories of behavior that describe consistent operator techniques that supports the Safety II concept. With this information, airlines may enable learning from these positive actions, or address procedures that need to be changed to avoid having pilots implement a workaround. These insights can provide a lens into what is “going right” in the operations that may otherwise not be known widely within the community. It is envisioned that this approach can be extended to other narrative programs such as Line Operation Safety Audit or Learning Improvement Team reports where similar observed behavior can be analyzed to extract positive actions and inform the overall operations.

NLP↗

An Approach to Identifying Aspects of Positive Pilot Behavior within the Aviation Safety Reporting System

The National Airspace System (NAS) is constantly evolving as air traffic continues to ramp up to pre-pandemic numbers and projected to grow to unprecedented levels in the coming years. As well as increasing demand to the current system, emerging operations such as Unmanned Autonomous Systems are also expected to add to complexity in the airspace. To address these issues, the industry and government agencies supporting the NAS will need to rely upon additional automation and new technologies to address future operational requirements, while continuing to be a world-leading safe transportation system. As these new technologies are implemented, the system continues to rely on human pilots and controllers in the loop to monitor the system and intervene in situations the automation cannot handle. The goal of proactively addressing safety is of foremost concern to ensure passenger confidence. The industry has implemented various Safety Monitoring Systems to identify safety risks and proactively address them before they result in a serious incident or accident. One such program is the Aviation Safety Reporting System (ASRS). ASRS is a long-established system where pilots and controllers voluntarily and anonymously report safety incidents they experienced and observed during line operations by providing rich text narratives describing the events, the environment, and conditions leading to the safety event of concern. These narratives provide insight and context around events of interest and can be used to identify emerging problems. They can trigger investigations within Flight Operational Quality Assurance or Flight Data Monitoring programs. However, this process typically focuses on the adverse events and the unsafe aspects of the operations surrounding the reported or detected events. This perspective of investigating factors that went wrong around an adverse event is commonly referred to as Safety I. Alternatively, characterizing successful actions that operators perform every day under varying conditions that keep the system within safe operating bounds is a concept referred to as Safety II. The benefit of the Safety II view is that the scope is much larger than that of Safety I since a vast majority of the operations result in successful flights. Many of the successful techniques used to manage operational threats are not documented in standard operating procedures or taught during training. They are typically acquired over time by working with experienced pilots during line operations or in many cases after experiencing a problem for the first time and reacting to it in situ, drawing from years of experience to manage the threat. In an attempt to quantify these positive actions, we are proposing an approach to extracting key behaviors within ASRS reports that can support the Safety II concept. Our analysis assumes that ASRS reports contain some descriptions of corrective actions that operators performed to prevent a situation from leading to an accident. Leveraging recent advances in Natural Language Process modeling, we have developed an approach to extract positive sentiment from reports, embed these positive statements in a vector space where they can be numerically analyzed, and clustering these statements into similar contextual categories. From these contextualized categories we can attempt to summarized and distilled aspects of the positive behavior. The goal is to identify categories of behavior that describe consistent operator techniques that supports the Safety II concept. With this information, airlines may enable learning from these positive actions, or address procedures that need to be changed to avoid having pilots implement a workaround. These insights can provide a lens into what is “going right” in the operations that may otherwise not be known widely within the community. It is envisioned that this approach can be extended to other narrative programs such as Line Operation Safety Audit or Learning Improvement Team reports where similar observed behavior can be analyzed to extract positive actions and inform the overall operations.

NLP↗

An Approach to Identifying Aspects of Positive Pilot Behavior within the Aviation Safety Reporting System

The National Airspace System (NAS) is constantly evolving as air traffic continues to ramp up to pre-pandemic numbers and projected to grow to unprecedented levels in the coming years. As well as increasing demand to the current system, emerging operations such as Unmanned Autonomous Systems are also expected to add to complexity in the airspace. To address these issues, the industry and government agencies supporting the NAS will need to rely upon additional automation and new technologies to address future operational requirements, while continuing to be a world-leading safe transportation system. As these new technologies are implemented, the system continues to rely on human pilots and controllers in the loop to monitor the system and intervene in situations the automation cannot handle. The goal of proactively addressing safety is of foremost concern to ensure passenger confidence. The industry has implemented various Safety Monitoring Systems to identify safety risks and proactively address them before they result in a serious incident or accident. One such program is the Aviation Safety Reporting System (ASRS). ASRS is a long-established system where pilots and controllers voluntarily and anonymously report safety incidents they experienced and observed during line operations by providing rich text narratives describing the events, the environment, and conditions leading to the safety event of concern. These narratives provide insight and context around events of interest and can be used to identify emerging problems. They can trigger investigations within Flight Operational Quality Assurance or Flight Data Monitoring programs. However, this process typically focuses on the adverse events and the unsafe aspects of the operations surrounding the reported or detected events. This perspective of investigating factors that went wrong around an adverse event is commonly referred to as Safety I. Alternatively, characterizing successful actions that operators perform every day under varying conditions that keep the system within safe operating bounds is a concept referred to as Safety II. The benefit of the Safety II view is that the scope is much larger than that of Safety I since a vast majority of the operations result in successful flights. Many of the successful techniques used to manage operational threats are not documented in standard operating procedures or taught during training. They are typically acquired over time by working with experienced pilots during line operations or in many cases after experiencing a problem for the first time and reacting to it in situ, drawing from years of experience to manage the threat. In an attempt to quantify these positive actions, we are proposing an approach to extracting key behaviors within ASRS reports that can support the Safety II concept. Our analysis assumes that ASRS reports contain some descriptions of corrective actions that operators performed to prevent a situation from leading to an accident. Leveraging recent advances in Natural Language Process modeling, we have developed an approach to extract positive sentiment from reports, embed these positive statements in a vector space where they can be numerically analyzed, and clustering these statements into similar contextual categories. From these contextualized categories we can attempt to summarized and distilled aspects of the positive behavior. The goal is to identify categories of behavior that describe consistent operator techniques that supports the Safety II concept. With this information, airlines may enable learning from these positive actions, or address procedures that need to be changed to avoid having pilots implement a workaround. These insights can provide a lens into what is “going right” in the operations that may otherwise not be known widely within the community. It is envisioned that this approach can be extended to other narrative programs such as Line Operation Safety Audit or Learning Improvement Team reports where similar observed behavior can be analyzed to extract positive actions and inform the overall operations.

NLP↗