SEARCH · Engineering Papers

Results for “data-driven method”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

NASA NTRS1992

Teamwork for Oversight of Processes and Systems (TOPS). Implementation guide for TOPS version 2.0, 10 August 1992

As the nation redefines priorities to deal with a rapidly changing world order, both government and industry require new approaches for oversight of management systems, particularly for high technology products. Declining defense budgets will lead to significant reductions in government contract management personnel. Concurrently, defense contractors are reducing administrative and overhead staffing to control costs. These combined pressures require bold approaches for the oversight of management systems. In the Spring of 1991, the DPRO and TRW created a Process Action Team (PAT) to jointly prepare a Performance Based Management (PBM) system titled Teamwork for Oversight of Processes and Systems (TOPS). The primary goal is implementation of a performance based management system based on objective data to review critical TRW processes with an emphasis on continuous improvement. The processes are: Finance and Business Systems, Engineering and Manufacturing Systems, Quality Assurance, and Software Systems. The team established a number of goals: delivery of quality products to contractual terms and conditions; ensure that TRW management systems meet government guidance and good business practices; use of objective data to measure critical processes; elimination of wasteful/duplicative reviews and audits; emphasis on teamwork--all efforts must be perceived to add value by both sides and decisions are made by consensus; and synergy and the creation of a strong working trust between TRW and the DPRO. TOPS permits the adjustment of oversight resources when conditions change or when TRW systems performance indicate either an increase or decrease in surveillance is appropriate. Monthly Contractor Performance Assessments (CPA) are derived from a summary of supporting system level and process-level ratings obtained from objective process-level data. Tiered, objective, data-driven metrics are highly successful in achieving a cooperative and effective method of measuring performance. The teamwork-based culture developed by TOPS proved an unequaled success in removing adversarial relationships and creating an atmosphere of continuous improvement in quality processes at TRW. The new working relationship does not decrease the responsibility or authority of the DPRO to ensure contract compliance and it permits both parties to work more effectively to improve total quality and reduce cost. By emphasizing teamwork in developing a stronger approach to efficient management of the defense industrial base TOPS is a singular success.

Strand, Albert A.↗

NASA NTRS2018

Development of a Ground Test and Analysis Protocol for NASA's NextSTEP Phase 2 Habitation Concepts

The NASA Next Space Technologies for Exploration Partnerships (NextSTEP) program is a public-private partnership model that seeks commercial development of deep space exploration capabilities to support human spaceflight missions around and beyond cislunar space. NASA first issued the Phase 1 NextSTEP Broad Agency Announcement to U.S. industries in 2014, which called for innovative cislunar habitation concepts that leveraged commercialization plans for low-Earth orbit. These habitats will be part of the Deep Space Gateway (DSG), the cislunar space station planned by NASA for construction in the 2020s. In 2016, Phase 2 of the NextSTEP program selected five commercial partners to develop ground prototypes. A team of NASA research engineers and subject matter experts (SMEs) have been tasked with developing the ground-test protocol that will serve as the primary means by which these Phase 2 prototypes will be evaluated. Since 2008, this core test team has successfully conducted multiple spaceflight analog mission evaluations utilizing a consistent set of operational tools, methods, and metrics to enable the iterative development, testing, analysis, and validation of evolving exploration architectures, operations concepts, and vehicle designs. The purpose of implementing a similar evaluation process for the Phase 2 Habitation Concepts is to consistently evaluate different commercial partner ground prototypes to provide data-driven, actionable recommendations for Phase 3. This paper describes the process by which the ground test protocol was developed and the objectives, methods, and metrics by which the NextSTEP Phase 2 Habitation Concepts will be rigorously and systematically evaluated. The protocol has been developed using both a top-down and bottom-up approach. Top-down development began with the Human Exploration and Operations Mission Directorate (HEOMD) exploration objectives and ISS Exploration Capability Study Team (IECST) candidate flight objectives. Strategic questions and associated rationales, derived from these candidate architectural objectives, provide the framework by which the ground-test protocol will address the DSG stack elements and configurations, systems and subsystems, and habitation, science, and EVA functions. From these strategic questions, high-level functional requirements for the DSG were drafted and associated ground-test objectives and analysis protocols were established. Bottom-up development incorporated objectives from NASA SMEs in autonomy, avionics and software, communication, environmental control and life support systems, exercise, extravehicular activity, exploration medical operations, guidance navigation and control, human factors and behavioral performance, human factors and habitability, logistics, Mission Control Center operations, power, radiation, robotics, safety and mission assurance, science, simulation, structures, thermal, trash management, and vehicle health. Top-down and bottom-up objectives were integrated to form overall functional requirements - ground-test objectives and analysis mapping. From this mapping, ground-test objectives were organized into those that will be evaluated through inspection, demonstration, analysis, subsystem standalone testing, and human-in-the-loop (HITL) testing. For the HITL tests, mission-like timelines, procedures, and flight rules have been developed to directly meet ground test objectives and evaluate specific functional requirements. Data collected from these assessments will be analyzed to determine the acceptability of habitation element configurations and the combinations of capabilities that will result in the best habitation platform to be recommended by the test team for Phase 3.

Gernhardt, Michael L.↗

NASA NTRS2003

Vector-Ordering Filter Procedure for Data Reduction

The vector-ordering filter (VOF) technique involves a procedure for sampling a large population of data vectors to select a subset of data vectors that fully characterize the state space of the large population. The VOF technique enables a large reduction of the volume of data that must be handled in the automated monitoring system and method discussed in the two immediately preceding articles. In so doing, the VOF technique enables the development of data-driven mathematical models of a monitored asset from sets of data that would otherwise exceed the memory capacities of conventional engineering computers. Data-driven mathematical models have been shown to offer high fidelity for purposes of control and monitoring of assets. In practice, a collection of asset-operating observations is acquired with the intention that the collection contain observations characteristic of the full dynamic range of operation of the asset. Often, such a collection contains an extremely large number of observations, many of which are redundant. The VOF technique fills the need for a means to extract, from the original collection of observational data, a reduced data matrix that excludes redundant data while maintaining the full statistical character and dynamic range of the original data. The reduced data matrix can then be used as the input data for development of a mathematical model of the monitored asset, or as training data for a neural-network substitute for an explicit mathematical model of the asset. Alternatively, the reduced data matrix can, itself, be used directly as a mathematical model of the monitored asset, as is commonly done in multivariate state-estimation techniques. The original data are collected from the asset over a range of operating states and are put in matrix form. Each column vector in the original data matrix represents the signal values acquired at a particular operational state of the asset. Thus, the number of columns of the original data matrix equals the number of observed states and the number of rows in this matrix equals the number of signals acquired at each observation. In the VOF technique, one extracts the reduced data matrix from the original data matrix through the selection of a representative subset of the column (state) vectors.

Bickford, Randall L.↗

NASA NTRS2007

Fundamental Technology Development for Gas-Turbine Engine Health Management

Integrated vehicle health management technologies promise to dramatically improve the safety of commercial aircraft by reducing system and component failures as causal and contributing factors in aircraft accidents. To realize this promise, fundamental technology development is needed to produce reliable health management components. These components include diagnostic and prognostic algorithms, physics-based and data-driven lifing and failure models, sensors, and a sensor infrastructure including wireless communications, power scavenging, and electronics. In addition, system assessment methods are needed to effectively prioritize development efforts. Development work is needed throughout the vehicle, but particular challenges are presented by the hot, rotating environment of the propulsion system. This presentation describes current work in the field of health management technologies for propulsion systems for commercial aviation.

Mercer, Carolyn R.↗

NASA NTRS2011

Prognostics of Power Mosfets Under Thermal Stress Accelerated Aging Using Data-Driven and Model-Based Methodologies

An approach for predicting remaining useful life of power MOSFETs (metal oxide field effect transistor) devices has been developed. Power MOSFETs are semiconductor switching devices that are instrumental in electronics equipment such as those used in operation and control of modern aircraft and spacecraft. The MOSFETs examined here were aged under thermal overstress in a controlled experiment and continuous performance degradation data were collected from the accelerated aging experiment. Dieattach degradation was determined to be the primary failure mode. The collected run-to-failure data were analyzed and it was revealed that ON-state resistance increased as die-attach degraded under high thermal stresses. Results from finite element simulation analysis support the observations from the experimental data. Data-driven and model based prognostics algorithms were investigated where ON-state resistance was used as the primary precursor of failure feature. A Gaussian process regression algorithm was explored as an example for a data-driven technique and an extended Kalman filter and a particle filter were used as examples for model-based techniques. Both methods were able to provide valid results. Prognostic performance metrics were employed to evaluate and compare the algorithms.

Electrolytic capacitors↗

NASA NTRS1992

Advanced software development workstation. Comparison of two object-oriented development methodologies

This report is an attempt to clarify some of the concerns raised about the OMT method, specifically that OMT is weaker than the Booch method in a few key areas. This interim report specifically addresses the following issues: (1) is OMT object-oriented or only data-driven?; (2) can OMT be used as a front-end to implementation in C++?; (3) the inheritance concept in OMT is in contradiction with the 'pure and real' inheritance concept found in object-oriented (OO) design; (4) low support for software life-cycle issues, for project and risk management; (5) uselessness of functional modeling for the ROSE project; and (6) problems with event-driven and simulation systems. The conclusion of this report is that both Booch's method and Rumbaugh's method are good OO methods, each with strengths and weaknesses in different areas of the development process.

Izygon, Michel E.↗

NASA NTRS2013

Semi-Supervised Eigenbasis Novelty Detection

Recent discoveries in high-time-resolution radio astronomy data have focused attention on a new class of events. Fast transients are rare pulses of radio frequency energy lasting from microseconds to seconds that might be produced by a variety of exotic astrophysical phenomena. For example, X-ray bursts, neutron stars, and active galactic nuclei are all possible sources of short-duration, transient radio signals. It is difficult to anticipate where such signals might appear, and they are most commonly discovered through analysis of high-time- resolution data that had been collected for other purposes. Transients are often faint and difficult to detect, so improved detection algorithms can directly benefit the science yield of all such commensal monitoring. A new detection algorithm learns a low-dimensional linear manifold for describing the normal data. High reconstruction error indicates a novel signal that does not match the patterns of normal data. One unsupervised portion of the manifold model adapts its representation in response to recent data. A second supervised portion of the model is made of a basis trained in advance using labeled examples of RFI; this prevents false positives due to these events. For a linear model, an orthonormalization operation is used to combine these bases prior to the anomaly detection decision. Another novel aspect of the approach lies in combining basis vectors learned in an unsupervised, online fashion from the data stream with supervised basis vectors learned in advance from known examples of false alarms. Adaptive, data-driven detection is achieved that is also informed by existing domain knowledge about signals that may be statistically anomalous, but are not interesting and should therefore be ignored. The method was evaluated using data from the Parkes Multibeam Survey. This data set was originally collected to search for pulsars, which are astronomical sources that emit radio pulses at regular periods. However, several non-pulsar anomalies have recently been discovered in this dataset, making it a compelling test case. By explicitly filtering known false alarm patterns, the approach yields significantly better performance than current transient detection methods.

Wagstaff, Kiri L.↗

NASA NTRS2018

Development of a Ground Test and Analysis Protocol to Support NASA's NextSTEP Phase 2 Habitation Concepts

Beaton, Kara H.↗

NASA NTRS2014

Review and Analysis of Algorithmic Approaches Developed for Prognostics on CMAPSS Dataset

Benchmarking of prognostic algorithms has been challenging due to limited availability of common datasets suitable for prognostics. In an attempt to alleviate this problem several benchmarking datasets have been collected by NASA's prognostic center of excellence and made available to the Prognostics and Health Management (PHM) community to allow evaluation and comparison of prognostics algorithms. Among those datasets are five C-MAPSS datasets that have been extremely popular due to their unique characteristics making them suitable for prognostics. The C-MAPSS datasets pose several challenges that have been tackled by different methods in the PHM literature. In particular, management of high variability due to sensor noise, effects of operating conditions, and presence of multiple simultaneous fault modes are some factors that have great impact on the generalization capabilities of prognostics algorithms. More than 70 publications have used the C-MAPSS datasets for developing data-driven prognostic algorithms. The C-MAPSS datasets are also shown to be well-suited for development of new machine learning and pattern recognition tools for several key preprocessing steps such as feature extraction and selection, failure mode assessment, operating conditions assessment, health status estimation, uncertainty management, and prognostics performance evaluation. This paper summarizes a comprehensive literature review of publications using C-MAPSS datasets and provides guidelines and references to further usage of these datasets in a manner that allows clear and consistent comparison between different approaches.

Uncertainty↗

NASA NTRS2005

NASA Tech Briefs, March 2005

Topics covered include: Scheme for Entering Binary Data Into a Quantum Computer; Encryption for Remote Control via Internet or Intranet; Coupled Receiver/Decoders for Low-Rate Turbo Codes; Processing GPS Occultation Data To Characterize Atmosphere; Displacing Unpredictable Nulls in Antenna Radiation Patterns; Integrated Pointing and Signal Detector for Optical Receiver; Adaptive Thresholding and Parameter Estimation for PPM; Data-Driven Software Framework for Web-Based ISS Telescience; Software for Secondary-School Learning About Robotics; Fuzzy Logic Engine; Telephone-Directory Program; Simulating a Direction-Finder Search for an ELT; Formulating Precursors for Coating Metals and Ceramics; Making Macroscopic Assemblies of Aligned Carbon Nanotubes; Ball Bearings Equipped for In Situ Lubrication on Demand; Synthetic Bursae for Robots; Robot Forearm and Dexterous Hand; Making a Metal-Lined Composite-Overwrapped Pressure Vessel; Ex Vivo Growth of Bioengineered Ligaments and Other Tissues; Stroboscopic Goggles for Reduction of Motion Sickness; Articulating Support for Horizontal Resistive Exercise; Modified Penning-Malmberg Trap for Storing Antiprotons; Tumbleweed Rovers; Two-Photon Fluorescence Microscope for Microgravity Research; Biased Randomized Algorithm for Fast Model-Based Diagnosis; Fast Algorithms for Model-Based Diagnosis; Simulations of Evaporating Multicomponent Fuel Drops; Formation Flying of Tethered and Nontethered Spacecraft; and Two Methods for Efficient Solution of the Hitting- Set Problem.

Source record↗

NASA NTRS1994

EDOS operations concept and development approach

The Earth Observing System (EOS) Data and Operations System (EDOS) is being developed by the National Aeronautics and Space Administration (NASA) Goddard Space Flight Center (GSFC) for the capture, level zero processing, distribution, and backup archiving of high speed telemetry data received from EOS spacecraft. All data received will conform to the Consultative Committee for Space Data Standards (CCSDS) recommendations. The major EDOS goals are to: (1) minimize EOS program costs to implement and operate EDOS; (2) respond effectively to EOS growth requirements; and (3) maintain compatibility with existing and enhanced versions of NASA institutional systems required to support EOS spacecraft. In order to meet these goals, the following objectives have been defined for EDOS: (1) standardize EDOS interfaces to maximize utility for future requirements; (2) emphasize life-cycle cost (LCC) considerations (rather than procurement costs) in making design decisions and meeting reliability, maintainability, availability (RMA) and upgradability requirements; (3) implement data-driven operations to the maximum extent possible to minimize staffing requirements and to maximize system responsiveness; (4) provide a system capable of simultaneously supporting multiple spacecraft, each in different phases of their life-cycles; (5) provide for technology insertion features to accommodate growth and future LCC reductions during the operations phase; and (6) provide a system that is sufficiently robust to accommodate incremental performance upgrades while supporting operations. Operations concept working group meetings were facilitated to help develop the EDOS operations concept. This provided a cohesive concept that met with approval of responsible personnel from the start. This approach not only speeded up the development process by reducing review cycles, it also provided a medium for generating good ideas that were immediately molded into feasible concepts. The operations concept was then used as a basis for the EDOS specification. When it was felt that concept elements did not support detailed requirements, the facilitator process was used to resolve discrepancies or to add new concept elements to support the specification. This method provided an ongoing revisal of the operations concept and prevented large revisions at the end of the requirement analysis phase of system development.

Knoble, G.↗

NASA NTRS2009

Solder Joint Health Monitoring Testbed

A method of monitoring the health of selected solder joints, called SJ-BIST, has been developed by Ridgetop Group Inc. under a Small Business Innovative Research (SBIR) contract. The primary goal of this research program is to test and validate this method in a flight environment using realistically seeded faults in selected solder joints. An additional objective is to gather environmental data for future development of physics-based and data-driven prognostics algorithms. A test board is being designed using a Xilinx FPGA. These boards will be tested both in flight and on the ground using a shaker table and an altitude chamber.

Delaney, Michael M.↗

NASA NTRSDate not supplied

Combining Disparate Measures of Metabolic Rate During Simulated Spacewalks

Scientists from NASA's Extravehicular Activities (EVA) Physiology Systems and Performance Project help design space suits for future missions, during which astronauts are expected to perform EVA activities on the Lunar or Martian surface. During an EVA, an astronaut's integrated metabolic rate is used to predict how much longer the activity can continue and still provide a safe margin of remaining consumables. For EVAs in the Apollo era, NASA physicians monitored live data feeds of heart rate, O 2 consumption, and liquid cooled garment (LCG) temperatures, which were subjectively combined or compared to produce an estimate of metabolic rate. But these multiple data feeds sometimes provided conflicting estimates of metabolic rate, making real-time calculations of remaining time difficult for physician/monitors. Currently, designs planned for the Constellation Program EVAs utilize an automated, but largely heuristic methodology for incorporating the above three measurements, plus an additional one - CO 2 production, ignoring data that appears in conflict; however a more rigorous model-based approach is desirable. In this study, we show how principal axis factor analysis, in combination with OLS regression and LOWESS smoothing can be used to estimate metabolic rate as a data-driven weighted average of heart rate, O 2 consumption, LCG temperature data, and CO 2 production. Preliminary results suggest less sensitivity to occasional spikes in observed data feeds, and reasonable within-subject reproducibility when applied to subsequent tasks. These methods do not require physician monitoring and as such can be automated in the electronic components of future space suits. With additional validation, our models show promise for increasing astronaut safety, while reducing the need for and potential errors associated with human monitoring of multiple systems.

Alan H Feiveson↗

NASA NTRS2008

Algorithms for Spectral Decomposition with Applications to Optical Plume Anomaly Detection

The analysis of spectral signals for features that represent physical phenomenon is ubiquitous in the science and engineering communities. There are two main approaches that can be taken to extract relevant features from these high-dimensional data streams. The first set of approaches relies on extracting features using a physics-based paradigm where the underlying physical mechanism that generates the spectra is used to infer the most important features in the data stream. We focus on a complementary methodology that uses a data-driven technique that is informed by the underlying physics but also has the ability to adapt to unmodeled system attributes and dynamics. We discuss the following four algorithms: Spectral Decomposition Algorithm (SDA), Non-Negative Matrix Factorization (NMF), Independent Component Analysis (ICA) and Principal Components Analysis (PCA) and compare their performance on a spectral emulator which we use to generate artificial data with known statistical properties. This spectral emulator mimics the real-world phenomena arising from the plume of the space shuttle main engine and can be used to validate the results that arise from various spectral decomposition algorithms and is very useful for situations where real-world systems have very low probabilities of fault or failure. Our results indicate that methods like SDA and NMF provide a straightforward way of incorporating prior physical knowledge while NMF with a tuning mechanism can give superior performance on some tests. We demonstrate these algorithms to detect potential system-health issues on data from a spectral emulator with tunable health parameters.

Srivastava, Askok N.↗

NASA NTRS2012

Particle Filtering for Model-Based Anomaly Detection in Sensor Networks

A novel technique has been developed for anomaly detection of rocket engine test stand (RETS) data. The objective was to develop a system that postprocesses a csv file containing the sensor readings and activities (time-series) from a rocket engine test, and detects any anomalies that might have occurred during the test. The output consists of the names of the sensors that show anomalous behavior, and the start and end time of each anomaly. In order to reduce the involvement of domain experts significantly, several data-driven approaches have been proposed where models are automatically acquired from the data, thus bypassing the cost and effort of building system models. Many supervised learning methods can efficiently learn operational and fault models, given large amounts of both nominal and fault data. However, for domains such as RETS data, the amount of anomalous data that is actually available is relatively small, making most supervised learning methods rather ineffective, and in general met with limited success in anomaly detection. The fundamental problem with existing approaches is that they assume that the data are iid, i.e., independent and identically distributed, which is violated in typical RETS data. None of these techniques naturally exploit the temporal information inherent in time series data from the sensor networks. There are correlations among the sensor readings, not only at the same time, but also across time. However, these approaches have not explicitly identified and exploited such correlations. Given these limitations of model-free methods, there has been renewed interest in model-based methods, specifically graphical methods that explicitly reason temporally. The Gaussian Mixture Model (GMM) in a Linear Dynamic System approach assumes that the multi-dimensional test data is a mixture of multi-variate Gaussians, and fits a given number of Gaussian clusters with the help of the wellknown Expectation Maximization (EM) algorithm. The parameters thus learned are used for calculating the joint distribution of the observations. However, this GMM assumption is essentially an approximation and signals the potential viability of non-parametric density estimators. This is the key idea underlying the new approach.

Solano, Wanda↗

NASA NTRS2011

Towards a State Based Control Architecture for Large Telescopes: Laying a Foundation at the VLT

Large telescopes are characterized by a high level of distribution of control-related tasks and will feature diverse data flow patterns and large ranges of sampling frequencies; there will often be no single, fixed server-client relationship between the control tasks. the architecture is also challenged by the task of integrating heterogeneous subsystems which will be delivered by multiple different contractors. Due to the high number of distributed components, the control system needs to effectively detect errors and faults, impede their propagation, and accurately mitigate them in the shortest time possible, enabling the service to be restored. The presented Data-Driven Architecture is based on a decentralized approach with an end-to-end integration of disparate, independently developed software components. These components employ a high-performance standards-based communication middle-ware infrastructure, based on the Data Distribution Service. A set of rules and principles, based on JPL's State Analysis method and architecture, are use to constrain component-to component interactions, where the Control System and System Under Control are clearly separated. State Analysis provide a model-based process for capturing system and software requirements and design, greatly reducing the gap between the requirements on software specified by systems engineers and the implementation by software engineers. The method and architecture has been field tested at the Very Large Telescope, where it has been integrated into an operational system.

European Extremely Large Telescope (E-ELT)↗

NASA NTRS2016

A Satellite Data-Driven, Client-Server Decision Support Application for Agricultural Water Resources Management

Water cycle extremes such as droughts and floods present a challenge for water managers and for policy makers responsible for the administration of water supplies in agricultural regions. In addition to the inherent uncertainties associated with forecasting extreme weather events, water planners need to anticipate water demands and water user behavior in a typical circumstances. This requires the use decision support systems capable of simulating agricultural water demand with the latest available data. Unfortunately, managers from local and regional agencies often use different datasets of variable quality, which complicates coordinated action. In previous work we have demonstrated novel methodologies to use satellite-based observational technologies, in conjunction with hydro-economic models and state of the art data assimilation methods, to enable robust regional assessment and prediction of drought impacts on agricultural production, water resources, and land allocation. These methods create an opportunity for new, cost-effective analysis tools to support policy and decision-making over large spatial extents. The methods can be driven with information from existing satellite-derived operational products, such as the Satellite Irrigation Management Support system (SIMS) operational over California, the Cropland Data Layer (CDL), and using a modified light-use efficiency algorithm to retrieve crop yield from the synergistic use of MODIS and Landsat imagery. Here we present an integration of this modeling framework in a client-server architecture based on the Hydra platform. Assimilation and processing of resource intensive remote sensing data, as well as hydrologic and other ancillary information occur on the server side. This information is processed and summarized as attributes in water demand nodes that are part of a vector description of the water distribution network. With this architecture, our decision support system becomes a light weight 'app' that connects to the server to retrieve the latest information regarding water demands, land use, yields and hydrologic information required to run different management scenarios. Furthermore, this architecture ensures all agencies and teams involved in water management use the same, up-to-date information in their simulations.

Agricultural↗

NASA NTRS2019

Human Performance Contributions to Safety in Commercial Aviation

Every day in aviation, pilots, air traffic controllers, and other front-line personnel perform countless correct judgments and actions in a variety of operational environments. These judgments and actions are often the difference between an accident and a non-event. Ironically, data on these behaviors are rarely collected or analyzed. Data-driven decisions about safety management and design of safety-critical systems are limited by the available data, which influence how decision makers characterize problems and identify solutions. Large volumes of data are collected on the failures and errors that result in infrequent incidents and accidents, but in the absence of data on behaviors that result in routine successful outcomes, safety management and system design decisions are based on a small sample of nonrepresentative safety data. This assessment aimed to find and document “safety successes” made possible by human operators. With many Aeronautics Research Mission Directorate (ARMD) Programs and Projects focusing on increased automation and autonomy and decreased human involvement, failure to fully consider the human contributions to successful system performance in civil aviation represents a significant risk — a risk that has not been recognized to date. Without understanding how humans contribute to safety, any estimate of predicted safety of autonomous capabilities is incomplete and inherently suspect. Furthermore, understanding the ways in which humans contribute to safety can promote strategic interactions among safety technologies, functions, procedures and the people using them. Without this understanding, the full benefits of an integrated, optimized human/technology or autonomous system will not be realized. Historically, safety has been consistently defined in terms of the occurrence of accidents or recognized risks (i.e., in terms of things that go wrong). These adverse outcomes are explained by identifying their causes, and safety is restored by eliminating or mitigating these causes. An alternative to this approach is to focus on what goes right and identify how to replicate that process. Focusing on the rare cases of failures attributed to “human error” provides little information about why human performance routinely prevents adverse events. Hollnagel has proposed that things go right because people continuously adjust their work to match their operating conditions. These adjustments become increasingly important as systems continue to grow in complexity. Thus, the definition of safety should reflect not only “avoiding things that go wrong” but “ensuring that things go right.” The basis for safety management requires developing an understanding of everyday activities. However, few mechanisms to monitor everyday work exist in the aviation domain, which limits opportunities to learn how designs function in reality. This concept of safety thinking and safety management is reflected in the emerging field of resilience engineering. According to Hollnagel, a system is resilient if it can sustain required operations under expected and unexpected conditions by adjusting its functioning prior to, during, or following changes, disturbances, and opportunities. To explore “positive” behaviors that contribute to resilient performance in commercial aviation, the assessment team examined a range of existing sources of data about pilot and air traffic control (ATC) tower controller performance, including subjective interviews with domain experts and objective aircraft flight data records. These data were used to identify strategies that support resilient performance, methods for exploring and refining those strategies in existing data, and proposed methods for capturing and analyzing new data.

Null, Cynthia H.↗