Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “learned priors”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

Featureless adaptive optimization accelerates functional electronic materials design

Electronic materials that exhibit phase transitions between metastable states (e.g., metal-insulator transition materials with abrupt electrical resistivity transformations) are challenging to decode. For these materials, conventional machine learning methods display limited predictive capability due to data scarcity and the absence of features that impede model training. In this article, we demonstrate a discovery strategy based on multi-objective Bayesian optimization to directly circumvent these bottlenecks by utilizing latent variable Gaussian processes combined with high-fidelity electronic structure calculations for validation in the chalcogenide lacunar spinel family. We directly and simultaneously learn phase stability and bandgap tunability from chemical composition alone to efficiently discover all superior compositions on the design Pareto front. Previously unidentified electronic transitions also emerge from our featureless adaptive optimization engine. Our methodology readily generalizes to optimization of multiple properties, enabling co-design of complex multifunctional materials, especially where prior data is sparse.

36 MATERIALS SCIENCE↗

Reinforcement Learning for feedback-enabled cyber resilience

The rapid growth in the number of devices and their connectivity has enlarged the attack surface and made cyber systems more vulnerable. As attackers become increasingly sophisticated and resourceful, mere reliance on traditional cyber protection, such as intrusion detection, firewalls, and encryption, is insufficient to secure the cyber systems. Cyber resilience provides a new security paradigm that complements inadequate protection with resilience mechanisms. A Cyber-Resilient Mechanism (CRM) adapts to the known or zero-day threats and uncertainties in real-time and strategically responds to them to maintain the critical functions of the cyber systems in the event of successful attacks. Feedback architectures play a pivotal role in enabling the online sensing, reasoning, and actuation process of the CRM. Reinforcement Learning (RL) is an important gathering of algorithms that epitomize the feedback architectures for cyber resilience. It allows the CRM to provide dynamic and sequential responses to attacks with limited or without prior knowledge of the environment and the attacker. In this work, we review the literature on RL for cyber resilience and discuss the cyber-resilient defenses against three major types of vulnerabilities, i.e., posture-related, information-related, and human-related vulnerabilities. Here we introduce moving target defense, defensive cyber deception, and assistive human security technologies as three application domains of CRMs to elaborate on their designs. The RL algorithms also have vulnerabilities themselves. We explain the major vulnerabilities of RL and present develop several attack models where the attacker target the information exchanged between the environment and the agent: the rewards, the state observations, and the action commands. We show that the attacker can trick the RL agent into learning a nefarious policy with minimum attacking effort. The paper introduces several defense methods to secure the RL-enabled systems from these attacks. However, there is still a lack of works that focuses on the defensive mechanisms for RL-enabled systems. Last but not least, we discuss the future challenges of RL for cyber security and resilience and emerging applications of RL-based CRMs.

97 MATHEMATICS AND COMPUTING↗

Investigating motion sickness using the conditioned taste aversion paradigm

The avoidance of foods which are associated with uncomfortable or aversive internal states has long been recognized. Many people are aware, either directly or via anecdotal reports, of individuals who avoid foods which were eaten just before the onset of sickness. Awareness of this phenomenon can be traced to the writings of John Locke. The disruption of diet during cancer therapy is sometimes ascribed to the attribution of an unpleasant quality to foods eaten preceding the sickness induced by therapy itself. In addition, it has long been recognized by the manufacturers of rodent poisons that animals avoid the injection of food treated with nonlethal doses of poison. An important part of the laboratory study of this phenomenon was directed toward studying the role learning plays in this type of avoidance behavior. Following the lead of Garcia and his associates, this avoidance has come to be interpreted as arising from a form of classical conditioning. In typical laboratory studies of this behavior, a novel food is ingested just prior to exposure to some stimulus, commonly poisoning or irradiation, which produces illness. Following the terminology of classical conditioning, it is common to describe this procedure as one of 'pairing' a conditioned stimulus (CS), the novel food, with an unconditioned stimulus (US), the illness induced by toxicosis or irradiation. Avoidance of the food in succeeding feeding opportunities is viewed as a learned response or a conditioned taste aversion (CTA). Garcia et al. asserted that motion sickness could produce 'gustatory' aversions, but passive motion was first reported as an US to establish CTA by Green and Rachlin. The purpose is to review the manner in which CTA has been used to study motion sickness. Numerous reviews concentrating on other aspects of CTA are available in the existing literature. Readers are encouraged to consult the various papers and edited books for extensive information on other aspects of this literature.

Robert A Fox↗

A Machine Learning Approach for Hourly Traffic Prediction Used in EV-Charging Sites

Reliable forecasting of hourly traffic volumes on highways is critical for planning and operating electric-vehicle charging infrastructure without overloading the grid. In this work, we develop and evaluate a station-specific machine-learning approach based on NeuralProphet, enhanced with conditional seasonality to better distinguish weekday, weekend, and holiday patterns. For each station, the model automatically retrieves the same calendar day from the prior years as an AR-Net initialization, fits trend and Fourier-based seasonality components, and then applies short-term auto-regressive corrections. We train and test on 2021 and 2022 TMAS data, respectively, and validate performance over the whole year. We chose to demonstrate how the model performs on a typical weekday (3/15/2022), weekend (3/27/2022), and a special holiday (12/25/2022). Our results yield MAPE of 7.4%, 23.6%, and 32.0%, respectively. Over the entire year 2022, the overall MAPE was 17%. This demonstrates that station-specific models with conditional seasonality can achieve accurate, scalable hourly forecasts for EV-charging load planning.

99 - GENERAL AND MISCELLANEOUS↗

Deep Learning for Automated Detection and Identification of Migrating American Eel Anguilla rostrata from Imaging Sonar Data

Adult American eels (Anguilla rostrata) are vulnerable to hydropower turbine mortality during outmigration from growth habitat in inland waters to the ocean where they spawn. Imaging sonar is a reliable and proven technology for monitoring of fish passage and migration; however, there is no efficient automated method for eel detection. We designed a deep learning model for automated detection of adult American eels from sonar data. The method employs convolution neural network (CNN) to distinguish between 14 images of eels and non-eel objects. Prior to image classification with CNN, background subtraction and wavelet denoising were applied to enhance sonar images. The CNN model was first trained and tested on data obtained from a laboratory experiment, which yielded overall accuracies of >98% for image-based classification. Then, the model was trained and tested on field data that were obtained near the Iroquois Dam located on the St. Lawrence River; the accuracy achieved was commensurate with that of human experts.

13 HYDRO ENERGY↗

Implementing machine learning methods on QICK hardware for qubit readout & control

Quantum readout and control is a fundamental aspect of quantum computing that requires accurate measurement of qubit states. Errors emerge in all stages, from initialization to readout, and identifying errors in post-processing necessitates resource-intensive statistical analysis. In our work, we use a lightweight fully-connected neural network (NN) to classify states of a transmon system with no prior processing. Our NN accelerator yields higher fidelities (92%) than the classical matched filter method (84%). By exploiting the natural parallelism of NNs and their placement near the source of data on field-programmable gate arrays (FPGAs), we can achieve ultra-low latency on the Quantum Instrumentation Control Kit (QICK). Integrating machine learning methods on QICK opens several pathways for efficient real-time processing of quantum circuits.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Designing for cooperative grain boundary segregation in multicomponent alloys

Tailoring the nanoscale distribution of chemical species at grain boundaries is a powerful method to dramatically influence the properties of polycrystalline materials. However, classical approaches to the problem have tacitly assumed that only competition is possible between solute species. In this paper, we show that solute elements can cooperate in the way they segregate to grain boundaries: In properly targeted alloys, the different chemical species cooperate to each fill complementary grain boundary sites disfavored by the other. By developing a theoretical “spectral” approach to this problem based on quantum-accurate grain boundary site distributions, we show how grain boundaries can be cooperatively alloyed, whether by depletion or enrichment. We provide machine-learned cosegregation information for over 700 ternary aluminum-based alloys and experimentally validate the concept in one ternary alloy where cosegregation is not expected by prior models but is expected based on the cooperative model.

grain boundaries↗

Physics in the Machine: Integrating Physical Knowledge in Autonomous Phase-Mapping

Application of artificial intelligence (AI), and more specifically machine learning, to the physical sciences has expanded significantly over the past decades. In particular, science-informed AI, also known as scientific AI or inductive bias AI, has grown from a focus on data analysis to now controlling experiment design, simulation, execution and analysis in closed-loop autonomous systems. The CAMEO (closed-loop autonomous materials exploration and optimization) algorithm employs scientific AI to address two tasks: learning a material system’s composition-structure relationship and identifying materials compositions with optimal functional properties. By integrating these, accelerated materials screening across compositional phase diagrams was demonstrated, resulting in the discovery of a best-in-class phase change memory material. Key to this success is the ability to guide subsequent measurements to maximize knowledge of the composition-structure relationship, or phase map. In this work we investigate the benefits of incorporating varying levels of prior physical knowledge into CAMEO’s autonomous phase-mapping. This includes the use of ab-initio phase boundary data from the AFLOW repositories, which has been shown to optimize CAMEO’s search when used as a prior.

97 MATHEMATICS AND COMPUTING↗

Engineering Design Unit Primary Mirror Segment

The James Webb Space Telescope Primary Mirror Engineering Design Unit (EDU) recently demonstrated the final manufacturing process at L-3-SSG/Tinsley in Richmond, California. Shown in the picture below, the EDU is a prototype 1.4 meter diameter Beryllium mirror that is identical to the 18 flight mirror segments and is used to check out all mirror manufacturing processes prior to the actual flight mirror processing. This recent result gives confidence that all manufacturing processes are in place to figure the 18 flight mirrors. Lessons learned from the EDU have been applied to the flight mirrors improving the performance and processing time and all 18 flight mirrors are well along in their processing at L3-SSG/Tinsley. As a result of completing the final manufacturing demonstration, the EDU mirror was sent to Ball Aerospace in Boulder, Colorado where it is now undergoing integration with the hexapod mount assembly and optical testing checkout. After integration to mount assemblies, flight mirrors will be sent for cryogenic (50 degrees Kelvin) optical testing where the cryogenic distortions will be measured. After cryogenic testing, mirrors will eventually be returned to Tinsley for final cryogenic polishing. During final cryogenic polishing, the inverse of the measured cryogenic distortions will be polished into the mirror to assure the mirror works at it's cryogenic operating temperature. Once cryogenic polishing is completed to final specifications, the mirror will be coated and sent for final cryogenic testing.

Alongi, Chris↗

Monitoring of the Atmosphere on the International Space Station with the Air Quality Monitor

During the early years of human spaceflight, short duration missions allowed for monitoring of the spacecraft environment to be performed via archival sampling, in which samples were returned to Earth for analysis. With the construction of the International Space Station (ISS) and the accompanying extended mission durations, the need for enhanced, real-time monitors became apparent. The Volatile Organic Analyzer (VOA) operated on ISS for 7 years, where it assessed trace volatile organic compounds in the cabin air. The large and fixed-position VOA was eventually replaced with the smaller Air Quality Monitor (AQM). Since March 2013, the atmosphere of the U.S. Operating Segment (USOS) has been monitored in near real-time by a pair of AQMs. These devices consist of a gas chromatograph (GC) coupled with a differential mobility spectrometer (DMS) and currently target detection list of 22 compounds. These targets are of importance to both crew health and the Environmental Control and Life Support Systems (ECLSS) on ISS. Data is collected autonomously every 73 hours, though the units can be controlled remotely from mission control to collect data more frequently during contingency or troubleshooting operations. Due to a nominal three-year lifetime on-orbit, the initial units were replaced in February 2016. This paper will focus on the preparation and use of the AQMs over the past several years. A description of the technical aspects of the AQM will be followed by lessons learned from the deployment and operation of the first set of AQMs. These lessons were used to improve the already-excellent performance of the instruments prior to deployment of the replacement units. Data trending over the past several years of operation on ISS will also be discussed, including data obtained during a survey of the USOS modules. Finally, a description of AQM use for contingency and investigative studies will be presented.

Wallace William T.↗

Lean Model-Based Systems Engineering on the NASA High-Density Vertiplex Subproject

The High Density Vertiplex (HDV) subproject of NASA’s Advanced Air Mobility (AAM) project adopted Model-Based Systems Engineering (MBSE)in July of 2020, prior to subproject formulation. A small and lean team of HDV Systems Engineers(SE) are utilizing MagicDraw to execute NASA SE processes via MBSE. The SEs learned how to use MagicDraw from scratch and HDV is the first project for which the SEs have utilized MagicDraw. This paper will demonstrate project technical execution via MBSE, utilizing the digital elements built into the SysML (Systems Modeling Language). SysML provides a model-centric means of carrying out the NASA SE common technical processes by providing tools for complete system modeling, including requirements and interface management and design capture. The authors also leverage and extend SysML to perform other SE tasks, such as Verification and Validation (V&V)tracking. MBSE has two main purposes for HDV: 1) documenting the subproject’s logical architecture for distribution outside of the subproject, 2) capturing the subproject’s physical architecture in a single-source-of-truth for use by the subproject’s members. This paper details the challenges, lessons learned, and solutions that were encountered in implementing MBSE in the first iteration on a multi-iteration, full-lifecycle design, build, fly project.

Demetrios Katsaduros↗

Lean Model-Based Systems Engineering on the NASA High-Density Vertiplex Subproject

The High Density Vertiplex (HDV) subproject of NASA’s Advanced Air Mobility (AAM) project adopted Model-Based Systems Engineering (MBSE) in July of 2020, prior to subproject formulation. A small and lean team of HDV Systems Engineers (SE) are utilizing MagicDraw to execute NASA SE processes via MBSE. The SEs learned how to use MagicDraw from scratch and HDV is the first project for which the SEs have utilized MagicDraw. This presentation will demonstrate project technical execution via MBSE, utilizing the digital elements built into the SysML (Systems Modeling Language). SysML provides a model-centric means of carrying out the NASA SE common technical processes by providing tools for complete system modeling, including requirements and interface management and design capture. The authors also leverage and extend SysML to perform other SE tasks, such as Verification and Validation (V&V) tracking. MBSE has two main purposes for HDV: 1) documenting the subproject’s logical architecture for distribution outside of the subproject, 2) capturing the subproject’s physical architecture in a single-source-of-truth for use by the subproject’s members. This presentation details the challenges, lessons learned, and solutions that were encountered in implementing MBSE in the first iteration on a multi-iteration, full-lifecycle design, build, fly project.

systems engineering↗

Probabilistic Day-Ahead Forecasting Using an Analog Ensemble Approach for Wind Farm Grid Services

Wind resource assessment and wind power forecasting are used in research and industry to anticipate future power output at scales ranging from individual wind turbines to entire wind farms. Probabilistic day-ahead wind forecasting is useful for anticipating how a wind farm could potentially participate in the day-ahead market by providing upper and lower bounds for expected power generation, thus informing grid operators of its uncertainty. Understanding this uncertainty is part of a larger project focused on building a platform that combines efforts in weather forecasting, aerodynamic and economic modeling to create maximum value of a wind plant to better provide services to the grid. This effort is also known as the Atmosphere to Electrons to Grid (A2E2G) project. One method for producing a probabilistic forecast is through the analog ensemble approach (Delle Monache et al., 2011). This method leverages historical forecasts and their corresponding observations as a training data set from which future forecasts can be made. For some future forecast, the most similar historical forecasts (analogs) are identified on a regular time basis such as once per a 3-hour window. The most similar analogs, based on a metric such as root mean square error (RMSE), are recorded and their corresponding verifying observations are used as an ensemble member for this future forecast. Prior work in this area demonstrates improvements over raw Numerical Weather Prediction (NWP) forecasts and shows skill similar to techniques such as logistic regression and machine learning (Delle Monache et al., 2013; Alessandrini et al., 2015). Here, we take the High-Resolution Rapid Refresh model (HRRR) day-ahead forecast (0-36 hours) to create a probabilistic day-ahead forecast using an analog ensemble approach. The HRRR has an hourly temporal resolution, with a spatial resolution of 3 km. The 12 UTC HRRR model run is downloaded every day for one year from August 2019 - July 2020, with the first 11 months serving as a bank of analogs from which the forecasting algorithm can create a probabilistic forecast. Once downloaded, the original HRRR forecast is temporally interpolated to 5-minutes, aligning with both the temporal resolution of the observations as well as the timescale relevant for day-ahead power forecasts. The forecast is validated at the M2 tower at the Flatirons Campus of the National Renewable Energy Laboratory (NREL) at a typical wind turbine height of 80 m. Variables such as wind speed, wind direction, and turbulence intensity are incorporated into the probabilistic forecast model and weighted according to their relative importance to the forecast. Based on metrics such as mean bias error (MBE), mean absolute error (MAE), and root mean square error, the analog ensemble forecast outperforms the raw HRRR forecast during the testing period of July 2020. Figure 1 illustrates an example day-ahead forecast compared against the verifying observations. The general variability and ramps are captured throughout the day, with potential to further improve the analog ensemble model through machine learning techniques.

numerical weather prediction↗

Safe Physics-Informed Machine Learning for Dynamics and Control

This tutorial paper focuses on safe physics-informed machine learning in the context of dynamics and control, providing a comprehensive overview of how to integrate physical models and safety guarantees. As machine learning techniques enhance the modeling and control of complex dynamical systems, ensuring safety and stability remains a critical challenge, especially in safety-critical applications like autonomous vehicles, robotics, medical decision-making, and energy systems. We explore various approaches for embedding and ensuring safety constraints, including structural priors, Lyapunov and Control Barrier Functions, predictive control, projections, and robust optimization techniques. Additionally, we delve into methods for uncertainty quantification and safety verification, including reachability analysis and neural network verification tools, which help validate that control policies remain within safe operating bounds even in uncertain environments. The paper includes illustrative examples demonstrating the implementation aspects of safe learning frameworks that combine the strengths of data-driven approaches with the rigor of physical principles, offering a path toward the safe control of complex dynamical systems.

Drgona, Jan↗

Towards Off-policy Evaluation as a Prerequisite for Real-world Reinforcement Learning in Building Control

We present an initial study of off-policy evaluation (OPE), a problem prerequisite to real-world reinforcement learning (RL), in the context of building control. OPE is the problem of estimating a policy's performance without running it on the actual system, using historical data from the existing controller. It enables the control engineers to ensure a new, pretrained policy satisfies the performance requirements and safety constraints of a real-world system, prior to interacting with it. While many methods have been developed for OPE, no study has evaluated which ones are suitable for building operational data, which are generated by deterministic policies and have limited coverage of the state-action space. After reviewing existing works and their assumptions, we adopted the approximate model (AM) method. Furthermore, we used bootstrapping to quantify uncertainty and correct for bias. In a simulation study, we evaluated the proposed approach on 10 policies pretrained with imitation learning. On average, the AM method estimated the energy and comfort costs with 1.84% and 14.1% error, respectively.

Chen, B↗

Spaceflight Associated Neuro-Ocular Syndrome: A Brief for Astronaut Candidates

INTRODUCTION: Spaceflight Associated Neuro-ocular Syndrome (SANS) was first described in 2011 as a clinical process involving degradation of visual acuity and structural changes of the eye when exposed to microgravity. This syndrome poses a risk to astronaut vision, safety, and mission performance, especially during extended-duration missions. The modern astronaut cohort is a diverse group of men and women including scientists, engineers, teachers, pilots, business professionals, and artists. Given their diverse backgrounds, it is imperative all crews are educated on SANS risks and potential countermeasures. This education must be comprehensive and easily understood, regardless of academic background or specific expertise. TOPIC: A PowerPoint was created to serve as a presentation and stand-alone educational aid for all astronaut candidates (ASCANS). This medium was chosen as it can be updated dynamically as new information emerges. The brief was split into sections to facilitate organized delivery of complex content in easy-to-understand components. Sections included descriptive statistics, signs and symptoms, outcomes, myths and ongoing research, countermeasures, ongoing surveillance, and unknown and future directions. More research is necessary in order to understand this syndrome, and without continued ASCAN participation in research, acquisition of data will not be possible. These efforts will become increasingly critical with deep space and exploration missions. APPLICATION: We recommend regular use of this brief for all ASCANS, and potentially veteran crewmembers, given the high risks and potentially poor outcomes associated with SANS. For ASCANS, recommendations include that this brief be provided at the start of their training, with a bi-annual refresher course for all astronauts. This ensures they are current on SANS information, given the dynamic nature of this syndrome. Additionally, this brief will be made available to all as a self-study aid, utilized as a refresher immediately prior to launch. It is anticipated that this brief will promote the health, safety, and wellbeing of professional and private crews alike, during and after long-duration spaceflight. RESOURCES: Brunstetter, T., Tarver, B., & Tsung, A. (2022, October). Spaceflight Associated Neuro-ocular Syndrome (Sans) and its Risk to Nasa Astronauts. JSC Aerospace Medicine Clerkship. Houston, TX; Lyndon B. Johnson Space Center. Laurie, S. S., Macias, B. R., Pardon, L. P., Brunstetter, T., Tarver, W. J., Gibson, C. R., Greenwald, S. H., Jasien, K. V., Mason, S., & Tsung, A. (2022). (rep.). Evidence Report: Risk of Spaceflight Associated Neuro-ocular Syndrome (SANS). Houston, TX: National Aeronautics and Space Administration. Mader, T. H., Gibson, C. R., Pass, A. F., Kramer, L. A., Lee, A. G., Fogarty, J., ... & Polk, J. D. (2011). Optic disc edema, globe flattening, choroidal folds, and hyperopic shifts observed in astronauts after long-duration space flight. Ophthalmology, 118(10), 2058-2069. LEARNING OBJECTIVES: • The audience will be able to describe current Spaceflight Associated Neuro-ocular Syndrome (SANS) findings, significance, and countermeasures. • The audience will understand the importance of astronauts’ familiarity with SANS prior to flying and the large risks associated with this syndrome. CME QUESTIONS (3 QUESTIONS, MULTIPLE CHOICE A-D): 1. What is the approximate prevalence rate of SANS in astronauts? A. 22% B. 50% C. 66% D. 80% Answer: C. 2. Which of the ocular findings are NOT associated with SANS? A. Optic disc edema B. Retinal detachment C. Globe flattening D. Chorioretinal folds Answer: B. 3. Which of the following diagnostic technologies are NOT available in-flight? A. Ocular ultrasound B. Retinal photography C. MRI Orbit D. Optical Coherence Tomography (OCT) Answer: C.

Sara Stroble Mason↗

Exploring Sentinel-1 and Sentinel-2 diversity for Flood inundation mapping using deep learning

Identification of flood water extent from satellite images has historically relied on either synthetic aperture radar (SAR) or multi-spectral (MS) imagery. MS sensors are limited to cloud free conditions, whereas SAR imagery is plagued by noise-like speckle. Prior studies that use combinations of MS and SAR data to overcome individual limitations of these sensors have not fully examined sensitivity of flood mapping performance to different combinations of SAR and MS derived spectral indices or band transformations in color space. This study explores the use of diverse bands of Sentinel 2 (S2) through well-established water indices and Sentinel 1 (S1) derived SAR imagery along with their combinations to assess their capability for generating accurate flood inundation maps. The robustness in performance of S-1 and S-2 band combinations was evaluated using 446 hand labeled flood inundation images spanning across 11 flood events from Sen1Floods11 dataset which are highly diverse in terms of land cover as well as location. A modified K-fold cross validation approach is used to evaluate the performance of 32 combinations of S1 and S2 bands using a fully connected deep convolutional neural network known as U-Net. Our results indicated that usage of elevation information has improved the capability of S1 imagery to produce more accurate flood inundation maps. Compared to a median F1 score of 0.62 when using only S1 bands, the combined use of S1 and elevation information led to an improved median F1 score of 0.73. Water extraction indices based on S2 bands have a statistically significant superior performance in comparison to S1. Among all the band combinations, HSV (Hue, Saturation, Value) transformation of S2 bands provides a median F1 score of 0.9, outperforming the commonly used water spectral indices owing to HSV’s transformation’s superior contrast distinguishing abilities. Additionally, U-Net algorithm was able to learn the relationship between raw S2 based water extraction indices and their corresponding raw S2 bands, but not of HSV owing to relatively complex computation involved in the latter. Results of the paper establishes important benchmarks for the extension of S1 and S2 data-based flood inundation mapping efforts over large spatial extents.

Goutam Konapala↗

Evaluating the Use of High-Fidelity Simulator Research Methods to Study Airline Flight Crew Resilience

As it evolves, aviation will continue to require integration of a wide range of safety systems and practices, some of which are already in place and others that are yet to be developed. New concepts in system safety thinking have emerged to consider not only what may go wrong, but also what can be learned when things go right during commercial flight operations. Taken together, these complementary perspectives form a more comprehensive approach to systemsafety thinking that can help to recognize and preserve the resilient performance capabilities currently provided by humans. A need exists, however, for research methods to enable better understanding of the human contributions to aviation safety. NASA’s System-Wide Safety Project supports research on using flight simulation methods to study operator resilience and safety-producing behaviors. Building on prior NASA efforts investigating procedural non-adherences during area navigation standard terminal route arrivals, a high-fidelity commercial aviation line operational simulation (LOS) experiment has been designed to study how flight crews anticipate, monitor for, respond to, and learn from expected and unexpected disturbances during these operations. A diverse set of LOS scenarios were developed to simulate highly realistic, complex, but routinely encountered operational situations. Each scenario provided multiple opportunities to collect data on how flight crews manage threats and errors, as well as novel opportunities to observe resilient and safety-producing behaviors. The experimental design, implications for the study of safety-producing behaviors using simulation, and considerations for airline pilot training will be discussed.

Chad L Stephens↗