Results for “operator decision support” — Page 49

DOE OSTI2022

Application and evaluation of a pattern-based building energy model calibration method using public building datasets

Building performance simulation has been adopted to support decision making in the building life cycle. An essential issue is to ensure a building energy simulation model can capture the reality and complexity of buildings and their systems in both the static characteristics and dynamic operations. Building energy model calibration is a technique that takes various types of measured performance data (e.g., energy use) and tunes key model parameters to match the simulated results with the actual measurements. This study performed an application and evaluation of an automated pattern-based calibration method on commercial building models that were generated based on characteristics of real buildings. A public building dataset that includes high-level building attributes (e.g., building type, vintage, total floor area, number of stories, zip code) of 111 buildings in San Francisco, California, USA, was used to generate building models in EnergyPlus. Monthly level energy use calibrations were then conducted by comparing building model results against the actual buildings' monthly electricity and natural gas consumption. The results showed 57 out of 111 buildings were successfully calibrated against actual buildings, while the remaining buildings showed opportunities for future calibration improvements. Enhancements to the pattern-based model calibration method are identified to expand its use for: (1) central heating, ventilation and air conditioning (HVAC) systems with chillers, (2) space heating and hot water heating with electricity sources, (3) mixed-use building types, and (4) partially occupied buildings.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2018

Integrating quantitative defense-in-depth metrics into new reactor designs

Risk-informed, performance-based (RIPB) methods have progressed to the point where high-level guidance can be used to augment traditional, deterministic, nuclear safety design practices in areas important to nuclear reactor safety. This paper describes an approach for augmenting the traditional defense-in-depth (DID) qualitative approach with quantitative risk information from a plant-specific probabilistic risk assessment (PRA) in a way that is structured, can be applied on a consistent basis, and allows for clear acceptance criteria. Adding performance-based targets that should be achieved is expected to result in safer and more economical plant designs. Evaluations of DID can be conducted throughout the design process as well as in support of design certification and operating license applications to identify where defense protections could be enhanced or relaxed. Consistent with the United States Nuclear Regulatory Commission's policy statement encouraging greater use of PRA to improve safety decision making and regulatory efficiency, this scenario-based DID method can be used to evaluate changes and overall plant design as part of the normal design control process. Although the RIPB method presented in this paper was developed for application to advanced passive light water reactor designs, the metrics could be tailored to other reactor designs. This risk-informed approach to DID helps to ensure that public and worker risk insights are integrated into the design process holistically.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2021

AGGREGATE: dAta-driven modelinG preservinG contRollable dEr for outaGe mAnagemenT and rEsiliency (Final Report)

The AGGREGATE project team successfully developed and validated various modules for outage management. Brief summaries of each module are provided to showcase their strength for outage management and restoration for a distribution system with a high penetration of connected distribution energy resources (DERs). In recent years, inverter-based DERs have been widely deployed in distribution system. A most of behind-the-meter (BTM) solar power generation is not visible to the utility. The data-driven DER and load estimation modules are using machine learning (ML) and artificial intelligence (AI) to manage this issue, which provides an opportunity for distribution system operators (DSOs) to operate systems and make decisions in real-time for a distribution system with a high penetration of DERs deployed. Also, the estimated DER and true load can be further leveraged in network aggregation and cold-load pick up estimation for reducing the computing complexity and providing for fast restoration. After load demand and DER power generations have been estimated, the information will support topology and state estimation (SE). The topology estimation module demonstrated the viability of mixed integer linear programming (MILP) formulation to estimate the most likely operational radial topology and outage sections using power flow measurements, historical/estimated load and DERs data and smart meter ping measurements. Formulation includes continuous (power flow, load and DERs data) and binary measurements (smart meter ping measurements) in a single formulation. Errors in continuous data and binary data are modeled as normal distribution and Bernoulli distribution, respectively. In the future distribution grid, the power injection from controllable DERs will be essential for efficient and resilient grid operation. However, determining the optimal DER injections and restoration actions is dependent on knowledge of the system states. State estimation (SE), already the cornerstone of transmission energy management systems, will become commonplace in distribution management systems as more measurements become available from deployment of automated metering infrastructure (AMI). Observability analysis is the first step in SE, as it determines the sufficiency of the available measurements for accurately estimating the current system states. A new type of pseudo-measurement called a Correlational Measurement (CM) is introduced in this module, to enhance the observability of the system to enable more accurate SE. CMs encapsulate knowledge of correlation between demand patterns for similar classes of loads as well as injection patterns for same-technology renewable DERs. During grid contingency scenarios, DERs have been traditionally disconnected, without any fault ride-through capabilities. However, with new regulations and better technology, it is feasible for these resources to contribute to the grid’s restoration after an adverse event and hence enhance resilience. The controllability module proposes a two-step restoration scheme for the power system restoration process by leveraging additional degrees of freedom in power electronics interfaced DERs for mitigating voltage problems. In a resilience mode without the utility system, the distribution grid relies on DERs to serve critical load. In such a severe event with multiple faults on the distribution feeders, actuation of various protective devices (PDs) divides the distribution system into electrical islands. The undetected actuated PDs due to fault current contributions from DERs can delay the restoration process, thereby reducing the system resilience. The Advanced Outage Management (AOM) and the Advanced Feeder Restoration (AFR) modules developed in this project provide improved system resilience with multiple DERs. AOM identifies the faulted sections and actuated PDs in a distribution system with DERs by incorporating smart meter data. The most credible outage scenario including fault locations, PD actuations, and fault indicator (FI) failures is identified by a set of binary integer linear programming incorporating hypotheses. The AFR module serves to restore a distribution system with available energy resources taking into consideration the availability of utility sources and DERs. By partitioning the system into islands, critical load will be served with the available generation resources within islands based on the solution of a MILP. When the utility systems become available, the optimal path will be determined by a spanning tree search algorithm that reconnects these islands back to substations and restores the remaining load. The transmission and distribution (T&D) co-simulation module was used to validate the effect of a control action performed on the distribution side assets as it propagates to the transmission side. This ensures that the control action performed results in a feasible operating point on both the transmission and the distribution system. In addition to validation, the team used the T&D co-simulation module to demonstrate how distribution system assets can be used to mitigate issues on the transmission system. Specifically, the team demonstrated that appropriate switching operations on the distribution side can alleviate the line overload condition on the transmission side without causing new operational constraint violations.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS1981

Transportation systems evaluation methodology development and applications, phase 3

Transportation systems or proposed changes in current systems are evaluated. Four principal evaluation criteria are incorporated in the process, operating performance characteristics as viewed by potential users, decisions based on the perceived impacts of the system, estimating what is required to reduce the system to practice; and predicting the ability of the concept to attract financial support. A series of matrix multiplications in which the various matrices represent evaluations in a logical sequence of the various discrete steps in a management decision process is used. One or more alternatives are compared with the current situation, and the result provides a numerical rating which determines the desirability of each alternative relative to the norm and to each other. The steps in the decision process are isolated so that contributions of each to the final result are readily analyzed. The ability to protect against bias on the part of the evaluators, and the fact that system parameters which are basically qualitative in nature can be easily included are advantageous.

Kuhlthau, A. R.↗

NASA NTRS1987

The implementation and use of Ada on distributed systems with high reliability requirements

Performance analysis was begin on the Ada implementations. The goal is to supply the system designer with tools that will allow a rational decision to be made about whether a particular implementation can support a given application early in the design cycle. Primary activities were: analysis of the original approach to recovery in distributed Ada programs using the Advanced Transport Operating System (ATOPS) example; review and assessment of the original approach which was found to be capable of improvement; preparation and presentation of a paper at the 1987 Washington DC Ada Symposium; development of a refined approach to recovery that is presently being applied to the ATOPS example; and design and development of a performance assessment scheme for Ada programs based on a flexible user-driven benchmarking system.

Knight, J. C.↗

NASA NTRS2014

Traffic Aware Planner (TAP) Flight Evaluation

NASA's Traffic Aware Planner (TAP) is a cockpit decision support tool that has the potential to achieve significant fuel and time savings when it is embedded in the data-rich Next Generation Air Transportation System (NextGen) airspace. To address a key step towards the operational deployment of TAP and the NASA concept of Traffic Aware Strategic Aircrew Requests (TASAR), a system evaluation was conducted in a representative flight environment in November, 2013. Numerous challenges were overcome to achieve this goal, including the porting of the foundational Autonomous Operations Planner (AOP) software from its original simulation-based, avionics-embedded environment to an Electronic Flight Bag (EFB) platform. A flight-test aircraft was modified to host the EFB, the TAP application, an Automatic Dependent Surveillance Broadcast (ADS-B) processor, and a satellite broadband datalink. Nine Evaluation Pilots conducted 26 hours of TAP assessments using four route profiles in the complex eastern and north-eastern United States airspace. Extensive avionics and video data were collected, supplemented by comprehensive inflight and post-flight questionnaires. TAP was verified to function properly in the live avionics and ADS-B environment, characterized by recorded data dropouts, latency, and ADS-B message fluctuations. Twelve TAP-generated optimization requests were submitted to ATC, of which nine were approved, and all of which resulted in fuel and/or time savings. Analysis of subjective workload data indicated that pilot interaction with TAP during flight operations did not induce additional cognitive loading. Additionally, analyses of post-flight questionnaire data showed that the pilots perceived TAP to be useful, understandable, intuitive, and easy to use. All program objectives were met, and the next phase of TAP development and evaluations with partner airlines is in planning for 2015.

Maris, John M.↗

NASA NTRSDate not supplied

Initial Validation of a Simulation System for Studying Interoperability in Future Air Traffic Management Systems

Future air traffic management systems will need to accommodate large numbers of increasingly diverse air vehicles with different operating paradigms. To support this trend, they will digitally share copious amounts of information via a common communication architecture. Operators will deploy programs that create and negotiate flight plans via the architecture’s communication protocols. These programs will autonomously make decisions that must be arbitrated by the architecture and robust to uncertainty. To study interoperability in air traffic management, a new airspace simulation system was composed by integrating a legacy airspace simulation, an air traffic control model, and a new research communication architecture. It was used to evaluate air traffic management concepts by adapting it to handle congested arrival traffic at Newark Liberty International Airport and executing simulations. Results demonstrated the ability of the simulation system to model in detail strategic traffic flow management, predeparture flight planning, and air traffic control working in concert. Subsequent studies can use the simulation system to study interoperability, autonomy, digital communication, and uncertainty in future air traffic management systems.

aircraft scheduling,traffic flow management,autono↗

NASA NTRSDate not supplied

Impact of ATD-2 Tools on Human Factors Metrics at Charlotte Douglas International Airport

New decision support tools were introduced at Charlotte Douglas International Airport (CLT) as part of NASA’s Air Traffic Demonstration 2 (ATD-2) project. ATD-2 is based on concepts which enable integrated arrival, departures, and surface (IADS) operations in a metroplex environment. Metrics on environmental benefits from ATD-2’s surface metering at CLT have shown impressive reductions in fuel consumption and CO2 emissions. Questions have remained on 1) the human factors impact of these new ATD-2 tools and surface metering and 2) the users’ perceptions of how these tools affect operations. To address these questions, post-bank surveys were designed for the four types of users of the new tools at CLT: Ramp Controllers, Ramp Managers, Tower Traffic Management Coordinators (TMCs) and Terminal Radar Approach Control (TRACON) TMCs. These surveys were administered during five-day blocks at three different times. Three types of banks were compared: No ATD-2 tool use, ATD-2 tool use, and ATD-2 use plus surface metering. Results indicate that higher workload or decreased situation awareness was not significantly associated with ATD-2 tool use more than no ATD-2 tool use--in fact for many users the opposite was true. Also, acceptability and operational efficiency measures indicate that the users did not perceive that banks with ATD-2 tools were significantly less acceptable or less efficient than banks without ATD-2 tools.

Airspace Technology Demonstration 2↗

NASA NTRS2020

Impact of ATD-2 Tools on Human Factor Metrics at Charlotte Douglas International Airport

New decision support tools were introduced at Charlotte Douglas International Airport (CLT) as part of NASA’s Air Traffic Demonstration 2 (ATD-2) project. ATD-2 is based on concepts which enable integrated arrival, departures, and surface (IADS) operations in a metroplex environment. Metrics on environmental benefits from ATD-2’s surface metering at CLT have shown impressive reductions in fuel consumption and CO2 emissions. Questions have remained on 1) the human factors impact of these new ATD-2 tools and surface metering and 2) the users’ perceptions of how these tools affect operations. To address these questions, post-bank surveys were designed for the four types of users of the new tools at CLT: Ramp Controllers, Ramp Managers, Tower Traffic Management Coordinators (TMCs) and Terminal Radar Approach Control (TRACON) TMCs. These surveys were administered during five-day blocks at three different times. Three types of banks were compared: No ATD-2 tool use, ATD-2 tool use, and ATD-2 use plus surface metering. Results indicate that higher workload or decreased situation awareness was not significantly associated with ATD-2 tool use more than no ATD-2 tool use--in fact for many users the opposite was true. Also, acceptability and operational efficiency measures indicate that the users did not perceive that banks with ATD-2 tools were significantly less acceptable or less efficient than banks without ATD-2 tools.

Bonny K Parke↗

NASA NTRSDate not supplied

Initial Validation of a Simulation System for Studying Interoperability in Future Air Traffic Management Systems

Future air traffic management systems will need to accommodate large numbers of increasingly diverse air vehicles with different operating paradigms. To support this trend, they will digitally share copious amounts of information via a common communication architecture. Operators will deploy programs that create and negotiate flight plans via the architecture’s communication protocols. These programs will autonomously make decisions that must be arbitrated by the architecture and robust to uncertainty. To study interoperability in air traffic management, a new airspace simulation system was composed by integrating a legacy airspace simulation, an air traffic control model, and a new research communication architecture. It was used to evaluate air traffic management concepts by adapting it to handle congested arrival traffic at Newark Liberty International Airport and executing simulations. Results demonstrated the ability of the simulation system to model in detail strategic traffic flow management, predeparture flight planning, and air traffic control working in concert. Subsequent studies can use the simulation system to study interoperability, autonomy, digital communication, and uncertainty in future air traffic management systems.

air traffic control↗

NASA NTRS2021

Initial Validation of a Simulation System for Studying Interoperability in Future Air Traffic Management Systems

Future air traffic management systems will need to accommodate large numbers of increasingly diverse air vehicles with different operating paradigms. To support this trend, they will digitally share copious amounts of information via a common communication architecture. Operators will deploy programs that create and negotiate flight plans via the architecture’s communication protocols. These programs will autonomously make decisions that must be arbitrated by the architecture and robust to uncertainty. To study interoperability in air traffic management, a new airspace simulation system was composed by integrating a legacy airspace simulation, an air traffic control model, and a new research communication architecture. It was used to evaluate air traffic management concepts by adapting it to handle congested arrival traffic at Newark Liberty International Airport and executing simulations. Results demonstrated the ability of the simulation system to model in detail strategic traffic flow management, predeparture flight planning, and air traffic control working in concert. Subsequent studies can use the simulation system to study interoperability, autonomy, digital communication, and uncertainty in future air traffic management systems.

air traffic control↗

NASA NTRSDate not supplied

Airspace Integration Considerations for Increasingly Autonomous Flight and Operations

There is much interest in autonomous systems and their operations. When it comes to autonomous systems, it is critical to understand various levels of proposed autonomy, reasons for selecting autonomous capabilities, certification approaches and various challenges or research needs associated with the integration of autonomous systems in the National Airspace System (NAS). Furthermore, there are levels of autonomous systems proposed for different types of aircraft including: single pilot operations, remotely piloted operations, fully autonomous operations, and multiple aircraft managed by a single, remote pilot. This preliminary article aims to provide a broad overview rather than concentrate on a specific element of autonomous flight. Hence, the focus of this article is on enabling various types of increasingly autonomous aircraft and their operations routinely within the next five to ten years, consistent with the FAA’s Info-Centric NAS (ICN) vision. Under ICN, a fully integrated information environment supports collaboration across diverse traffic management services and shifts decision-making to the most appropriate actor supported by automation. However, the roles and responsibilities of Air Traffic Services (ATS) personnel providing traditional services are likely to be very similar to today in that humans will still be responsible for majority of decision making. There is another NASA research effort beyond ICN is underway called Sky For All which is not covered in this article as it is in the planning stages.

autonomy↗

NASA NTRS2023

Progressively Enabling Earth Independent Medical Operations (EIMO)

This panel presents the findings from a series of Technical Interchange Meetings (TIMs) hosted by the Exploration Medical Capability Element (ExMC) in NASA’s Human Research Program. The topics for the TIMs were derived from a 2-day conference of senior leaders and subject matters experts that collectively outlined a multi-faceted strategy designed to optimize crew health and performance through an increasingly autonomous medical approach. The first abstract in this panel outlines the scope of issues related to data collection, usage, transmission and computing capacity to facilitate EIMO. The second presentation provides an overview of the challenges in developing curricula and advanced training tools to baseline knowledge, skills and abilities (KSA), verify clinical competency and assure retention during prolonged durations inherent in exploration-class missions. An overview of the complicated medical supply and resource chain necessary to facilitate EIMO is provided in the third presentation of this panel. The final presentation in this EIMO panel surveys the breadth and depth of demands on cognitive load expected to be experienced by crew on an exploration mission and proposes strategies to mitigate the prospect of cognitive overload through methods to shift task load from the crew to multi-modal artificial intelligence based medical support systems. Taken together, these presentations summarize the challenges to be expected and potential solution spaces to be explored and developed to progressively enable increasing autonomous medical operations to support crewed missions beyond low earth orbit. Through EIMO focused pre-mission planning, integrated data architecture design, innovative training development and AI-assisted task load management, the gradual transition of medical care and decision making from terrestrial to space-based assets enabling support of astronaut health and performance and reducing overall mission risk is achievable.

Jay Lemery↗

NASA NTRSDate not supplied

Progressively Enabling Earth Independent Medical Operations (EIMO)

This panel presents the findings from a series of Technical Interchange Meetings (TIMs) hosted by the Exploration Medical Capability Element (ExMC) in NASA’s Human Research Program. The topics for the TIMs were derived from a 2-day conference of senior leaders and subject matters experts that collectively outlined a multi-faceted strategy designed to optimize crew health and performance through an increasingly autonomous medical approach. The first abstract in this panel outlines the scope of issues related to data collection, usage, transmission and computing capacity to facilitate EIMO. The second presentation provides an overview of the challenges in developing curricula and advanced training tools to baseline knowledge, skills and abilities (KSA), verify clinical competency and assure retention during prolonged durations inherent in exploration-class missions. An overview of the complicated medical supply and resource chain necessary to facilitate EIMO is provided in the third presentation of this panel. The final presentation in this EIMO panel surveys the breadth and depth of demands on cognitive load expected to be experienced by crew on an exploration mission and proposes strategies to mitigate the prospect of cognitive overload through methods to shift task load from the crew to multi-modal artificial intelligence based medical support systems. Taken together, these presentations summarize the challenges to be expected and potential solution spaces to be explored and developed to progressively enable increasing autonomous medical operations to support crewed missions beyond low earth orbit. Through EIMO focused pre-mission planning, integrated data architecture design, innovative training development and AI-assisted task load management, the gradual transition of medical care and decision making from terrestrial to space-based assets enabling support of astronaut health and performance and reducing overall mission risk is achievable.

John Lemery↗

DOE OSTI2015

Proof-of-Concept Demonstrations for Computation-Based Human Reliability Analysis. Modeling Operator Performance During Flooding Scenarios

The United States (U.S.) Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) program has the overall objective to help sustain the existing commercial nuclear power plants (NPPs). To accomplish this program objective, there are multiple LWRS “pathways,” or research and development (R&D) focus areas. One LWRS focus area is called the Risk-Informed Safety Margin and Characterization (RISMC) pathway. Initial efforts under this pathway to combine probabilistic and plant multi-physics models to quantify safety margins and support business decisions also included HRA, but in a somewhat simplified manner. HRA experts at Idaho National Laboratory (INL) have been collaborating with other experts to develop a computational HRA approach, called the Human Unimodel for Nuclear Technology to Enhance Reliability (HUNTER), for inclusion into the RISMC framework. The basic premise of this research is to leverage applicable computational techniques, namely simulation and modeling, to develop and then, using RAVEN as a controller, seamlessly integrate virtual operator models (HUNTER) with 1) the dynamic computational MOOSE runtime environment that includes a full-scope plant model, and 2) the RISMC framework PRA models already in use. The HUNTER computational HRA approach is a hybrid approach that leverages past work from cognitive psychology, human performance modeling, and HRA, but it is also a significant departure from existing static and even dynamic HRA methods. This report is divided into five chapters that cover the development of an external flooding event test case and associated statistical modeling considerations.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

DOE OSTI2026

Flow Forecasting on New England’s Great River Hydro (CRADA 549) (Final Report)

Great River Hydro (GRH) is New England's largest conventional hydropower generator and operates sustainable hydropower systems in Vermont, New Hampshire, and Massachusetts. Several of GRH’s hydropower projects on the Connecticut River are largely constrained to operate as run-of-river; however, though a new FERC license they have opportunity in the form of flex hours where they are allowed to deviate operations to capture favorable market conditions or emergency operations. To maximize value from these new operations, GRH is considering new sources of forecasted inflow. Pacific Northwest National Laboratory (PNNL) is evaluating the quality of two commercial inflow forecasts for GRH under DOE Water Power Technology Office’s Notice of Opportunity for Technical Assistance: Improving Hydropower’s Value through Informed Decision Making. This project will provide an unbiased evaluation of the accuracy of inflow forecasts available through commercial vendors. GRH will use this information to support their decision on which inflow forecast to use under their new FERC license and to assess the overall value of the available inflow forecasting tools and practices. In this talk we will discuss details of the near real-time forecast evaluation process, the metrics involved in the evaluation, and some preliminary results. In addition, we’ll touch on the second phase of the project which involves evaluating scheduling strategies that identify windows of opportunity to change operations in ways that increase profit through participation in day ahead and real time markets.

13 HYDRO ENERGY↗

NASA NTRS2002

NASA 2nd Generation RLV Program Introduction, Status and Future Plans

The Space Launch Initiative (SLI), managed by the Second Generation Reusable Launch Vehicle (2ndGen RLV) Program, was established to examine the possibility of revolutionizing space launch capabilities, define conceptual architectures, and concurrently identify the advanced technologies required to support a next-generation system. Initial Program funds have been allocated to design, evaluate, and formulate realistic plans leading to a 2nd Gen RLV full-scale development (FSD) decision by 2006. Program goals are to reduce both risk and cost for accessing the limitless opportunities afforded outside Earth's atmosphere fo civil, defense, and commercial enterprises. A 2nd Gen RLV architecture includes a reusable Earth-to-orbit launch vehicle, an on-orbit transport and return vehicle, ground and flight operations, mission planning, and both on-orbit and on-the-ground support infrastructures All segments of the architecture must advance in step with development of the RLV if a next-generation system is to be fully operational early next decade. However, experience shows that propulsion is the single largest contributor to unreliability during ascent, requires the largest expenditure of time for maintenance, and takes a long time to develop; therefore, propulsion is the key to meeting safety, reliability, and cost goals. For these reasons, propulsion is SLI's top technology investment area.

Dumbacher, Dan L.↗

NASA NTRS1998

Shared Situation Awareness in the Flight Deck-ATC System

New technologies and operational concept changes have been proposed for implementation in the National Airspace System (NAS). These changes include improved datalink (CPDLC) technologies for providing improved weather, traffic, Flight Object (FO) and navigation information to the pilot and controller, and new forms of automation for both the flight deck and air traffic management system. In addition, the way business is conducted in the NAS is under consideration. Increases in the discretion provided to pilots (and dispatchers in commercial airlines) are being contemplated in an effort to increase system capacity and flexibility. New concepts of operation (e.g., Collaborative Decision Making and Free Flight) allow for more control to be given to the cockpit or airline with correspondingly greater monitoring responsibilities on the ground. In addition, new technologies and displays make possible much greater information flow between the ground and the cockpit and also dramatic changes in the type of information provided. Designing to support these changes suggests two integrally linked questions: (1) What display technologies and information are needed to support desired changes responsibilities? (2) How will the changes in information availability influence the negotiation process between the cockpit and the ground? Each of these proposed changes (both in technology and operational concept) will have a marked impact on the performance, workload, and Situation Awareness (SA) of both pilots and controllers. Typically such changes are evaluated independently in terms of the effects of the proposed change on either pilot performance or ATC performance. It is proposed here, however, that in order to fully understand the effects of such changes, the joint pilot/controller system must be considered.

Endsley, Mica R.↗