SEARCH · Engineering Papers

Results for “Human-Machine Teaming”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

34 records · Page 2

All sources NASA NTRS DOE OSTI

DOE OSTI2022

A Methodology for Evaluating Operator Usage of Machine Learning Recommendations for Power Grid Contingency Analysis

This work presents the application of a methodology to measure domain expert trust and workload, elicit feedback, and understand the technological usability and impact when a machine learning assistant is introduced into contingency analysis for real-time power grid simulation. The goal of this framework is to rapidly collect and analyze a broad variety of human factors data in order to accelerate the development and evaluation loop for deploying machine learning applications. We describe our methodology and analysis, and we discuss insights gained from a pilot participant about the current usability state of an early technology readiness level (TRL) artificial neural network (ANN) recommender.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2022

Operator Insights and Usability Evaluation of Machine Learning Assistance for Power Grid Contingency Analysis

Introducing machine learning (ML) assistance into any established process comes with adoption barriers, including entrenched procedures, technological and human readiness levels, human-machine trust, and work culture resistance to change. These barriers are even greater in critical operations such as operating a national or regional power grid, in which both regulatory frameworks and the importance of maintaining reliability levels causes additional resistance to the adoption of new computational support. Developers of future systems and job aides must consider not only technical aspects, but also whether new systems are usable by power system operators. This work presents the methodology and results of a study to evaluate the usability and readiness of a prototype recommender system for power grid contingency analysis. We explore operator cognitive load and evaluate operator performance when solving a collection of scenarios both with and without recommender assistance. We also examine operator trust in the system. We report insights gained on the readiness of the system using a collection of evaluation techniques.

Human-Machine Teaming, Power Systems, usability ev↗

NASA NTRSDate not supplied

Resilience Modeling in Complex Engineered Systems with Human-Machine Interactions

In recent times, there has been a growing interest in resilience-based design. Resilience-based design operates on the concept that failures and unexpected events will happen, and when they occur, complex engineered systems should be able to operate within acceptable bounds and recover reasonably. Humans can contribute to the resilience of a system by quickly detecting unforeseen events and taking corrective measures. To this effect, researchers have proposed guidelines and design approaches that can help promote human-system resilience. However, there is no early design stage tool to validate if a system is indeed resilient after applying these guidelines and design methods. In this research, we integrate the Human Error and Functional Failure Reasoning (HEFFR) framework into the fmdtools toolkit to enable designers to model the combined (machine, human, and joint) failures, including their propagation and dynamic effects, during early design stages. This integrated tool also allows designers to model the effects of performance shaping factors, team dynamics, and human-machine interactions in systems of systems. A demonstrative example of a remotely operated rover is explored to demonstrate how this approach can be applied to understand resilience in complex engineered systems with human interactions.

Lukman Irshad↗

DOE OSTI2026

Control Room of the Future Testbed Workshop – After-Action Report

The U.S. Department of Energy’s Office of Electricity is supporting a one-year, multi-laboratory effort to define the needs and requirements for a Control Room of the Future testbed, or CROFT. The effort responds to increasing grid complexity driven by large new loads, dynamic generation resources, and the growing adoption of advanced technologies and tools, including artificial intelligence (AI) and machine learning (ML). To support safe, secure, and effective grid modernization, CROFT will focus on how emerging technologies and tools can be rigorously evaluated in realistic operational settings, with attention to human-machine interaction, cognitive load, and workforce readiness. The project team includes Argonne National Laboratory, Idaho National Laboratory, National Laboratory of the Rockies, and Pacific Northwest National Laboratory. As part of the scoping effort, the team conducted two industry-focused workshops: one at DTECH on February 5, 2026, informed by prior industry interviews, and a second on May 4, 2026, adjacent to IEEE T&D. These engagements brought together utilities, vendors, consultants, national laboratories, academia, and government stakeholders to identify and prioritize use cases, barriers, validation needs, data-sharing constraints, and near- and longer-term requirements. This feedback will directly inform CROFT’s architecture and research focus areas, ensuring the testbed is grounded in real-world operational needs and designed to evaluate emerging technologies and tools in realistic control-room environments.

artificial intelligence↗

NASA NTRSDate not supplied

Defining Collaborative Control Interactions Using Systems Theory

Human teams collaborate by establishing roles, changing functional authorities, maintaining team cognition, coordinating, and mutually helping one another close control loops. These complex inter-actions are inspiring novel concepts to improve human-machine and multi-machine collaboration. However, these new systems face engineering gaps in modeling, analysis, design, and assurance. As such, few have been fielded in safety-critical domains like aerospace. To analyze safety, this paper introduces a system-theoretic framework to describe interactions that are—or are planned to be—used in multi-controller systems. It outlines a taxonomy of seven structural dimensions that influence controller interactions and nine dynamics observed in collaborative control that are defined using Systems Theory. An analyzed set of 101 controller interactions in aerospace systems demonstrates how to apply the framework and that designers are trying to create more sophisticated systems. This framework provides the foundation to extend system-theoretic hazard analysis techniques to systematically find collabora-tive-control causal factors.

Human machine teaming↗

NASA NTRS2007

Telerobotic Surgery: An Intelligent Systems Approach to Mitigate the Adverse Effects of Communication Delay

An extremely innovative approach has been presented, which is to have the surgeon operate through a simulator running in real-time enhanced with an intelligent controller component to enhance the safety and efficiency of a remotely conducted operation. The use of a simulator enables the surgeon to operate in a virtual environment free from the impediments of telecommunication delay. The simulator functions as a predictor and periodically the simulator state is corrected with truth data. Three major research areas must be explored in order to ensure achieving the objectives. They are: simulator as predictor, image processing, and intelligent control. Each is equally necessary for success of the project and each of these involves a significant intelligent component in it. These are diverse, interdisciplinary areas of investigation, thereby requiring a highly coordinated effort by all the members of our team, to ensure an integrated system. The following is a brief discussion of those areas. Simulator as a predictor: The delays encountered in remote robotic surgery will be greater than any encountered in human-machine systems analysis, with the possible exception of remote operations in space. Therefore, novel compensation techniques will be developed. Included will be the development of the real-time simulator, which is at the heart of our approach. The simulator will present real-time, stereoscopic images and artificial haptic stimuli to the surgeon. Image processing: Because of the delay and the possibility of insufficient bandwidth a high level of novel image processing is necessary. This image processing will include several innovative aspects, including image interpretation, video to graphical conversion, texture extraction, geometric processing, image compression and image generation at the surgeon station. Intelligent control: Since the approach we propose is in a sense predictor based, albeit a very sophisticated predictor, a controller, which not only optimizes end effector trajectory but also avoids error, is essential. We propose to investigate two different approaches to the controller design. One approach employs an optimal controller based on modern control theory; the other one involves soft computing techniques, i.e. fuzzy logic, neural networks, genetic algorithms and hybrids of these.

Cardullo, Frank M.↗

NASA NTRS2023

Human Factors Support for On-Orbit Servicing, Assembly and Manufacturing Mission 1 (OSAM-1)

The NASA Engineering and Safety Center (NESC) Human Factors Technical Discipline Team was requested by the Satellite Servicing Projects Division at Goddard Space Flight Center to provide support in assessing the design of the On-Orbit Servicing, Assembly and Manufacturing Mission 1 (OSAM-1) Mission Operations Center in light of concerns about overcrowding and potential for distractions, as well as assistance in creating a Human-Machine Interface style guide for OSAM-1 user interface design. This report contains the outcome of the NESC assessment.

On-Orbit Servicing, Assembly and Manufacturing Mis↗

NASA NTRSDate not supplied

A Distributed Simulation-to-Flight Framework to Support Investigating Trust/Trustworthiness in Multi-Agent Systems

As autonomous systems continue to grow both in use and complexity, the necessity for robust and extensible simulation-to-flight frameworks is paramount for establishing an effective architecture for autonomous systems. Hardware test flights are time-consuming and cost prohibitive during early system design and development. Simulation environments can be useful tools to accelerate algorithm development and testing. However, transitions from simulation to flight (sim-to-flight) can be challenging, unless systems are designed with this transition in mind and with the necessary capabilities built into the architecture and framework. One of the objectives of Autonomy Teaming and TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR) was to design and develop a distributed mixed-reality simulation environment to begin establishing a basis for certification of autonomous systems via research into trust and trustworthiness. ATTRACTOR’s objective was to construct computational concepts of trustworthiness and justifiable trust in multi-agent autonomous teams, to inform future certification of safety-critical and time-critical autonomous systems in aviation. In this paper, we present an autonomous systems architecture and development framework paired with a persistent distributed modeling and simulation (ModSim) environment for test and evaluation of autonomous systems. They were designed under ATTRACTOR in order to measure and establish trustworthiness and trust in single-and multi-agent human-machine systems whether these machines are fixed-wing general aviation, rotary-wing Unmanned Aerial Vehicles (UAVs), ground rovers, or even spacecraft. The Autonomous Entity Operational Network (AEON) framework enables autonomous system development with an easily extensible collection of libraries and plug-n-play nodes facilitated by the Data Distribution Service (DDS) communication protocol standard. The Baseline Environment for Autonomous Modeling (BEAM) simulation environment is a distributed mixed-reality Unity™-based environment built around the same DDS communication paradigm allowing for easy integration with AEON-based autonomous applications, enabling sim-to-flight with minimal configuration changes. Using AEON and BEAM, source code that runs in simulation ports directly to hardware and has successfully flown in the National Airspace System (NAS) at NASA LaRC many times over the lifetime of ATTRACTOR.

Benjamin N Kelley↗

NASA NTRSDate not supplied

ATTRACTOR: Toward Trustworthy and Trusted Autonomous Systems

The question of what it means and what it takes for an autonomous system to consider another autonomous system justifiably trustworthy must be addressed by all who seek to integrate intelligent machine agents into real-world operations. A satisfactory answer to this question is an essential component in accepting autonomous machine decision-making in safety-critical and time-critical environments, such as aviation. Historically, simulation platforms for test and evaluation of complex systems have proven to be effective in assessing performance and contributing to decisions on the fitness of systems to operate in current general and commercial aviation airspace. Moreover, simulations have informed the definition of safety-critical constraints. However, as machine systems progressively take on responsibilities for decision-making traditionally supplied by humans, simulations require enhancement. Mixed reality simulation that integrates real-world platforms and data or high-fidelity simulation data in a sim-to-flight paradigm provides insight into agent interaction and the rationale behind autonomous agent decision-making as well as the capacity for seamless integrated implementation, testing, and operation of systems. Strong simulation capabilities are especially important in the presence of algorithms that hold great promise in decision-making yet increase the uncertainty in the system. Autonomy Teaming and TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR) is a subproject of NASA’s Convergent Aeronautics Solutions (CAS) Project. ATTRACTOR’s objective is to build a basis for understanding trust and trustworthiness in multi-agent autonomous teams, and thus to inform future certification of safety-critical and time-critical autonomous systems in aviation. Because the concepts of trust and trustworthiness must be addressed in a context, ATTRACTOR has chosen Search and Rescue (SAR) in dynamic and unstructured environments, with emphasis on search, as its design reference mission (DRM). During dynamic planning and execution of trajectory-based operations, autonomous agents determine their trajectories given an assigned mission or missions and call for assistance from an appropriate teammate when needed. This experience along with the attendant human-machine and machine-machine interactions, serve as a platform for developing approaches to identifying and measuring trustworthiness and increasing trust. In this paper, we give an overview of some of ATTRACTOR’s research and development activities, findings, and ongoing work.

ATTRACTOR↗

NASA NTRSDate not supplied

A Distributed Simulation-to-Flight Framework to Support Investigating Trust/Trustworthiness in Multi-Agent Systems

As autonomous systems continue to grow both in use and complexity, the necessity for robust and extensible simulation-to-flight methods is paramount for establishing an effective architecture for autonomous systems. A fundamental objective of the ATTRACTOR (Autonomy Teaming and TRAjectories for Complex Trusted Operational Reliability) project was to design and develop a distributed mixed-reality simulation environment to begin establishing a basis for certification of autonomous systems via research into trust and trustworthiness. In this paper, we present an autonomous systems architecture and development framework paired with a persistent distributed modeling and simulation environment for test and evaluation of autonomous systems. The Autonomous Entity Operations Network (AEON) framework enables autonomous system development with an easily extensible collection of libraries and plug-n-play nodes facilitated by the Data Distribution Service (DDS) communication protocol standard. The Baseline Environment for Autonomous Modeling (BEAM) simulation environment is a distributed mixed-reality Unity™-based environment built around the same DDS communication paradigm allowing for easy integration with AEON-based autonomous applications. They were designed under ATTRACTOR in order to measure and establish trustworthiness and trust in single- and multi-agent human-machine systems whether these machines are fixed-wing general aviation, rotary-wing Unmanned Aerial Vehicles (UAVs), ground rovers, or even spacecraft. Together AEON and BEAM enable sim-to-flight with minimal configuration changes. By using AEON and BEAM, source code that runs in simulation ports directly to hardware and has successfully flown in the lab and in the National Airspace System (NAS) at NASA LaRC many times over the lifetime of ATTRACTOR.

Benjamin N Kelley↗

NASA NTRSDate not supplied

A Distributed Simulation-to-Flight Framework to Support Investigating Trust/Trustworthiness in Multi-Agent Systems

Benjamin N Kelley↗

DOE OSTI2023

Advancements in Development and Testing of Thermal Power Dispatch Simulators

Flexible plant operations and generation (FPOG) offer nuclear power plants (NPPs) the chance to leverage alternative, non-electric revenue streams while ensuring their continued role as reliable, clean, and constant sources of baseload electrical power. The excess thermal energy generated from NPPs during periods of low electricity demand can be channeled as raw materials to numerous industrial processes via a thermal power dispatch (TPD) system. Hydrogen production via high-temperature steam electrolysis (HTSE) is an optimal use case based on technical and economic feasibility. Researchers at Idaho National Laboratory (INL) have conducted previous works that developed and implemented TPD system models within the GSE Solutions Generic Pressurized Water Reactor (GPWR) simulator to support human-in-the-loop (HITL) scenario-based evaluations. The first part of this report documents modifications made to the GPWR TPD model and HMI from the previous iteration in line with a new Sargent and Lundy (S&L) TPD design with an automatic control system. The was done in collaboration with Westinghouse using their three-loop pressurizer water reactor (W3LPWR) simulator which contains an industrial grade automatic control system for the TPD. This was installed in the Human Systems Simulation Laboratory (HSSL) at INL. The second part of the report documents findings from an all-hands-on-deck integration and verification workshop that was conducted in the HSSL over several days. The research team comprised INL human factors and TPD experts, a nuclear engineer from GSE Solutions who implemented the revised TPD model for GPWR, the human-machine interface (HMI) prototyping and human factors team from the University of Idaho, and personnel with operations experience with pressurized water reactors. The workshop provided time and expertise to conduct the final activities to bring the operations, HMI, and simulator into a functional state. The goals of the integration and verification workshop were: 1. to install the revised GPWR TPD model into the HSSL 2. verify the TPD HMI prototype was functional 3. integrate the HTSE Simulink model to GPWR. 3. Issues were identified for resolution, but overall the workshop accomplished its goal to integrated and verify the majority of the intended functional. Future work will resolve the identified issues and use the integrated simulation to support an evaluation and demonstration in the next fiscal year.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

NASA NTRS1998

Learning from Automation Surprises and "Going Sour" Accidents: Progress on Human-Centered Automation

Advances in technology and new levels of automation on commercial jet transports has had many effects. There have been positive effects from both an economic and a safety point of view. The technology changes on the flight deck also have had reverberating effects on many other aspects of the aviation system and different aspects of human performance. Operational experience, research investigations, incidents, and occasionally accidents have shown that new and sometimes surprising problems have arisen as well. What are these problems with cockpit automation, and what should we learn from them? Do they represent over-automation or human error? Or instead perhaps there is a third possibility - they represent coordination breakdowns between operators and the automation? Are the problems just a series of small independent glitches revealed by specific accidents or near misses? Do these glitches represent a few small areas where there are cracks to be patched in what is otherwise a record of outstanding designs and systems? Or do these problems provide us with evidence about deeper factors that we need to address if we are to maintain and improve aviation safety in a changing world? How do the reverberations of technology change on the flight deck provide insight into generic issues about developing human-centered technologies and systems (Winograd and Woods, 1997)? Based on a series of investigations of pilot interaction with cockpit automation (Sarter and Woods, 1992; 1994; 1995; 1997a, 1997 b), supplemented by surveys, operational experience and incident data from other studies (e.g., Degani et al., 1995; Eldredge et al., 1991; Tenney et al., 1995; Wiener, 1989), we too have found that the problems that surround crew interaction with automation are more than a series of individual glitches. These difficulties are symptoms that indicate deeper patterns and phenomena concerning human-machine cooperation and paths towards disaster. In addition, we find the same kinds of patterns behind results from studies of physician interaction with computer-based systems in critical care medicine (e.g., Moll van Charante et al., 1993; Obradovich and Woods, 1996; Cook and Woods, 1996). Many of the results and implications of this kind of research are synthesized and discussed in two comprehensive volumes, Billings (1996) and Woods et al. (1994). This paper summarizes the pattern that has emerged from our research, related research, incident reports, and accident investigations. It uses this new understanding of why problems arise to point to new investment strategies that can help us deal with the perceived "human error" problem, make automation more of a team player, and maintain and improve safety.

Woods, David D.↗

NASA NTRS1997

Demonstration of a Safety Analysis on a Complex System

For the past 17 years, Professor Leveson and her graduate students have been developing a theoretical foundation for safety in complex systems and building a methodology upon that foundation. The methodology includes special management structures and procedures, system hazard analyses, software hazard analysis, requirements modeling and analysis for completeness and safety, special software design techniques including the design of human-machine interaction, verification, operational feedback, and change analysis. The Safeware methodology is based on system safety techniques that are extended to deal with software and human error. Automation is used to enhance our ability to cope with complex systems. Identification, classification, and evaluation of hazards is done using modeling and analysis. To be effective, the models and analysis tools must consider the hardware, software, and human components in these systems. They also need to include a variety of analysis techniques and orthogonal approaches: There exists no single safety analysis or evaluation technique that can handle all aspects of complex systems. Applying only one or two may make us feel satisfied, but will produce limited results. We report here on a demonstration, performed as part of a contract with NASA Langley Research Center, of the Safeware methodology on the Center-TRACON Automation System (CTAS) portion of the air traffic control (ATC) system and procedures currently employed at the Dallas/Fort Worth (DFW) TRACON (Terminal Radar Approach CONtrol). CTAS is an automated system to assist controllers in handling arrival traffic in the DFW area. Safety is a system property, not a component property, so our safety analysis considers the entire system and not simply the automated components. Because safety analysis of a complex system is an interdisciplinary effort, our team included system engineers, software engineers, human factors experts, and cognitive psychologists.

Leveson, Nancy↗

NASA NTRS1994

What to Do Until the Money Runs Out: A Refinement Framework for Cognitive Engineering in the Real World

A case study is presented to illustrate some of the problems of applying cognitive science to complex human-machine systems. Disregard for facts about human cognition often undermines the safety, reliability, and cost-effectiveness of complex systems. Yet single-point methods (for example, better user-interface design), whether rooted in computer science or in experimental psychology, fall far short of addressing systems-level problems in a timely way using realistic resources. A model-based methodology is proposed for organizing and prioritizing the cognitive engineering effort, focusing appropriate expertise on major problems first, then moving to more sophisticated refinements if time and resources permit. This case study is based on a collaborative effort between the Human Factors Division at NASA-Ames and the Spaceborne Imaging Radar SIR-C/X-Band Synthetic Aperture Radar (SIR-C/X-SAR) Project at the Jet Propulsion Laboratory (JPL), California institute of Technology. The first SIR-C/X-SAR Shuttle mission flew successfully in April, 1994. A series of such missions is planned to provide radar data to study Earth's ecosystems, climatic and geological processes, hydrologic cycle, and ocean circulation. In addition to JPL and NASA personnel, the SIR-C/X-SAR operations team included Scientists and engineers from the German and Italian space agencies.

Shafto, Michael G.↗

NASA NTRSDate not supplied

Examining the Changing Roles and Responsibilities of Humans in Envisioned Future In-Time Aviation Safety Management Systems

Advances in technology are enabling new concepts of operations that will trans-form aviation including increasingly autonomous capabilities to handle evolving complex dynamic ecosystems like those associated with Advanced Aerial Mobility. A major challenge is how to ensure today’s safety levels are maintained as the system scales for rapid detection and timely mitigation of safety issues. NASA has developed a concept of operation for In-Time Aviation Safety Management Systems (IASMS) that represents a system-of-system perspective on interconnected capabilities needed to proactively reduce risk in complex operational environments where unknown hazards may exist. As a result, NASA research priorities include under-standing how the balance between humans and automation changes in such envisioned systems, which may lead to novel human-machine interaction paradigms and human-autonomy teaming for informed contingency management.

Lawrence Prinzel↗