Engineering PapersSearch

SEARCH · Engineering Papers

Results for “human in the loop”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Agentic traffic intelligence: Augmented human-in-the-loop scenario generation for microscopic traffic simulation

Traditional microscopic traffic simulation generation often relies on static datasets and manual design, limiting its ability to simulate complex conditions easily. This paper presents a novel framework, Agentic Traffic Intelligence, which combines human approval large language models (LLMs), the Real-Twin tool, and multi-agent systems to perform realistic microscopic traffic simulation scenario generation. The proposed framework incorporates human-in-the-loop (HIL) control, retrieval-augmented generation (RAG), and multi-agent control mechanisms. HIL mechanisms are used to guide multiple LLMs focused on attributes for microscopic simulation generation and to improve the interpretability and transparency of LLM execution for users. RAG enhances context extraction by dynamically integrating external knowledge sources for traffic scenario generation foundations. A multi-agent architecture with supervisory control coordinates the interaction of simulation components, including traffic simulators, control logic, and calibration tools. This enables the synthesis of simulation-ready scenarios that reflect dynamic demand profiles and behavior controls. Furthermore, the framework fuses multisource traffic data with unstructured context and supports iterative refinement through interactive user feedback. Validated through microscopic simulation using Simulation of Urban Mobility, the generated scenarios demonstrate high-fidelity network generation with inflow and turn movement and behavioral calibration, offering a robust and efficient tool for stress-testing and optimizing urban mobility systems.

Hierarchical multi-agent control

Objective Measurement Assessment of Departure Advisories for Ramp Controllers from a Human-In-The-Loop Simulation

NASA has developed and demonstrated an integrated arrival, departure, and surface concept and technology for efficient air traffic management in busy terminal environments under the Airspace Technology Demonstration 2 (ATD-2) subproject. In April/May 2019, NASA conducted a human-in-the-loop (HITL) simulation with ramp and tower controllers having experience at Dallas/Fort Worth International Airport (DFW). The purpose of this HITL was to evaluate the impacts of various surface metering goals in ramp operations at DFW and test new features of the ATD-2 ramp controller decision support tools. This paper evaluates the quantitative metrics from the simulation results related to airport performance and surface metering given various departure scheduling advisories for ramp controllers. The objective measurements compared include the controller’s compliance with the target times for pushback and spot arrival, the number of metered flights, aircraft gate hold and taxi time, runway throughput, and the number of aircraft on the surface. The simulation results show that there were no statistically significant benefits from surface metering at DFW for airport performance due to several simulation artifacts notably different from real operations. Considering controller workload and situation awareness, following either gate pushback or spot arrival advisory could be a better metering option than utilizing both advisories when surface metering is applied at DFW.

airport ramp traffic control decision support

Case Study: Influences of Uncertainties and Traffic Scenario Difficulties in a Human-in-the-Loop Simulation

This paper presents a case study of how factors such as wind prediction errors and metering delays can influence controller performance and workload in Human-In-The-Loop simulations. Retired air traffic controllers worked two arrival sectors adjacent to the terminal area. The main tasks were to provide safe air traffic operations and deliver the aircraft to the metering fix within +/- 25 seconds of the scheduled arrival time with the help of provided decision support tools. Analyses explore the potential impact of metering delays and system uncertainties on controller workload and performance. The results suggest that trajectory prediction uncertainties impact safety performance, while metering fix accuracy and workload appear subject to the scenario difficulty.

air traffic control

Unmanned Aircraft Systems Human-in-the-Loop Controller and Pilot Acceptability Study: Collision Avoidance, Self-Separation, and Alerting Times (CASSAT)

The Federal Aviation Administration (FAA) has been mandated by the Congressional funding bill of 2012 to open the National Airspace System (NAS) to Unmanned Aircraft Systems (UAS). With the growing use of unmanned systems, NASA has established a multi-center "UAS Integration in the NAS" Project, in collaboration with the FAA and industry, and is guiding its research efforts to look at and examine crucial safety concerns regarding the integration of UAS into the NAS. Key research efforts are addressing requirements for detect-and-avoid (DAA), self-separation (SS), and collision avoidance (CA) technologies. In one of a series of human-in-the-loop experiments, NASA Langley Research Center set up a study known as Collision Avoidance, Self-Separation, and Alerting Times (CASSAT). The first phase assessed active air traffic controller interactions with DAA systems and the second phase examined reactions to the DAA system and displays by UAS Pilots at a simulated ground control station (GCS). Analyses of the test results from Phase I and Phase II are presented in this paper. Results from the CASSAT study and previous human-in-the-loop experiments will play a crucial role in the FAA's establishment of rules, regulations, and procedures to safely, efficiently, and effectively integrate UAS into the NAS.

Comstock, James R., Jr.

A dual-loop model of the human controller

A representative model of the human controller in single-axis compensatory tracking tasks that exhibits an internal feedback loop which is not evident in single-loop models now in common use is presented. This hypothetical inner-loop involves a neuromuscular command signal derived from the time rate of change of controlled element output which is due to control activity. It is not contended that the single-loop human controller models now in use are incorrect, but that they contain an implicit but important internal loop closure, which, if explicitly considered, can account for a good deal of the adaptive nature of the human controller in a systematic manner.

Hess, R. A.

Flight Deck Robustness/Conformance Testing with a Surface Management System: An Integrated Pilot-Controller Human-in-the-Loop Surface Operations Simulation

An integrated flight deck and controller human-in-the-loopsimulation was conducted with a total of 120 Dallas-Ft.Worth (DFW) taxi-out operations. In this first integratedPilot-Controller Spot and Runway Departure Advisor(SARDA) simulation, ATC Ground and Local Controllersused the SARDA decision support tool to plan and issuespot release clearances and departure clearances. TheAirport and Terminal Area Simulator (ATAS), a simulatedB737NG piloted, in turn, by 10 commercial transportparticipant pilots, was integrated into the realisticsimulation traffic environment. In the simulation,controllers used SARDA advisories to issue spot release,taxi route, and runway/departure radio voice clearances toall aircraft on the airport surface. Simulation resultsindicated that under a variety of observed pilot/aircraftperformance variations, SARDA yielded controlleradvisories that were: Supportive of current-day time-basedoperations; Compatible with controllers’ expectations;Predictive of actual take-off times; and, Adaptable to offnominalevents. An Information Sharing Display, thatpresented SARDA sequence and timing information on theflight deck, was considered useful for both NextGenoperations and current-day time-based Traffic ManagementInitiative (TMI) operations.

Foyle, David C.

Identifying Functional Requirements for Flexible Airspace Management Concept Using Human-In-The-Loop Simulations

Flexible Airspace Management (FAM) is a mid- term Next Generation Air Transportation System (NextGen) concept that allows dynamic changes to airspace configurations to meet the changes in the traffic demand. A series of human-in-the-loop (HITL) studies have identified procedures and decision support requirements needed to implement FAM. This paper outlines a suggested FAM procedure and associated decision support functionality based on these HITL studies. A description of both the tools used to support the HITLs and the planned NextGen technologies available in the mid-term are presented and compared. The mid-term implementation of several NextGen capabilities, specifically, upgrades to the Traffic Management Unit (TMU), the initial release of an en route automation system, the deployment of a digital data communication system, a more flexible voice communications network, and the introduction of a tool envisioned to manage and coordinate networked ground systems can support the implementation of the FAM concept. Because of the variability in the overall deployment schedule of the mid-term NextGen capabilities, the dependency of the individual NextGen capabilities are examined to determine their impact on a mid-term implementation of FAM. A cursory review of the different technologies suggests that new functionality slated for the new en route automation system is a critical enabling technology for FAM, as well as the functionality to manage and coordinate networked ground systems. Upgrades to the TMU are less critical but important nonetheless for FAM to be fully realized. Flexible voice communications network and digital data communication system could allow more flexible FAM operations but they are not as essential.

dynamic airspace configuration

Registering Articulated Objects With Human-in-the-loop Corrections

Remotely programming robots to execute tasks often relies on registering objects of interest in the robot’s environment. Frequently, these tasks involve articulating objects such as opening or closing a valve. However, existing human-in-the-loop methods for registering objects do not consider articulations and the corresponding impact to the geometry of the object, which can cause the methods to fail. In this work, we present an approach where the registration system attempts to automatically determine the object model, pose, and articulation for user-selected points using a nonlinear iterative closest point algorithm. When the automated fitting is incorrect, the operator can iteratively intervene with corrections after which the system will refit the object. We present an implementation of our fitting procedure and evaluate it with a user study that shows that it can improve user performance, in measures of time on task and task load, ease of use, and usefulness compared to a manual registration approach. We also present a situated example that demonstrates the integration of our method in an end-to-end system for articulating a remote valve.

teleoperation

A human-in-the-loop explanation framework for morphologically transparent AI predictions from whole-slide images

Deep learning models enable the prediction of clinical endpoints from whole-slide images (WSIs), but many such models function as “black boxes”, lacking transparency about whether and which histomorphological patterns drive their predictions, hindering interpretability and clinical adoption. Here we propose a human-in-the-loop explanation framework, MorphoXAI, which provides both local and global interpretability for deep learning models by incorporating human-expert interpretations. At the global level, it reveals the histomorphological patterns on which the model consistently relies to distinguish between classes of WSIs, as well as the patterns associated with confusion between classes. At the local level, it indicates which of these patterns are used in the prediction of an individual WSI and which regions within the slide correspond to such patterns. We validated our method across multiple deep learning–based WSI analysis tasks spanning different tissue types. The results show that our framework generates explanations that accurately reflect the histomorphology underlying the model’s predictions at both global and local levels. For interpretability and clinical utility in diagnostic contexts, human evaluation results showed that our explanations were easy to interpret, rich in diagnostic features, and directly helpful for diagnostic decision-making, thereby enhancing pathologist-AI collaboration. Our work highlights that unifying global and local explanations and grounding them in expert-interpreted morphology enhances the interpretability and verifiability of deep learning models, thereby facilitating the transparent deployment of such models in clinical practice.

Lou, Peiliang

Evaluation of a Tactical Surface Metering Tool for Charlotte Douglas International Airport via Human-in-the-Loop Simulation

NASA has been working with the FAA and aviation industry partners to develop and demonstrate new concepts and technologies that integrate arrival, departure, and surface traffic management capabilities. In March 2017, NASA conducted a human-in-the-loop (HITL) simulation for integrated surface and airspace operations, modeling Charlotte Douglas International Airport, to evaluate the operational procedures and information requirements for the tactical surface metering tool, and data exchange elements between the airline controlled ramp and ATC Tower. In this paper, we focus on the calibration of the tactical surface metering tool using various metrics measured from the HITL simulation results. Key performance metrics include gate hold times from pushback advisories, taxi-in-out times, runway throughput, and departure queue size. Subjective metrics presented in this paper include workload, situational awareness, and acceptability of the metering tool and its calibration.

Verma, Savita

Comparison of Ground-Based and Airborne Function Allocation Concepts for NextGen Using Human-In-The-Loop Simulations

Investigation of function allocation for the Next Generation Air Transportation System is being conducted by the National Aeronautics and Space Administration (NASA). To provide insight on comparability of different function allocations for separation assurance, two human-in-the-loop simulation experiments were conducted on homogeneous airborne and ground-based approaches to four-dimensional trajectory-based operations, one referred to as ground-based automated separation assurance (groundbased) and the other as airborne trajectory management with self-separation (airborne). In the coordinated simulations at NASA s Ames and Langley Research Centers, controllers for the ground-based concept at Ames and pilots for the airborne concept at Langley managed the same traffic scenarios using the two different concepts. The common scenarios represented a significant increase in airspace demand over current operations. Using common independent variables, the simulations varied traffic density, scheduling constraints, and the timing of trajectory change events. Common metrics were collected to enable a comparison of relevant results. Where comparisons were possible, no substantial differences in performance or operator acceptability were observed. Mean schedule conformance and flight path deviation were considered adequate for both approaches. Conflict detection warning times and resolution times were mostly adequate, but certain conflict situations were detected too late to be resolved in a timely manner. This led to some situations in which safety was compromised and/or workload was rated as being unacceptable in both experiments. Operators acknowledged these issues in their responses and ratings but gave generally positive assessments of the respective concept and operations they experienced. Future studies will evaluate technical improvements and procedural enhancements to achieve the required level of safety and acceptability and will investigate the integration of airborne and ground-based capabilities within the same airspace to leverage the benefits of each concept.

Wing, David J.

Human-In-The-Loop Experimental Research for Detect and Avoid

This paper describes a Detect and Avoid (DAA) concept for integration of UAS into the NAS developed by the National Aeronautics and Space Administration (NASA) and provides results from recent human-in-the-loop experiments performed to investigate interoperability and acceptability issues associated with these vehicles and operations. The series of experiments was designed to incrementally assess critical elements of the new concept and the enabling technologies that will be required.

Consiglio, Maria

UAS-NAS Integrated Human in the Loop: Test Environment Report

The desire and ability to fly Unmanned Aircraft Systems (UAS) in the National Airspace System (NAS) is of increasing urgency. The application of unmanned aircraft to perform national security, defense, scientific, and emergency management are driving the critical need for less restrictive access by UAS to the NAS. UAS represent a new capability that will provide a variety of services in the government (public) and commercial (civil) aviation sectors. The growth of this potential industry has not yet been realized due to the lack of a common understanding of what is required to safely operate UAS in the NAS. NASA's UAS Integration in the NAS Project is conducting research in the areas of Separation Assurance/Sense and Avoid Interoperability (SSI), Human Systems Integration (HSI), and Communication to support reducing the barriers of UAS access to the NAS. This research was broken into two research themes namely, UAS Integration and Test Infrastructure. UAS Integration focuses on airspace integration procedures and performance standards to enable UAS integration in the air transportation system, covering Sense and Avoid (SAA) performance standards, command and control performance standards, and human systems integration. The focus of the Test Infrastructure theme was to enable development and validation of airspace integration procedures and performance standards, including the execution of integrated test and evaluation. In support of the integrated test and evaluation efforts, the Project developed an adaptable, scalable, and schedulable relevant test environment incorporating live, virtual, and constructive elements capable of validating concepts and technologies for unmanned aircraft systems to safely operate in the NAS. To accomplish this task, the Project planned to conduct three integrated events: a Human-in-the-Loop simulation and two Flight Test series that integrated key concepts, technologies and/or procedures in a relevant air traffic environment. Each of the integrated events were built on the technical achievements, fidelity and complexity of previous simulations and tests, resulting in research findings that support the development of regulations governing the access of UAS into the NAS. The purpose of this document is to describe how well the system under test was representative

Human in the Loop

Managing Demand and Capacity Using Multi-Sector Planning and Flexible Airspace: Human-in-the-Loop Evaluation of NextGen

When demand for an airspace sector exceeds capacity, the balance can be re-established by reducing the demand, increasing the capacity, or both. The Multi-Sector Planner (MSP) concept has been proposed to better manage traffic demand by modifying trajectories across multiple sectors. A complementary approach to MSP, called Flexible Airspace Management (FAM), reconfigures the airspace such that capacity can be reallocated dynamically to balance the traffic demand across multiple sectors, resulting in fewer traffic management initiatives. The two concepts have been evaluated with a series of human-in-the-loop simulations at the Airspace Operations Laboratory to examine and refine the roles of the human operators in these concepts, as well as their tools and procedural requirements. So far MSP and FAM functions have been evaluated individually but the integration of the two functions is desirable since there are significant overlaps in their goals, geographic/temporal scope of the problem space, and the implementation timeframe. Ongoing research is planned to refine the humans roles in the integrated concept.

Lee, Paul U.

Human-in-the-loop evaluation of RMS Active Damping Augmentation

Active Damping Augmentation is the insertion of Controls-Structures Integration Technology to benefit the on-orbit performance of the Space Shuttle Remote Manipulator System. The goal is to reduce the vibration decay time of the Remote Manipulator System following normal payload maneuvers and operations. Simulation of Active Damping Augmentation was conducted in the realtime human-in-the-loop Systems Engineering Simulator at the NASA Johnson Space Center. The objective of this study was to obtain a qualitative measure of operational performance improvement from astronaut operators and to obtain supporting quantitative performance data. Sensing of vibratory motions was simulated using a three-axis accelerometer mounted at the end of the lower boom of the Remote Manipulator System. The sensed motions were used in a feedback control law to generate commands to the joint servo mechanisms which reduced the unwanted oscillations. Active damping of the Remote Manipulator System with an attached 3990 lb. payload was successfully demonstrated. Six astronaut operators examined the performance of an Active Damping Augmentation control law following single-joint and coordinated six-joint translational and rotational maneuvers. Active Damping Augmentation disturbance rejection of Orbiter thruster firings was also evaluated. Significant reductions in the dynamic response of the 3990 lb. payload were observed. Astronaut operators recommended investigation of Active Damping Augmentation benefits to heavier payloads where oscillations are a bigger problem (e.g. Space Station Freedom assembly operators).

Demeo, Martha E.

Teal Cycle Ergometer Vibration Isolation and Stabilization System Human-in-the-Loop Ground Evaluation for Long-Duration Spaceflight

Purpose: To counter the deleterious effects of weightlessness on the cardiopulmonary system, astronauts living on the International Space Station exercise on a variety of countermeasure equipment including the Cycle Ergometer Vibration Isolation and Stabilization System (CEVIS). Operational since 2001, the onboard CEVIS will be replaced by a new model, known as Teal CEVIS (TC). As a part of ground evaluation, TC hardware underwent human-in-the-loop (HITL) testing to verify the TC hardware produces workloads that elicit physiologic responses comparable to a laboratory cycle ergometer (LAB). Methods: Seven subjects (5 M/2 F) performed submaximal cycle ergometer testing with indirect calorimetry measures on TC and LAB on separate test days. Testing consisted of graded 30 watt increases in workload until subjects reached 85% of age-predicted max heart rate (HR). Exercise outcomes included rate of oxygen uptake (VO2; liters/min), rate of energy expenditure (REE; kcal/min), and HR (beats/min). Linear mixed models (LMM) were fitted to compare VO2, REE, and HR responses between devices across power outputs with fixed effects for power (P) and device (D) and with random effects for subject. LMM effect coefficients (β), std errors (SE), pseudo-partial R2 of effects (pR2), and model likelihood ratio statistics (χ2, Pr(> chisq); α<.05) are provided. Results: LMM main effects for P and D were observed for VO2 (βp=.0107, SE = .0002, pR2=0.97; βd = -0.075, SE = 0.019, pR2=0.14; χ2(1)= 13.57, p<.001), such that VO2 was higher across stages on TC. Main effects of P and D were observed for REE (βp= .0586, SE = 0.001, pR2= 0.97; βd = -0.33208, SE = 0.099, pR2= 0.056; χ2(1)=10.481, p < .01), such that REE was higher across power outputs on TC. A P x D interaction was observed for HR, along with main effects for P and D (βp x d =-0.048, SE = 0.02, pR2=0.048; βp= 0.42, SE = 0.016, pR2=0.87; χ2(1) = 5.398, p = 0.02), such that higher workloads elicited a greater difference in HR between devices. Conclusions: HITL results show, TC elicits greater physiologic responses across power outputs compared to LAB. However, pR2 for device effects show small differences between devices. Therefore, TC can be expected to provide appropriate physiological stimulus across workloads and be considered a reliable tool to mitigate the effects of weightlessness.

Cycle Ergometer Vibration and Isolation System

Space Suit Portable Life Support System (PLSS) 2.0 Human-in-the-Loop (HITL) Testing

The space suit Portable Life Support System (PLSS) 2.0 represents the second integrated prototype developed and tested to mature a design that uses advanced technologies to reduce consumables, improve robustness, and provide additional capabilities over the current state of the art. PLSS 2.0 was developed in 2012, with extensive functional evaluations and system performance testing through mid-2014. In late 2014, PLSS 2.0 was integrated with the Mark III space suit in an ambient laboratory environment to facilitate manned testing, designated PLSS 2.0 Human-in-the-Loop (HITL) testing, in which the PLSS prototype performed the primary life support functions, including suit pressure regulation, ventilation, carbon dioxide control, and cooling of the test subject and PLSS avionics. The intent of this testing was to obtain subjective test subject feedback regarding qualitative aspects of PLSS 2.0 performance such as thermal comfort, sounds, smells, and suit pressure fluctuations due to the cycling carbon dioxide removal system, as well as to collect PLSS performance data over a range of human metabolic rates from 500-3000 Btu/hr. Between October 27 and December 18, 2014, nineteen two-hour simulated EVA test points were conducted in which suited test subjects walked on a treadmill to achieve a target metabolic rate. Six test subjects simulated nominal and emergency EVA conditions with varied test parameters including metabolic rate profile, carbon dioxide removal control mode, cooling water temperature, and Liquid Cooling and Ventilation Garment (state of the art or prototype). The nineteen test points achieved more than 60 hours of test time, with 36 hours accounting for simulated EVA time. The PLSS 2.0 test article performed nominally throughout the test series, confirming design intentions for the advanced PLSS. Test subjects' subjective feedback provided valuable insight into thermal comfort and perceptions of suit pressure fluctuations that will influence future advanced PLSS design and testing strategies.

Watts, Carly

Space Suit Portable Life Support System (PLSS) 2.0 Human-in-the-Loop (HITL) Testing

The space suit Portable Life Support System (PLSS) 2.0 represents the second integrated prototype developed and tested to mature a design that uses advanced technologies to reduce consumables, improve robustness, and provide additional capabilities over the current state of the art. PLSS 2.0 was developed in 2012, with extensive functional evaluations and system performance testing through mid-2014. In late 2014, PLSS 2.0 was integrated with the Mark III space suit in an ambient laboratory environment to facilitate manned testing, designated PLSS 2.0 Human-in-the-Loop (HITL) testing, in which the PLSS prototype performed the primary life support functions, including suit pressure regulation, ventilation, carbon dioxide control, and cooling of the test subject and PLSS avionics. The intent of this testing was to obtain subjective test subject feedback regarding qualitative aspects of PLSS 2.0 performance such as thermal comfort, sounds, smells, and suit pressure fluctuations due to the cycling carbon dioxide removal system, as well as to collect PLSS performance data over a range of human metabolic rates from 500-3000 Btu/hr. Between October 27 and December 18, 2014, nineteen two-hour simulated EVA test points were conducted in which suited test subjects walked on a treadmill to achieve a target metabolic rate. Six test subjects simulated nominal and emergency EVA conditions with varied test parameters including metabolic rate profile, carbon dioxide removal control mode, cooling water temperature, and Liquid Cooling and Ventilation Garment (state of the art or prototype). The nineteen test points achieved more than 60 hours of test time, with 36 hours accounting for simulated EVA time. The PLSS 2.0 test article performed nominally throughout the test series, confirming design intentions for the advanced PLSS. Test subjects' subjective feedback provided valuable insight into thermal comfort and perceptions of suit pressure fluctuations that will influence future advanced PLSS design and testing strategies.

Watts, Carly