Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “usability studies”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Dynamic Tunnel Usability Study: Format Recommendations for Synthetic Vision System Primary Flight Displays

A usability study evaluating dynamic tunnel concepts has been completed under the Aviation Safety and Security Program, Synthetic Vision Systems Project. The usability study was conducted in the Visual Imaging Simulator for Transport Aircraft Systems (VISTAS) III simulator in the form of questionnaires and pilot-in-the-loop simulation sessions. Twelve commercial pilots participated in the study to determine their preferences via paired comparisons and subjective rankings regarding the color, line thickness and sensitivity of the dynamic tunnel. The results of the study showed that color was not significant in pilot preference paired comparisons or in pilot rankings. Line thickness was significant for both pilot preference paired comparisons and in pilot rankings. The preferred line/halo thickness combination was a line width of 3 pixels and a halo of 4 pixels. Finally, pilots were asked their preference for the current dynamic tunnel compared to a less sensitive dynamic tunnel. The current dynamic tunnel constantly gives feedback to the pilot with regard to path error while the less sensitive tunnel only changes as the path error approaches the edges of the tunnel. The tunnel sensitivity comparison results were not statistically significant.

Arthur, Jarvis J., III↗

Usability Study of Two Collocated Prototype System Displays

Currently, most of the displays in control rooms can be categorized as status screens, alerts/procedures screens (or paper), or control screens (where the state of a component is changed by the operator). The primary focus of this line of research is to determine which pieces of information (status, alerts/procedures, and control) should be collocated. Two collocated displays were tested for ease of understanding in an automated desktop survey. This usability study was conducted as a prelude to a larger human-in-the-loop experiment in order to verify that the 2 new collocated displays were easy to learn and usable. The results indicate that while the DC display was preferred and yielded better performance than the MDO display, both collocated displays can be easily learned and used.

Trujillo, Anna C.↗

Earthdata Search Usability Study Process

User experience (UX) design is the process of enhancing user satisfaction by improving various aspects of the user's interaction with an application or website. One aspect of UX design is usability, or the extent to which an application can be used to to accomplish tasks efficiently, effectively, and with satisfaction. NASA's Earthdata Search Client recently underwent a focused usability testing project to measure usability and gain valuable user feedback and insights to increase usability for its end-users. This presentation focuses on the process by which the usability tests were administered and the lessons learned throughout the process.

UI↗

Airborne Precision Spacing for Dependent Parallel Operations Interface Study

This paper describes a usability study of proposed cockpit interfaces to support Airborne Precision Spacing (APS) operations for aircraft performing dependent parallel approaches (DPA). NASA has proposed an airborne system called Pair Dependent Speed (PDS) which uses their Airborne Spacing for Terminal Arrival Routes (ASTAR) algorithm to manage spacing intervals. Interface elements were designed to facilitate the input of APS-DPA spacing parameters to ASTAR, and to convey PDS system information to the crew deemed necessary and/or helpful to conduct the operation, including: target speed, guidance mode, target aircraft depiction, and spacing trend indication. In the study, subject pilots observed recorded simulations using the proposed interface elements in which the ownship managed assigned spacing intervals from two other arriving aircraft. Simulations were recorded using the Aircraft Simulation for Traffic Operations Research (ASTOR) platform, a medium-fidelity simulator based on a modern Boeing commercial glass cockpit. Various combinations of the interface elements were presented to subject pilots, and feedback was collected via structured questionnaires. The results of subject pilot evaluations show that the proposed design elements were acceptable, and that preferable combinations exist within this set of elements. The results also point to potential improvements to be considered for implementation in future experiments.

Volk, Paul M.↗

Systematization of Password Manager Use Cases and Design Paradigms

Despite efforts to replace them, passwords remain the primary form of authentication on the web. Password managers seek to address many of the problems with passwords by helping users generate, store, and fill strong and unique passwords. Even though experts frequently recommend password managers, there is limited information regarding their usability. To aid in designing such usability studies, we systematize password manager use cases, identifying ten essential use cases, three recommended use cases, and four extended use cases. We also systematize the system designs employed to satisfy these use cases, designs that should be examined in usability studies to understand their relative strengths and weaknesses. Finally, we describe observations from 136 cognitive walkthroughs exploring the identified essential use cases in eight popular managers. Ultimately, we expect that this work will serve as the foundation for an explosion of new research into the usability of password managers.

Simmons, James↗

JSC Search System Usability Case Study

The advanced nature of "search" has facilitated the movement from keyword match to the delivery of every conceivable information topic from career, commerce, entertainment, learning... the list is infinite. At NASA Johnson Space Center (JSC ) the Search interface is an important means of knowledge transfer. By indexing multiple sources between directorates and organizations, the system's potential is culture changing in that through search, knowledge of the unique accomplishments in engineering and science can be seamlessly passed between generations. This paper reports the findings of an initial survey, the first of a four part study to help determine user sentiment on the intranet, or local (JSC) enterprise search environment as well as the larger NASA enterprise. The survey is a means through which end users provide direction on the development and transfer of knowledge by way of the search experience. The ideal is to identify what is working and what needs to be improved from the users' vantage point by documenting: (1) Where users are satisfied/dissatisfied (2) Perceived value of interface components (3) Gaps which cause any disappointment in search experience. The near term goal is it to inform JSC search in order to improve users' ability to utilize existing services and infrastructure to perform tasks with a shortened life cycle. Continuing steps include an agency based focus with modified questions to accomplish a similar purpose

Meza, David↗

Replication Package for "Union-Find and Usability: A Case Study and Analysis of Rust Formal Verifier

This replication package is a case study on automated deductive verification for Rust for practical programs. It is a companion artifact to a corresponding usability study on verification titled "Union-Find and Usability: A Case Study and Analysis of Rust Formal Verifiers". It seeks to answer the question "Can Rust developers today use Rust verifiers to verify their code?". To answer this question, the study contrasts the verification experience of two mature Rust verifiers, Creusot and Prusti, by using the tools to develop a verified implementation of union-find in Rust. The union-find implementation is based on real-world code as used in the popular egg E-graph library. The artifact consists of two different verified libraries, one using Creusot and one using Prusti. The libraries have similar Rust interfaces and high-level proofs but differ in their details: Creusot and Prusti have different annotation languages and support different proof styles. Each implementation can be verified with its respective tool and compiles as a traditional Rust development.

Sarracino, John↗

Integrating Mission Timelines and Procedures to Enhance Situational Awareness in Human Spaceflight Operations

Future human spaceflight missions to the Moon, Mars, and beyond will have significant transmission delays and reduced communications bandwidth compared to operations in Low Earth Orbit, requiring increased crew autonomy due to the limited availability of real-time support from Mission Control Center (MCC). In this paper, we argue for the development of technology to enhance situational awareness (SA) for a remote MCC and astronauts onboard the vehicle, describe a usability study conducted towards this aim, and identify key usability features for the design of future integrated timeline tools. By tightly integrating timelines and procedures, future crew systems can predict future constraint violations, suggest resolutions to these violations, and ultimately support more successful missions.

User experience↗

Integrating Mission Timelines and Procedures to Enhance Situational Awareness in Human Spaceflight Operations

Future human spaceflight missions to the Moon, Mars, and beyond will have significant transmission delays and reduced communications bandwidth compared to operations in Low Earth Orbit, requiring increased crew autonomy due to the limited availability of real-time support from Mission Control Center (MCC). In this paper, we argue for the development of technology to enhance situational awareness (SA) for a remote MCC and astronauts onboard the vehicle, describe a usability study conducted towards this aim, and identify key usability features for the design of future integrated timeline tools. By tightly integrating timelines and procedures, future crew systems can predict future constraint violations, suggest resolutions to these violations, and ultimately support more successful missions.

user experience↗

Usability Evaluation of Fleet Management Interface for High Density Vertiplex Environments

To meet the rising demand for an Advanced Air Mobility (i.e. urban and rural unmanned aircraft systems) ecosystem, NASA Aeronautics Research Mission Directorate (ARMD) is hosting a series of simulations and flight tests under the High Density Vertiplex sub-project (HDV). HDV aims to develop an integrated automation architecture to support terminal area flight operations. The HDV simulations and flight tests address safety, integration, and operational challenges, while integrated systems and software demonstrate design readiness, robustness, and interoperability. During the initial HDV simulation in 2021, a prototype traffic management tool developed by NASA called Fleet Management Interface (FMI) was tested. FMI was designed to introduce an advanced level of human-automation interaction to aid both Ground Control Station Operators (GCSOs) and Fleet Managers (FMs) in remotely managing flights under their ownership. In a human-in-the-loop simulation, a usability study was conducted with FMI to identify optimal approaches for displaying information for human operators using subjective measures of usability, workload, situation awareness, risk, and trust, along with qualitative feedback. This study consisted of task analysis in which GCSO and FM subjects used an Urban Air Mobility (UAM) environment to develop and execute a plan for two different traffic scenarios of remotely controlled vehicles. In each scenario, a controlled vehicle completed a takeoff, active flight, and landing sequence while automated traffic flew in the background at a rate of 20 operations per hour. In the first scenario, the controlled vehicle flew a nominal route with takeoff and landing at the same vertiport. In the second scenario, the controlled vehicle started on the nominal route, then diverted to an unplanned location mid-flight. Results showed that self-reported performance, usability, trust, and situation awareness ratings of FMI were moderately to strongly high. There were small differences between scenarios, with Scenario 2 being perceived as more unstable, complex, variable, risky, and potentially harmful than Scenario 1. Furthermore, participants described improvements that could be made to create a better user experience. For example, users would like greater configurability of the interface based on their personal information requirements, and they would like the opportunity to review routes before assigning them. The results from this study will inform future development of the FMI with the end goal of creating a reference automation tool for airspace management procedures in AAM. The FMI could be introduced as a potential way to reduce dependency on traditional air navigation services through increased automation in high density vertiplex environments.

vertiplex↗

Development of a Safety Hazards Risk Assessment Tool for Uncrewed Aircraft System Traffic Management during Preflight Planning

Tremendous growth in the uncrewed and remotely piloted vehicle market is expected in low-altitude, uncontrolled airspace, resulting in potential decreases in safety without systems that support monitoring, assessing, and mitigating risk. At NASA, the System-Wide Safety (SWS) project has been developing a suite of data-driven tools to predict hazards so that the potential risks that these hazards pose can be mitigated. Services to predict various hazards have been developed, including battery capacity, proximity to static obstacles, population risks, global positioning system signal strength, radio frequency spectrum interference risk, and vertiport congestion. These services can monitor hazards along a flight path and if any risks posed by these hazards exceed a threshold, the uncrewed aircraft system (UAS) fleet manager can be alerted to mitigate the risk by modifying the flight path, changing the scheduled departure or arrival times, and/or diverting the vehicle to an alternate vertiport. These services were originally developed to monitor and assess risks during flight, but they have been adapted to assess hazard risks prior to departure so that a fleet manager can evaluate the potential risks for a fleet of UAS along their planned flight paths. These services have been integrated into a prototype tool called the Supplemental Data Service Provider-Consolidated Dashboard (SDSP-CD), developed at NASA Ames Research Center. The tool consists of a dashboard which provides a comprehensive overview for a number of risks and a map display that shows the details of the hazards along each flight’s path. Based on the findings from three previous studies, the SDSP-CD has been updated with new design elements and functions. In this paper, we describe lessons learned from the previous studies, changes made to the interface, and the feedback received during a follow-up usability study. Overall, participants reported that there is a substantial benefit of having a fleet manager use a consolidated dashboard to assess hazards for the vehicles in their fleet and to provide situational awareness to potential risks so that they can be mitigated prior to flight. Once the SDSP-CD matures, it will need to be integrated into flight and mission planning tools. Some initial thoughts on how this integration should be accomplished are shared in this paper. Finally, the functional differences between preflight vs. in-flight risk assessment and the differences in fleet manager vs. UAS pilot roles that may require different information and user interactions are discussed.

preflight↗

How Usability Testing Resulted in Improvements to Ground Collision Software for General Aviation: Improved Ground Collision Avoidance System (IGCAS)

The NASA improved Ground Collision Avoidance System (iGCAS) team conducted an onsite usability study at Experimental Aircraft Association (EAA) Air Venture in Oshkosh, Wisconsin from July 19 through July 26, 2015. EAA Air Venture had approximately 550,000 attendees from which the sample pool of pilots were selected. The objectives of this study were to assess the overall appropriateness and acceptability of iGCAS as a warning system for General Aviation aircraft, usability of the iGCAS displays and audio cues, test terrain avoidance characteristics, performance, functionality, pilot response time, and correlate terrain avoidance performance and pilot response time data.

collision avoidance↗

Improving Aviation Safety with information Visualization: A Flight Simulation Study

Many aircraft accidents each year are caused by encounters with invisible airflow hazards. Recent advances in aviation sensor technology offer the potential for aircraft-based sensors that can gather large amounts of airflow velocity data in real-time. With this influx of data comes the need to study how best to present it to the pilot - a cognitively overloaded user focused on a primary task other than that of information visualization. In this paper, we present the results of a usability study of an airflow hazard visualization system that significantly reduced the crash rate among experienced helicopter pilots flying a high fidelity, aerodynamically realistic fixed-base rotorcraft flight simulator into hazardous conditions. We focus on one particular aviation application, but the results may be relevant to user interfaces in other operationally stressful environments.

Aragon, Cecilia R.↗

Aircraft Icing Weather Data Reporting and Dissemination System

The long-term operational concept of this research is to develop an onboard aircraft system that assesses and reports atmospheric icing conditions automatically and in a timely manner in order to improve aviation safety and the efficiency of aircraft operations via improved real-time and forecast weather products. The idea is to use current measurement capabilities on aircraft equipped with icing sensors and in-flight data communication technologies as a reporting source. Without requiring expensive avionics upgrades, aircraft data must be processed and available for downlink. Ideally, the data from multiple aircraft can then be integrated (along with other real-time and modeled data) on the ground such that aviation-centered icing hazard metrics for volumes of airspace can be assessed. As the effect of icing on different aircraft types can vary, the information should be displayed in meaningful ways such that multiple types of users can understand the information. That is, information must be presented in a manner to allow users to understand the icing conditions with respect to individual concerns and aircraft capabilities. This research provides progress toward this operational concept by: identifying an aircraft platform capable of digitally capturing, processing, and downlinking icing data; identifying the required in situ icing data processing; investigating the requirements for routing the icing data for use by weather products; developing an icing case study in order to gain insight into major air carrier needs; developing and prototyping icing display concepts based on the National Center for Atmospheric Research's existing diagnostic and forecast experimental icing products; and conducting a usability study for the prototyped icing display concepts.

Bass, Ellen J.↗

Earthdata Search: The Relevance of Relevance

Through recent usability studies, the issue of relevance became increasingly clear in the Earthdata Search Client. After all, if a user can't find the data they are looking for, nothing else we do matters. This presentation walks through usability testing findings and recent relevance improvements made to the Earthdata Search Client.

search↗

Data Collection and Analysis Challenges and Mitigation Strategies for Quantitative Human Factors Research Studies in Nuclear Power Plant Modernization

The United States (U.S.) Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) program Plant Modernization Pathway is conducting targeted research and development (R&D) to address aging and reliability concerns with the legacy instrumentation and control and related information systems of the U.S. LWR fleet. In this effort, the application of human factors engineering (HFE) provides an important role in ensuring new digital plant technologies enable broad innovation and business improvement with continued operational safety. Evaluation is a key activity in HFE, which often occurs iteratively through the system design lifecycle. While qualitative methods are important in collecting information of how users perform tasks through observations, quantitative methods are equally important in assessing system design based on performance. In collecting and analyzing this quantitative data, there are notable challenges in the nuclear HFE domain that may threaten the validity and reliability of the inferences made in these studies. Notable challenges include small sample size and limited resources, large error variance and small effect size, an “adding test features to losing degrees of freedom” dilemma, non-normal distribution, and heterogeneity of variance. The results in control room usability studies are often statistically non-significant, which makes it hard to interpret. This work discusses these challenges across different scientific viewpoints and provides real-world examples of these challenges in practice. Collectively, the objective of this work is to position these challenges to the larger data science community as a means of identifying future opportunities to address these issues.

99 GENERAL AND MISCELLANEOUS↗

Airflow Hazard Visualization for Helicopter Pilots: Flight Simulation Study Results

Airflow hazards such as vortices or low level wind shear have been identified as a primary contributing factor in many helicopter accidents. US Navy ships generate airwakes over their decks, creating potentially hazardous conditions for shipboard rotorcraft launch and recovery. Recent sensor developments may enable the delivery of airwake data to the cockpit, where visualizing the hazard data may improve safety and possibly extend ship/helicopter operational envelopes. A prototype flight-deck airflow hazard visualization system was implemented on a high-fidelity rotorcraft flight dynamics simulator. Experienced helicopter pilots, including pilots from all five branches of the military, participated in a usability study of the system. Data was collected both objectively from the simulator and subjectively from post-test questionnaires. Results of the data analysis are presented, demonstrating a reduction in crash rate and other trends that illustrate the potential of airflow hazard visualization to improve flight safety.

Aragon, Cecilia R.↗

Usability of Pre-Flight Planning Interfaces for Supplemental Data Service Provider Tools to Support Uncrewed Aircraft System Traffic Management

Small uncrewed aircraft systems (sUASs) operate in low-altitude, uncontrolled airspace – where support services for their operators (UASOs) are not currently provided. NASA’s System-Wide Safety (SWS) project is identifying the potential risks and hazards to sUAS operations to provide, inform, and improve the designs of In-time Aviation Safety Management Systems (IASMS). The IASMS will include a suite of data-driven tools that compile and analyze data collected from aviation systems and environmental sources to predict hazards, and provide information to allow operators to mitigate these risks (Young et al., 2020). These risk and hazard services can be run and displayed to operators on graphical user interfaces (GUIs), as they relate to a vehicle(s)’ route of flight. These interfaces offer both a means to present hazard service output and offer an opportunity to test user understanding of the information, user decision making, and the best ways to present such data to an operator. Based on these future technologies and intended missions, it is important to investigate interface requirements and evaluate how operators might use these tools. Presenting salient and meaningful risk assessment information to operators is necessary to increase situation awareness and ultimately safety. Building on previous research (Feldman et al., 2022), a usability study comparing two GUIs was conducted to explore how individuals interacted with different styles of information displays. A series of pre-flight hazard and risk-assessment tasks were developed to evaluate participant performance using the Supplemental Data Service Provider Consolidated Dashboard and the Human Automation Team Interface System interfaces. Participants were trained to use both GUIs and their performance was analysed across different scenarios involving multiple sUASs. Performance on simple tasks and the System Usability Scale scores were reported by Feldman et al., 2023. Additional analyses and evaluations on more complex tasks (e.g., risk assessment, prioritization), workload and response times are examined in this paper.

sUAV interfaces↗