Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “TRUST”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
NASA NTRS2020

Designing and Training for Appropriate Trust in Increasingly Autonomous Advanced Air Mobility Operations: A Mental Model Approach: Version 1

To enable effective human-autonomy teaming (HAT) in Advanced Air Mobility (AAM) operations, the current paper presents a theoretical framework to design and train for appropriate trust in automation. The novel contribution of this work resides in connecting the construct of trust to mental models and showing how this method could be used to enable emerging HAT concepts such as Adaptive Trust Calibration. To contextualize this framework, in section 2 we discuss simplified vehicle operations (SVO) and remote vehicle operations (RVO), which are leading operational concepts within AAM. In section 3 we describe our perspective on automation and increasingly autonomous systems and present a brief discussion on human-automation interaction and human-autonomy teaming. In section 4 we provide a detailed discussion on the construct of trust in automation. In section 5 we present a framework that associates mental models with trust through principles of transparent design. Finally, in section 6 we present three descriptive models for designing and training for appropriate trust in increasingly autonomous systems.

Human-Autonomy Teaming↗
DOE OSTI2025

Trustworthiness and Trust: Identifying Factors that Drive Successful Human-AI Interaction in Nuclear Power Plant Applications

Emerging technologies such as artificial intelligence (AI) and machine learning (ML) are rapidly evolving and considered a promising tool for efficient and continued safe operations of the U.S. nuclear power plants (NPPs). Emerging AI techniques like large language models (LLMs) are one such technology that may support personnel at existing NPPs perform work more efficiently. For example, operators may query the current operational status of a power plant via a chat interface leveraging LLMs to access plant-related information in an interactive manner rather than manually collecting various sensor data for tasks such as surveillances or completing work orders. This is a fundamental shift in the way operators currently perform their tasks today. The literature of human-automation interaction indicates that trust is a crucial factor that drives successful interaction between a human operator and an automated system, like an AI-infused NPP application. This work presents the results of a literature review on key factors that relate to trust in AI/LLM technologies for NPP applications. The relevant literature of human factors and cognitive engineering has identified various factors related to trust including trustworthiness, performance characteristics, operator skill and perceived risk. This preliminary literature review will guide development and evaluation of models involving the identified factors influencing trust in AI and develop a framework for human-centered design for interface between humans and AI. By addressing trust, this work supports developing a technical basis for designing key characteristics of AI/LLM to support calibrated trust, which will ultimately support wide-scale adoption of AI/LLM technologies, as well as ensure safe, effective, and reliable use.

99 - GENERAL AND MISCELLANEOUS↗
DOE OSTI2021

Trust-but-Verify in Cyber-Physical Systems

Cyber-physical systems span a wide spectrum, from long-lived legacy systems to more modern installations. Trust is an issue that arises across the spectrum, albeit with different variants of goals and constraints. On the one end of the spectrum, legacy systems are characterized by function-based designs in which trust is an implicitly in-built concept -- the operation is historically designed, implemented, and optimized in a benign stance with respect to intended use. On the other end of the spectrum, modern systems are characterized by offerings from manufacturers, vendors, and system installers -- the devices and deployments use a variety of security features that offer promises of increased trust. All along this spectrum of cyber-physical systems, extending trust beyond the traditional cyber portions to the arteries that connect the physical portions to the cyber portions is a major challenge. Here, we identify a Trust-but-Verify approach that spans this spectrum in addressing trust.

Perumalla, Kalyan↗
DOE OSTI2024

TRUST Sensors in Environments: Fiber Optic Displacement (SE-FOD) Report, Release FY24

The objective of the Delivery Environments (DE) Testbeds to Reduce Uncertainty in Simulations and Tests (TRUST) project is to quantify and help increase confidence in specific areas of computational and experimental capabilities that are applicable to the development, on-target assessment, and qualification of current and future delivery environments. More complete quantification of confidence in experimental and computational capabilities and the sufficient increase of confidence in those capabilities is critical to improving weapons engineering design, qualification, and assessment efforts that are critical to the current and future stockpile. This work uses and provides feedback on analysis tools and experimental results databases for efficient and responsive engineering which are currently under development. Under the TRUST work package, there are five testbeds and their associated engineering analysis baseline models (EABMs): 1. Contact Thermal Conductivity (CTC), 2. Nonlinear Dynamics (ND), 3. Sensors in Environments: Accelerometers (SE-A), 4. Sensors in Environments: Fiber Optic Displacement Gauges (SE-FOD), and 5. Sensors in Environments: Thermocouples (SE-TC). The TRUST project uses single-feature testbeds to quantify uncertainties in specific models and experiments and to identify capability development needs that can help to reduce these uncertainties. Each testbed is designed, configured, and tested in collaboration with groups with design and experimental capability: E-14 and MPA-CINT. The complementary simulations are conducted using W-13 analysis tools and stored in model repositories with plans for incremental progress toward EABM requirements. W-13 extends and exercises the testbed simulations in collaboration with experimentalists for uncertainty quantification of current and future materials, geometries, and environments. Additionally, TRUST is intended to provide engineers in W-13 and E-14 with experience in both numerical simulations and experimental methods through cross-discipline collaborations. Following the introduction to the TRUST project, the remainder of this report focuses on experimental, modeling and simulation, and analysis efforts conducted in FY24 relevant to the TRUST Sensors in Environments: Fiber Optic Displacement (SE-FOD) testbed.

42 ENGINEERING↗
NASA NTRS2018

Serious Gaming for Building a Basis of Certification via Trust and Trustworthiness of Autonomous Systems

Autonomous systems governed by a variety of adaptive and nondeterministic algorithms are being planned for inclusion into safety-critical environments, such as unmanned aircraft and space systems in both civilian and military applications. However, until autonomous systems are proven and perceived to be capable and resilient in the face of unanticipated conditions, humans will be reluctant or unable to delegate authority, remaining in control aided by machine-based information and decision support. Proving capability, or trustworthiness, is a necessary component of certification. Perceived capability is a component of trust. Trustworthiness is an attribute of a cyber-physical system that requires context-driven metrics to prove and certify. Trust is an attribute of the agents participating in the system and is gained over time and multiple interactions through trustworthy behavior and transparency. Historically, artificial intelligence and machine learning systems provide answers without explanation - without a rationale or insight into the machine “thinking”. In order to function as trusted teammates, machines must be able to explain their decisions and actions. This transparency is a product of both content and communication. NASA’s Autonomy Teaming & TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR) project seeks to build a basis for certification of autonomous systems via establishing metrics for trustworthiness and trust in multi-agent team interactions, using AI (Artificial Intelligence) explainability and persistent modeling and simulation, in the context of mission planning and execution, with analyzable trajectories. Inspired by Massively Multiplayer Online Role Playing Games (MMORPG) and Serious Gaming, the proposed ATTRACTOR modeling and simulation environment is similar to online gaming environments in which player (aka agent) participants interact with each other, affect their environment, and expect the simulation to persist and change regardless of any individual agent’s active participation. This persistent simulation environment will accommodate individual agents, groups of self-organizing agents, and large-scale infrastructure behavior. The effects of the emerging adaptation and coevolution can be observed and measured to building a basis of measurable trustworthiness and trust, toward certification of safety-critical autonomous systems.

Allen, B. Danette↗
NASA NTRS2022

Adapting a Trusted AI Framework to Space Mission Autonomy

As artificial intelligence (AI) is increasingly pro- posed for new and future capabilities in space missions, the question of how to trust AI-enabled space autonomy has been explored. Recently, a collaboration between The Aerospace Corporation (Aerospace) and NASA’s Jet Propulsion Labora- tory (JPL) investigated how Aerospace’s Trusted AI Frame- work could be applied to two JPL projects that planned on lev- eraging AI for critical autonomous tasks. This combined effort led to many insights in the practical implementation of trusted AI along with considerable updates to the Trusted AI Frame- work that tailored its topic threads to space exploration. This document cohesively summarizes the enhanced framework as tailored to space missions as well as estimation of the level of trust required as a function of mission criticality and key stakeholders. The goal of this work is to provide a set of best practices to inform autonomy researchers, flight engineers, mission and proposal reviewers, and instrument and mission principal investigators (PI’s) to drive AI-based autonomy that maximizes trust and lowers the barriers to mission adoption for both science and engineering applications.

Amini, Rashied↗
DOE OSTI2022

Newton trust-region methods with primary variable switching for simulating high temperature multiphase porous media flow

Coupling multiphase flow with energy transport due to high temperature heat sources introduces significant new challenges since boiling and condensation processes can lead to dry-out conditions with subsequent re-wetting. The transition between two-phase and single-phase behavior can require changes to the primary dependent variables adding discontinuities as well as extending constitutive nonlinear relations to extreme physical conditions. Practical simulations of large-scale engineered domains lead to Jacobian systems with a very large number of unknowns that must be solved efficiently using iterative methods in parallel on high-performance computers. Performance assessment of potential nuclear repositories, carbon sequestration sites and geothermal reservoirs can require numerous Monte-Carlo simulations to explore uncertainty in material properties, boundary conditions, and failure scenarios. Due to the numerical challenges, standard NR iteration may not converge over the range of required simulations and require more sophisticated optimization method like trust-region. In this study, we use the open-source simulator PFLOTRAN for the important practical problem of the safety assessment of future nuclear waste repositories in the U.S. DOE geologic disposal safety assessment Framework. The simulator applies the PETSc parallel framework and a backward Euler, finite volume discretization. We demonstrate failure of the conventional NR method and the success of trust-region modifications to Newton’s method for a series of test problems of increasing complexity. Trust-region methods essentially modify the Newton step size and direction under some circumstances where the standard NR iteration can cause the solution to diverge or oscillate. Furthermore, we show how the Newton Trust-Region method can be adapted for Primary Variable Switching (PVS) when the multiphase state changes due to boiling or condensation. The simulations with high-temperature heat sources which led to extreme nonlinear processes with many state changes in the domain did not converge with NR, but they do complete successfully with the trust-region methods modified for PVS. This implementation effectively decreased weeks of simulation time needing manual adjustments to complete a simulation down to a day. Finally, we show the strong scalability of the methods on a single node and multiple nodes in an HPC cluster.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2016

Effects of Transparency on Pilot Trust and Agreement in the Autonomous Constrained Flight Planner

We performed a human-in-the-loop study to explore the role of transparency in engendering trust and reliance within highly automated systems. Specifically, we examined how transparency impacts trust in and reliance upon the Autonomous Constrained Flight Planner (ACFP), a critical automated system being developed as part of NASA's Reduced Crew Operations (RCO) Concept. The ACFP is designed to provide an enhanced ground operator, termed a super dispatcher, with recommended diversions for aircraft when their primary destinations are unavailable. In the current study, 12 commercial transport rated pilots who played the role of super dispatchers were given six time-pressured all land scenarios where they needed to use the ACFP to determine diversions for multiple aircraft. Two factors were manipulated. The primary factor was level of transparency. In low transparency scenarios the pilots were given a recommended airport and runway, plus basic information about the weather conditions, the aircraft types, and the airport and runway characteristics at that and other airports. In moderate transparency scenarios the pilots were also given a risk evaluation for the recommended airport, and for the other airports if they requested it. In the high transparency scenario additional information including the reasoning for the risk evaluations was made available to the pilots. The secondary factor was level of risk, either high or low. For high-risk aircraft, all potential diversions were rated as highly risky, with the ACFP giving the best option for a bad situation. For low-risk aircraft the ACFP found only low-risk options for the pilot. Both subjective and objective measures were collected, including rated trust, whether the pilots checked the validity of the automation recommendation, and whether the pilots eventually flew to the recommended diversion airport. Key results show that: 1) Pilots trust increased with higher levels of transparency, 2) Pilots were more likely to verify ACFPs recommendations with low levels of transparency and when risk was high, 3) Pilots were more likely to explore other options from the ACFP in low transparency conditions and when risk was high, and 4) Pilots decision to accept or reject ACFPs recommendations increased as a function of the transparency in the explanation. The finding that higher levels of transparency was coupled with higher levels of trust, a lower need to verify other options, and higher levels of agreement with ACFP recommendations, confirms the importance of transparency in aiding reliance on automated recommendations. Additional analyses of qualitative data gathered from subjects through surveys and during debriefing interviews also provided the basis for new design recommendations for the ACFP.

trust↗
NASA NTRSDate not supplied

The Role of Trust and Usability in Enabling Spaceflight Crew Autonomy

Future long duration exploration missions will require an increased use of onboard automated systems as spaceflight crews venture further than before and have longer communications delays with ground support. The design of these systems must support appropriate crew trust and have sufficient usability to enable spaceflight crew autonomy or risk being misused while crews wait to communicate with ground support. We evaluated trust & usability in our self-scheduling tool, Playbook, for crew mission timelines. Data was collected in a controlled lab experiment with 31 participants. Participants in the study conducted two tasks: scheduling, where participants were responsible for scheduling a majority of a day's operational tasks, and rescheduling, where participants were provided a schedule and asked to reschedule higher priority activities. We found a significant correlation between system trust and usability, irrespective of self-scheduling tasks. We conclude that system usability may play a bigger role in how trust is learned while conducting novel crew autonomy tasks such as self-scheduling. Future research should investigate the role of usability to encourage appropriate trust in onboard automated crew systems and enable crew autonomy.

crew autonomy↗
NASA NTRS2010

Effects of Secondary Task Modality and Processing Code on Automation Trust and Utilization During Simulated Airline Luggage Screening

The purpose of this research was to examine the impact of environmental distractions on human trust and utilization of automation during the process of visual search. Participants performed a computer-simulated airline luggage screening task with the assistance of a 70% reliable automated decision aid (called DETECTOR) both with and without environmental distractions. The distraction was implemented as a secondary task in either a competing modality (visual) or non-competing modality (auditory). The secondary task processing code either competed with the luggage screening task (spatial code) or with the automation's textual directives (verbal code). We measured participants' system trust, perceived reliability of the system (when a target weapon was present and absent), compliance, reliance, and confidence when agreeing and disagreeing with the system under both distracted and undistracted conditions. Results revealed that system trust was lower in the visual-spatial and auditory-verbal conditions than in the visual-verbal and auditory-spatial conditions. Perceived reliability of the system (when the target was present) was significantly higher when the secondary task was visual rather than auditory. Compliance with the aid increased in all conditions except for the auditory-verbal condition, where it decreased. Similar to the pattern for trust, reliance on the automation was lower in the visual-spatial and auditory-verbal conditions than in the visual-verbal and auditory-spatial conditions. Confidence when agreeing with the system decreased with the addition of any kind of distraction; however, confidence when disagreeing increased with the addition of an auditory secondary task but decreased with the addition of a visual task. A model was developed to represent the research findings and demonstrate the relationship between secondary task modality, processing code, and automation use. Results suggest that the nature of environmental distractions influence interaction with automation via significant effects on trust and system utilization. These findings have implications for both automation design and operator training.

Phillips, Rachel↗
NASA NTRS2021

Accessible Telemetry Streams using a Zero Trust Architecture for the Flight Operations Directorate

As a result of information technology based work becoming increasingly distributed, unique challenges have been presented within the realm of defined network perimeters, namely with respect to secure access to resources. Historically, and from a simplistic abstract perspective, the common approach has been to adopt the, so-called, moat model whereby a physical network perimeter (or interconnected perimeters) is defined to encapsulate resources behind a boundary protected by a firewall. Users are provisioned access through a virtual private network (VPN) and may be further constrained to resources through specific firewall allow and disallow rulesets. Virtual Private Networks and firewall rulesets lead to common problems, particularly at scale and, as a result, perimeter-less architectures provided over the public internet are increasingly becoming prevalent, particularly with its more popular implementation, the Zero Trust Architecture. We present a proposed implementation of the Zero Trust Architecture with a particular concrete example utilizing a de-perimeterized network that requires authentication and authorization for each action between nodes and does not operate within an implicit trust boundary. It should be noted that this paper is not an attempt at providing comprehensive resolutions for the specific problem space with respect to perimeter based security and is more directed at providing information with regard to our proposed implementation of a Zero Trust Architecture for the Flight Operations Directorate. We direct the reader to our Introduction and Background section for more details on specific documentation and where it can be located as it relates to de-perimeterization and Zero Trust.

Paul Shoemaker↗
NASA NTRS2023

Space Trusted Autonomy Readiness Levels

Technology Readiness Levels are a mainstay for organizations that fund, develop, test, acquire, or use technologies. Technology Readiness Levels provide a standardized assessment of a technology’s maturity and enable consistent comparison among technologies. They inform decisions throughout a technology’s development life cycle, from concept, through development, to use. A variety of alternative Readiness Levels have been developed, including Algorithm Readiness Levels, manufacturing Readiness Levels, Human Readiness Levels, Commercialization Readiness Levels, Machine Learning Readiness Levels, and Technology Commitment Levels. However, while Technology Readiness Levels have been increasingly applied to emerging disciplines, there are unique challenges to assessing the rapidly developing capabilities of autonomy. This paper adopts the moniker of Space Trusted Autonomy Readiness Levels to identify a two-dimensional scale of readiness and trust appropriate for the special challenges of assessing autonomy technologies that seek space use. It draws inspiration from other readiness levels’ definitions, and from the rich field of trust and trustworthiness. The Space Trusted Autonomy Readiness Levels were developed by a collaborative Space Trusted Autonomy subgroup, which was created from The Space Science and Technology Partnership Forum between the United States Space Force, the National Aeronautics and Space Administration, and the National Reconnaissance Office.

Kerianne L Hobbs↗
NASA NTRSDate not supplied

What is the Role of Usability and Trust in Autonomy?

Usability encompasses learnability, efficiency, memorability, effectiveness, and satisfaction. NASA’s standards for usability acceptance criteria focus on interfaces that help operators achieve their tasks efficiently, effectively, and with satisfaction. However, discussions on usability, especially regarding future highly automated and autonomous systems, rarely include trust. As NASA plans for long-duration exploration missions, it envisions astronauts operating more independently from Mission Control on Earth. This independence will drive the development of these highly automated and autonomous systems that astronauts will use daily. To prepare for this future, our team has developed a scheduling and execution software tool that facilitates self-scheduling, allowing astronauts to independently manage their own schedule without Mission Control’s involvement. Over many years, we have developed, matured, and evaluated our software tool in extreme environments, prioritizing user-centered design and high usability. These evaluations have included multiple campaigns in NASA analogs, including NEEMO, BASALT, and HERA, as well as technology demonstrations onboard the International Space Station. Our recent research on software interfaces for future astronaut autonomy revealed a strong correlation between usability and trust measures. In a controlled lab experiment, we asked novice users to perform a complex scheduling task, during which the software immediately validated the schedule’s constraints and checked for violations. We collected usability (User Experience Questionnaire, UEQ) and trust (Trust in Automated Systems scale, TAS) measures; significant, strong, and moderate correlations emerged between several of the UEQ metrics and TAS. These results support the argument for investing in usability early to enable and sustain trust in highly automated and autonomous systems.

usability↗
DOE OSTI2021

5G Zero Trust Architecture: A Testable and Phased Approach

With the emphasis being placed on Zero Trust Architecture in technology and by Executive Order (Executive Order on Improving the Nation’s Cybersecurity ), an investigation was performed as to the feasibility of implementing this approach into 5G. Reference documents include NIST Special Publication 800-207 and Department of Defense (DoD) Zero Trust Reference Architecture , and these provided guidance as to the overall approach. Progress is being made within 5G to address the lack of Zero Trust Architecture; a June 2021 DoD press release indicated that “The prototype 5G network is built on the next generation of Open Radio Network standards and designed to comply with DOD specifications for zero-trust architecture for native security and secure connectivity with other networks.”

5G↗
DOE OSTI2021

Adaptive Cognitive Mechanisms to Maintain Calibrated Trust and Reliance in Automation

Trust calibration for a human–machine team is the process by which a human adjusts their expectations of the automation’s reliability and trustworthiness; adaptive support for trust calibration is needed to engender appropriate reliance on automation. Herein, we leverage an instance-based learning ACT-R cognitive model of decisions to obtain and rely on an automated assistant for visual search in a UAV interface. This cognitive model matches well with the human predictive power statistics measuring reliance decisions; we obtain from the model an internal estimate of automation reliability that mirrors human subjective ratings. The model is able to predict the effect of various potential disruptions, such as environmental changes or particular classes of adversarial intrusions on human trust in automation. Finally, we consider the use of model predictions to improve automation transparency that account for human cognitive biases in order to optimize the bidirectional interaction between human and machine through supporting trust calibration. The implications of our findings for the design of reliable and trustworthy automation are discussed.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Customer trust in their utility company and interest in household-level battery storage

Here we seek to contribute to understanding household interest in battery storage, particularly the role of customer trust in their utility company. Using data from a survey of California households, we examine whether customers’ trust in their utility company is associated with their interest in battery storage. We find that customer trust that their utility company acts in their best interest is associated with lower levels of interest. Trust that the utility company is competent has some effect. Perceptions that battery storage has financial and environmental benefits and low nonmonetary costs are also associated with higher levels of interest.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Developing a Distributed Trust Model for Distributed Energy Resources

Electric power system operators can manage distribution system reliability by coordinating end customer usage of distributed energy resources. The end customers in this regard are Service Provisioning Customers, who provide their energy resources to a Grid Service Provider, which in turn dispatches large aggregations of distributed energy resources to provide reliable service to the power system.The security of this system relies upon information protection mechanisms, as described in IEEE 2030.5. However, in addition to preventive security measures, a monitoring function is required to ensure trustworthiness. Trust models are a method to detect and respond to both expected and unexpected behavior. Different trust models are required for various types and characteristics of each situation. This paper will describe the topics that must be considered when developing a trust model as it applies to distributed energy resources. The major contribution of this paper is the creation and application of a Distributed Trust Model applied to distributed energy resources.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Yes, No, Maybe So: Human Factors Considerations for Fostering Calibrated Trust in Foundation Models Under Uncertainty

High-stakes analytical environments require analysts to evaluate evidence and generate conclusions to inform critical decisions often under conditions of uncertainty. Probabilistic decision-making based on incomplete or inaccurate information can reduce productivity, compromise national interests, and endanger public safety. Researchers are developing expert systems built on foundation models (FMs) to support analysts’ decision-making processes by enabling human-artificial intelligence (AI) teaming, in part through the quantification and expression of uncertainty information. As FMs continue to mature, it is imperative to correspondingly consider analysts’ needs for appropriately interpreting and using uncertainty information. However, prior research indicates that it remains unclear how analysts engage with FM-generated uncertainty information and the extent to which these interactions influence trust in, and reliance on, expert systems. We plan to review the state of the science and conduct an exploratory, qualitative study to (a) understand how properly communicated uncertainty can foster calibrated trust and appropriate reliance and (b) identify approaches for effectively conveying FM-generated uncertainty information during analytical workflows. We will administer semi-structured interviews with analysts from a specific high-stakes analytical environment to collect their current experiences with job-related uncertainty and their impressions when viewing FM-generated uncertainty information. During the interview protocol, participants will be presented with several different FM outputs and invited to discuss their thoughts and beliefs about the uncertainty information displayed. Participants may provide insights into how trust and reliance may be influenced by uncertainty. The results of this study will help us to better understand how analysts currently interpret and use uncertainty information. Our findings may inform human factors recommendations for effectively conveying uncertainty information to foster calibrated trust in, and appropriate reliance on, expert systems. Interaction designers and FM developers can use this knowledge to enhance human-AI teaming and ensure the responsible deployment of FM-based expert systems in analytical workflows.

97 MATHEMATICS AND COMPUTING↗