Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Trustworthiness”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

105 records · Page 6
All sourcesNASA NTRSDOE OSTI
NASA NTRS2021

Automated Data Accountability for Missions in Mars Rover Data

As the Mars Curiosity Rover transmits data to the JPL Ground Data System (GDS), it frequently observes data loss and corruption, requiring re-transmits from the rover and Ground Data System Analysts (GDSA) to monitor the downlink process. As new missions are launched, the GDSA team redistributes analysts to these new missions, causing shortages in previous missions. The GDSA team can significantly benefit from the automation and optimization of the downlink process of telemetry data. In fact, there is a need for a better understanding of why the data is corrupted, so that the GDSA team can best determine the root cause of the issues in the GDS. This paper presents machine learning and deep learning based approaches to automate and optimize the detection of data loss. We first created a pipeline to automatically accumulate data from the telemetry databases (MAROS, Telemetry Data Storage, and GDS Elastic Search Database) in the downlink process. With our newly created datasets, we perform feature selection to supplement the GDSA understanding of the downlink process and provide supplemental analysis on the importance of different features. We implement various machine learning and deep learning based models, including support vector machines, ensemble methods, and deep neural networks and evaluate their accuracies in identifying whether a downlink process is complete or incomplete. We utilize fast hyperparameter optimization methods that allow our models to quickly be re-trained, allowing them to quickly be tuned and optimized on daily incoming data in real time. This hyperparameter optimization also allows our methods to be quickly integrated into other JPL missions. Our results show that our best-performing machine learning and deep learning based models outperform the existing GDSA detection software by 6 accuracy points and can aid analysts by providing insights into the data accountability problem. Since these various machine learning and deep learning approaches vary significantly in interpretability, we provide a discussion on the tradeoffs between their performance and trustworthiness in helping detect issues in data transmission.

Divsalar, Dariush↗
NASA NTRS2023

Space Trusted Autonomy Readiness Levels

Technology Readiness Levels are a mainstay for organizations that fund, develop, test, acquire, or use technologies. Technology Readiness Levels provide a standardized assessment of a technology’s maturity and enable consistent comparison among technologies. They inform decisions throughout a technology’s development life cycle, from concept, through development, to use. A variety of alternative Readiness Levels have been developed, including Algorithm Readiness Levels, manufacturing Readiness Levels, Human Readiness Levels, Commercialization Readiness Levels, Machine Learning Readiness Levels, and Technology Commitment Levels. However, while Technology Readiness Levels have been increasingly applied to emerging disciplines, there are unique challenges to assessing the rapidly developing capabilities of autonomy. This paper adopts the moniker of Space Trusted Autonomy Readiness Levels to identify a two-dimensional scale of readiness and trust appropriate for the special challenges of assessing autonomy technologies that seek space use. It draws inspiration from other readiness levels’ definitions, and from the rich field of trust and trustworthiness. The Space Trusted Autonomy Readiness Levels were developed by a collaborative Space Trusted Autonomy subgroup, which was created from The Space Science and Technology Partnership Forum between the United States Space Force, the National Aeronautics and Space Administration, and the National Reconnaissance Office.

Kerianne L Hobbs↗
NASA NTRS2022

NASA's Responsible AI Use Cases

This submission consists of NASA's Responsible Artificial Intelligence (RAI) Use Cases. These RAI Use Cases are to be made public, pursuant to the Presidential Executive Order 13960, Promoting the Use of Trustworthy Artificial Intelligence in the Federal Government. They were collected from NASA's practicing AI research community and cover the gamut of NASA's AI activities. These Use Cases will be updated annually as required by the Executive Order. The information included consists of the NASA Center, a summary of the goal, the AI techniques being applied, information on training data, and information on source code.

Artificial Intelligence↗
NASA NTRSDate not supplied

Filling the Gaps in Understanding Solar Flares

Solar flares provide a laboratory for plasma physics and magnetohydrodynamic processes. While the mechanism of transferring magnetic flux between topological domains via reconnection is accepted in a general sense, some of the finer details remain to be worked out: What elusive thresholds, pertaining to which physical quantities, determine the initiation of flux transfer that cascades into the energy release observed in the impulsive phase of a flare? What role does turbulence play in accelerating or prolonging magnetic reconnection? To what degree do magnetic waves contribute to the transport of energy, or to the acceleration of particles in the impulsive phase? While our innate human curiosity and our instinctive impulse to explore compel us to investigate these intricacies of million-degree magnetized plasmas, the impacts of space weather on our technology-dependent society and economy obligate us to pursue a quantitative understanding, with the ultimate goal being trustworthy predictive capabilities. In this presentation, I will touch upon a few aspects of solar flares where improvements in observational and analytical capabilities can be expected to help us fill gaps in our understanding, as well as some of the recent, current, and near-term developments that are carrying the field forward.

David E. McKenzie↗
NASA NTRS2023

NASA's Responsible AI Use Cases

This submission consists of summary use cases for NASA's Responsible Artificial Intelligence (RAI). These RAI Use Cases are to be made public, pursuant to the Presidential Executive Order 13960, Promoting the Use of Trustworthy Artificial Intelligence in the Federal Government. They were collected from NASA's practicing AI research community and cover the gamut of NASA's AI activities. These Use Cases will be updated annually as required by the Executive Order. The information included consists of the NASA Center, a summary of the goal, the AI techniques being applied, information on training data, and information on source code.

Artificial Intelligence↗
NASA NTRSDate not supplied

NASA Aeronautics Research Mission Directorate System Security Engineering Approaches

System security engineering (SSE) is a set of formal engineering methods and is considered a subset of systems engineering. It is a relatively new development in systems engineering with the initial NIST (National Institute of Standards) standard published in November of 2016 with updates in 2018, and 2022. The guiding principles in our methodology are based in NIST Special Publication 800-160 Vol. 1 “Systems Security Engineering: Considerations For A Multidisciplinary Approach In The Engineering Of Trustworthy Secure Systems” and integrate methodologies from common IT (Information Technology) threat modeling approaches utilizing MBSE (Model-Based Systems Engineering). The presentation will discuss how our teams utilize SSE and MBSE (Model-Based Systems Engineering) to develop secure architectures for systems under development in our NASA aeronautics research environment. This includes the activities to develop Protection Needs (PN) that, in turn result in security requirements in the design context and policies for the future state operational context for system protection. The process of applying SSE to analyze project architectures and ConOps (Concept of Operations) is intended to ensure the transferred research is both secure and securable in a “real-world” setting.

Systems Security Engineering↗
NASA NTRS2020

Human-in-the-loop Robotic Control for Human-Robot Collaboration

The prospect of a collaborative work environment between humans and robotic automation in a manufacturing setting [1] provides the motivation for finding innovative solutions to human-in-the-loop control for safe, efficient, and trustworthy human-robot collaboration (HRC), or HR interaction (HRI), in cyberphysical human systems (CPHSs) [2], [3]. Studies in [4] show that collaborative automation can be beneficial to 90% of approximately 300,000 small-to-medium-scale enterprises in the United States. In the paradigm of human-centered automation [5], human safety, ergonomics, and the collaborative efficiency of the work are given the utmost importance. Traditional methods to ensure the safety of humans around factory robots involve the use of cages. Recent work looked beyond cage-based safety to provide robot control and sensing-driven solutions for human safety around robots [6]-[8].

Ashwin P. Dani↗
NASA NTRS2024

Discrete Event Simulation-Based Timeline Validation Using R2U2

The Gateway Vehicle Systems Manager (VSM), the top-level software control system in a distributed, hierarchical Autonomous System Management Architecture is, like most modern spacecraft software control systems, heavily data-driven. For example, schedules (timelines) will be developed on the ground and, due to the high degree of autonomy, contain complex procedures involving conditional branching, variable timing, and resource contention resolution. In order to verify that an uploaded timeline will function correctly, it is necessary to explore the feasible set of possible executions. While it is possible to test a timeline using a mission simulation, the complexity of the system and duration of a timeline limits the number of trials and therefore the test coverage. To address this problem, the VSM team is using a discrete event system model that can rapidly generate from a timeline sets of event sequences using Monte Carlo techniques. To achieve rapid and trustworthy checking of the event sequences, we use an offline version of the runtime model checking tool R2U2. This presentation describes the approach the VSM team is using to implement the discrete event simulation and evaluate event sequences using R2U2. The presentation will discuss: 1. Description of the timelines by VSM in the context of VSM operations 2. Expansion of a timeline into a sequence of atomic events 3. Adjustment, in the Monte Carlo environment, of an event sequence to account for uncertainty, external events, and failures 4. Definition of R2U2 input and mission-time linear temporal logic files 5. Generation and use of R2U2 verdict sequences 6. Lessons learned and future work

Verification↗
NASA NTRS2023

Decoupling in the Vertical Shape of HCHO During a Sea Breeze Event: the Effect on Trace Gas Satellite Retrievals and Column-to-Surface Translation

The effect of sea breeze circulation on stratification of the vertical formaldehyde (HCHO) concentration vertical profiles is explored using a regional atmospheric chemical transport model (CTM) for three synoptically stagnant days focused on the east coast of the U.S in June 2018. During this event, a significant thermal contrast between the Atlantic Ocean and the terrestrial regions (12–17 °C), observed by moderate resolution imaging spectroradiometer (MODIS) and well-captured by the WRF-CMAQ model (15–18 °C), is conducive to monsoon-like flow, perpendicular to the shorelines, carrying clean marine air masses over the land within a few hundreds of meters above the surface. In contrast, the westerly continental polluted air masses prevail in higher altitudes. These two conflicting flows result in atypical vertical shapes of HCHO concentrations increasing with altitude. This decoupling pattern is so pronounced that we observe total column HCHO negatively correlate with surface concentrations. Comparisons of an accredited global model, GEOS-CF, to surface wind measurements and MODIS skin temperature indicate its poor representation of the sea breeze timing and strength, resulting in GEOS-CF HCHO vertical shapes being drastically different from the WRF-CMAQ. Based on radiative transfer calculations, the differences in the vertical distribution of HCHO between the WRF-CMAQ and that of GEOS-CF in the first 3 km are sufficient to induce a 20–30% error in air mass factors (thus total vertical HCHO column abundances). Through an experiment involving converting HCHO total columns to surface mixing ratios, we demonstrate that GEOS-CF allocates noticeably more HCHO molecules (40–150%) to the surface layer due to the misrepresentation of the vertical shape of HCHO during the sea breeze event. It is known that a significant fraction of the human population lives in coastal areas prone to detrimental effects caused by air pollution, and elevated pollutant concentrations usually occur in synoptically stagnant atmospheric conditions when local circulation patterns come into play; accordingly, our experiments emphasize the importance of the effect a priori profiles can have on satellite-derived applications under such conditions. To ensure that the quantitative representation of satellite-based trace gas retrievals on a daily basis is trustworthy and useable for air quality applications, atmospheric models providing a priori profiles for satellite retrievals should be well-tuned to reproduce complex local circulation such as sea-land breezes.

sea breeze↗
NASA NTRSDate not supplied

In-Depth Modeling and Simulation Analysis of Artemis Missions Using the Impact Probabilistic Risk Assessment Tool

BACKGROUND The Artemis campaign is a Moon exploration program with a series of six planned missions, five of which will be crewed. These five crewed missions will contain a single mission segment (space flight), or multiple mission segments involving space flight (Orion), lunar landing (LTV) and/or space habitat (Gateway). Each crewed segment faces the risk of unique medical conditions, necessitating medical sets/kits tailored to those specificities. To support and enable a data-driven and evidence-based decision-making process through out a mission’s life cycle, a software tool called IMPACT was developed. Using probabilistic risk assessment (PRA) methodologies, IMPACT (Informing Mission Planning via Analysis of Complex Tradespaces) is a novel tool built for analyzing the possibility of encountering complex medical risks during space flight, and for identifying the medical resources and capabilities needed to treat those potential at-risk medical conditions. IMPACT achieves this by performing hundreds of thousands of Monte Carlo simulations of missions to build aggregate pictures of medical risk. During an extended simulation modeling phase, IMPACT generated analytical results for medical risks, and the medical resources and capabilities to address those risks, for every segment of every crewed Artemis mission. This presentation will highlight the reliability, consistency and validity of IMPACT’s computational modeling techniques and will showcase the library of analytical outcomes generated for the Artemis missions. OVERVIEW During the early stages of IMPACT’s design, architecture and technical requirements collection, “scenarios” (use cases) - achievement goals required for acceptance testing, were identified by stakeholders. IMPACT successfully completed the scenario testing requirements and undertook an extensive operational run phase utilizing a wide range of input combinations with a goal of delivering a cohesive, trustworthy, reliable, vast, and diverse body of evidence. The intent of these modeling runs was to validate consistency in output, ensure solidity of executable operations and to streamline processes by identifying areas requiring efficiency improvements. Using the many missions of Artemis, IMPACT ran variations of operational runs to assess the output for acceptable, as well as unusual characteristics. This rigorous long-term “shakedown” analysis was implemented to help build a collective body of evidence to aid in securing a high level of confidence, reliability, and validity in the output, whether from the applicational components of IMPACT, or the entirety of the operational process. ANTICIPATED ANALYSIS AND CONCLUSION This presentation will discuss the various categories of input criteria; the comparisons in the application of these input criteria to various Artemis missions; the preparation and collection of the body of evidence, and reliability of the computational modeling techniques. This paper serves as an initial analytical overview of IMPACT’s probabilistic risk assessment (PRA) medical risk outputs covering Artemis missions and is not intended to be deemed the official medical response for the Artemis campaign.

Crew Composition↗
NASA NTRSDate not supplied

Data Democratization: Challenges and Opportunities

Democratizing Earth data is one of the challenges many organizations around the world face in order to maximize the use of their Earth data for research, applications, education, and societal benefits. For example, at the NASA Goddard Earth Sciences (GES) Data and Information Services Center (DISC), over 1600 global and regional datasets in several NASA Earth science focus areas, including atmospheric composition, water and energy cycles, and climate variability, are archived and distributed to the public. Giovanni, the Geospatial Interactive Online Visualization and Analysis Infrastructure, was developed by GES DISC to facilitate data access and exploration, especially for novice users of Earth science. With Giovanni, users can analyze and visualize over 2000 Earth science variables (e.g., precipitation, aerosol, surface wind) without downloading data, software, the expert understanding of data formats and structures, and coding skills, lowering the barrier to data analysis/comparison by preprocessing and accessing to the data. Results of data analysis and visualization can be accessed in several popular formats (e.g., NetCDF, CSV). As a result of Giovanni's efforts, more than 3000 referral papers have been published in various fields. In spite of this, Giovanni is still difficult to use for some users. For instance, if one searches for "precipitation," it will return over 150 related variables. The question is, which one to use? Furthermore, variables from different data providers (e.g., satellites and models) are named differently with different units, further confusing users, especially those outside the communities. Data democratization is complex and multifaceted. Challenges include service and data discovery, user experiences, visualization, data quality, trustworthiness, and more. In this presentation, we will examine Giovanni as an example of challenges and opportunities in developing data democratization services.

data democratization↗
NASA NTRS2023

AI Ethics Appendix: A Novel Approach to AI Ethics Workforce Development

The "AI Ethics Appendix" is a game-based design fiction for the deliberation of uncertain artificial intelligence (AI) ethical scenarios. The game is intended to be used as a tool for AI practitioners and industry professionals to grow responsible and ethical AI knowledge as they integrate this technology into their development. As AI/ML ethical considerations and governmental compliance develop, it is important to encourage teams to encourage teams to incorporate diverse thinking early in the development cycle and consider how different stakeholders may be affected by the technology. This game accomplishes these goals through storytelling and meaningful game interaction based on methods from game design as well as speculative and design fiction in the field of Human-Centered Design. The game mechanics were informed by the NASA Framework for the Ethical Use of Artificial Intelligence, Executive Order 13960, examples of AI use cases, and colleagues' work experiences with AI/ML.

trustworthy↗