Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “trustworthy”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17
All sourcesNASA NTRSDOE OSTI
NASA NTRSDate not supplied

Filling the Gaps in Understanding Solar Flares

Solar flares provide a laboratory for plasma physics and magnetohydrodynamic processes. While the mechanism of transferring magnetic flux between topological domains via reconnection is accepted in a general sense, some of the finer details remain to be worked out: What elusive thresholds, pertaining to which physical quantities, determine the initiation of flux transfer that cascades into the energy release observed in the impulsive phase of a flare? What role does turbulence play in accelerating or prolonging magnetic reconnection? To what degree do magnetic waves contribute to the transport of energy, or to the acceleration of particles in the impulsive phase? While our innate human curiosity and our instinctive impulse to explore compel us to investigate these intricacies of million-degree magnetized plasmas, the impacts of space weather on our technology-dependent society and economy obligate us to pursue a quantitative understanding, with the ultimate goal being trustworthy predictive capabilities. In this presentation, I will touch upon a few aspects of solar flares where improvements in observational and analytical capabilities can be expected to help us fill gaps in our understanding, as well as some of the recent, current, and near-term developments that are carrying the field forward.

David E. McKenzie↗
NASA NTRS2023

NASA's Responsible AI Use Cases

This submission consists of summary use cases for NASA's Responsible Artificial Intelligence (RAI). These RAI Use Cases are to be made public, pursuant to the Presidential Executive Order 13960, Promoting the Use of Trustworthy Artificial Intelligence in the Federal Government. They were collected from NASA's practicing AI research community and cover the gamut of NASA's AI activities. These Use Cases will be updated annually as required by the Executive Order. The information included consists of the NASA Center, a summary of the goal, the AI techniques being applied, information on training data, and information on source code.

Artificial Intelligence↗
NASA NTRSDate not supplied

NASA Aeronautics Research Mission Directorate System Security Engineering Approaches

System security engineering (SSE) is a set of formal engineering methods and is considered a subset of systems engineering. It is a relatively new development in systems engineering with the initial NIST (National Institute of Standards) standard published in November of 2016 with updates in 2018, and 2022. The guiding principles in our methodology are based in NIST Special Publication 800-160 Vol. 1 “Systems Security Engineering: Considerations For A Multidisciplinary Approach In The Engineering Of Trustworthy Secure Systems” and integrate methodologies from common IT (Information Technology) threat modeling approaches utilizing MBSE (Model-Based Systems Engineering). The presentation will discuss how our teams utilize SSE and MBSE (Model-Based Systems Engineering) to develop secure architectures for systems under development in our NASA aeronautics research environment. This includes the activities to develop Protection Needs (PN) that, in turn result in security requirements in the design context and policies for the future state operational context for system protection. The process of applying SSE to analyze project architectures and ConOps (Concept of Operations) is intended to ensure the transferred research is both secure and securable in a “real-world” setting.

Systems Security Engineering↗
NASA NTRS2020

Human-in-the-loop Robotic Control for Human-Robot Collaboration

The prospect of a collaborative work environment between humans and robotic automation in a manufacturing setting [1] provides the motivation for finding innovative solutions to human-in-the-loop control for safe, efficient, and trustworthy human-robot collaboration (HRC), or HR interaction (HRI), in cyberphysical human systems (CPHSs) [2], [3]. Studies in [4] show that collaborative automation can be beneficial to 90% of approximately 300,000 small-to-medium-scale enterprises in the United States. In the paradigm of human-centered automation [5], human safety, ergonomics, and the collaborative efficiency of the work are given the utmost importance. Traditional methods to ensure the safety of humans around factory robots involve the use of cages. Recent work looked beyond cage-based safety to provide robot control and sensing-driven solutions for human safety around robots [6]-[8].

Ashwin P. Dani↗
NASA NTRS2024

Discrete Event Simulation-Based Timeline Validation Using R2U2

The Gateway Vehicle Systems Manager (VSM), the top-level software control system in a distributed, hierarchical Autonomous System Management Architecture is, like most modern spacecraft software control systems, heavily data-driven. For example, schedules (timelines) will be developed on the ground and, due to the high degree of autonomy, contain complex procedures involving conditional branching, variable timing, and resource contention resolution. In order to verify that an uploaded timeline will function correctly, it is necessary to explore the feasible set of possible executions. While it is possible to test a timeline using a mission simulation, the complexity of the system and duration of a timeline limits the number of trials and therefore the test coverage. To address this problem, the VSM team is using a discrete event system model that can rapidly generate from a timeline sets of event sequences using Monte Carlo techniques. To achieve rapid and trustworthy checking of the event sequences, we use an offline version of the runtime model checking tool R2U2. This presentation describes the approach the VSM team is using to implement the discrete event simulation and evaluate event sequences using R2U2. The presentation will discuss: 1. Description of the timelines by VSM in the context of VSM operations 2. Expansion of a timeline into a sequence of atomic events 3. Adjustment, in the Monte Carlo environment, of an event sequence to account for uncertainty, external events, and failures 4. Definition of R2U2 input and mission-time linear temporal logic files 5. Generation and use of R2U2 verdict sequences 6. Lessons learned and future work

Verification↗
NASA NTRS2023

Decoupling in the Vertical Shape of HCHO During a Sea Breeze Event: the Effect on Trace Gas Satellite Retrievals and Column-to-Surface Translation

The effect of sea breeze circulation on stratification of the vertical formaldehyde (HCHO) concentration vertical profiles is explored using a regional atmospheric chemical transport model (CTM) for three synoptically stagnant days focused on the east coast of the U.S in June 2018. During this event, a significant thermal contrast between the Atlantic Ocean and the terrestrial regions (12–17 °C), observed by moderate resolution imaging spectroradiometer (MODIS) and well-captured by the WRF-CMAQ model (15–18 °C), is conducive to monsoon-like flow, perpendicular to the shorelines, carrying clean marine air masses over the land within a few hundreds of meters above the surface. In contrast, the westerly continental polluted air masses prevail in higher altitudes. These two conflicting flows result in atypical vertical shapes of HCHO concentrations increasing with altitude. This decoupling pattern is so pronounced that we observe total column HCHO negatively correlate with surface concentrations. Comparisons of an accredited global model, GEOS-CF, to surface wind measurements and MODIS skin temperature indicate its poor representation of the sea breeze timing and strength, resulting in GEOS-CF HCHO vertical shapes being drastically different from the WRF-CMAQ. Based on radiative transfer calculations, the differences in the vertical distribution of HCHO between the WRF-CMAQ and that of GEOS-CF in the first 3 km are sufficient to induce a 20–30% error in air mass factors (thus total vertical HCHO column abundances). Through an experiment involving converting HCHO total columns to surface mixing ratios, we demonstrate that GEOS-CF allocates noticeably more HCHO molecules (40–150%) to the surface layer due to the misrepresentation of the vertical shape of HCHO during the sea breeze event. It is known that a significant fraction of the human population lives in coastal areas prone to detrimental effects caused by air pollution, and elevated pollutant concentrations usually occur in synoptically stagnant atmospheric conditions when local circulation patterns come into play; accordingly, our experiments emphasize the importance of the effect a priori profiles can have on satellite-derived applications under such conditions. To ensure that the quantitative representation of satellite-based trace gas retrievals on a daily basis is trustworthy and useable for air quality applications, atmospheric models providing a priori profiles for satellite retrievals should be well-tuned to reproduce complex local circulation such as sea-land breezes.

sea breeze↗
NASA NTRSDate not supplied

In-Depth Modeling and Simulation Analysis of Artemis Missions Using the Impact Probabilistic Risk Assessment Tool

BACKGROUND The Artemis campaign is a Moon exploration program with a series of six planned missions, five of which will be crewed. These five crewed missions will contain a single mission segment (space flight), or multiple mission segments involving space flight (Orion), lunar landing (LTV) and/or space habitat (Gateway). Each crewed segment faces the risk of unique medical conditions, necessitating medical sets/kits tailored to those specificities. To support and enable a data-driven and evidence-based decision-making process through out a mission’s life cycle, a software tool called IMPACT was developed. Using probabilistic risk assessment (PRA) methodologies, IMPACT (Informing Mission Planning via Analysis of Complex Tradespaces) is a novel tool built for analyzing the possibility of encountering complex medical risks during space flight, and for identifying the medical resources and capabilities needed to treat those potential at-risk medical conditions. IMPACT achieves this by performing hundreds of thousands of Monte Carlo simulations of missions to build aggregate pictures of medical risk. During an extended simulation modeling phase, IMPACT generated analytical results for medical risks, and the medical resources and capabilities to address those risks, for every segment of every crewed Artemis mission. This presentation will highlight the reliability, consistency and validity of IMPACT’s computational modeling techniques and will showcase the library of analytical outcomes generated for the Artemis missions. OVERVIEW During the early stages of IMPACT’s design, architecture and technical requirements collection, “scenarios” (use cases) - achievement goals required for acceptance testing, were identified by stakeholders. IMPACT successfully completed the scenario testing requirements and undertook an extensive operational run phase utilizing a wide range of input combinations with a goal of delivering a cohesive, trustworthy, reliable, vast, and diverse body of evidence. The intent of these modeling runs was to validate consistency in output, ensure solidity of executable operations and to streamline processes by identifying areas requiring efficiency improvements. Using the many missions of Artemis, IMPACT ran variations of operational runs to assess the output for acceptable, as well as unusual characteristics. This rigorous long-term “shakedown” analysis was implemented to help build a collective body of evidence to aid in securing a high level of confidence, reliability, and validity in the output, whether from the applicational components of IMPACT, or the entirety of the operational process. ANTICIPATED ANALYSIS AND CONCLUSION This presentation will discuss the various categories of input criteria; the comparisons in the application of these input criteria to various Artemis missions; the preparation and collection of the body of evidence, and reliability of the computational modeling techniques. This paper serves as an initial analytical overview of IMPACT’s probabilistic risk assessment (PRA) medical risk outputs covering Artemis missions and is not intended to be deemed the official medical response for the Artemis campaign.

Crew Composition↗
NASA NTRSDate not supplied

Data Democratization: Challenges and Opportunities

Democratizing Earth data is one of the challenges many organizations around the world face in order to maximize the use of their Earth data for research, applications, education, and societal benefits. For example, at the NASA Goddard Earth Sciences (GES) Data and Information Services Center (DISC), over 1600 global and regional datasets in several NASA Earth science focus areas, including atmospheric composition, water and energy cycles, and climate variability, are archived and distributed to the public. Giovanni, the Geospatial Interactive Online Visualization and Analysis Infrastructure, was developed by GES DISC to facilitate data access and exploration, especially for novice users of Earth science. With Giovanni, users can analyze and visualize over 2000 Earth science variables (e.g., precipitation, aerosol, surface wind) without downloading data, software, the expert understanding of data formats and structures, and coding skills, lowering the barrier to data analysis/comparison by preprocessing and accessing to the data. Results of data analysis and visualization can be accessed in several popular formats (e.g., NetCDF, CSV). As a result of Giovanni's efforts, more than 3000 referral papers have been published in various fields. In spite of this, Giovanni is still difficult to use for some users. For instance, if one searches for "precipitation," it will return over 150 related variables. The question is, which one to use? Furthermore, variables from different data providers (e.g., satellites and models) are named differently with different units, further confusing users, especially those outside the communities. Data democratization is complex and multifaceted. Challenges include service and data discovery, user experiences, visualization, data quality, trustworthiness, and more. In this presentation, we will examine Giovanni as an example of challenges and opportunities in developing data democratization services.

data democratization↗
DOE OSTI2020

Probing for Artifacts: Detecting Imagenet Model Evasions

While deep learning models have made incredible progress across a variety of machine learning tasks, they remain vulnerable to adversarial examples crafted to fool otherwise trustworthy models. In this work we approach this problem through the lens of a detection framework. We propose a classification network that uses the hidden layer activations of a trained model as inputs to detect adversarial artifacts in an input. We train this classification network simultaneously against multiple adversarial algorithms to create a more robust detector and show higher detection rates than several alternatives. The novelty of our approach is in the scale and scope of probing Imagenet models for adversarial artifacts. In addition, we propose an improvement to feature squeezing, another common adversarial example detection method.

Rounds, Jeremiah↗
DOE OSTI2021

“Frequency” and the PMU standard

The Phasor Measurement Unit (PMU) has revolutionized power system management by providing high-resolution, time-stamped, spatial, and temporal measure¬ments of the state of power systems. However, the standards defining the measured quantities have ambiguities that cause uncertainty, irreproducibility, and loss of interoperability during periods of change where the measurements should have the greatest value. This paper examines PMU frequency measurement to highlight the problems. It develops some suggestions for the scope and the next evolutionary improvements to the documentary standards. The aim is to improve the meaningfulness and trustworthiness of dynamic power system measurements.

Frequency, rate of change of frequency, definition↗
DOE OSTI2021

Robustness of deep learning algorithms in astronomy -- galaxy morphology studies

Deep learning models are being increasingly adopted in wide array of scientific domains, especially to handle high-dimensionality and volume of the scientific data. However, these models tend to be brittle due to their complexity and overparametrization, especially to the inadvertent adversarial perturbations that can appear due to common image processing such as compression or blurring that are often seen with real scientific data. It is crucial to understand this brittleness and develop models robust to these adversarial perturbations. To this end, we study the effect of observational noise from the exposure time, as well as the worst case scenario of a one-pixel attack as a proxy for compression or telescope errors on performance of ResNet18 trained to distinguish between galaxies of different morphologies in LSST mock data. We also explore how domain adaptation techniques can help improve model robustness in case of this type of naturally occurring attacks and help scientists build more trustworthy and stable models.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Data Curation for Machine Learning Applied to Geothermal Power Plant Operational Data for GOOML: Geothermal Operational Optimization with Machine Learning: Preprint

Geothermal Operational Optimization with Machine Learning (GOOML) is a transferable and extensible component-based geothermal asset modeling framework that considers complex steamfield relationships and identifies optimization prospects using a data-driven approach to physics-guided, data-centric machine learning. This framework has been used to develop digital twins that provide steamfield operators with operational environments to analyze and understand historical and forecasted power production, explore new steamfield configuration possibilities, and seek optimal asset management in real world applications. To create, test, and apply the GOOML framework, diverse time-series datasets spanning multiple years were sourced from various geothermal power plant components within several complex real-world geothermal operations. These operations are based in the United States and New Zealand and include a variety of technologies, end-uses and configurations, collectively covering nearly all relevant operating conditions for modern geothermal fields. Datasets were acquired from multiple sources to ensure that machine learning experiments generalized properly to various operating conditions. It was found that the data varied in quality, format, and completeness. To ensure consistency between the various datasets, a standardized data curation process was developed to reliably streamline data preparation. This paper will discuss best practices as learned from the GOOML data curation process which takes the following steps: 1) acquisition of large quantities of data from power plant operators, 2) digestion of data to gain an initial understanding of what is included, 3) data transformation, which includes converting the data into a standardized machine-readable format so that they can be visualized, quality checked, and cleaned, 4) quality assurance and quality control, involving identification of significant data gaps and apparent anomalies through mapping of data features to real world componentry via the GOOML historical model, followed by discussion with modelers and power plant operators to identify additional data needs and to resolve issues, 5) use in machine learning algorithms, and 6) repetition of steps one through five until all data needs are met and data are deemed suitable for producing trustworthy modeling results which may be disseminated, ideally along with the curated dataset. This iterative process is focused on improving the quality of the data rather than tuning machine learning model parameters and supports a shift towards data-centric AI as a means to improving real-world applicability of geothermal machine learning projects.

access↗
DOE OSTI2022

Application of Orthogonal Defect Classification for Software Reliability Analysis

The modernization of existing and new nuclear power plants with digital instrumentation and control systems (DI&C) is a recent and highly trending topic. However, there lacks strong consensus on best-estimate reliability methodologies by both the United States (U.S.) Nuclear Regulatory Commission (NRC) and the industry. This has resulted in hesitation for further modernization projects until a more unified methodology is realized. In this work, we develop an approach called Orthogonal-defect Classification for Assessing Software Reliability (ORCAS) to quantify probabilities of various software failure modes in a DI&C system. The method utilizes accepted industry methodologies for software quality assurance that are also verified by experimental or mathematical formulations. In essence, the approach combines a semantic failure classification model with a reliability growth model to predict (and quantify) the potential failure modes of a DI&C software system. The semantic classification model is used to address the question: How do latent defects in software contribute to different software failure root causes? The use of reliability growth models is then used to address the question: Given the connection between latent defects and software failure root causes, how can we quantify the reliability of the software? A case study was conducted on a representative I&C platform (ChibiOS) running a smart sensor acquisition software developed by Virginia Commonwealth University (VCU). The testing and evidence collection guidance in ORCAS was applied, and defects were uncovered in the software. Qualitative evidence, such as condition coverage, was used to gauge the completeness and trustworthiness of the assessment while quantitative evidence was used to determine the software failure probabilities. The reliability of the software was then estimated and compared to existing operational data of the sensor device. It is demonstrated that by using ORCAS, a semantic reasoning framework can be developed to justify if the software is reliable (or unreliable) while still leveraging the strength of the existing methods.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

Application of Orthogonal Defect Classification for Software Reliability Analysis

The modernization of existing and new nuclear power plants with digital instrumentation and control systems (DI&C) is a recent and highly trending topic. However, there lacks strong consensus on best-estimate reliability methodologies by both the United States (U.S.) Nuclear Regulatory Commission (NRC) and the industry. This has resulted in hesitation for further modernization projects until a more unified methodology is realized. In this work, we develop an approach called Orthogonal-defect Classification for Assessing Software Reliability (ORCAS) to quantify probabilities of various software failure modes in a DI&C system. The method utilizes accepted industry methodologies for software quality assurance that are also verified by experimental or mathematical formulations. In essence, the approach combines a semantic failure classification model with a reliability growth model to predict (and quantify) the potential failure modes of a DI&C software system. The semantic classification model is used to address the question: How do latent defects in software contribute to different software failure root causes? The use of reliability growth models is then used to address the question: Given the connection between latent defects and software failure root causes, how can we quantify the reliability of the software? A case study was conducted on a representative I&C platform (ChibiOS) running a smart sensor acquisition software developed by Virginia Commonwealth University (VCU). The testing and evidence collection guidance in ORCAS was applied, and defects were uncovered in the software. Qualitative evidence, such as condition coverage, was used to gauge the completeness and trustworthiness of the assessment while quantitative evidence was used to determine the software failure probabilities. The reliability of the software was then estimated and compared to existing operational data of the sensor device. It is demonstrated that by using ORCAS, a semantic reasoning framework can be developed to justify software reliability (or unreliability) while still leveraging the strength of the existing methods.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

Smart sensor for online situational awareness in power grids

Waveforms in power grids typically reveal a certain pattern with specific features and peculiarities driven by the system operating conditions, internal and external uncertainties, etc. This prompts an observation of different types of waveforms at the measurement points (substations). An innovative next-generation smart sensor technology includes a measurement unit embedded with sophisticated analytics for power grid online surveillance and situational awareness. The smart sensor brings additional levels of smartness into the existing phasor measurement units (PMUs) and intelligent electronic devices (IEDs). It unlocks the full potential of advanced signal processing and machine learning for online power grid monitoring in a distributed paradigm. Within the smart sensor are several interconnected units for signal acquisition, feature extraction, machine learning-based event detection, and a suite of multiple measurement algorithms where the best-fit algorithm is selected in real-time based on the detected operating condition. Embedding such analytics within the sensors and closer to where the data is generated, the distributed intelligence mechanism mitigates the potential risks to communication failures and latencies, as well as malicious cyber threats, which would otherwise compromise the trustworthiness of the end-use applications in distant control centers. The smart sensor achieves a promising classification accuracy on multiple classes of prevailing conditions in the power grid and accordingly improves the measurement quality across the power grid.

Dehghanian, Payman↗
DOE OSTI2021

On the Stochastic Stability of Deep Markov Models

Deep Markov models (DMM) are generative models which are scalable and expressive generalization of Markov models for representation, learning, and inference problems. DMMs using deep neural networks to parametrize the transition of Markov probability distributions have recently been shown to provide more expressiveness in modeling sequential data and dynamical system responses. However, the fundamental stochastic stability guarantees of such models have not been thoroughly investigated. In this paper, we present a rigorous analytical method to prove the necessary and sufficient conditions of DMM's stochastic stability. This task is achieved by spectral analysis of the efficiently computed Jacobians of probabilistic maps modeled by deep neural networks. We make theoretical connections between the eigenvalues of neural network's weights and the different activation function types used on the stability and overall dynamic behavior of DMMs with Gaussian distributions. We empirically substantiate our theoretical results on stochastic stability and eigenvalue spectra via several numerical experiments. Formal stability guarantees of DMMs can substantially improve their robustness and trustworthiness, necessary for reliable use in safety-critical real-world applications.

Drgona, Jan↗
DOE OSTI2023

Opportunities and Challenges for Remote Microreactor Operations

The nuclear industry is developing new advanced reactor technologies, and many companies are embracing this advancement by pursuing the development of microreactors. The term microreactor generally refers to a nuclear reactor with an operating power of 20 MW or less. The power range of microreactors makes them appealing for many use cases, such as powering remote communities, mining sites, and military bases. Most of the microreactor designs being pursued will incorporate remote facility operations into the final product. However, no framework has yet been developed to determine what remote operations systems require for reliable, resilient, and secure operation of a microreactor. This work identifies the challenges unique to remote operations and monitoring for microreactors specifically regarding instrumentation and control, communication methods, regulatory requirements, and operational policies. The types of commands and sensor measurements that must be transmitted between the facilities as well as methods for verifying the trustworthiness of these signals are assessed. This work evaluates the security, reliability, and performance requirements that must be met when considering the selection of communication hardware and protocols for use in remote operations. Also, an assessment was performed to study how remote operations fit within current regulatory requirements and what may need to be updated in regulatory policy to allow for remote operation. Finally, the operational contingencies unique to remote operations that must be in place for response to abnormal events are identified. This paper details these challenges and research opportunities to provide a foundation for the design of remote operation systems.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2023

Uncertainty Quantified Machine Learning for Street Level Flooding Predictions in Norfolk, Virginia

Everyday citizens, emergency responders, and critical infrastructure can be dramatically affected by street flooding caused by the increasing tidal levels and major storm events magnified by climate change. Low-level flooding, such as nuisance flooding, continues to increase in frequency, especially in cities like Norfolk, Virginia, which can expect nearly 200 flooding events by 2050 [1]. Recently, machine learning (ML) models have been leveraged to produce real-time predictions based on local weather and geographic conditions. However, ML models are known to produce unusual results when presented with data that varies from their training set. For decision-makers to determine the trustworthiness of the model's predictions, ML models need to quantify their prediction uncertainty. This study applies Deep Quantile Regression (DQR) to a previously published, LSTM-based model for hourly water depth predictions [2], and analyzes its out-of-distribution (OOD) performance.

Goldenberg, Steven↗