Results for “Confidence” — Page 43

NASA NTRSDate not supplied

Supervised Autonomous Assembly to Create and Evolve Persistent Assets

Supervised autonomous assembly (SAA) will create a paradigm shift in the planning and design of future persistent assets (PAs), both in near zero-g environments and on planetary surfaces. SAA refers to an autonomy approach that has the benefits of autonomous assembly as well as the benefits provided by a supervisor (operator) who is available to resolve unexpected situations. SAA provides both increased design freedom as well as reduced programmatic risk. SAA enables evolution of future PAs over decades as in-space operations transition from single purpose missions to creation of PAs, such as laboratories and experimental stations which more closely resembling terrestrial laboratories that can easily adapt and evolve to new missions leveraging repeated visits to the PA. The ability to evolve enables PAs to rapidly respond to changing objectives resulting from new questions as our understanding improves. A recently initiated National Aeronautics and Space Administration (NASA) project in the Space Technology Mission Directorate (STMD) Game Changing Development (GCD) Program called the Precision Assembled Space Structure (PASS), leverages the advantages of SAA to develop technologies that enable efficient creation and evolution of hexagonal topologies; both planar (example: fuel depots) and curved (examples: telescopes and shelters). PASS will be used to provide context for the philosophy and concepts discussed as well as the decision and selections made. PASS objectives are: a) Develop confidence in SAA and on-orbit servicing, assembly and manufacturing (OSAM) technologies by executing a test campaign that uses a path-to-flight autonomous precision assembly process directly applicable to future space telescopes. b) Test autonomous technologies including automated path planning and error recovery, to emphasize a robust approach that relies on generic robots and special purpose tools. c) Validate critical component models using a digital twin that includes the assembled primary mirror support structure and assembly process. A digital twin is a high-fidelity simulation of the asset capable of predicting the on-orbit performance. The paper concludes after identifying the critical need for a modest assembly flight experiment to validate and develop confidence in the SAA paradigm, thus accelerating adoption of the benefits described. SAA is a game changing paradigm that enhances the ability of an organization to infuse new technology through rapid evolution of PAs while leveraging OSAM technologies.

Structural Modeling↗

NASA NTRSDate not supplied

Revisiting Building-Block Approaches for Structures

The building-block approach is a method for building confidence in designs, working from lower-level components to more complex assemblies, using knowledge gained at each step to inform higher fidelity representations. The goals of this paper are to revisit the basic techniques of the building-block approach as it applies to structural design and analysis and illustrate some best practices that should be considered when developing structural analysis models. The paper demonstrates the process of leveraging existing information to build confidence in the design/development of analysis models, followed by examples of how the building-block approach for analysis was used in NASA hardware and spacecraft development efforts. The paper will also discuss virtual testing and hybrid simulation techniques, which are showing promise for more efficient/cost-effective verification of aerospace structures.

Kenneth R. Hamm Jr.↗

NASA NTRS2021

Radar-Based Bayesian Estimation of Ice Crystal Growth Parameters within a Microphysical Model

The potential for polarimetric Doppler radar measurements to improve predictions of ice microphysical processes within an idealized model–observational framework is examined. In an effort to more rigorously constrain ice growth processes (e.g., vapor deposition) with observations of natural clouds, a novel framework is developed to compare simulated and observed radar measurements, coupling a bulk adaptive-habit model of vapor growth to a polarimetric radar forward model. Bayesian inference on key microphysical model parameters is then used, via a Markov chain Monte Carlo sampler, to estimate the probability distribution of the model parameters. The statistical formalism of this method allows for robust estimates of the optimal parameter values, along with (non-Gaussian) estimates of their uncertainty. To demonstrate this framework, observations from Department of Energy radars in the Arctic during a case of pristine ice precipitation are used to constrain vapor deposition parameters in the adaptive habit model. The resulting parameter probability distributions provide physically plausible changes in ice particle density and aspect ratio during growth. A lack of direct constraint on the number concentration produces a range of possible mean particle sizes, with the mean size inversely correlated to number concentration. Consistency is found between the estimated inherent growth ratio and independent laboratory measurements, increasing confidence in the parameter PDFs and demonstrating the effectiveness of the radar measurements in constraining the parameters. The combined Doppler and polarimetric observations produce the highest-confidence estimates of the parameter PDFs, with the Doppler measurements providing a stronger constraint for this case.

Robert S. Schrom↗

NASA NTRS2022

A Global Evaluation of IMERG Precipitation Occurrence Using SMAP Detected Soil Moisture Change

A globally consistent ground validation method for remotely sensed precipitation products is crucial for building confidence in these products. This study develops a new methodology to validate the IMERG precipitation products through the use of SMAP soil moisture changes as a proxy for precipitation occurrence. Using a standard 2x2 contingency table method, preliminary results provide confidence in SMAP’s ability to be utilized as a validation tool for IMERG as results are comparable to previous validation studies. However, the method allows for an overestimate of false alarm frequency due to light precipitation events that can evaporate before the subsequent SMAP overpass and changes in overpass-to-overpass SMAP soil moisture that are within the range of SMAP uncertainty. To counter these issues, a 3x3 contingency table is used to reduce noise and extract more signal from the detection method. Through the use of this novel approach, the validation method produces a global mean POD of 0.64 and global mean FAR of 0.40, the first global-scale ground validation skill scores for the IMERG products. Advancing the method to validate precipitation quantity and the development of a real-time validation for the IMERG Early product are the crucial next developments.

Precipitation↗

NASA NTRS2022

Run Time Assurance for Electric Vertical Takeoff and Landing Aircraft

NASA is conducting research to demonstrate and evaluate the application of Run Time Assurance (RTA) as a means to assure safety in Electric Vertical Takeoff and Landing (eVTOL) aircraft with highly automated or autonomous flight capability supervised by a single onboard pilot. The work described in this report demonstrates an application of RTA and examines the implications for design and analysis of aircraft functions and systems; aircraft safety hazards; safety assurance; development assurance; and pilot tasks and performance. This research effort also seeks to assess the efficacy of the combined application of traditional Functional Hazard Analysis (FHA) and the more modern System Theoretic Process Analysis (STPA) techniques to perform hazard analyses on aircraft with complex automated and autonomous systems and an onboard pilot. During the research effort we developed architectural designs of two alternate eVTOL aircraft, generally following the process characterized in the SAE standards ARP4754 and ARP4761. The design has focused on the control architectures of these aircraft, which are identical except that one incorporates RTA techniques to reduce the criticality of some key software components. Artifacts of this process include a taxonomy of aircraft-level functions, aircraft-level architecture diagrams, aircraft-level functional hazard assessments (AFHA), function allocations onto aircraft systems and subsystems, functional block diagrams for a select set of control-related functions, and system-level functional hazard assessments (SFHA) for those functions. This project has highlighted the notion that DAL D is something of a sweet spot for low-confidence controllers in an RTA-based design. Among the many activities described in DO-178C, the activities related to requirement verifiability, algorithmic accuracy, and test coverage can be the most challenging for the kinds of advanced control techniques that may be desirable in novel UAM designs, such as adaptive control, machine-learning, artificial intelligence, numerical search, and Monte Carlo based algorithms. Moreover, the standard requires that development teams demonstrate that errors leading to unacceptable failure conditions have been removed from the software. The RTA architecture, which cordons off the low-confidence function, makes it much easier to show this for these kinds of algorithms. With regard to the use of STPA and FHA as complementary hazard analysis techniques, our research effort led us to the conclusion that STPA should be used to derive requirements for hardware and software systems and/or components. Also, STPA is a natural complement to other processes in ARP4754A involving design studies and iteration.

Run-time assurance↗

NASA NTRSDate not supplied

Mars Ascent Vehicle GNC Targeting Routines with Considerations for Flight Software Development

The Mars Ascent Vehicle (MAV) will be the first vehicle to perform an ascent from the surface of another atmospheric planetary body outside of the Earth-Moon system. Significant light-time delay requires complete autonomy of flight throughout ascent, and naturally a high level of reliability is desired in both MAV’s hardware and software subsystems. The MAV Guidance, Navigation and Controls (GNC) team and the MAV Flight Software (FSW) team have partnered together to improve the efficiency of algorithm integration onto the MAV flight processor, and to increase confidence that said integration is successful and without human error. An interface architecture is proposed for the GNC suite that allows both the guidance and navigation subsystems to provide code algorithms directly in C++, and the controls subsystem to provide MATLAB Simulink auto-coded algorithms. Several continuous integration/deployment (CI/CD) methodologies have been considered for ease of transition of algorithm code from the GNC team to the FSW team. The GNC/FSW teams also worked together to develop a cFS-friendly wrapper which abstracts the integration of the GNC algorithm code into an interface-level API that is compatible with cFS. Several iterations of vehicle GNC code have been produced between the GNC/FSW team’s partnership, and this strong interface between these two teams have allowed the GNC/FSW teams to greatly increase confidence of efficient and error-free implementation of the GNC code onto MAV for a successful flight.

Jason Everett↗

NASA NTRSDate not supplied

Unlocking the Mysteries of the Moon’s Shadowed Regions

The Moon poles host large quantities of water-ice deposits in the permanently shadowed regions (PSRs), which are vital for enabling sustainable human space exploration, making these regions high-priority targets for upcoming Artemis missions [1]. Unfortunately, today, the best available orbital lunar imagery [2, 3] lacks the meter-scale resolution and signal needed to understand the geomorphology and trafficability of PSRs, complicating the planning and execution of future missions seeking to explore PSRs. We have developed an image enhancement tool called HORUS (Hyper-effective nOise Removal Unet Software) [4, 5], designed to enhance LRO Narrow-Angle Camera (NAC) optical low-light imagery of permanently shadowed regions by effectively removing the CCD-related, photon, and other residual noises that corrupt the images. The tool is composed of two deep learning neural networks trained on environmental metadata and real and synthetic imagery, the latter generated by a physical noise model (LROC). We demonstrated that HORUS effectively produces low-noise, high-resolution images (~1.5m/px), achieving a 5 to 10x improvement over existing long-exposure images of PSRs. HORUS allows scientists and engineers to identify geomorphic features (e.g., craters and boulders) in shadowed regions as small as 3 meters across as well as to peek inside of small shadowed regions, for the first time. The tool was deployed and thoroughly validated for NASA's VIPER mission [6], where it was applied to 20 candidate target regions across the lunar South Pole. Additionally, we conducted different approaches to validate the resulting HORUS-processed images. With HORUS denoised images, VIPER scientists can increase their confidence on what surface features (previously unseen) exist in the shadowed regions, helping them plan rover traverses more safely and efficiently (e.g., Fig. 1) In this manuscript, we will describe how VIPER scientists are utilizing HORUS denoised images to extract new information from the terrain and increase their confidence in what surface features exist in the shadowed regions. In combination with other high-resolution images and digital elevation maps, HORUS images are helping the team analyze potential lading and science sites, as well as planning traverses more safely and efficiently (e.g., Fig. 1). Additionally, we will describe how HORUS tool unlocks a broad range of scientific and exploration applications to other Artemis and CPLS missions to the lunar poles, including (but not limited to) geomorphic analysis, change detection, surface hazard detection, and terrain relative navigation.

artificial intelligence↗

NASA NTRS2022

Uncertainty Models for the Hybrid Parametric Variation Method of Uncertainty Quantification; Analysis

There is some level of uncertainty in every finite element model (FEM), which flows to a level of uncertainty in predicted results. The purpose of uncertainty quantification (UQ) is to provide statistical bounds on prediction accuracy based on model uncertainty. This is distinct from model updating, which attempts to modify models to improve their accuracy. UQ does not improve the accuracy of models, but accepts that the models are inherently inaccurate and attempts to quantify the impact of that inaccuracy on predicted results. Previously, an alternate method for UQ, called the Hybrid Parametric Variation (HPV) method, was applied to Space Launch System (SLS) Hurty/Craig-Bampton (HCB) components to predict system-level statistics for launch vehicle attitude control transfer functions and core stage section loads due to buffet. The HPV method combines a parametric variation of the HCB fixed-interface (FI) modal frequencies with a nonparametric variation (NPV) method that randomly varies the HCB mass and stiffness matrices as Wishart random matrix distributions using random matrix theory (RMT). Alternatively, the most common method for modeling uncertainty in the structural dynamics community is a parametric approach, which varies physical parameters in the model. However, there are several disadvantages associated with the parametric method. Determining a reduced set of parameters that have a significant impact on the system response can be time consuming, and the selected parameter probability distributions are rarely reliably known. Therefore, in practice, the parameters are surrogates for the actual errors, and the link to parameter uncertainty is unknown. Another major drawback is that the uncertainty that can be represented is limited to the form of the nominal FEM. It is the experience of the authors that based on numerous aerospace programs, almost all FEM errors are in form rather than parameter values. This hypothesis is supported by the observation of the authors that it is almost never possible to ‘tune’ a FEM to match modal test results by only modifying model parameters. Model-form uncertainty cannot be directly represented by FEM input parameters nor included in a parametric approach. However, model-form uncertainty can be modeled using RMT, where a probability distribution is developed for the matrix ensemble of interest. The major advantage of the NPV method is that it covers errors in model form. The HPV method anchors uncertainty at the HCB component level to component modal test results by matching the HCB and test modes based on mode descriptions or other methods, and then applying differing levels of frequency variation. The specific variations depend on the confidence to which a component FEM has been validated through modal testing. The NPV method is layered on the frequency variation to match modal test self-orthogonality and cross-orthogonality (XO) results. Once the component uncertainty models are identified, they are assembled, and the uncertainty is propagated to the system level using a Monte Carlo (MC) analysis approach that generates statistics for system-level predictions This provides a UQ method that can be traced to test data, which can be updated as additional data and improved correlated models become available. The purpose of this paper is to collect and present all of the theory for HPV that has been previously published in reports and papers and to present examples of its application. Specifically, component uncertainty models based on the dispersion of corresponding mass and stiffness matrices using proposed test/analysis correlation metrics are investigated. The first example is purely academic so that the true answers are known, and the validity of the HPV method and the corresponding uncertainty models can be determined. The purpose of this paper is to collect and present all of the theory for HPV that has been previously published in reports and papers and to present examples of its application. Specifically, component uncertainty models based on the dispersion of corresponding mass and stiffness matrices using proposed test/analysis correlation metrics are investigated. The first example is purely academic so that the true answers are known, and the validity of the HPV method and the corresponding uncertainty models can be determined. The second example is an application to a component that is design specific to the SLS. Based on this work and other assessments, the HPV method provides another tool to the toolset used for complex system UQ analysis. From experience gathered to date using the HPV method, additional design specific applications must be investigated to provide further confidence in the validity of the HPV method of UQ analysis.

Uncertainty quantification↗

NASA NTRSDate not supplied

Exploitation of a Validation Hierarchy for Modeling and Simulation

Across engineering there is an evolving need to increase reliance on physics-based simulation to develop, design and optimize engineering systems. This increased reliance on modeling and simulation has highlighted a growing need to transform the confidence that modeling and simulation analysts have in their results into credibility for systems engineers to design and field systems more quickly and with less physical testing. For isolated components of a complex system, where a single discipline may drive product design, this is less of a concern as the relationship is often straightforward and easy to explain. However, when these isolated components are integrated, and are expected to operate in a multi-disciplinary context in which safety critical systems are involved, new concepts and model assurance standards are required. In this paper we address this challenge by showing how a model validation hierarchy can be exploited to identify those model validation experiments that will contribute most to increasing confidence and credibility of modeling and simulation predictions. The approach that is adopted contains four main steps. The first step is the construction of a model validation hierarchy that links subsystems, assemblies, and components to a hierarchy of physical experiments that can be used support model validation. This hierarchy connects the concerns of systems engineers to those of the modeling and simulation analyst in a clear and logical way. The structure and content of this hierarchy is then used in a second step to establish which physical phenomena have the greatest impact on overall system performance metrics. A gap analysis technique, based upon modeling and simulation concerns, is then used to prioritize the important physical phenomenon. Unfortunately, a common outcome of such gap analyses is the identification of many important gaps and so, in the final step of our process, we advocate the use of a global sensitivity analysis as a means to complete the prioritization.

Verification and Validation↗

NASA NTRSDate not supplied

Exploitation of a Validation Hierarchy for Modeling and Simulation

Across engineering there is an evolving need to increase reliance on physics-based simulation to develop, design and optimize engineering systems. This increased reliance on modeling and simulation has highlighted a growing need to transform the confidence that modeling and simulation analysts have in their results into credibility for systems engineers to design and field systems more quickly and with less physical testing. For isolated components of a complex system, where a single discipline may drive product design, this is less of a concern as the relationship is often straightforward and easy to explain. However, when these isolated components are integrated, and are expected to operate in a multi-disciplinary context in which safety critical systems are involved, new concepts and model assurance standards are required. In this paper we address this challenge by showing how a model validation hierarchy can be exploited to identify those model validation experiments that will contribute most to increasing confidence and credibility of modeling and simulation predictions. The approach that is adopted contains four main steps. The first step is the construction of a model validation hierarchy that links subsystems, assemblies, and components to a hierarchy of physical experiments that can be used support model validation. This hierarchy connects the concerns of systems engineers to those of the modeling and simulation analyst in a clear and logical way. The structure and content of this hierarchy is then used in a second step to establish which physical phenomena have the greatest impact on overall system performance metrics. A gap analysis technique, based upon modeling and simulation concerns, is then used to prioritize the important physical phenomenon. Unfortunately, a common outcome of such gap analyses is the identification of many important gaps and so, in the final step of our process, we advocate the use of a global sensitivity analysis as a means to complete the prioritization.

Verification and Validation↗

NASA NTRS2019

Simultaneous X-Ray and Infrared Observations of Sagittarius A*'s Variability

Emission from Sagittarius A* is highly variable at both X-ray and infrared (IR) wavelengths. Observations over the last ∼20 yr have revealed X-ray flares that rise above a quiescent thermal background about once per day, while faint X-ray flares from Sgr A* are undetectable below the constant thermal emission. In contrast, the IR emission of Sgr A* is observed to be continuously variable. Recently, simultaneous observations have indicated a rise in IR flux density around the same time as every distinct X-ray flare, while the opposite is not always true (peaks in the IR emission may not be coincident with an X-ray flare). Characterizing the behavior of these simultaneous X-ray/IR events and measuring any time lag between them can constrain models of Sgr A* ʼs accretion flow and the flare emission mechanism. Using 100+ hours of data from a coordinated campaign between the Spitzer Space Telescope and the Chandra X-ray Observatory, we present results of the longest simultaneous IR and X-ray observations of Sgr A* taken to date. The cross-correlation between the IR and X-ray light curves in this unprecedented data set, which includes four modest X-ray/IR flares, indicates that flaring in the X-ray may lead the IR by approximately 10–20 min with 68% confidence. However, the 99.7% confidence interval on the time-lag also includes zero, i.e., the flaring remains statistically consistent with simultaneity. Long-duration and simultaneous multi-wavelength observations of additional bright flares will improve our ability to constrain the flare timing characteristics and emission mechanisms, and must be a priority for Galactic Center observing campaigns.

H. Boyce↗

NASA NTRS2022

Multiwavelength Variability of Sagittarius A* in 2019 July

We report a timing analysis of near-infrared (NIR), X-ray, and submillimeter data during a 3 day coordinated campaign observing Sagittarius A* . Data were collected at 4.5 μm with the Spitzer Space Telescope, 2–8 keV with the Chandra X-ray Observatory, 3–70 keV with NuSTAR, 340 GHz with ALMA, and 2.2 μm with the GRAVITY instrument on the Very Large Telescope Interferometer. Two dates show moderate variability with no significant lags between the submillimeter and the infrared at 99% confidence. A moderately bright NIR flare (FK ∼ 15 mJy) was captured on July 18 simultaneous with an X-ray flare (F2−10 keV ∼ 0.1 counts s−1 ) that most likely preceded bright submillimeter flux (F340 GHz ∼ 5.5 Jy) by about + - + 34 33 14 minutes at 99% confidence. The uncertainty in this lag is dominated by the fact that we did not observe the peak of the submillimeter emission. A synchrotron source cooled through adiabatic expansion can describe a rise in the submillimeter once the synchrotron self-Compton NIR and X-ray peaks have faded. This model predicts high GHz and THz fluxes at the time of the NIR/X-ray peak and electron densities well above those implied from average accretion rates for Sgr A* . However, the higher electron density postulated in this scenario would be in agreement with the idea that 2019 was an extraordinary epoch with a heightened accretion rate. Since the NIR and X-ray peaks can also be fit by a nonthermal synchrotron source with lower electron densities, we cannot rule out an unrelated chance coincidence of this bright submillimeter flare with the NIR/X-ray emission.

Galactic center↗

NASA NTRSDate not supplied

Analysis of Pull Force Test Results for Crimped Connections

Crimped electrical contacts are reliable when strict process controls are followed during manufacturing and accompanied by continuous process verification through pull force testing. Cable and wire harness assemblies’ standards are developed and refined over time to provide the minimum pull force that a crimp contact must meet before it breaks from the wire. However, in practice the failures occur at a much higher tensile strength than the minimum required. The first section of this paper reviewed 780 pull force test results provided by NASA Centers were collected and analyzed to determine how the data compares to NASA’s pre-existing requirements from the cable/harness standards NASA-STD-8739.4 and IPC/WHMA-A-620B-S. The measured tensile strength of most of the contact/conductor pairs (i.e. Contact/Wires or C/W) exceeded the minimum pull force values of NASA-STD-8739.4 and IPC/WHMA-A-620 standards by at least 100 %. The tensile strength samples of the C/W pairs values followed a normal distribution with an average tensile strength value that was at least 182 % of the minimum requirement, and all the samples analyzed passed the pull force testing. In addition, the 95 % confidence interval of the average tensile strength distributions for several C/W pairs was determined and plotted as error bars to show that there is good likelihood to observe C/W pair normal distribution with a 95 % confidence interval that will meet and surpass the standard requirements. The frequency of pull force testing can be problematic for projects because of the cost and availability of spare contacts for the destructive test. It could be possible to reduce the frequency of pull force testing if at the beginning of the production run, the conditions of the crimp tool and materials are verified, and the settings of the tool remain unchanged throughout the process. However, the project must evaluate the impact to risk from reducing the frequency of testing.

Crimped connections↗

NASA NTRS2022

Classifying Unidentified X-Ray Sources in the Chandra Source Catalog Using A Multiwavelength Machine-Learning Approach

The rapid increase in serendipitous X-ray source detections requires the development of novel approaches to efficiently explore the nature of X-ray sources. If even a fraction of these sources could be reliably classified, it would enable population studies for various astrophysical source types on a much larger scale than currently possible. Classification of large numbers of sources from multiple classes characterized by multiple properties (features) must be done automatically and supervised machine learning (ML) seems to provide the only feasible approach. We perform classification of Chandra Source Catalog version 2.0 (CSCv2) sources to explore the potential of the ML approach and identify various biases, limitations, and bottlenecks that present themselves in these kinds of studies. We establish the framework and present a flexible and expandable Python pipeline, which can be used and improved by others. We also release the training data set of 2941 X-ray sources with confidently established classes. In addition to providing probabilistic classifications of 66,369 CSCv2 sources (21% of the entire CSCv2 catalog), we perform several narrower-focused case studies (high-mass X-ray binary candidates and X-ray sources within the extent of the H.E.S.S. TeV sources) to demonstrate some possible applications of our ML approach. We also discuss future possible modifications of the presented pipeline, which are expected to lead to substantial improvements in classification confidences.

Hui Yang↗

NASA NTRS2022

Comparisons Between the Distributions of Dust and Combustion Aerosols in MERRA-2, FLEXPART and CALIPSO and Implications for Deposition Freezing Over Wintertime Siberia

Aerosol distributions have a potentially large influence on climate-relevant cloud properties but can be difficult to observe over the Arctic given pervasive cloudiness, long polar nights, data paucity over remote regions, and periodic diamond dust events that satellites can misclassify as aerosol. We compared Arctic 2008–2015 mineral dust and combustion aerosol distributions from the Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation (CALIPSO) satellite, the Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2) reanalysis products, and the FLEXible PARTicle (FLEXPART) dispersion model. Based on coincident, seasonal Atmospheric Infrared Sounder (AIRS) Arctic satellite meteorological data, diamond dust may occur up to 60 % of the time in winter, but it hardly ever occurs in summer. In its absence, MERRA-2 and FLEXPART each predict the vertical and horizontal distribution of large-scale patterns in combustion aerosols with relatively high confidence (Kendall tau rank correlation > 0.6), although a sizable amount of variability is still unaccounted for. They do the same for dust, except in conditions conducive to diamond dust formation where CALIPSO is likely misclassifying diamond dust as mineral dust and near the surface (< ∼ 2 km) where FLEXPART may be overpredicting local dust emissions. Comparisons to ground data suggest that MERRA-2 Arctic dust concentrations can be improved by the addition of local dust sources. All three products predicted that wintertime dust and combustion aerosols occur most frequently over the same Siberian regions where diamond dust is most common in the winter. This suggests that dust aerosol impacts on ice phase processes may be particularly high over Siberia, although further wintertime model validation with non-CALIPSO observations is needed. This assessment paves the way for applying the model-based aerosol simulations to a range of regional-scale Arctic aerosol–cloud interaction studies with greater confidence.

MERRA-2↗

NASA NTRS2023

Adaptive Independent Verification and Validation (IV&V) Reduces Risk of Software Impacting Safety in Artemis Missions

The National Aeronautics and Space Administration (NASA) is asking more of its human spaceflight programs than ever before through the collective Artemis Missions. The NASA Independent Verification and Validation (IV&V) Program contributes to NASA’s human spaceflight goals by providing IV&V services for NASA’s critical spacecraft and ground software. The IV&V Program is tasked with providing assurance from both individual and integrated mission software perspectives. The Artemis IV&V organization is actively supporting six distinct development efforts: Orion, the Space Launch System (SLS), Exploration Ground Systems (EGS), Mission Control Center (MCC), the Lunar Gateway, and the Human Landing System (HLS), representing a wide diversity of developer organizations, management structures, and development approaches. With much of this extremely complex flight and ground software being essential to human safety both on the ground and in space, Artemis IV&V is likewise challenged to provide more value-added assurance to future Artemis missions within a constrained budget. To meet this challenge, Artemis IV&V employs a variety of novel and evolving “Adaptive IV&V” approaches for planning and executing IV&V analysis to increase both the efficiency and effectiveness of the IV&V Program’s assurance activities, and to address the difficulties imposed by assuring software for a large, highly integrated, multi-mission enterprise managed and executed by physically and organizationally distinct programs. Instilling agile principles like iterative planning cycles, self-organizing teams, and regular retrospectives, into IV&V planning and execution has led to a more rapid turnaround of a minimum viable assurance product and allowed for increased alignment of assurance activities with development progress. Adopting an assurance case methodology has led to greater consistency and clearer communication of assurance design and provided a foundation for long-term maintenance of assurance plans, products, and results across missions. The IV&V-developed Assurance / Safety Case Analytical Network (A-SCAN) framework and tool has enabled the quantification and tracking of system/software risk and confidence. These confidence measures provide a means to repeatedly express the impact of planned and completed assurance work and the remaining residual risk. Applied as part of a “Follow-the-Risk” organizational ethos, this allows consistent rightsizing of analysis rigor and intensity commensurate with the perceived risk of defects, as well as appropriate targeting of the highest risk areas of the software to find safety issues before they can manifest. Finally, the development of the IV&V Advanced Risk Reduction Integrated Software Test and Operations Tri-program Lightweight Environment (ARRISTOTLE), an integrated software-only simulation of Orion, SLS, and EGS systems, has made it possible to independently test integrated pad and flight scenarios and inject faults to observe how the Artemis multi-program, mission software behaves in degraded modes and in response to hazards. These adaptive IV&V investments have enabled Artemis IV&V to become more efficient and effective in IV&V planning and execution and respond more readily to changes in the risk landscape, increasing the breadth and depth of risk reduction possible within the available resources. Residual risk tracking allows IV&V to communicate more effectively with stakeholders, both internal and external at all levels, and inform key decision-making personnel. This evolving assurance design approach provides IV&V surety that work is performed in the highest risk, most value-added areas of the software, to keep our astronauts and ground crews safe and ensure mission success.

Gerek A Whitman↗

NASA NTRS2022

Validation of the Mars 2020 Fault Protection Design: Navigating the Infinity of the Off-Nominal

On July 30th 2020, the Mars 2020 mission successfully launched out of Cape Canaveral, Florida, passed through the Earth’s shadow, and began its short cruise to Mars. Less than seven months later, the Perseverance rover touched down safely in Jezero Crater to begin its ambitious mission that includes looking for signs of ancient life and collecting samples for future return to Earth. Getting to the successful landing, or “Tango Delta Nominal,” could not have been achieved without also considering the off-nominal. One of the teams supporting this ambitious mission is the fault protection (FP) team. This team is tasked with assessing the various failures, or faults, that could prevent mission success and with ensuring that the autonomous behaviors built into the software and hardware can detect faults and recover the vehicle to a safe state. As part of its charter, the FP team designed a test campaign to provide confidence in the system’s robustness to off-nominal scenarios across all of Mars 2020’s mission phases. The greatest challenge associated with designing such a validation campaign was reducing the infinite number of anomalous scenarios into a finite test suite. In addition, the tests needed to be executed efficiently in order to utilize the team’s limited test venue access, but still needed to maintain a level of rigor that guaranteed confidence in the test outcomes. Given that each test scenario generated massive amounts of data, the team also developed methods for quickly ascertaining whether the autonomous fault protection behaviors maintained vehicle safety in the presence of an anomaly. This paper summarizes the processes that the Mars 2020 fault protection team employed to execute its off-nominal validation campaign. It captures both the methods of generating a suite of off-nominal tests, as well as reducing it to a subset that can be realistically executed within schedule and resource constraints. It also describes the various processes and philosophies that the team utilized to execute the tests efficiently, including creating a standardized procedure template, keeping the test cases modular so that they could be easily interchanged, and capturing common fault injections in a change-controlled database. Finally, it will describe the tools and processes for assessing the test data, focusing in particular on a tool that evaluated vehicle state using “secondary” sources of data to validate that the software had truly configured the spacecraft to the expected safe state.

Morantz, Chaz↗

NASA NTRS2023

Adaptive Independent Verification and Validation (IV&V) Reduces Risk of Software Impacting Safety in Artemis Missions

The National Aeronautics and Space Administration (NASA) is asking more of its human spaceflight programs than ever before through the collective Artemis Missions. The NASA Independent Verification and Validation (IV&V) Program contributes to NASA’s human spaceflight goals by providing IV&V services for NASA’s critical spacecraft and ground software. The IV&V Program is tasked with providing assurance from both individual and integrated mission software perspectives. The Artemis IV&V organization is actively supporting six distinct development efforts: Orion, the Space Launch System (SLS), Exploration Ground Systems (EGS), Mission Control Center (MCC), the Lunar Gateway, and the Human Landing System (HLS), representing a wide diversity of developer organizations, management structures, and development approaches. With much of this extremely complex flight and ground software being essential to human safety both on the ground and in space, Artemis IV&V is likewise challenged to provide more value-added assurance to future Artemis missions within a constrained budget. To meet this challenge, Artemis IV&V employs a variety of novel and evolving “Adaptive IV&V” approaches for planning and executing IV&V analysis to increase both the efficiency and effectiveness of the IV&V Program’s assurance activities, and to address the difficulties imposed by assuring software for a large, highly integrated, multi-mission enterprise managed and executed by physically and organizationally distinct programs. Instilling agile principles like iterative planning cycles, self-organizing teams, and regular retrospectives, into IV&V planning and execution has led to a more rapid turnaround of a minimum viable assurance product and allowed for increased alignment of assurance activities with development progress. Adopting an assurance case methodology has led to greater consistency and clearer communication of assurance design and provided a foundation for long-term maintenance of assurance plans, products, and results across missions. The IV&V-developed Assurance / Safety Case Analytical Network (A-SCAN) framework and tool has enabled the quantification and tracking of system/software risk and confidence. These confidence measures provide a means to repeatedly express the impact of planned and completed assurance work and the remaining residual risk. Applied as part of a “Follow-the-Risk” organizational ethos, this allows consistent rightsizing of analysis rigor and intensity commensurate with the perceived risk of defects, as well as appropriate targeting of the highest risk areas of the software to find safety issues before they can manifest. Finally, the development of the IV&V Advanced Risk Reduction Integrated Software Test and Operations Tri-program Lightweight Environment (ARRISTOTLE), an integrated software-only simulation of Orion, SLS, and EGS systems, has made it possible to independently test integrated pad and flight scenarios and inject faults to observe how the Artemis multi-program, mission software behaves in degraded modes and in response to hazards. These adaptive IV&V investments have enabled Artemis IV&V to become more efficient and effective in IV&V planning and execution and respond more readily to changes in the risk landscape, increasing the breadth and depth of risk reduction possible within the available resources. Residual risk tracking allows IV&V to communicate more effectively with stakeholders, both internal and external at all levels, and inform key decision-making personnel. This evolving assurance design approach provides IV&V surety that work is performed in the highest risk, most value-added areas of the software, to keep our astronauts and ground crews safe and ensure mission success.

Gerek Whitman↗