SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
To examine the challenges of spaceborne computing systems and the past, present and future approaches to verification and validation in hardware and software systems.
The Orbital Debris Engineering Model (ORDEM) developed by the NASA Orbital Debris Program Office (ODPO) is a data-driven model — extensive radar, optical, laboratory, and in situ measurement data sets have been used to build the model since its earliest versions. A salient aspect of professional software development is the verification and validation (V&V) process. Verification answers the question “Is the model built correctly?” while validation addresses the question “Did we build the correct model?” Less extensive, reserved, or independent data sets serve the validation requirement. Due to the dynamic nature of the orbital debris environment, it is critical to use contemporaneous data sources that represent the current environment to support ORDEM development and validation. ORDEM has utilized in situ data collected from Space Shuttle and Hubble Space Telescope surface inspections, now over a decade old. This historical dataset is fundamental for providing baseline in situ measurement data for sizes between 10 to 300 microns, but new data sources are being evaluated using returned surfaces from or near the International Space Station (ISS). This paper reviews a general microscopic survey of ISS soft goods, the Pressurized Mating Adapter 2 (PMA-2) blanket, and a limited-scope feasibility study conducted on the Space Exploration Technologies Corporation (SpaceX) Dragon capsule’s Thermal Protection System (TPS) material. The PMA-2 blanket, exposed to the space environment between 09 July 2013 and 25 February 2015, is an approximately 3.7 m2-area blanket composed of a betacloth outer layer and multiple ballistic fabric inner layers. The SpaceX Cargo Dragon capsule regularly visited the ISS from 2012 through 2020 and potentially provides a timely and well-characterized source of data for modeling purposes. The capsule’s lateral surfaces use SpaceX Proprietary Ablative Material (SPAM) TPS material, a syntactic foam, for thermal management during all mission phases. Seven SPAM extracted samples have been analyzed to date. This paper will provide an overview of the characterization completed for impact features by size, depth, impactor diameter, and the impactor residues chemical analyses, allowing a differentiation between micrometeoroids and orbital debris and a categorization by mass density and density class. Impactor diameter is estimated using damage equations generated from ground-based hypervelocity impact testing. The orbital debris impactors are compared to the current ORDEM 3.2 model of the environment at ISS altitudes. We briefly discuss the meteoroid impactors, including constituents and mass densities, in the general context of current models.
The Orbital Debris Engineering Model (ORDEM) developed by the NASA Orbital Debris Program Office (ODPO) is a data-driven model — extensive radar, optical, laboratory, and in situ measurement data sets have been used to build the model since its earliest versions. A salient aspect of professional software development is the verification and validation (V&V) process. Verification answers the question “Is the model built correctly?” while validation addresses the question “Did we build the correct model?” Less extensive, reserved, or independent data sets serve the validation requirement. Due to the dynamic nature of the orbital debris environment, it is critical to use contemporaneous data sources that represent the current environment to support ORDEM development and validation. ORDEM has utilized in situ data collected from Space Shuttle and Hubble Space Telescope surface inspections, now over a decade old. This historical dataset is fundamental for providing baseline in situ measurement data for sizes between 10 to 300 microns, but new data sources are being evaluated using returned surfaces from or near the International Space Station (ISS). This paper reviews a general microscopic survey of ISS soft goods, the Pressurized Mating Adapter 2 (PMA-2) blanket, and a limited-scope feasibility study conducted on the Space Exploration Technologies Corporation (SpaceX) Dragon capsule’s Thermal Protection System (TPS) material. The PMA-2 blanket, exposed to the space environment between 09 July 2013 and 25 February 2015, is an approximately 3.7 m2-area blanket composed of a betacloth outer layer and multiple ballistic fabric inner layers. The SpaceX Cargo Dragon capsule regularly visited the ISS from 2012 through 2020 and potentially provides a timely and well-characterized source of data for modeling purposes. The capsule’s lateral surfaces use SpaceX Proprietary Ablative Material (SPAM) TPS material, a syntactic foam, for thermal management during all mission phases. Seven SPAM extracted samples have been analyzed to date. This paper will provide an overview of the characterization completed for impact features by size, depth, impactor diameter, and the impactor residues chemical analyses, allowing a differentiation between micrometeoroids and orbital debris and a categorization by mass density and density class. Impactor diameter is estimated using damage equations generated from ground-based hypervelocity impact testing. The orbital debris impactors are compared to the current ORDEM 3.2 model of the environment at ISS altitudes. We briefly discuss the meteoroid impactors, including constituents and mass densities, in the general context of current models.
The purpose of this ongoing research and development program is to develop software tools which enable the rapid development, upgrading, and maintenance of embedded real-time artificial intelligence systems. The goals of this phase of the research were to investigate the feasibility of developing software tools which automatically translate expert system rules into Ada code and develop methods for performing validation and verification testing of the resultant expert system. A prototype system was demonstrated which automatically translated rules from an Air Force expert system was demonstrated which detected errors in the execution of the resultant system. The method and prototype tools for converting AI representations into Ada code by converting the rules into Ada code modules and then linking them with an Activation Framework based run-time environment to form an executable load module are discussed. This method is based upon the use of Evidence Flow Graphs which are a data flow representation for intelligent systems. The development of prototype test generation and evaluation software which was used to test the resultant code is discussed. This testing was performed automatically using Monte-Carlo techniques based upon a constraint based description of the required performance for the system.
Fundamental to vehicle health management is the deployment of systems incorporating advanced technologies for predicting and detecting anomalous conditions in highly complex and integrated environments. Integrated structural integrity health monitoring, statistical algorithms for detection, estimation, prediction, and fusion, and diagnosis supporting adaptive control are examples of advanced technologies that present considerable verification and validation challenges. These systems necessitate interactions between physical and software-based systems that are highly networked with sensing and actuation subsystems, and incorporate technologies that are, in many respects, different from those employed in civil aviation today. A formidable barrier to deploying these advanced technologies in civil aviation is the lack of enabling verification and validation tools, methods, and technologies. The development of new verification and validation capabilities will not only enable the fielding of advanced vehicle health management systems, but will also provide new assurance capabilities for verification and validation of current generation aviation software which has been implicated in anomalous in-flight behavior. This paper describes the research focused on enabling capabilities for verification and validation underway within NASA s Integrated Vehicle Health Management project, discusses the state of the art of these capabilities, and includes a framework for prioritizing activities.
Modern spacecraft (as well as most other complex mechanisms like aircraft, automobiles, and chemical plants) rely more and more on software, to a point where software failures have caused severe accidents and loss of missions. Software failures during a manned mission can cause loss of life, so there are severe requirements to make the software as safe and reliable as possible. Typically, verification and validation (V&V) has the task of making sure that all software errors are found before the software is deployed and that it always conforms to the requirements. Experience, however, shows that this gold standard of error-free software cannot be reached in practice. Even if the software alone is free of glitches, its interoperation with the hardware (e.g., with sensors or actuators) can cause problems. Unexpected operational conditions or changes in the environment may ultimately cause a software system to fail. Is there a way to surmount this problem? In most modern aircraft and many automobiles, hardware such as central electrical, mechanical, and hydraulic components are monitored by IVHM (Integrated Vehicle Health Management) systems. These systems can recognize, isolate, and identify faults and failures, both those that already occurred as well as imminent ones. With the help of diagnostics and prognostics, appropriate mitigation strategies can be selected (replacement or repair, switch to redundant systems, etc.). In this short paper, we discuss some challenges and promising techniques for software health management (SWHM). In particular, we identify unique challenges for preventing software failure in systems which involve both software and hardware components. We then present our classifications of techniques related to SWHM. These classifications are performed based on dimensions of interest to both developers and users of the techniques, and hopefully provide a map for dealing with software faults and failures.
This paper discusses the verification and validation (V&2) of advanced software used for integrated vehicle health monitoring (IVHM), in the context of NASA's next-generation space shuttle. We survey the current VBCV practice and standards used in selected NASA projects, review applicable formal verification techniques, and discuss their integration info existing development practice and standards. We also describe two verification tools, JMPL2SMV and Livingstone PathFinder, that can be used to thoroughly verify diagnosis applications that use model-based reasoning, such as the Livingstone system.
Research into complexities of software systems Fault Management (FM) and how architectural design decisions affect safety, preservation of assets, and maintenance of desired system functionality has coalesced into a technical reference (TR) suite that advances the provision of safety and mission assurance. The NASA Independent Verification and Validation (IV&V) Program, with Software Assurance Research Program support, extracted FM architectures across the IV&V portfolio to evaluate robustness, assess visibility for validation and test, and define software assurance methods applied to the architectures and designs. This investigation spanned IV&V projects with seven different primary developers, a wide range of sizes and complexities, and encompassed Deep Space Robotic, Human Spaceflight, and Earth Orbiter mission FM architectures. The initiative continues with an expansion of the TR suite to include Launch Vehicles, adding the benefit of investigating differences intrinsic to model-based FM architectures and insight into complexities of FM within an Agile software development environment, in order to improve awareness of how nontraditional processes affect FM architectural design and system health management. The identification of particular FM architectures, visibility, and associated IV&V techniques provides a TR suite that enables greater assurance that critical software systems will adequately protect against faults and respond to adverse conditions. Additionally, the role FM has with regard to strengthened security requirements, with potential to advance overall asset protection of flight software systems, is being addressed with the development of an adverse conditions database encompassing flight software vulnerabilities. Capitalizing on the established framework, this TR suite provides assurance capability for a variety of FM architectures and varied development approaches. Research results are being disseminated across NASA, other agencies, and the software community. This paper discusses the findings and TR suite informing the FM domain in best practices for FM architectural design, visibility observations, and methods employed for IV&V and mission assurance.
Verification and validation of expert systems is very important for the future success of this technology. Software will never be used in non-trivial applications unless the program developers can assure both users and managers that the software is reliable and generally free from error. Therefore, verification and validation of expert systems must be done. The primary hindrance to effective verification and validation is the use of methodologies which do not produce testable requirements. An extension of the flight technique panels used in previous NASA programs should provide both documented requirements and very high levels of verification for expert systems.
The goal of this paper is to emphasize the importance of developing complete and unambiguous requirements early in the project cycle (prior to Preliminary Design Phase). Having a complete set of requirements early in the project cycle allows sufficient time to generate a traceability matrix. Requirements traceability and analysis are the key elements in improving verification and validation process, and thus overall software quality. Traceability can be most beneficial when the system changes. If changes are made to high-level requirements it implies that low-level requirements need to be modified. Traceability ensures that requirements are appropriately and efficiently verified at various levels whereas analysis ensures that a rightly interpreted set of requirements is produced.
This project has focused on the use, assessment, and development of the SAS4A/SASSYS-1 (SAS) safety analysis software. Although SAS was originally intended as a safety analysis tool for Liquid Metal cooled Fast Reactors (LMFRs), which includes both Sodium Fast Reactors (SFRs) and Lead Fast Reactors (LFRs), the vast majority of its recent development and customization has been tailored to SFRs. In general, enhancements that are made to the software for SFRs are applicable to LFRs, however, the fuel composition and corrosive nature of lead requires careful consideration when performing safety analysis of an LFR. In this project an emphasis was placed on closing gaps that are associated with modeling LFRs using SAS. The principal objective of the project was to enhance the ability of SAS as a licensing tool for LFRs. This objective was to be accomplished through three tasks: 1) Enhance the ability of SAS to couple with external software; 2) Improve the underlying physics models in SAS, with priority given to models that are highly relevant to LFRs; 3) Extend the Verification and Validation (V&V) basis of the software
The Jet Propulsion Laboratory (JPL) Advanced Multi-Mission Operations System (AMMOS) in partnership with NASA’s Katherine Johnson Independent Verification and Validation (NASA IV&V) Jon McBride Software Testing and Research (JSTAR) laboratory have jointly developed a command encryption capability that provides a software-only encryption solution utilizing the CCSDS Space Data Link Security Protocol. This solution, named Key Management and Cryptography (KMC), incorporates JPL’s encryption service with the JSTAR open-source CryptoLib library, providing a solution for spacecraft command encryption, which is required per NASA standard NASA-STD-1006-A. This paper will focus on its design, usage, and operation for two SmallSat missions, along with its future expanded functionality of end-to-end spacecraft encryption.
A preliminary verification and validation of a new material response model is presented. This model, Icarus, is intended to serve as a design tool for the thermal protection systems of re-entry vehicles. Currently, the capability of the model is limited to simulating the pyrolysis of a material as a result of the radiative and convective surface heating imposed on the material from the surrounding high enthalpy gas. Since the major focus behind the development of Icarus has been model extensibility, the hope is that additional physics can be quickly added. This extensibility is critical since thermal protection systems are becoming increasing complex, e.g. woven carbon polymers. Additionally, as a three-dimensional, unstructured, finite-volume model, Icarus is capable of modeling complex geometries. In this paper, the mathematical and numerical formulation is presented followed by a discussion of the software architecture and some preliminary verification and validation studies.
A preliminary verification and validation of a new material response model is presented. This model, Icarus, is intended to serve as a design tool for the thermal protection systems of re-entry vehicles. Currently, the capability of the model is limited to simulating the pyrolysis of a material as a result of the radiative and convective surface heating imposed on the material from the surrounding high enthalpy gas. Since the major focus behind the development of Icarus has been model extensibility, the hope is that additional physics can be quickly added. The extensibility is critical since thermal protection systems are becoming increasing complex, e.g. woven carbon polymers. Additionally, as a three-dimensional, unstructured, finite-volume model, Icarus is capable of modeling complex geometries as well as multi-dimensional physics, which have been shown to be important in some scenarios and are not captured by one-dimensional models. In this paper, the mathematical and numerical formulation is presented followed by a discussion of the software architecture and some preliminary verification and validation studies.
In this study, the reproducibility of scientific research is crucial to the success of the scientific method. Here, we review the current best practices when publishing mechanistic models in systems biology. We recommend, where possible, to use software engineering strategies such as testing, verification, validation, documentation, versioning, iterative development, and continuous integration. In addition, adhering to the Findable, Accessible, Interoperable, and Reusable modeling principles allows other scientists to collaborate and build off of each other’s work. Existing standards such as Systems Biology Markup Language, CellML, or Simulation Experiment Description Markup Language can greatly improve the likelihood that a published model is reproducible, especially if such models are deposited in well-established model repositories. Where models are published in executable programming languages, the source code and their data should be published as open-source in public code repositories together with any documentation and testing code. For complex models, we recommend container-based solutions where any software dependencies and the run-time context can be easily replicated.
With the rapidly increasing performance of information technology, i.e., computer hardware and software systems, as well as networks and communication systems, a new capability is being developed that holds the clear promise of greatly increased exploration capability, along with dramatically reduced design, development, and operating costs. These new intelligent systems technologies, utilizing knowledge-based software and very high performance computer systems, will provide new design and development tools, scheduling mechanisms, and vehicle and system health monitoring capabilities. In addition, specific technologies such as neural nets will provide a degree of machine intelligence and associated autonomy which has previously been unavailable to the mission and spacecraft designer and to the system operator. One of the most promising applications of these new information technologies is to the area of in situ resource utilization. Useful resources such as oxygen, compressed carbon dioxide, water, methane, and buffer gases can be extracted and/or generated from planetary atmospheres, such as the Martian atmosphere. These products, when used for propulsion and life-support needs can provide significant savings in the launch mass and costs for both robotic and crewed missions. In the longer term the utilization of indigenous resources is an enabling technology that is vital to sustaining long duration human presence on Mars. This paper will present the concepts that are currently under investigation and development for mining the Martian atmosphere, such as temperature-swing adsorption, zirconia electrolysis etc., to create propellants and life-support materials. This description will be followed by an analysis of the information technology and control needs for the reliable and autonomous operation of such processing plants in a fault tolerant manner, as well as the approach being taken for the development of the controlling software. Finally, there will be a brief discussion of the verification and validation process so crucial to the implementation of mission-critical software.
Over the past few years, the CTH multiphysics hydrocode has overhauled its software quality and testing processes, implementing current best practices in software quality and building a robust V&V test suite comprised of traditional hydrocode verification problems, including ASC Tri-Lab Test Suite and Enhanced Tri-Lab Test Suite problems, as well as validation problems for some of CTH’s most frequently used equations of state, materials models, and other key capabilities. Substantial progress towards building this new test suite was made in FY19 and FY20. In FY21, the test suite has been expanded to include verification and validation tests of the Steinberg-Guinan-Lund (ST) viscoplastic model and the Johnson Cook (JFRAC) fracture model. Additionally, two new verification tests were added, covering hydrodynamics and high explosive (HE) modeling capabilities: the Kidder Gaussian density problem and the Escape of HE Products (EHEP) problem from the Tri-Lab Test Suite. This report discusses each of these test problems in detail. Verification test results are compared to analytic solutions. Validation test results are compared to experimental data. Wherever possible, convergence or mesh refinement studies are included. Additionally, while implementing the Kidder verification problem, a bug was identified that affects the use of tables to initialize pressure or density in 1D or 2D calculations. A brief discussion of the bug and its fix is included. CTH demonstrates good performance overall on the new test suite problems. Simulation results showed good agreement with analytic solutions for the Kidder problem, with convergence rates ranging between 1.8 and sub-linear, and relatively good agreement for the EHEP problem, though convergence rates for pressure and density were nearly 0. The ST and JFRAC strain rate loading verification tests show good agreement with analytic solutions. Likewise, CTH simulation results show good agreement with experimental validation data, including Taylor rod impact testing, for the materials tested. Future V&V work will focus on adding 2D and 3D versions of existing verification tests as well as adding validation tests of other frequently used capabilities such as other fracture models.