Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38

An Overview of Distributed Spacecraft Autonomy at NASA Ames

Autonomous decision-making significantly increases mission effectiveness by mitigating the effects of communication constraints, like latency and bandwidth, and mission complexity on multi-spacecraft operations. To advance the state of the art in autonomous Distributed Space Systems (DSS), the Distributed Spacecraft Autonomy (DSA) team at NASA's Ames Research Center is developing within five relevant technical areas: distributed resource and task management, reactive operations, system modeling and simulation, human-swarm interaction, and ad hoc network communications. DSA is maturing these technologies - critical for future large autonomous DSS - from concept to launch via simulation studies and orbital deployments. A 100-node heterogenous Processor-in-the-Loop (PiL) testbed aids distributed autonomy capability development and verification of multi-spacecraft missions. The DSA software payload deployed to the D-Orbit SCV-004 spacecraft demonstrates multi-agent reconfigurability and reliability as part of an ESA-sponsored in-orbit technology demonstration. Finally, DSA's primary flight mission showcases collaborative resource allocation for multipoint science data collection with four small spacecraft as a payload on NASA's Starling 1.0 satellites.

Caleb Ashmore Adams↗

ARRISTOTLE Simulation Demo

NASA’s Katherine Johnson Independent Verification and Validation (NASA IV&V) Jon McBride Software Testing and Research (JSTAR) laboratory has developed the Advanced Risk Reduction Integrated Software Test and Operations Tri-program Lightweight Environment (ARRISTOTLE). ARRISTOTLE is a digital twin of NASA’s Artemis mission. ARRISTOLE executes the true SLS and Orion flight binaries integrated on a single laptop computer. This demo will showcase the execution of the full Artemis liftoff. Once airborne, an inflight anomaly will be injected, resulting in Orion detaching from the booster. The Launch Abort System (LAS) will safely bring Orion back to earth. The Unity game engine is used to visualize the flyout as flight software executes.

emulation↗

Cylinder Occupancy and Measurement Suite

This software automatically records data from the Unattended Cylinder Verification Station, determines the state of the instrument, captures images of uranium hexafluoride shipment cylinders, and determines their weight. The code has been developed primarily for the International Atomic Energy Agency, but this type of instrument could be of interest to any facility operator in the uranium supply chain that handles uranium hexafluoride shipment cylinders (e.g. the 30B and 48Y cylinder). Operators may be interested in having this instrument confirm shipper declared measurement values upon receipt of a cylinder. They may also use it to aid in process monitoring if the quantity of material in the cylinder changes due to a processing activity.

Stewart, Scott L↗

Formal Specification and Parametric Verification of the ICAROUS Distributed Merging Protocol for Autonomous Aircraft Systems

ICAROUS is a software architecture that provides highly assured core software modules for building safety-centric autonomous unmanned aircraft applications. One of its core components is the ICAROUS distributed merging (IDM) protocol, which allows for decentralized merging of autonomous aircrafts through a designated intersection. This report presents initial results on formal specification and parametric verification of the IDM protocol. We present the development of a formal, discrete-time specification of the ICAROUS distributed merging protocol in TLA+. The developed TLA+ specification includes an abstracted model of the physical aircraft dynamics, the consensus machinery for leader election and coordination, and the computation of merging schedules. In addition, we present details on a command line tool we developed verimerge, that utilizes the TLC model checker for doing bounded, parametric verification and allows for plotting of these results in 2D parameter spaces. The tool also provides functionality for visualization of concrete protocol behaviors, to aid debugging and understanding. We present preliminary, bounded time verification results for a finite number of aircraft. Limitations of the current techniques and possible future extensions of this work are also discussed.

ICAROUS↗

A specification-based approach to concurrent structure verification in multiprocessor systems

A recently initiated research project concerned with the concurrent detection of software errors and errors due to physical failures in the hardware of multiprocessor systems is described in this paper. An approach to error detection is described, which is specification based and relies on the structural verification of program control flow and data structure integrity. The techniques discussed utilize the hardware redundancy inherent in parallel processing systems to provide verification of both program structure and data concurrently with program execution.

Fuchs, W. Kent↗

Verification and Validation of Safety-Critical Aircraft Systems Operating under Off-Nominal, Contingency, and Emergency Conditions

Verification and validation (V&V) of safety-critical technologies developed for loss of control (LOC) prevention and recovery and other aviation safety concerns pose significant challenges. Aircraft LOC can result from a wide spectrum of hazards, often occurring in combination, which cannot be fully replicated during evaluation. Technologies developed for LOC prevention and recovery must therefore be effective under a wide variety of hazardous and uncertain conditions, and the verification and validation of these technologies must provide some measure of assurance that the new vehicle safety technologies do no harm (i.e., that they themselves do not introduce new safety risks). V&V technologies must also enable the identification of system limitations and constraints, as well as enable the identification of safe and unsafe operating conditions (and their boundaries). Additionally, the V&V of complex, increasingly autonomous systems is a fundamental concern. Scalable, reproducible and cost-effective techniques for the assurance of safety critical systems during their design and operation is a key barrier to fielding new systems or updating current systems. Moreover, these techniques need to provide artifacts that enable a comprehensive evidence-based approach to certification. This briefing summarizes research performed under NASA’s Aviation Safety Program and follow-on research for the V&V of safety-critical aircraft system technologies developed for LOC prevention and recovery and increasingly autonomous systems, and for a broad assurance capability in both current and emerging aviation applications. Note that, in this briefing, the term “validation” refers to a confirmation that the system implementation (e.g., algorithms etc.) is performing the intended function(s), as well as an affirmation of effectiveness in these functions. “Verification” refers to a confirmation that the system implementation in the software and hardware meets its (hopefully validated) specifications (e.g., correctly executes algorithms as designed).

Validation↗

Use of Semi-Autonomous Tools for ISS Commanding and Monitoring

As the International Space Station (ISS) has moved into a utilization phase, operations have shifted to become more ground-based with fewer mission control personnel monitoring and commanding multiple ISS systems. This shift to fewer people monitoring more systems has prompted use of semi-autonomous console tools in the ISS Mission Control Center (MCC) to help flight controllers command and monitor the ISS. These console tools perform routine operational procedures while keeping the human operator "in the loop" to monitor and intervene when off-nominal events arise. Two such tools, the Pre-positioned Load (PPL) Loader and Automatic Operators Recorder Manager (AutoORM), are used by the ISS Communications RF Onboard Networks Utilization Specialist (CRONUS) flight control position. CRONUS is responsible for simultaneously commanding and monitoring the ISS Command & Data Handling (C&DH) and Communications and Tracking (C&T) systems. PPL Loader is used to uplink small pieces of frequently changed software data tables, called PPLs, to ISS computers to support different ISS operations. In order to uplink a PPL, a data load command must be built that contains multiple user-input fields. Next, a multiple step commanding and verification procedure must be performed to enable an onboard computer for software uplink, uplink the PPL, verify the PPL has incorporated correctly, and disable the computer for software uplink. PPL Loader provides different levels of automation in both building and uplinking these commands. In its manual mode, PPL Loader automatically builds the PPL data load commands but allows the flight controller to verify and save the commands for future uplink. In its auto mode, PPL Loader automatically builds the PPL data load commands for flight controller verification, but automatically performs the PPL uplink procedure by sending commands and performing verification checks while notifying CRONUS of procedure step completion. If an off-nominal condition occurs during procedure execution, PPL Loader notifies CRONUS through popup messages, allowing CRONUS to examine the situation and choose an option of how PPL loader should proceed with the procedure. The use of PPL Loader to perform frequent, routine PPL uplinks offloads CRONUS to better monitor two ISS systems. It also reduces procedure performance time and decreases risk of command errors. AutoORM identifies ISS communication outage periods and builds commands to lock, playback, and unlock ISS Operations Recorder files. Operation Recorder files are circular buffer files of continually recorded ISS telemetry data. Sections of these files can be locked from further writing, be played back to capture telemetry data that occurred during an ISS loss of signal (LOS) period, and then be unlocked for future recording use. Downlinked Operation Recorder files are used by mission support teams for data analysis, especially if failures occur during LOS. The commands to lock, playback, and unlock Operations Recorder files are encompassed in three different operational procedures and contain multiple user-input fields. AutoORM provides different levels of automation for building and uplinking the commands to lock, playback, and unlock Operations Recorder files. In its automatic mode, AutoORM automatically detects ISS LOS periods, then generates and uplinks the commands to lock, playback, and unlock Operations Recorder files when MCC regains signal with ISS. AutoORM also features semi-autonomous and manual modes which integrate CRONUS more into the command verification and uplink process. AutoORMs ability to automatically detect ISS LOS periods and build the necessary commands to preserve, playback, and release recorded telemetry data greatly offloads CRONUS to perform more high-level cognitive tasks, such as mission planning and anomaly troubleshooting. Additionally, since Operations Recorder commands contain numerical time input fields which are tedious for a human to manually build, AutoORM's ability to automatically build commands reduces operational command errors. PPL Loader and AutoORM demonstrate principles of semi-autonomous operational tools that will benefit future space mission operations. Both tools employ different levels of automation to perform simple and routine procedures, thereby offloading human operators to perform higher-level cognitive tasks. Because both tools provide procedure execution status and highlight off-nominal indications, the flight controller is able to intervene during procedure execution if needed. Semi-autonomous tools and systems that can perform routine procedures, yet keep human operators informed of execution, will be essential in future long-duration missions where the onboard crew will be solely responsible for spacecraft monitoring and control.

Brzezinski, Amy S.↗

Comparison of a Simple Patched Conic Trajectory Code to Commercially Available Software

Often in spaceflight proposal development, mission designers must eva luate numerous trajectories as different design factors are investiga ted. Although there are numerous commercial software packages availab le to help develop and analyze trajectories, most take a significant amount of time to develop the trajectory itself, which isn't effectiv e when working on proposals. Thus a new code, PatCon, which is both q uick and easy to use, was developed to aid mission designers to condu ct trade studies on launch and arrival times for any given target pla net. The code is able to run quick analyses, due to the incorporation of the patched conic approximation, to determine the trajectory. PatCon provides a simple but accurate approximation of the four body moti on problem that would be needed to solve any planetary trajectory. P atCon has been compared to a patched conic test case for verification, with limited validation or comparison with other COTS software. This paper describes the patched conic technique and its implementation i n PatCon. A description of the results and comparison of PatCon to ot her more evolved codes such as AGI#s Satellite Tool Kit and JAQAR As trodynamics# Swingby Calculator is provided. The results will include percent differences in values such as C3 numbers, and Vinfinity at a rrival, and other more subjective results such as the time it takes to build the simulation, and actual calculation time.

AndersonPark, Brooke M.↗

TU-C-BRE-11: 3D EPID-Based in Vivo Dosimetry: A Major Step Forward Towards Optimal Quality and Safety in Radiation Oncology Practice

Purpose: To develop a 3D in vivo dosimetry method that is able to substitute pre-treatment verification in an efficient way, and to terminate treatment delivery if the online measured 3D dose distribution deviates too much from the predicted dose distribution. Methods: A back-projection algorithm has been further developed and implemented to enable automatic 3D in vivo dose verification of IMRT/VMAT treatments using a-Si EPIDs. New software tools were clinically introduced to allow automated image acquisition, to periodically inspect the record-and-verify database, and to automatically run the EPID dosimetry software. The comparison of the EPID-reconstructed and planned dose distribution is done offline to raise automatically alerts and to schedule actions when deviations are detected. Furthermore, a software package for online dose reconstruction was also developed. The RMS of the difference between the cumulative planned and reconstructed 3D dose distributions was used for triggering a halt of a linac. Results: The implementation of fully automated 3D EPID-based in vivo dosimetry was able to replace pre-treatment verification for more than 90% of the patient treatments. The process has been fully automated and integrated in our clinical workflow where over 3,500 IMRT/VMAT treatments are verified each year. By optimizing the dose reconstruction algorithm and the I/O performance, the delivered 3D dose distribution is verified in less than 200 ms per portal image, which includes the comparison between the reconstructed and planned dose distribution. In this way it was possible to generate a trigger that can stop the irradiation at less than 20 cGy after introducing large delivery errors. Conclusion: The automatic offline solution facilitated the large scale clinical implementation of 3D EPID-based in vivo dose verification of IMRT/VMAT treatments; the online approach has been successfully tested for various severe delivery errors.

60 APPLIED LIFE SCIENCES↗

Simulation verification techniques study. Task report 4: Simulation module performance parameters and performance standards

Shuttle simulation software modules in the environment, crew station, vehicle configuration and vehicle dynamics categories are discussed. For each software module covered, a description of the module functions and operational modes, its interfaces with other modules, its stored data, inputs, performance parameters and critical performance parameters is given. Reference data sources which provide standards of performance are identified for each module. Performance verification methods are also discussed briefly.

Source record↗

FAME, a microprocessor based front-end analysis and modeling environment

Higher order software (HOS) is a methodology for the specification and verification of large scale, complex, real time systems. The HOS methodology was implemented as FAME (front end analysis and modeling environment), a microprocessor based system for interactively developing, analyzing, and displaying system models in a low cost user-friendly environment. The nature of the model is such that when completed it can be the basis for projection to a variety of forms such as structured design diagrams, Petri-nets, data flow diagrams, and PSL/PSA source code. The user's interface with the analyzer is easily recognized by any current user of a structured modeling approach; therefore extensive training is unnecessary. Furthermore, when all the system capabilities are used one can check on proper usage of data types, functions, and control structures thereby adding a new dimension to the design process that will lead to better and more easily verified software designs.

Rosenbaum, J. D.↗

Full Life-Cycle Defect Management Assessment: Initial Inspection Data Collection Results and Research Questions for Further Study

It is often the case in software projects that when schedule and budget resources are limited, the Verification and Validation (V&V) activities suffer. Fewer V&V activities can be afforded and moreover, short-term challenges can result in V&V activities being scaled back or dropped altogether. As a result, too often the default solution is to save activities for improving software quality until too late in the life-cycle, relying on late-term code inspections followed by thorough testing activities to reduce defect counts to acceptable levels. As many project managers realize, however, this is a resource-intensive way of achieving the required quality for software. The Full Life-cycle Defect Management Assessment Initiative, funded by NASA s Office of Safety and Mission Assurance under the Software Assurance Research Program, aims to address these problems by: Improving the effectiveness of early life-cycle V&V activities to make their benefits more attractive to team leads. Specifically, we focus on software inspection, a proven method that can be applied to any software work product, long before executable code has been developed; Better communicating this effectiveness to software development teams, along with suggestions for parameters to improve in the future to increase effectiveness; Analyzing the impact of early life-cycle V&V on the effectiveness and cost required for late life-cycle V&V activities, such as testing, in order to make the tradeoffs more apparent. This white paper reports on an initial milestone in this work, the development of a preliminary model of inspection effectiveness across multiple NASA Centers. This model contributes toward reaching our project goals by: Allowing an examination of inspection parameters, across different types of projects and different work products, for an analysis of factors that impact defect detection effectiveness. Allowing a comparison of this NASA-specific model to existing recommendations in the literature regarding how to plan effective inspections. Forming a baseline model which can be extended to incorporate factors describing: the numbers and types of defects that are missed by inspections; how such defects flow downstream through software development phases; how effectively they can be caught by testing activities in the late stages of development. The model has been implemented in a prototype web-enabled decision-support tool which allows developers to enter their inspection data and receive feedback based on a comparison against the model. The tool also allows users to access reusable materials (such as checklists) from projects included in the baseline. Both the tool itself and the model underlying it will continue to be extended throughout the remainder of this initiative. As results of analyzing inspection effectiveness for defect containment are determined, they can be shared via the tool and also via updates to existing training courses on metrics and software inspections. Moreover, the tool will help satisfy key CMMI requirements for the NASA Centers, as it will enable NASA to take a global view across peer review results for various types of projects to identify systemic problems. This analysis can result in continuous improvements to the approach to verification.

Shull, Forrest↗

Optimum-AIV: A planning and scheduling system for spacecraft AIV

A project undertaken for the European Space Agency (ESA) is presented. The project is developing a knowledge based software system for planning and scheduling of activities for spacecraft assembly, integration, and verification (AIV). The system extends into the monitoring of plan execution and the plan repair phase. The objectives are to develop an operational kernel of a planning, scheduling, and plan repair tool, called OPTIMUM-AIV, and to provide facilities which will allow individual projects to customize the kernel to suit its specific needs. The kernel shall consist of a set of software functionalities for assistance in initial specification of the AIV plan, in verification and generation of valid plans and schedules for the AIV activities, and in interactive monitoring and execution problem recovery for the detailed AIV plans. Embedded in OPTIMUM-AIV are external interfaces which allow integration with alternative scheduling systems and project databases. The current status of the OPTIMUM-AIV project, as of Jan. 1991, is that a further analysis of the AIV domain has taken place through interviews with satellite AIV experts, a software requirement document (SRD) for the full operational tool was approved, and an architectural design document (ADD) for the kernel excluding external interfaces is ready for review.

Arentoft, M. M.↗

SAM Co-Development to Support Fluoride-salt-cooled High-temperature Reactor Design and Licensing (CRADA 2018-18177 Final Report)

This project aims to raise the SAM code's technical and commercial maturity level to enable the Kairos Power ("Kairos") to use SAM to support its KP-FHR design analysis and licensing application. Argonne National Laboratory ("Argonne") has actively developed and maintained SAM, a modern system-level analysis tool for advanced nonlight water reactors safety analysis. Kairos is currently actively pursuing development of an FHR design and associated technology, and consequently requires a pedigreed safety analysis tool. The purpose of this project is to increase the maturity of the SAM code for the modeling and simulation of the KP-FHR design, thus enabling its use in safety analyses that support licensing application. Argonne will primarily be responsible for software design, development, and testing; while Kairos will be responsible for software requirements, assessment of software capabilities, needs, gaps, and priorities, and development of proprietary models. The research and development activities for the joint SAM development project include: a series of identification and prioritization studies on design characteristics, event sequences, relevant phenomena, and software capabilities; SAM capability enhancements for specific KP-FHR systems and components; performing code verification and validations; integrating uncertainty quantification (UQ), model calibration, and sensitivity analysis (SA) techniques in safety analyses; and raising the software quality rigor level for commercial-grade applications.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Software assurance standard

This standard specifies the software assurance program for the provider of software. It also delineates the assurance activities for the provider and the assurance data that are to be furnished by the provider to the acquirer. In any software development effort, the provider is the entity or individual that actually designs, develops, and implements the software product, while the acquirer is the entity or individual who specifies the requirements and accepts the resulting products. This standard specifies at a high level an overall software assurance program for software developed for and by NASA. Assurance includes the disciplines of quality assurance, quality engineering, verification and validation, nonconformance reporting and corrective action, safety assurance, and security assurance. The application of these disciplines during a software development life cycle is called software assurance. Subsequent lower-level standards will specify the specific processes within these disciplines.

Source record↗

Verification of a Multiphysics Code with Method of Manufactured Solutions

To push the modeling and simulation to a new level in simulating realistic problems, a common practice is to couple different single-physics codes together to simulate problems involving multiple physics. Although the method of manufactured solutions (MMS) has been shown to be a powerful tool in code verification, there has been little work on applying MMS to a multi-physics code system. Additionally, due to the scarcity of multi-physics validation data, the significance of code verification becomes more prominent as an important part in software quality assurance. With this, the application of MMS to a coupled code system, e.g. VERA-CS, has become a crucial step, which helps identify bugs affecting the order of accuracy, thus increasing the credibility and reliability of the code. In this work, we investigate the application of MMS to verify a multi-physics code system and establish a framework for this application, and we demonstrate this with a neutronics code coupled with a thermal hydraulics (TH) code. The convection module is not yet added to this work, so by TH, only conduction is considered.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Performance Evaluation of a Data Validation System

Online data validation is a performance-enhancing component of modern control and health management systems. It is essential that performance of the data validation system be verified prior to its use in a control and health management system. A new Data Qualification and Validation (DQV) Test-bed application was developed to provide a systematic test environment for this performance verification. The DQV Test-bed was used to evaluate a model-based data validation package known as the Data Quality Validation Studio (DQVS). DQVS was employed as the primary data validation component of a rocket engine health management (EHM) system developed under NASA's NGLT (Next Generation Launch Technology) program. In this paper, the DQVS and DQV Test-bed software applications are described, and the DQV Test-bed verification procedure for this EHM system application is presented. Test-bed results are summarized and implications for EHM system performance improvements are discussed.

Wong, Edmond↗

PRIME: a real-time cyber-physical systems testbed: from wide-area monitoring, protection, and control prototyping to operator training and beyond

As the power grid continues to evolve with advanced wide-area monitoring, protection, and control (WAMPAC) algorithms, there is an increasing need for realistic testbed environments with industry-grade software and hardware-in-the-loop (HIL) to perform verification and validation studies. Such testbed environments serve as ideal platforms to perform WAMPAC prototyping, operator training, and also to study the impacts of different types of cyberattack scenarios on the operation of the grid. In this paper, we introduce PRIME (PNNL cybeR physIcal systeMs tEstbed): the testbed that integrates real-time transmission system simulator with commercial industry grade energy management system (EMS) software and remote hardware-in-the-loop (RHIL). PRIME is an end-to-end, modular testbed that allows high-fidelity RHIL experimentation of a power system. We present two detailed case studies (fault location and clearing in transmission system, and operator training) to showcase the capabilities of our PRIME testbed. Finally, we briefly discuss some of the potential limitations of our testbed in terms of scalability and flexibility to setup larger test systems and identify directions for future work to address those limitations.

42 ENGINEERING↗