Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “verification and validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 487 records · Page 27

Workshop on the Role of Design Assurance in System-Wide Safety’s Safety Demonstrator Series

This report summarizes a three-hour hybrid in-person/online workshop on June 7, 2024 on the topic of design assurance and the Safety Demonstrator Series (SDS), which was held at NASA Ames Research Center. The SDS provides an operational demonstration of, and recommendations for, requirements and standards necessary to monitor, assess, and mitigate risks to assure safety in disaster-oriented operations. Over sixty NASA personnel participated in the workshop. Four main topics were discussed: (1) assurance needs for the Safety Demonstrator Series, (2) assurance and the In-Time Aviation Safety Management System (IASMS), (3) in-time assurance: existing efforts and future opportunities, and (4) demonstrating assurance tools in the Safety Demonstrators. Key takeaways are as follows: 1. Design assurance tools can be used to assure an In-Time Aviation Safety Management System, the systems that comprise it, and other systems or missions. Assurance must consider both systems and components and include the interactions between elements in both a systems/aircraft context and a systems-of-systems/airspace context. 2. Design-time assurance activities can support the identification of monitors needed for operational assurance activities (i.e., the “monitor” function in the monitor-assess-mitigate paradigm at the heart of the IASMS concept). 3. A major opportunity for design-time assurance tools to contribute to the IASMS concept is to support rapid re-validation of systems. This will be particularly important for (1) supporting novel operations in the IASMS, where operational data may disprove design-time assumptions (motivating re-analysis of system safety) and (2) adapting technologies (e.g., AI/ML for autonomous operations) to new operational domains, where there may be new or different safety considerations not included in the initial scope of operations. 4. There are several design assurance tools under development in the System-Wide Safety project that can support assurance of Services, Functions, and Capabilities (SFCs) in the Safety Demonstrators. Transitioning these tools from one-off research projects into a functioning part of IASMS assurance will require closer integration between these tools. 5. It is not clear whether the role of design assurance tools is primarily as a part of IASMS architecture or as an external check on IASMS. The workshop consisted of four discussion topics initiated via four lightning talks by System-Wide Safety researchers. Discussions utilized Mural to engage both in-person and online participants in the hybrid format. Polls and surveys were also utilized to gather participant input. The workshop closed with a reflection activity for participants, as well as new ideas for collaboration and coordination of ongoing System-Wide Safety research.

design assurance↗

The Pursuit of K: Reflections on the Current State-of-the-Art in Stress Intensity Factor Solutions for Practical Aerospace Applications

The elastic stress intensity factor (SIF, commonly denoted as K) is the foundation of practical fracture mechanics (FM) analysis for aircraft structures. This single parameter describes the first-order effects of stress magnitude and distribution as well as the geometry of both structure/component and crack. Hence, the calculation of K is often the most significant step in fatigue analysis based on FM. This presentation will provide several reflections on the current state-of-the-art in SIF solution methods used for practical aerospace applications, including a brief historical perspective, descriptions of some recent and ongoing advances, and comments on some remaining challenges. Newman and Raju made significant early contributions to practical structural analysis by developing closed-form SIF equations for surface and corner cracks in simplified geometries, often based on empirical fits of finite element (FE) solutions. Those solutions (and others like them) were sometimes revised as new analyses were conducted or limitations discovered. The foundational solutions have exhibited striking longevity, despite the relatively "coarse" FE models employed many decades ago. However, in recent years, the accumulation of different generations of solutions for the same nominal geometry has led to some confusion (which solution is correct?), and steady increases in computational capabilities have facilitated the discovery of inaccuracies in some (not all!) of the legacy solutions. Some examples of problems and solutions are presented and discussed, including the challenge of maintaining consistency with legacy design applications. As computational power has increased, the prospect of calculating large numbers of SIF solutions for specific complex geometries with advanced numerical methods has grown more attractive. Fawaz and Andersson, for example, have been generating literally millions of new SIF solutions for different combinations of multiple cracks under simplified loading schemes using p-version FE methods. These data are invaluable, but questions remain about their practical use, because the tabular databases of key results needed to support practical life analysis can occupy gigabytes of storage for only a few classes of geometries. The prospect of using such advanced numerical methods to calculate in real time only those K solutions actually needed to support a specific crack growth analysis is also tempting, but the stark reality is that the computational cost is still so high that the approach is not practical except for specific, critical application problems. Some thoughts are offered about alternative paradigms. Compounding approaches are some of the earliest building blocks of SIF development for more complex geometries. These approaches are especially attractive because of their very low computational cost and their conceptual robustness; they are, in some ways, an intriguing contrast and complement to the brute-force numerical methods. In recent years, researchers at NRC-Canada have published remarkable results showing how compounding approaches can be used to generate accurate solutions for very difficult problems. Examples are provided of some successes--and some limitations--using this approach. These closed-form, tabulated numerical, and compounding approaches have typically been used for simple remote loading with simple load paths to the crack. However, many significant cracks occur in complex stress gradient fields. This is a job for weight function (WF) methods, where the arbitrary stress distribution on the crack plane in the corresponding uncracked body (typically determined using FE methods) is used to determine K. Several significant recent advances in WF methods and solutions are highlighted here. Fueled by advanced 3D numerical methods, many new solutions have been generated for classic geometries such as surface and corner cracks with wide ranges of geometrical validity. A new WF formulation has also be developed for part-through cracks considering the arbitrary stress gradients in all directions in the crack plane (so-called bivariant solutions). Basic WF methods have recently been combined with analytical expressions for crack plane stresses to develop a large family of accurate SIF solutions for corner, surface, and through cracks at internal or external notches with very wide ranges of shapes, sizes, acuities, and offsets. Finally, WF solutions are much faster than FE or boundary element solutions, but can still be much slower than simple closed-form solutions, especially for bivariant solutions that can require 2D numerical integration. Novel pre-integration and dynamic tabular methods have been developed that substantially increase the speed of these advanced WF solutions. The practical utility of advanced SIF methods, including both WF and direct numerical methods, is greatly enhanced if the FM life analysis can be directly and efficiently linked with digital models of the actual structure or component (e.g., FE models for stress analysis). Two recent advances of this type will be described. One approach directly interfaces the FM life analysis with the FE model of the uncracked component (including stress results). Through a powerful graphical user interface, simplified FM life models can be constructed (and visualized) directly on the component model, with the computer collecting the geometry and stress gradient information needed for the life calculation. An even more powerful paradigm uses expert logic to automatically build an optimum simple fracture model at any and every desired location in the component model, perform the life calculation, and even generate fatigue crack growth life contour maps, all with minimal user intervention. This paradigm has also been extended to the automatic calculation of fracture risk, considering uncertainty or variability in key input parameters such as initial crack size or location. Another new integrated approach links the engineering life analysis, the component model, and a 3D numerical fracture analysis built with the same component model to generate a table of SIF values at a specific location that can then be employed efficiently to perform the life calculation. Some attention must be given to verification and validation (V&V) issues and challenges: how good are these SIF solutions, how good is good enough, and does anyone believe the life answer? It is important to think critically about the different sources of error or uncertainty and to perform V&V in a hierarchal, building-block manner. Some accuracy issues for SIF solutions, for example, may actually involve independent material behavior issues, such as constraint loss effects for crack fronts near component surfaces, and can be a source of confusion. Recommendations are proposed for improved V&V approaches. This presentation will briefly but critically survey the range of issues and advances mentioned above, with a particular view towards assembling an integrated approach that combines different methods to create practical tools for real-world design and analysis problems. Examples will be selectively drawn from the recent literature, from recent enhancements in the NASGRO and DARWIN computer codes, and from previously unpublished research

CraigMcClung, R.↗

MARGInS: Model-Based Analysis of Realizable Goals in Systems

Under NASAs Constellation effort, the Exploration Technology Development Program funded research toward a system validation capability that applied machine learning and test-case generation techniques to the analysis of black-box system behavior. The behavior analysis capability scaled to spaces of hundreds of input parameters and tens of thousands of test cases. Aerospace systems at the vehicle level, especially those systems which contain some level of autonomy, are best described by hybrid and non-linear mathematics. Even simplified models of such systems need parameter dimensionalities in the hundreds or thousands of parameters in order to capture sufficient fidelity. The System Safety Assessments (such as those described in the SAE ARP 4761A Safety Assessment Process guidelines) for these systems are prone to errorinteractions between the vehicles subsystems are complex, and can display emergent behaviors. NASA captured this new analysis in the Model-based Analysis of Realizable Goals in Systems (MARGInS) tool and applied it to the Pad Abort 1 (PA-1) simulation as part of the independent validation and verification cycle before the PA-1 flight test in May of 2010. MARGInS evaluated the adherence of the high-fidelity simulation to its requirements, and deter- mined the margins to failure from the expected nominal input conditions. Following the PA-1 test, the capabilities within the MARGInS framework have been extended with sophisticated statistical and white-box test case generation techniques and applied to other NASA missions. The frame- work now includes a critical factors analysis that was applied to NASAs Orion simulation and design. NASAs Aeronautics Research Mission Directorate (ARMD) leveraged the existing MARGInS framework for work on aviation safety for civil transport vehicles and for research on autonomy issues. The NASA ARMD effort created a time series output prediction capability that has been used to characterize trajectories for a plane with an adaptive control system, and a safety boundary detection capability that has been applied to an air traffic control concept of operation for the Federal Aviation Administration. The statistical and machine- learning based techniques within MARGInS have been successfully combined with concolic execution to improve the coverage of a critical unit by driving system-level inputs. The use case driving the concolic execution and MARGInS integration was inspired by the Air France 447 disaster in which the loss of a critical functionality (the airspeed calculation from the pitot tubes) led to loss of the entire plane with the people aboard. To illustrate capabilities and limitations, we will highlight the analyses for the applications listed above. We will then discuss the future plans for MARGInS and its interfaces with other tools.

Validation↗

Full Life-Cycle Defect Management Assessment: Initial Inspection Data Collection Results and Research Questions for Further Study

It is often the case in software projects that when schedule and budget resources are limited, the Verification and Validation (V&V) activities suffer. Fewer V&V activities can be afforded and moreover, short-term challenges can result in V&V activities being scaled back or dropped altogether. As a result, too often the default solution is to save activities for improving software quality until too late in the life-cycle, relying on late-term code inspections followed by thorough testing activities to reduce defect counts to acceptable levels. As many project managers realize, however, this is a resource-intensive way of achieving the required quality for software. The Full Life-cycle Defect Management Assessment Initiative, funded by NASA s Office of Safety and Mission Assurance under the Software Assurance Research Program, aims to address these problems by: Improving the effectiveness of early life-cycle V&V activities to make their benefits more attractive to team leads. Specifically, we focus on software inspection, a proven method that can be applied to any software work product, long before executable code has been developed; Better communicating this effectiveness to software development teams, along with suggestions for parameters to improve in the future to increase effectiveness; Analyzing the impact of early life-cycle V&V on the effectiveness and cost required for late life-cycle V&V activities, such as testing, in order to make the tradeoffs more apparent. This white paper reports on an initial milestone in this work, the development of a preliminary model of inspection effectiveness across multiple NASA Centers. This model contributes toward reaching our project goals by: Allowing an examination of inspection parameters, across different types of projects and different work products, for an analysis of factors that impact defect detection effectiveness. Allowing a comparison of this NASA-specific model to existing recommendations in the literature regarding how to plan effective inspections. Forming a baseline model which can be extended to incorporate factors describing: the numbers and types of defects that are missed by inspections; how such defects flow downstream through software development phases; how effectively they can be caught by testing activities in the late stages of development. The model has been implemented in a prototype web-enabled decision-support tool which allows developers to enter their inspection data and receive feedback based on a comparison against the model. The tool also allows users to access reusable materials (such as checklists) from projects included in the baseline. Both the tool itself and the model underlying it will continue to be extended throughout the remainder of this initiative. As results of analyzing inspection effectiveness for defect containment are determined, they can be shared via the tool and also via updates to existing training courses on metrics and software inspections. Moreover, the tool will help satisfy key CMMI requirements for the NASA Centers, as it will enable NASA to take a global view across peer review results for various types of projects to identify systemic problems. This analysis can result in continuous improvements to the approach to verification.

Shull, Forrest↗

Microbial burden prediction model for unmanned planetary spacecraft

The technical development of a computer program for predicting microbial burden on unmanned planetary spacecraft is outlined. The discussion includes the derivation of the basic analytical equations, the selection of a method for handling several random variables, the macrologic of the computer programs and the validation and verification of the model. The prediction model was developed to (1) supplement the biological assays of a spacecraft by simulating the microbial accretion during periods when assays are not taken; (2) minimize the necessity for a large number of microbiological assays; and (3) predict the microbial loading on a lander immediately prior to sterilization and other non-lander equipment prior to launch. It is shown that these purposes not only were achieved but also that the prediction results compare favorably to the estimates derived from the direct assays. The computer program can be applied not only as a prediction instrument but also as a management and control tool. The basic logic of the model is shown to have possible applicability to other sequential flow processes, such as food processing.

Hoffman, A. R.↗

Automatic documentation system extension to multi-manufacturers' computers and to measure, improve, and predict software reliability

The DOMONIC system has been modified to run on the Univac 1108 and the CDC 6600 as well as the IBM 370 computer system. The DOMONIC monitor system has been implemented to gather data which can be used to optimize the DOMONIC system and to predict the reliability of software developed using DOMONIC. The areas of quality metrics, error characterization, program complexity, program testing, validation and verification are analyzed. A software reliability model for estimating program completion levels and one on which to base system acceptance have been developed. The DAVE system which performs flow analysis and error detection has been converted from the University of Colorado CDC 6400/6600 computer to the IBM 360/370 computer system for use with the DOMONIC system.

Simmons, D. B.↗

Simulation verification techniques study: Simulation performance validation techniques document

Techniques and support software for the efficient performance of simulation validation are discussed. Overall validation software structure, the performance of validation at various levels of simulation integration, guidelines for check case formulation, methods for real time acquisition and formatting of data from an all up operational simulator, and methods and criteria for comparison and evaluation of simulation data are included. Vehicle subsystems modules, module integration, special test requirements, and reference data formats are also described.

Duncan, L. M.↗

Simulation verification techniques study. Subsystem simulation validation techniques

Techniques for validation of software modules which simulate spacecraft onboard systems are discussed. An overview of the simulation software hierarchy for a shuttle mission simulator is provided. A set of guidelines for the identification of subsystem/module performance parameters and critical performance parameters are presented. Various sources of reference data to serve as standards of performance for simulation validation are identified. Environment, crew station, vehicle configuration, and vehicle dynamics simulation software are briefly discussed from the point of view of their interfaces with subsystem simulation modules. A detailed presentation of results in the area of vehicle subsystems simulation modules is included. A list of references, conclusions and recommendations are also given.

Duncan, L. M.↗

Formulation of consumables management models: Test plan for the mission planning processor working model

The test plan and test procedures to be used in the verification and validation of the software being implemented in the mission planning processor working model program are documented. The mission planning processor is a user oriented tool for consumables management and is part of the total consumables subsystem management concept. An overview of the working model is presented. Execution of the test plan will comprehensively exercise the working model software. An overview of the test plan, including a testing schedule, is presented along with the test plan for the unit, module, and system levels. The criteria used to validate the working model results for each consumables subsystem is discussed.

Connelly, L. C.↗

N2-Cs thermionic converter development

The possibility of using molecular nitrogen gas additive as an energy storage which in turn produces cesium ionization in advanced thermionic converter is first described theoretically. Experimental data confirmed for the first time such cesium ionization by vibrationally excited molecular nitrogen. Based on these experimental observations, the use of this scheme for the verification of validity of the recent Lam's theory and its application for the development of an advanced flowing thermionic converter are discussed.

Shaw, D. T.↗

Tools for Embedded Computing Systems Software

A workshop was held to assess the state of tools for embedded systems software and to determine directions for tool development. A synopsis of the talk and the key figures of each workshop presentation, together with chairmen summaries, are presented. The presentations covered four major areas: (1) tools and the software environment (development and testing); (2) tools and software requirements, design, and specification; (3) tools and language processors; and (4) tools and verification and validation (analysis and testing). The utility and contribution of existing tools and research results for the development and testing of embedded computing systems software are described and assessed.

Source record↗

Terminal configured vehicle program: Test facilities guide

The terminal configured vehicle (TCV) program was established to conduct research and to develop and evaluate aircraft and flight management system technology concepts that will benefit conventional take off and landing operations in the terminal area. Emphasis is placed on the development of operating methods for the highly automated environment anticipated in the future. The program involves analyses, simulation, and flight experiments. Flight experiments are conducted using a modified Boeing 737 airplane equipped with highly flexible display and control equipment and an aft flight deck for research purposes. The experimental systems of the Boeing 737 are described including the flight control computer systems, the navigation/guidance system, the control and command panel, and the electronic display system. The ground based facilities used in the program are described including the visual motion simulator, the fixed base simulator, the verification and validation laboratory, and the radio frequency anechoic facility.

Source record↗

Numerical derivation of linear perturbation model from nonlinear real-time vehicle simulations - A means of validation and increased credibility

A numerical technique has been developed which generates linear perturbation models from nonlinear real-time aircraft vehicle simulations at Langley Research Center. The technique is very general and can be applied to simulations of any system described by nonlinear differential equations. Linear models about various trim conditions have been obtained from simulations of the Boeing 737, Northrop F5-E, Cessna 180, Sikorsky CH-54, and NASA/Army Rotor Systems Research Aircraft (RSRA). One use of the derived linear models is as a validation and verification tool for the nonlinear simulation.

Dieudonne, J. E.↗

DSN data systems software methodology

A software methodology for JPL deep space network (DSN) data systems software implementations through transfer and delivery is presented. The DSN Data Systems Software Methodology is compatible with and depends on DSN software methodology and also incorporates the characteristics of real-time program development in a DSN environment. The DSN Data Systems software implementation consists of a series of six distinct phases. An Independent Group is responsible for verification and validation of the DSN Data Systems software during developing phases. The DSN data systems software methodology is applied to all development software provided for or by the DSN data systems section in Mark IV where there is a desire for reliability, maintainability, and usability within budget and schedule constraints.

Hung, C. K.↗

Management of the Galileo attitude and articulation control flight software development

Management concepts are presented for software development for a new technology area, i.e., real-time autonomous, computer-based spacecraft control. Flight computer selection and sizing are done initially to maximize performance within constraints of size, power, and cost. A higher order language is chosen to enhance productivity. Because the computer is embedded in the control systems hardware and is tied to the iterative design process of the spacecraft, the management and configuration control of the software is different from more typical applications. The development process must permit early coding but accept late changes. Margin management must be a continuing process in the development. Validation and verification is a special problem because it is not feasible to test the software in the actual operating environment prior to launch.

Pace, G. D.↗

Simulations used in the development and flight test of the HiMAT vehicle

Real-time simulations have been essential in the flight-test program of the highly maneuverable aircraft technology (HiMAT) remotely piloted research vehicle at the Dryden Flight Research Facility of NASA Ames Research Center. The HiMAT project makes extensive use of simulations in design, development, and qualification for flight, pilot training, and flight planning. Four distinct simulations, each with varying amounts of hardware in the loop, were developed for the HiMAT project. The use of simulations has been the key to flight qualification of the HiMAT vehicle. Specifically, they are useful in detecting anomalous behavior of the flight software and hardware at the various stages of development, verification, and validation.

Evans, M. B.↗

The role of simulation in the development and flight test of the HiMAT vehicle

Real time simulations have been essential in the flight test program of the highly maneuverable aircraft technology (HiMAT) remotely piloted research vehicle at NASA Ames Research Center's Dryden Flight Research Facility. The HiMAT project makes extensive use of simulations in design, development, and qualification for flight, pilot training, and flight planning. Four distinct simulations, each with varying amounts of hardware in the loop, were developed for the HiMAT project. The use of simulations in detecting anomalous behavior of the flight software and hardware at the various stages of development, verification, and validation has been the key to flight qualification of the HiMAT vehicle.

Evans, M. B.↗

Software control and system configuration management: A systems-wide approach

A comprehensive software control and system configuration management process for flight-crucial digital control systems of advanced aircraft has been developed and refined to insure efficient flight system development and safe flight operations. Because of the highly complex interactions among the hardware, software, and system elements of state-of-the-art digital flight control system designs, a systems-wide approach to configuration control and management has been used. Specific procedures are implemented to govern discrepancy reporting and reconciliation, software and hardware change control, systems verification and validation testing, and formal documentation requirements. An active and knowledgeable configuration control board reviews and approves all flight system configuration modifications and revalidation tests. This flexible process has proved effective during the development and flight testing of several research aircraft and remotely piloted research vehicles with digital flight control systems that ranged from relatively simple to highly complex, integrated mechanizations.

Petersen, K. L.↗