Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “verification and validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Towards verifiable cancer digital twins: tissue level modeling protocol for precision medicine

Cancer exhibits substantial heterogeneity, manifesting as distinct morphological and molecular variations across tumors, which frequently undermines the efficacy of conventional oncological treatments. Developments in multiomics and sequencing technologies have paved the way for unraveling this heterogeneity. Nevertheless, the complexity of the data gathered from these methods cannot be fully interpreted through multimodal data analysis alone. Mathematical modeling plays a crucial role in delineating the underlying mechanisms to explain sources of heterogeneity using patient-specific data. Intra-tumoral diversity necessitates the development of precision oncology therapies utilizing multiphysics, multiscale mathematical models for cancer. This review discusses recent advancements in computational methodologies for precision oncology, highlighting the potential of cancer digital twins to enhance patient-specific decision-making in clinical settings. We review computational efforts in building patient-informed cellular and tissue-level models for cancer and propose a computational framework that utilizes agent-based modeling as an effective conduit to integrate cancer systems models that encode signaling at the cellular scale with digital twin models that predict tissue-level response in a tumor microenvironment customized to patient information. Furthermore, we discuss machine learning approaches to building surrogates for these complex mathematical models. These surrogates can potentially be used to conduct sensitivity analysis, verification, validation, and uncertainty quantification, which is especially important for tumor studies due to their dynamic nature.

60 APPLIED LIFE SCIENCES↗
NASA NTRS2004

Systems engingeering for the Kepler Mission : a search for terrestrial planets

The Kepler mission will launch in 2007 and determine the distribution of earth-size planets (0.5 to 10 earth masses) in the habitable zones (HZs) of solar-like stars. The mission will monitor > 100,000 dwarf stars simultaneously for at least 4 years. Precision differential photometry will be used to detect the periodic signals of transiting planets. Kepler will also support asteroseismology by measuring the pressure-mode (p-mode) oscillations of selected stars. Key mission elements include a spacecraft bus and 0.95 meter, wide-field, CCD-based photometer injected into an earth-trailing heliocentric orbit by a 3-stage Delta II launch vehicle as well as a distributed Ground Segment and Follow-up Observing Program. The project is currently preparing for Preliminary Design Review (October 2004) and is proceeding with detailed design and procurement of long-lead components. In order to meet the unprecedented photometric precision requirement and to ensure a statistically significant result, the Kepler mission involves technical challenges in the areas of photometric noise and systematic error reduction, stability, and false-positive rejection. Programmatic and logistical challenges include the collaborative design, modeling, integration, test, and operation of a geographically and functionally distributed project. A very rigorous systems engineering program has evolved to address these challenges. This paper provides an overview of the Kepler systems engineering program, including some examples of our processes and techniques in areas such as requirements synthesis, validation & verification, system robustness design, and end-to-end performance modeling.

systems engineering↗
NASA NTRS2014

Evolution of Software-Only-Simulation at NASA IV and V

Software-Only-Simulations have been an emerging but quickly developing field of study throughout NASA. The NASA Independent Verification Validation (IVV) Independent Test Capability (ITC) team has been rapidly building a collection of simulators for a wide range of NASA missions. ITC specializes in full end-to-end simulations that enable developers, VV personnel, and operators to test-as-you-fly. In four years, the team has delivered a wide variety of spacecraft simulations that have ranged from low complexity science missions such as the Global Precipitation Management (GPM) satellite and the Deep Space Climate Observatory (DSCOVR), to the extremely complex missions such as the James Webb Space Telescope (JWST) and Space Launch System (SLS).This paper describes the evolution of ITCs technologies and processes that have been utilized to design, implement, and deploy end-to-end simulation environments for various NASA missions. A comparison of mission simulators are discussed with focus on technology and lessons learned in complexity, hardware modeling, and continuous integration. The paper also describes the methods for executing the missions unmodified flight software binaries (not cross-compiled) for verification and validation activities.

Embedded↗
NASA NTRS2008

Development of NASA's Models and Simulations Standard

From the Space Shuttle Columbia Accident Investigation, there were several NASA-wide actions that were initiated. One of these actions was to develop a standard for development, documentation, and operation of Models and Simulations. Over the course of two-and-a-half years, a team of NASA engineers, representing nine of the ten NASA Centers developed a Models and Simulation Standard to address this action. The standard consists of two parts. The first is the traditional requirements section addressing programmatics, development, documentation, verification, validation, and the reporting of results from both the M&S analysis and the examination of compliance with this standard. The second part is a scale for evaluating the credibility of model and simulation results using levels of merit associated with 8 key factors. This paper provides an historical account of the challenges faced by and the processes used in this committee-based development effort. This account provides insights into how other agencies might approach similar developments. Furthermore, we discuss some specific applications of models and simulations used to assess the impact of this standard on future model and simulation activities.

simulation↗
NASA NTRS2014

HRP's Healthcare Spin-Offs Through Computational Modeling and Simulation Practice Methodologies

Spaceflight missions expose astronauts to novel operational and environmental conditions that pose health risks that are currently not well understood, and perhaps unanticipated. Furthermore, given the limited number of humans that have flown in long duration missions and beyond low Earth-orbit, the amount of research and clinical data necessary to predict and mitigate these health and performance risks are limited. Consequently, NASA's Human Research Program (HRP) conducts research and develops advanced methods and tools to predict, assess, and mitigate potential hazards to the health of astronauts. In this light, NASA has explored the possibility of leveraging computational modeling since the 1970s as a means to elucidate the physiologic risks of spaceflight and develop countermeasures. Since that time, substantial progress has been realized in this arena through a number of HRP funded activates such as the Digital Astronaut Project (DAP) and the Integrated Medical Model (IMM). Much of this success can be attributed to HRP's endeavor to establish rigorous verification, validation, and credibility (VV&C) processes that ensure computational models and simulations (M&S) are sufficiently credible to address issues within their intended scope. This presentation summarizes HRP's activities in credibility of modeling and simulation, in particular through its outreach to the community of modeling and simulation practitioners. METHODS: The HRP requires all M&S that can have moderate to high impact on crew health or mission success must be vetted in accordance to NASA Standard for Models and Simulations, NASA-STD-7009 (7009) [5]. As this standard mostly focuses on engineering systems, the IMM and DAP have invested substantial efforts to adapt the processes established in this standard for their application to biological M&S, which is more prevalent in human health and performance (HHP) and space biomedical research and operations [6,7]. These methods have also generated substantial interest by the broader medical community though institutions like the National Institutes of Health (NIH) and the Food and Drug Administration (FDA) to develop similar standards and guidelines applicable to the larger medical operations and research community. DISCUSSION: Similar to NASA, many leading government agencies, health institutions and medical product developers around the world are recognizing the potential of computational M&S to support clinical research and decision making. In this light, substantial investments are being made in computational medicine and notable discoveries are being realized [8]. However, there is a lack of broadly applicable practice guidance for the development and implementation of M&S in clinical care and research in a manner that instills confidence among medical practitioners and biological researchers [9,10]. In this presentation, we will give an overview on how HRP is working with the NIH's Interagency Modeling and Analysis Group (IMAG), the FDA and the American Society of Mechanical Engineers (ASME) to leverage NASA's biomedical VV&C processes to establish a new regulatory standard for Verification and Validation in Computational Modeling of Medical Devices, and Guidelines for Credible Practice of Computational Modeling and Simulation in Healthcare.

Mulugeta, Lealem↗
NASA NTRS2016

Assurance of Fault Management: Risk-Significant Adverse Condition Awareness

Fault Management (FM) systems are ranked high in risk-based assessment of criticality within flight software, emphasizing the importance of establishing highly competent domain expertise to provide assurance for NASA projects, especially as spaceflight systems continue to increase in complexity. Insight into specific characteristics of FM architectures seen embedded within safety- and mission-critical software systems analyzed by the NASA Independent Verification Validation (IVV) Program has been enhanced with an FM Technical Reference (TR) suite. Benefits are aimed beyond the IVV community to those that seek ways to efficiently and effectively provide software assurance to reduce the FM risk posture of NASA and other space missions. The identification of particular FM architectures, visibility, and associated IVV techniques provides a TR suite that enables greater assurance that critical software systems will adequately protect against faults and respond to adverse conditions. The role FM has with regard to overall asset protection of flight software systems is being addressed with the development of an adverse condition (AC) database encompassing flight software vulnerabilities.Identification of potential off-nominal conditions and analysis to determine how a system responds to these conditions are important aspects of hazard analysis and fault management. Understanding what ACs the mission may face, and ensuring they are prevented or addressed is the responsibility of the assurance team, which necessarily should have insight into ACs beyond those defined by the project itself. Research efforts sponsored by NASAs Office of Safety and Mission Assurance defined terminology, categorized data fields, and designed a baseline repository that centralizes and compiles a comprehensive listing of ACs and correlated data relevant across many NASA missions. This prototype tool helps projects improve analysis by tracking ACs, and allowing queries based on project, mission type, domain component, causal fault, and other key characteristics. The repository has a firm structure, initial collection of data, and an interface established for informational queries, with plans for integration within the Enterprise Architecture at NASA IVV, enabling support and accessibility across the Agency. The development of an improved workflow process for adaptive, risk-informed FM assurance is currently underway.

Software Verification & Validation↗
NASA NTRS2001

Commercial Systems Validation Overview

The Commercial Systems Validation Laboratory (CSVL) is composed of the 1) Virtual Product Laboratory (VPL), 2) Commercial Instrument Validation Laboratory (CIVL), and 3) In-Flight Verification & Validation Network (IVVN). All three laboratories are described in this viewgraph presentation, which covers the characterization of CCD cameras, hyperspectral sensors, ground reference equipment, and the Multispectral Thermal Imager (MTI). The presentation also includes sections on thermal modeling, measurements taken from a float, and the Scientific Data Purchase (SDP) Program. The section of the SDP profiles four data providers: 1) EarthSat; 2) Intermap; 3) Positive Systems; 4) Space Imaging.

Ryan, Robert↗
NASA NTRS2006

Microthruster Propulsion for the Space Technology 7 (ST7) Technology Demonstration Mission

For future applications to precision formation flying missions, NASA's New Millennium Program is scheduled to test colloid micro-Newton thrusters (CMNTs) on the ST7 technology demonstration mission. These CMNTs are part of a disturbance reduction system (DRS) on the ESA SMART-2 Spacecraft or LISA Pathfinder. The goal of the ST7 DRS is to demonstrate technologies necessary to meet the nanometer precision positioning control requirements of the LISA mission. In order to achieve these goals, the CMNTs are required to demonstrate a thrust resolution of less than 0.1 micro-N and a thrust noise of less than 0.1 micro-N/[square root]Hz for thrust levels between 5 and 30 micro-N. Developed by Busek Co. with support from JPL in testing an design, the CMNT has been developed over the last four years into a flight-ready microthrust system. The development, validation testing, and flight unit production of the CMNTs are described. Development tests and analysis include preliminary wear tests, propellant loading process verification, flow testing, and performance verification. Validation and flight unit verification includes thermal and structural analysis, life testing, thermal and dynamic load testing, and performance verification. Final delivery of the units is planned in 2007 with and planned launch and flight demonstration 2009.

microthruster↗
NASA NTRS2005

Thermal System Verification and Model Validation for NASA's Cryogenic Passively Cooled James Webb Space Telescope

A thorough and unique thermal verification and model validation plan has been developed for NASA s James Webb Space Telescope. The JWST observatory consists of a large deployed aperture optical telescope passively cooled to below 50 Kelvin along with a suite of several instruments passively and actively cooled to below 37 Kelvin and 7 Kelvin, respectively. Passive cooling to these extremely low temperatures is made feasible by the use of a large deployed high efficiency sunshield and an orbit location at the L2 Lagrange point. Another enabling feature is the scale or size of the observatory that allows for large radiator sizes that are compatible with the expected power dissipation of the instruments and large format Mercury Cadmium Telluride (HgCdTe) detector arrays. This passive cooling concept is simple, reliable, and mission enabling when compared to the alternatives of mechanical coolers and stored cryogens. However, these same large scale observatory features, which make passive cooling viable, also prevent the typical flight configuration fully-deployed thermal balance test that is the keystone to most space missions thermal verification plan. JWST is simply too large in its deployed configuration to be properly thermal balance tested in the facilities that currently exist. This reality, when combined with a mission thermal concept with little to no flight heritage, has necessitated the need for a unique and alternative approach to thermal system verification and model validation. This paper describes the thermal verification and model validation plan that has been developed for JWST. The plan relies on judicious use of cryogenic and thermal design margin, a completely independent thermal modeling cross check utilizing different analysis teams and software packages, and finally, a comprehensive set of thermal tests that occur at different levels of JWST assembly. After a brief description of the JWST mission and thermal architecture, a detailed description of the three aspects of the thermal verification and model validation plan is presented.

Cleveland, Paul E.↗
NASA NTRS2005

Thermal System Verification and Model Validation for NASA's Cryogenic Passively Cooled James Webb Space Telescope (JWST)

A thorough and unique thermal verification and model validation plan has been developed for NASA s James Webb Space Telescope. The JWST observatory consists of a large deployed aperture optical telescope passively cooled to below 50 Kelvin along with a suite of several instruments passively and actively cooled to below 37 Kelvin and 7 Kelvin, respectively. Passive cooling to these extremely low temperatures is made feasible by the use of a large deployed high efficiency sunshield and an orbit location a! !he L2 Lagrange p~in!. Another enabling feature is the scale or size of the observatory that allows for large radiator sizes that are compatible with the expected power dissipation of the instruments and large format Mercury Cadmium Telluride (HgCdTe) detector arrays. This passive cooling concept is simple, reliable, and mission enabling when compared to the alternatives of mechanical coolers and stored cryogens. However, these same large scale observatory features, which make passive cooling viable, also prevent the typical flight configuration fully-deployed thermal balance test that is the keystone to most space missions thermal verification plan. JWST is simply too large in its deployed configuration to be properly thermal balance tested in the facilities that currently exist. This reality, when combined with a mission thermal concept with little to no flight heritage, has necessitated the need for a unique and alternative approach to thermal system verification and model validation. This paper describes the thermal verification and model validation plan that has been developed for JWST.

Parrish, Keith↗
NASA NTRS2006

Emerging technologies for V&V of ISHM software for space exploration

Systems1,2 required to exhibit high operational reliability often rely on some form of fault protection to recognize and respond to faults, preventing faults' escalation to catastrophic failures. Integrated System Health Management (ISHM) extends the functionality of fault protection to both scale to more complex systems (and systems of systems), and to maintain capability rather than just avert catastrophe. Forms of ISHM have been utilized to good effect in the maintenance phase of systems' total lifecycles (often referred to as 'condition-based mainte-nance'), but less so in a 'fault protection' role during actual operations. One of the impediments to such use lies in the challenges of verification, validation and certification of ISHM systems themselves. This paper makes the case that state-of-the-practice V&V and certification techniques will not suffice for emerging forms of ISHM systems; however, a number of maturing software engineering assurance technologies show particular promise for addressing these ISHM V&V challenges.

fault detection, isolation, and recovery↗
DOE OSTI2025

Robust Solution Verification Experiments on Nonuniform Meshes

The activities of verification, validation, and uncertainty quantification (VVUQ) provide a comprehensive means to assess the credibility of computational models. Within VVUQ, solution verification assesses numerical errors and evaluates whether the simulation is sufficiently accurate for its intended applications. As computational modeling gains traction in the development of complex, high-consequence systems, the need for robust solution verification intensifies, particularly because experimental data for these systems are often limited. This work examines improvements in the robustness of Richardson extrapolation (RE), a method commonly used in solution verification to study the discretization error of computational models using a power law. Nonuniform mesh refinement is discussed alongside other pollutants that affect the robustness of the power law model. Maximum likelihood estimation (MLE) is proposed as a robust strategy to address the uncertainty generated by nonuniform mesh refinement. An exploratory computational fluid dynamics (CFD) study of a 2D planar Poiseuille flow is conducted to determine if nonuniform mesh noise can be modeled with this MLE approach for more robust RE.

Weinmeister, Justin [ORNL] (ORCID:0000000160090237↗
NASA NTRS2000

The Sandmeier Field Goniometer: A Measurement Tool for Bi-Directional Reflectance

Recent activities at the Remote Sensing Program at Stennis Space Center have identified the need to properly verify and validate data provided by the remote sensing community. One important variable, which effects remote sensing data is bi-directional reflectance distribution (BRDF). In order to quantify the effects of BRDF on man-made and natural ground targets, the Stennis Verification & Validation (V&V) team commissioned the Systems Engineering Division at NASA Ames Research Center to develop a Field Goniometer for use at the V&V Large Target Range and for various ground truthing missions. The Swiss Field Goniometer (FIGOS) was used as a benchmark instrument to design the new state of the art Sandmeier Field Goniometer (SGF), named after Stefan Sandmeier, developer of FIGOS. After establishing requirements for the SFG, design efforts began in early May 1998. The design of the SFG was completed in September 1998. Manufacturing, construction, and testing was completed in May 1999. The SFG was shipped to NASA SSC and fully operational by June 1999.

Turner, Mark↗
DOE OSTI2025

Establishing model credibility for process-microstructure-property relationships in additive manufacturing using exascale computing

Additive Manufacturing (AM) of alloys holds significant promise as a disruptive technology in various industries, yet its adoption is often hindered by challenges in achieving consistent part quality. These issues are primarily due to the complex process-microstructure-property (PSP) relationships inherent to AM. Computational models can greatly aid in understanding these relationships, but their widespread impact and adoption has been limited by a lack of validated, open-source, and computationally efficient PSP modeling frameworks and hardware limitations. Here, this study leverages the ExaAM software suite and data from the AMBench-2018 series of laser powder bed fusion (LPBF) benchmark experiments to perform a comprehensive model assessment, including verification, validation, sensitivity analysis, and uncertainty quantification. The RADICAL-EnTK workflow manager was used to perform an ensemble of heat transport, solidification, and mechanical response simulations on the exascale computer Frontier, considering uncertainties in critical model inputs such as laser spot size and nucleation parameters, and consisting of 125 explicit grain structure simulations and 7875 crystal plasticity simulations. For a selected location within the Inconel 625 AMBench-2018 test artifact, sensitivity analysis and uncertainty quantification were performed using the predicted distributions of grain structure and mechanical properties. Qualitative agreement was found between the predicted grain size and texture and the observed AMBench-2018 microstructure, the mean predicted yield stress was within 5% of the experimental measurement mean, and the mean predicted engineering stress at 5% strain was within 10% of the experimental measurement mean. The insights gained from development and validation of the ExaAM PSP modeling framework will help guide future directions for enhancing the credibility and reliability of PSP models in AM, thereby accelerating the adoption of AM technologies in various industries.

Additive manufacturing↗
DOE OSTI2026

Fracture Analysis of Cohesive Zone Models for Modeling Residual Stress Induced Delamination in Composite Structures

A fracture study of coupon-scale composite cylinders with embedded defects was conducted with an objective to assess and validate a modeling approach using two available cohesive material models. The study included experimental and simulation evaluations of initiation of crack growth and progression. Interrupted thermal experiments used acoustic emissions monitoring to identify the onset of crack progression during each cooling interval and ultrasonic scanning provided images of defect growth. Verification, validation, and uncertainty quantification (VVUQ) processes were performed in the assessment of the simulation predicted temperature at which crack propagation begins (quantity of interest). The Sobol sensitivity analysis identified the hoop direction elastic modulus in the carbon fiber reinforced polymer (CFRP) plies as the most influential parameter for simulations using both cohesive models, accounting for at least 70% of the variation in the temperature at crack propagation. The UQ temperature range for the Tvergaard-Hutchinson model was higher (more conservative) than the experimental acoustic measurement indicators of crack progression, while the temperature range for the Thouless-Parmigiani model enveloped the experimental data points for the primary defect size of 0.75 x 1 in. The simulations could not capture the stable crack growth indicated in the experiments. This is likely due to the models’ inability to represent anisotropic fracture toughness attributed to the structure of the orthotropic fiber weave in a woven composite laminate.

42 ENGINEERING↗
DOE OSTI2020

Enhancement of risk informed validation framework for external hazard scenario

In recent years, the U.S. Nuclear Regulatory Commission (USNRC) and the International Atomic Energy Agency (IAEA) have developed methodologies to assess the vulnerabilities of nuclear plants against site specific extreme hazards. In many cases, advanced simulation tools are being considered to simulate multi-physics, multi-scale phenomena and to evaluate vulnerability of nuclear facilities. The credibility of advanced simulation tools is assessed based on a formal verification, validation, and uncertainty quantification procedure. One of the key limitations in validation is the lack of relevant experimental data at system-level. This limitation leads to a decrease in the confidence of system-level risk predictions. Therefore, a robust validation framework is needed to formalize the confidence in predictive capability of advanced simulation results. Additionally, this study enhances the existing risk informed validation methodology, originally proposed by Kwag et al. [1] and Bodda et al. [2], by developing additional attributes and a new set of validation indicies for a complete and wider applicability of the framework. In this manuscript, the methodology to identify the critical path that leads to the system-level failure is illustrated. The overall validation is checked for completeness and consistency by comparing the critical path for both the system-level simulation and experimental models. The applicability of the code for an intended application is represented in terms of various maturity levels and helps in the process of decision making.

42 ENGINEERING↗
NASA NTRS2016

Sensitivity Analysis of the Integrated Medical Model for ISS Programs

Sensitivity analysis estimates the relative contribution of the uncertainty in input values to the uncertainty of model outputs. Partial Rank Correlation Coefficient (PRCC) and Standardized Rank Regression Coefficient (SRRC) are methods of conducting sensitivity analysis on nonlinear simulation models like the Integrated Medical Model (IMM). The PRCC method estimates the sensitivity using partial correlation of the ranks of the generated input values to each generated output value. The partial part is so named because adjustments are made for the linear effects of all the other input values in the calculation of correlation between a particular input and each output. In SRRC, standardized regression-based coefficients measure the sensitivity of each input, adjusted for all the other inputs, on each output. Because the relative ranking of each of the inputs and outputs is used, as opposed to the values themselves, both methods accommodate the nonlinear relationship of the underlying model. As part of the IMM v4.0 validation study, simulations are available that predict 33 person-missions on ISS and 111 person-missions on STS. These simulated data predictions feed the sensitivity analysis procedures. The inputs to the sensitivity procedures include the number occurrences of each of the one hundred IMM medical conditions generated over the simulations and the associated IMM outputs: total quality time lost (QTL), number of evacuations (EVAC), and number of loss of crew lives (LOCL). The IMM team will report the results of using PRCC and SRRC on IMM v4.0 predictions of the ISS and STS missions created as part of the external validation study. Tornado plots will assist in the visualization of the condition-related input sensitivities to each of the main outcomes. The outcomes of this sensitivity analysis will drive review focus by identifying conditions where changes in uncertainty could drive changes in overall model output uncertainty. These efforts are an integral part of the overall verification, validation, and credibility review of IMM v4.0.

medical equipment↗
NASA NTRS1984

Assessment of Galileo modal test results for mathematical model verification

The modal test program for the Galileo Spacecraft was completed at the Jet Propulsion Laboratory in the summer of 1983. The multiple sine dwell method was used for the baseline test. The Galileo Spacecraft is a rather complex 2433 kg structure made of a central core on which seven major appendages representing 30 percent of the total mass are attached, resulting in a high modal density structure. The test revealed a strong nonlinearity in several major modes. This nonlinearity discovered in the course of the test necessitated running additional tests at the unusually high response levels of up to about 21 g. The high levels of response were required to obtain a model verification valid at the level of loads for which the spacecraft was designed. Because of the high modal density and the nonlinearity, correlation between the dynamic mathematical model and the test results becomes a difficult task. Significant changes in the pre-test analytical model are necessary to establish confidence in the upgraded analytical model used for the final load verification. This verification, using a test verified model, is required by NASA to fly the Galileo Spacecraft on the Shuttle/Centaur launch vehicle in 1986.

Trubert, M.↗