Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Qualitative and Quantitative verification”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

17 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS1985

Expert systems for Space Station automation

The expert systems required for automating key functions of the Manned Space Station (MSS) are explored. It is necessary that the expert systems developed be flexible, degrade gracefully in the case of a failure, and be able to work with incomplete data. The AI systems will have to perform interpretation and diagnosis, design, prediction and induction, and monitoring and control functions. Both quantitative and qualitative reasoning capabilities need improvements, as do automatic verification techniques, explanation and learning capabilities, and the use of metaknowledge, i. e., knowledge about the knowledge contained in the knowledge base. Information retrieval, fault isolation and manufacturing process control demonstrations are needed to validate expert systems for the MSS.

Georgeff, M. P.↗
NASA NTRS1973

The theory of charged particle temperatures in the upper atmosphere.

The thermal structure of the earth's upper atmosphere is examined in detail, with emphasis on the physical processes that govern the behavior of charged-particle temperatures. The characteristic features of and competition between the heating, cooling, and thermal conduction processes that govern electron and ion temperatures in the mid-latitude and auroral ionospheric regions are theoretically analyzed, and appropriate comparisons are made with experimental data. The proposed elaborate theory is considered qualitatively successful in accounting for the thermal structure of the ionosphere, and points requiring quantitative verification are delineated.

Schunk, R. W.↗
NASA NTRS1999

Real-Time Assimilation of Goes-Derived Products into A Mesoscale Model and It's Impact on Short-Term (06-36hr) Forecasts from 17 October 1998 through the Present

As the parameterizations of surface energy budgets in regional models have become more complete physically, models have the potential to be much more realistic in simulations of coupling between surface radiation, hydrology, and surface energy transfer. Realizing the importance of properly specifying the surface energy budget, many institutions are using land-surface models to represent the lower boundary forcing associated with biophysical processes and soil hydrology. However, the added degrees of freedom due to inclusion of such land-surface schemes require the specification of additional parameters within the model system such as vegetative resistances, green vegetation fraction, leaf area index, soil physical and hydraulic characteristics, stream flow, runoff, and the vertical distribution of soil moisture. Spatial heterogeneity of these parameters makes correct specification problematic since measurements are not routinely available. A technique has been developed for assimilating GOES-IR skin temperature tendencies, solar insolation, and surface albedo into the surface energy budget equation of a mesoscale model so that the simulated rate of temperature change closely agrees with the satellite observations. The technique has been successfully employed in a number of mesoscale models in case-study mode. We have taken the next step and developed a study to determine if assimilating these types of data into mesoscale models in real-time can improve short-term (648h) forecasts of temperature, relative humidity, and QPF on a daily basis over relatively large regions. Therefore, an operational modeling/assimilation system has been developed at the GHCC during the past summer that allows us to produce simulations out to 48 hours in a timely manor. The PSU/NCAR MM5 is used in a nested configuration with a 25 km grid covering the southeastern third of the US. The model has been on-line since 1 July 1998 and forecast products are posted on our web site. The satellite algorithms that generate data to be assimilated came on-line 17 October 1998. Quantitative assessment of the forecast quality is performed via traditional verification statistics. In addition, invaluable qualitative information is obtained through close collaboration with several NWSFO's who are using the MM5 products in real-time on a daily basis. The assimilation technique has been applied in an off-line mode since 17 October. Results based on bulk statistical verification of surface meteorology over the entire Southeastern US show that assimilating the GOES-derived land surface tendencies and solar radiation results in a significant reduction of the shelter air temperature and RH bias on a daily basis. In fact, the assimilation technique has produced improved temperature and RH forecasts for 97% of the 100 simulations performed to date. Work is currently underway to determine the sensitivity of the assimilation procedure to the availability of satellite data, length of assimilation period, model initialization, and synoptic-scale meteorological conditions. In addition, results from a detailed energy budget analysis using the Early Eta, our operational MM5, and the assimilation runs will help us to better understand the satellite assimilation the land-surface energy budge. Research during the spring-summer of 1999 will focus on the impact of the assimilation technique during the warm season where it is hypothesized that it can have a positive impact on QPF during conditions of weak synoptic-scale forcing.

Lapenta, William M.↗
NASA NTRS2014

OSIRIS-REx Touch-and-Go (TAG) Mission Design for Asteroid Sample Collection

The Origins Spectral Interpretation Resource Identification Security Regolith Explorer (OSIRIS-REx) mission is a NASA New Frontiers mission launching in September 2016 to rendezvous with the near-Earth asteroid Bennu in October 2018. After several months of proximity operations to characterize the asteroid, OSIRIS-REx flies a Touch-And-Go (TAG) trajectory to the asteroid's surface to collect at least 60 g of pristine regolith sample for Earth return. This paper provides mission and flight system overviews, with more details on the TAG mission design and key events that occur to safely and successfully collect the sample. An overview of the navigation performed relative to a chosen sample site, along with the maneuvers to reach the desired site is described. Safety monitoring during descent is performed with onboard sensors providing an option to abort, troubleshoot, and try again if necessary. Sample collection occurs using a collection device at the end of an articulating robotic arm during a brief five second contact period, while a constant force spring mechanism in the arm assists to rebound the spacecraft away from the surface. Finally, the sample is measured quantitatively utilizing the law of conservation of angular momentum, along with qualitative data from imagery of the sampling device. Upon sample mass verification, the arm places the sample into the Stardust-heritage Sample Return Capsule (SRC) for return to Earth in September 2023.

OSIRIS-REX TOUCH-AND-GO (TAG) MISSION DESIGN FOR A↗
NASA NTRS2012

The SPoRT-WRF: Evaluating the Impact of NASA Datasets on Convective Forecasts

The Short-term Prediction Research and Transition Center (SPoRT) is a collaborative partnership between NASA and operational forecasting entities, including a number of National Weather Service offices. SPoRT transitions real-time NASA products and capabilities to its partners to address specific operational forecast challenges. One challenge that forecasters face is applying convection-allowing numerical models to predict mesoscale convective weather. In order to address this specific forecast challenge, SPoRT produces real-time mesoscale model forecasts using the Weather Research and Forecasting (WRF) model that includes unique NASA products and capabilities. Currently, the SPoRT configuration of the WRF model (SPoRT-WRF) incorporates the 4-km Land Information System (LIS) land surface data, 1-km SPoRT sea surface temperature analysis and 1-km Moderate resolution Imaging Spectroradiometer (MODIS) greenness vegetation fraction (GVF) analysis, and retrieved thermodynamic profiles from the Atmospheric Infrared Sounder (AIRS). The LIS, SST, and GVF data are all integrated into the SPoRT-WRF through adjustments to the initial and boundary conditions, and the AIRS data are assimilated into a 9-hour SPoRT WRF forecast each day at 0900 UTC. This study dissects the overall impact of the NASA datasets and the individual surface and atmospheric component datasets on daily mesoscale forecasts. A case study covering the super tornado outbreak across the Ce ntral and Southeastern United States during 25-27 April 2011 is examined. Three different forecasts are analyzed including the SPoRT-WRF (NASA surface and atmospheric data), the SPoRT WRF without AIRS (NASA surface data only), and the operational National Severe Storms Laboratory (NSSL) WRF (control with no NASA data). The forecasts are compared qualitatively by examining simulated versus observed radar reflectivity. Differences between the simulated reflectivity are further investigated using convective parameters along with model soundings to determine the impacts of the various NASA datasets. Additionally, quantitative evaluation of select meteorological parameters is performed using the Meteorological Evaluation Tools model verification package to compare forecasts to in situ surface and upper air observations.

Zavodsky, Bradley↗
NASA NTRS2000

Comprehensive Design Reliability Activities for Aerospace Propulsion Systems

This technical publication describes the methodology, model, software tool, input data, and analysis result that support aerospace design reliability studies. The focus of these activities is on propulsion systems mechanical design reliability. The goal of these activities is to support design from a reliability perspective. Paralleling performance analyses in schedule and method, this requires the proper use of metrics in a validated reliability model useful for design, sensitivity, and trade studies. Design reliability analysis in this view is one of several critical design functions. A design reliability method is detailed and two example analyses are provided-one qualitative and the other quantitative. The use of aerospace and commercial data sources for quantification is discussed and sources listed. A tool that was developed to support both types of analyses is presented. Finally, special topics discussed include the development of design criteria, issues of reliability quantification, quality control, and reliability verification.

Christenson, R. L.↗
NASA NTRS2016

Space Suit Portable Life Support System (PLSS) 2.0 Pre-Installation Acceptance (PIA) Testing

Following successful completion of the space suit Portable Life Support System (PLSS) 1.0 development and testing in 2011, the second system-level prototype, PLSS 2.0, was developed in 2012 to continue the maturation of the advanced PLSS design which is intended to reduce consumables, improve reliability and robustness, and incorporate additional sensing and functional capabilities over the current Space Shuttle/International Space Station Extravehicular Mobility Unit (EMU) PLSS. PLSS 2.0 represents the first attempt at a packaged design comprising first generation or later component prototypes and medium fidelity interfaces within a flight-like representative volume. Pre-Installation Acceptance (PIA) is carryover terminology from the Space Shuttle Program referring to the series of test sequences used to verify functionality of the EMU PLSS prior to installation into the Space Shuttle airlock for launch. As applied to the PLSS 2.0 development and testing effort, PIA testing designated the series of 27 independent test sequences devised to verify component and subsystem functionality, perform in situ instrument calibrations, generate mapping data to define set-points for control algorithms, evaluate hardware performance against advanced PLSS design requirements, and provide quantitative and qualitative feedback on evolving design requirements and performance specifications. PLSS 2.0 PIA testing was carried out from 3/20/13 - 3/15/14 using a variety of test configurations to perform test sequences that ranged from stand-alone component testing to system-level testing, with evaluations becoming increasingly integrated as the test series progressed. Each of the 27 test sequences was vetted independently, with verification of basic functionality required before completion. Because PLSS 2.0 design requirements were evolving concurrently with PLSS 2.0 PIA testing, the requirements were used as guidelines to assess performance during the tests; after the completion of PIA testing, test data served to improve the fidelity and maturity of design requirements as well as plans for future advanced PLSS functional testing.

Watts, Carly↗
NASA NTRS2016

Space Suit Portable Life Support System (PLSS) 2.0 Pre-Installation Acceptance (PIA) Testing

Following successful completion of the space suit Portable Life Support System (PLSS) 1.0 development and testing in 2011, the second system-level prototype, PLSS 2.0, was developed in 2012 to continue the maturation of the advanced PLSS design. This advanced PLSS is intended to reduce consumables, improve reliability and robustness, and incorporate additional sensing and functional capabilities over the current Space Shuttle/International Space Station Extravehicular Mobility Unit (EMU) PLSS. PLSS 2.0 represents the first attempt at a packaged design comprising first generation or later component prototypes and medium fidelity interfaces within a flight-like representative volume. Pre-Installation Acceptance (PIA) is carryover terminology from the Space Shuttle Program referring to the series of test sequences used to verify functionality of the EMU PLSS prior to installation into the Space Shuttle airlock for launch. As applied to the PLSS 2.0 development and testing effort, PIA testing designated the series of 27 independent test sequences devised to verify component and subsystem functionality, perform in situ instrument calibrations, generate mapping data, define set-points, evaluate control algorithms, evaluate hardware performance against advanced PLSS design requirements, and provide quantitative and qualitative feedback on evolving design requirements and performance specifications. PLSS 2.0 PIA testing was carried out in 2013 and 2014 using a variety of test configurations to perform test sequences that ranged from stand-alone component testing to system-level testing, with evaluations becoming increasingly integrated as the test series progressed. Each of the 27 test sequences was vetted independently, with verification of basic functionality required before completion. Because PLSS 2.0 design requirements were evolving concurrently with PLSS 2.0 PIA testing, the requirements were used as guidelines to assess performance during the tests; after the completion of PIA testing, test data served to improve the fidelity and maturity of design requirements as well as plans for future advanced PLSS functional testing.

Anchondo, Ian↗
NASA NTRS2011

Adapting CALIPSO Climate Measurements for Near Real Time Analyses and Forecasting

The Cloud-Aerosol Lidar and Infrared Pathfinder satellite Observations (CALIPSO) mission was originally conceived and designed as a climate measurements mission, with considerable latency between data acquisition and the release of the level 1 and level 2 data products. However, the unique nature of the CALIPSO lidar backscatter profiles quickly led to the qualitative use of CALIPSO?s near real time (i.e., ? expedited?) lidar data imagery in several different forecasting applications. To enable quantitative use of their near real time analyses, the CALIPSO project recently expanded their expedited data catalog to include all of the standard level 1 and level 2 lidar data products. Also included is a new cloud cleared level 1.5 profile product developed for use by operational forecast centers for verification of aerosol predictions. This paper describes the architecture and content of the CALIPSO expedited data products. The fidelity and accuracy of the expedited products are assessed via comparisons to the standard CALIPSO data products.

Vaughan, Mark A.↗
NASA NTRS1981

Selecting a software development methodology

The state of the art analytical techniques for the development and verification of digital flight control software is studied and a practical designer oriented development and verification methodology is produced. The effectiveness of the analytic techniques chosen for the development and verification methodology are assessed both technically and financially. Technical assessments analyze the error preventing and detecting capabilities of the chosen technique in all of the pertinent software development phases. Financial assessments describe the cost impact of using the techniques, specifically, the cost of implementing and applying the techniques as well as the relizable cost savings. Both the technical and financial assessment are quantitative where possible. In the case of techniques which cannot be quantitatively assessed, qualitative judgements are expressed about the effectiveness and cost of the techniques. The reasons why quantitative assessments are not possible will be documented.

Jones, R. E.↗
NASA NTRS1992

CAPTIONALS: A computer aided testing environment for the verification and validation of communication protocols

This paper covers the verification and protocol validation for distributed computer and communication systems using a computer aided testing approach. Validation and verification make up the so-called process of conformance testing. Protocol applications which pass conformance testing are then checked to see whether they can operate together. This is referred to as interoperability testing. A new comprehensive approach to protocol testing is presented which address: (1) modeling for inter-layer representation for compatibility between conformance and interoperability testing; (2) computational improvement to current testing methods by using the proposed model inclusive of formulation of new qualitative and quantitative measures and time-dependent behavior; (3) analysis and evaluation of protocol behavior for interactive testing without extensive simulation.

Feng, C.↗
NASA NTRS1984

Flow-field investigation of a supercruise fighter model

A NASA Langley investigation was conducted in the 16-foot Transonic Tunnel to survey the flow field around a model of a Supersonic cruise fighter configuration. In this investigation, a model of a supersonic cruise fighter configuration formerly utilized in afterbody-nozzle performance investigations was surveyed with a single, multiholed probe to determine local values of angle of attack, side flow, and Mach number. The investigation was conducted at Mach numbers of 0.6, 0.9, and 1.2 at angles of attack from 0 to 10 deg. The purpose of the investigation was to provide a data base of experimental data for use in verification of theoretical methods, and to compare the experimental data with predictions from currently available theoretical techniques. Results from this investigation show that local angles of attack were generally greater than free stream above the wing and generally less than free stream below the wing. Also there were large spanwise gradients above the wing at the higher angles of attack. The comparisons of experimental data with theoretical predictions show that the theoretical techniques give a qualitative estimate of the flow-field but will require much work to give good quantitative results.

Reubush, D. E.↗
NASA NTRS2016

Functional Fault Model Development Process to Support Design Analysis and Operational Assessment

A functional fault model (FFM) is an abstract representation of the failure space of a givensystem. As such, it simulates the propagation of failure effects along paths between the origin ofthe system failure modes and points within the system capable of observing the failure effects. Asa result, FFMs may be used to diagnose the presence of failures in the modeled system. FFMsnecessarily contain a significant amount of information about the design, operations, and failuremodes and effects. One of the important benefits of FFMs is that they may be qualitative, ratherthan quantitative and, as a result, may be implemented early in the design process when there ismore potential to positively impact the system design. FFMs may therefore be developed andmatured throughout the monitored system's design process and may subsequently be used toprovide real-time diagnostic assessments that support system operations. This paper provides anoverview of a generalized NASA process that is being used to develop and apply FFMs. FFMtechnology has been evolving for more than 25 years. The FFM development process presented inthis paper was refined during NASA's Ares I, Space Launch System, and Ground SystemsDevelopment and Operations programs (i.e., from about 2007 to the present). Process refinementtook place as new modeling, analysis, and verification tools were created to enhance FFMcapabilities. In this paper, standard elements of a model development process (i.e., knowledgeacquisition, conceptual design, implementation & verification, and application) are describedwithin the context of FFMs. Further, newer tools and analytical capabilities that may benefit thebroader systems engineering process are identified and briefly described. The discussion isintended as a high-level guide for future FFM modelers.

Melcher, Kevin J.↗
NASA NTRS2016

Functional Fault Model Development Process to Support Design Analysis and Operational Assessment

A functional fault model (FFM) is an abstract representation of the failure space of a given system. As such, it simulates the propagation of failure effects along paths between the origin of the system failure modes and points within the system capable of observing the failure effects. As a result, FFMs may be used to diagnose the presence of failures in the modeled system. FFMs necessarily contain a significant amount of information about the design, operations, and failure modes and effects. One of the important benefits of FFMs is that they may be qualitative, rather than quantitative and, as a result, may be implemented early in the design process when there is more potential to positively impact the system design. FFMs may therefore be developed and matured throughout the monitored system's design process and may subsequently be used to provide real-time diagnostic assessments that support system operations. This paper provides an overview of a generalized NASA process that is being used to develop and apply FFMs. FFM technology has been evolving for more than 25 years. The FFM development process presented in this paper was refined during NASA's Ares I, Space Launch System, and Ground Systems Development and Operations programs (i.e., from about 2007 to the present). Process refinement took place as new modeling, analysis, and verification tools were created to enhance FFM capabilities. In this paper, standard elements of a model development process (i.e., knowledge acquisition, conceptual design, implementation & verification, and application) are described within the context of FFMs. Further, newer tools and analytical capabilities that may benefit the broader systems engineering process are identified and briefly described. The discussion is intended as a high-level guide for future FFM modelers.

Verification↗
NASA NTRS1996

Fluid Dynamic Verification Experiments on STS-70

Fluid dynamic experiments were flown on STS-70 as phase two of the engineering evaluation of the first bioreactor Engineering Development Unit (EDU#1). The phase one experiments were comparative cell cultures in identical units on earth and onboard STS-70. In phase two, two types of fluid dynamic experiments were performed. Qualitative comparisons of the basic flow patterns were evaluated with the use of 'dye' streaklines formed from alternate injections of either a mild acid or base solution into the external flow loop that was then perfused into the vessel. The presence of Bromothymol Blue in the fluid then caused color changes from yellow to blue or vice versa, indicating the basic fluid motions. This reversible change could be repeated as desired. In the absence of significant density differences in the fluid, the flow patterns in space should be the same as on earth. Video tape records of the flow patterns for a wide range of operating conditions were obtained. The second type of fluid dynamic experiment was the quantitative evaluation of the trajectories of solid beads of various densities and sizes. The beads were introduced into the vessel and the paths recorded on video tape, with the vessel operated at various rotation rates and flow perfusion rates. Because of space limitations, the video camera was placed as close as possible to the vessel, resulting in significant optical distortion. This report describes the analysis methods to obtain comparisons between the in-flight fluid dynamics and numerical models of the flow field. The methods include optical corrections to the video images and calculation of the bead trajectories for given operating conditions and initial bead locations.

Kleis, Stanley J.↗
NASA NTRS2018

Overview of the Exploration Exercise Device Validation Study Plans

The NASA has determined that a multi-functional exercise device will be developed for use as an exercise device during exploration missions. The device will allow for full body resistance and metabolic exercise necessary to minimize physiological losses during space flight and to maintain fitness necessary to perform critical mission tasks. Prior to implementation as an exercise device on an Exploration vehicle, there will be verification and validation testing completed to determine device efficacy at providing the necessary training stimuli to achieve desired goals. Because the exploration device will be new device that has yet be specified, specific Verification and Validation (V&V) protocols have yet to be developed. Upon delivery of an exploration exercise device training unit, stakeholders throughout NASA will develop V&V plans that include ground-based testing and testing on the International Space Station (ISS). Stakeholders will develop test protocols that include success criterion for the device. Ground tests will occur at NASA Johnson Space Station prior to flight testing. The intents of the ground tests are to allow crew, spaceflight medicine, science, engineering, Astronaut Strength, Conditioning, and Reconditioning staff, and others to gain experience in the best utilization of the device. The goal is to obtain an evidence base for recommending use of the device on the ISS. The developed protocol will be created to achieve multiple objectives, including determining if the device provides an adequate training stimulus for 5th - 95th percentile males and females, allows for exercise modalities that protect functional capability, and is robust and can withstand extensive human use. Although protocols are yet to be determined, current expectations include use of the device by test subjects and current crew in order to obtain quantitative and qualitative feedback. Information obtained during the ground tests may be used to influence device modifications during design iterations. Assuming successful ground tests, the device will be installed on the ISS for testing during space flight. Spaceflight testing is envisioned to include an activation and checkout (ACO) phase and a V&V phase. During the ACO phase, 1-2 crewmembers will exercise with the device to ensure proper function. ACO is expected to last multiple months because of the many modes and methods of exercise that need to be assessed. However, the goal is to complete the ACO as quickly as possible. Once successful ACO occurs, the crew will be free to use the device for normal exercise pending concurrence from stakeholders. V&V tests on the ISS will ideally consist of crew using the device for all of their exercise for an entire mission. Exercise prescriptions will be supplied that replicate expected prescriptions during exploration missions. Crew that are not enrolled in the V&V studies would be also free to use the device as their schedule permits. As experience is gained by users, exercise protocols could change. The intent of all V&V testing is to ensure that all have thorough understanding of experience at optimizing device capability

DeWitt, J. K.↗
NASA NTRS2003

Experimental Evaluation of Verification and Validation Tools on Martian Rover Software

To achieve its science objectives in deep space exploration, NASA has a need for science platform vehicles to autonomously make control decisions in a time frame that excludes intervention from Earth-based controllers. Round-trip light-time is one significant factor motivating autonomy capability, another factor is the need to reduce ground support operations cost. An unsolved problem potentially impeding the adoption of autonomy capability is the verification and validation of such software systems, which exhibit far more behaviors (and hence distinct execution paths in the software) than is typical in current deepspace platforms. Hence the need for a study to benchmark advanced Verification and Validation (V&V) tools on representative autonomy software. The objective of the study was to access the maturity of different technologies, to provide data indicative of potential synergies between them, and to identify gaps in the technologies with respect to the challenge of autonomy V&V. The study consisted of two parts: first, a set of relatively independent case studies of different tools on the same autonomy code, second a carefully controlled experiment with human participants on a subset of these technologies. This paper describes the second part of the study. Overall, nearly four hundred hours of data on human use of three different advanced V&V tools were accumulated, with a control group that used conventional testing methods. The experiment simulated four independent V&V teams debugging three successive versions of an executive controller for a Martian Rover. Defects were carefully seeded into the three versions based on a profile of defects from CVS logs that occurred in the actual development of the executive controller. The rest of the document is structured a s follows. In section 2 and 3, we respectively describe the tools used in the study and the rover software that was analyzed. In section 4 the methodology for the experiment is described; this includes the code preparation, seeding of defects, participant training and experimental setup. Next we give a qualitative overview of how the experiment went from the point of view of each technology; model checking (section 5), static analysis (section 6), runtime analysis (section 7) and testing (section 8). The find section gives some preliminary quantitative results on how the tools compared.

Brat, Guillaume↗