SEARCH · Engineering Papers

Results for “system validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 775 records · Page 43

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

Towards a More Realistic Representation of NASA CERES-derived Surface Radiative Fluxes during Polar Night: A Comparison with the MOSAiC Field Campaign

The Arctic is one of the most sensitive regions of Earth to climate change, and yet remains one of the more difficult regions to both observe and simulate. The remoteness of the Arctic from most of civilization makes large volumes of in situ observations difficult to collect, and so satellite observations are a critical tool for observing the region, such as those from Clouds and the Earth’s Radiant Energy System (CERES). But validating the satellite radiative flux estimates is difficult because of the lack of in situ measurements. A useful remedy is utilizing measurements from field campaigns, such as the year-long Multidisciplinary drifting Observatory for the Study of Arctic Climate (MOSAiC) expedition during 2019-2020. The extensive high-quality surface radiative flux and meteorological measurements collected from MOSAiC provide a useful check on flux retrievals from CERES instruments. We compare MOSAiC and CERES surface radiative fluxes during October through March of 2019-2020. As this time period is mostly during polar night and twilight, the focus of the study is on longwave surface fluxes. To better understand the reasons for errors in CERES fluxes, we use the large set of meteorological measurements also collected by MOSAiC, including surface temperature, thermodynamic vertical profiles, surface turbulent fluxes, and cloud properties. Previous work identified a significant source of error in the CERES estimate of surface downwelling longwave flux as the estimate of low level cloud amount when compared with MOSAiC W-band radar measurements. An overestimate of low cloud amount yielded an underestimate of the downwelling flux on the order of tens of W m2. Continuing this work, we test other meteorological properties such as the vertical thermodynamic profile and surface turbulent fluxes. In addition, we examine different estimates of surface fluxes derived from CERES measurements. Previous work used the SYN1deg product, which had the limitation of having a 1º x 1 º horizontal resolution. A newer product estimates the surface fluxes at the CERES footprint resolution, which allows a more precise colocation, and thus a representative flux estimate, with the MOSAiC site.

J Brant Dodson↗

NASA NTRSDate not supplied

Towards A More Realistic Representation of Nasa Ceres-Derived Surface Radiative Fluxes During Polar Night: A Comparison With the Mosaic Field Campaign

The Arctic remains one of the more difficult regions observe, and so satellite observations are a critical tool for observing the region, such as those from Clouds and the Earth’s Radiant Energy System (CERES). But validating the satellite surface radiative flux estimates is difficult because of the lack of in situ measurements. The extensive high-quality surface radiative flux and meteorological measurements collected from MOSAiC provide a useful check on flux retrievals from CERES instruments. We compare MOSAiC and CERES surface radiative fluxes during October through March of 2019-2020 using the large set of meteorological measurements also collected by MOSAiC, specifically cloud properties. Previous work identified a significant source of error in the CERES estimate of surface downwelling longwave flux as the estimate of low level cloud amount when compared with MOSAiC W-band radar measurements. Continuing this work, we examine the effects of errors in cloud water path on surface radiative fluxes. When using all cloud conditions, errors in cloud water are also significantly correlated with surface radiative flux errors, though the size off the effect is only about half that of cloud amount. But when considering low cloud conditions only, the effects of cloud water and cloud amount are comparable.

J. Brant Dodson↗

NASA NTRSDate not supplied

Guiding Integration of Formal Verification in Assurance Cases

Assurance cases are being increasingly acknowledged as away to build trust in complex systems with autonomous capabilities. An assurance case is a comprehensive, defensible, and valid justification that a system will function as intended for a specific mission and operating environment. Formal verification is often reserved for the most critical components of such systems. However, formal verification tools are often complex, and their usage is subject to many constraints and contextual dependencies. This can raise challenges both for performing the verification as well as reflecting the verification results appropriately in the assurance case, especially for non-expert users of the verification tool. To address these challenges, we present a tool-supported methodology for integrating formal verification results in an assurance case by capturing key verification method information in a rigorously constructed assurance case. In particular, we capture the tool specification in terms of its inputs, outputs, and assurance constraints as assumptions over inputs and guarantees provided over its outputs. The tool specification is parametrized over the inputs and outputs to both guide the intended application of the tool, as well as to check that the tool has been applied following the stated assumptions and that the guarantees hold. We define a generic tool assurance argument pattern that enables integration of the verification results in the assurance case by allowing custom refinement and automated instantiation for each tool use. We demonstrate our methodology on two formal verification tools and their applications to the verification of neural network properties for the aircraft domain.

Assurance Cases↗

NASA NTRSDate not supplied

Towards a More Realistic Representation of NASA CERES-derived Surface Radiative Fluxes during Polar Night: A Comparison with the MOSAiC Field Campaign

J Brant Dodson↗

NASA NTRS2023

Level-1 Calibration Assessment of Spire’s LEMUR-2 GNSS-R Ocean Normalized Bistatic Radar Cross Section Estimates

An assessment of the ocean surface Level-1 Normalized Bistatic Radar Cross Section products provided by the two batch 1 ‘LEMUR-2’ GNSS-R receivers of Spire, Inc. is reported. The analysis uses datasets extending from DOY 345, 2020 to DOY 329, 2021. Initial assessments indicate a highly consistent overall inter-channel response with mean NBRCS differences estimated to be at the 1.00% level over a 7-12 m/s ECMWF reference wind speed range. Efforts to validate the observation systems’ aggregate response relative to ≈3 million co-located CYGNSS measurements suggest a highly complementary behavior with an overall NBRCS correlation of 79.03% highlighting the potential utility of ‘LEMUR-2’ measurements for ocean surface wind sensing and related applications. Non-physical NBRCS dependencies on various Level-1 calibration variables, also observed with GNSS-R previous systems, are nonetheless noted and are explored in detail.

Mohammad M. Al-Khaldi↗

NASA NTRSDate not supplied

The Troupe System: an Autonomous Multi-Agent Rover Swarm

Autonomous cooperative robotic systems are the future of space exploration. The complexity of such systems makes their development, verification and assurance challenging. The Robust Software Engineering group at NASA Ames has developed the Troupe project that aims to explore the design and development of a swarm of autonomous rovers tasked to perform autonomous exploration and mapping of an unknown terrain. In this paper, we showcase the system design, and accompanying verification and validation tools integrated in the Troupe system development life-cycle.

TROUPE↗

NASA NTRS1988

The LANDSAT/global positioning system project

A GPSPAC/LANDSAT-D Interface (GLI) Ground Support System was built to validate the performance and to calibrate the accuracy of the experimental navigation package, GPSPAC, flown on the LANDSAT-4 and 5 spacecraft. Although the GLI system operated successfully to give the orbit information needed to validate the GPSPAC, it also detected two anomalies: one is characteristic of the GLI system and the other is characteristic of the pre-operational phase of GPS. Several methods were applied to resolve or reduce the anomalies. This paper presents a description of the problems, the methods applied to resolve or reduce them, and the results.

Wood, Terri↗

NASA NTRS2013

Distilling the Verification Process for Prognostics Algorithms

The goal of prognostics and health management (PHM) systems is to ensure system safety, and reduce downtime and maintenance costs. It is important that a PHM system is verified and validated before it can be successfully deployed. Prognostics algorithms are integral parts of PHM systems. This paper investigates a systematic process of verification of such prognostics algorithms. To this end, first, this paper distinguishes between technology maturation and product development. Then, the paper describes the verification process for a prognostics algorithm as it moves up to higher maturity levels. This process is shown to be an iterative process where verification activities are interleaved with validation activities at each maturation level. In this work, we adopt the concept of technology readiness levels (TRLs) to represent the different maturity levels of a prognostics algorithm. It is shown that at each TRL, the verification of a prognostics algorithm depends on verifying the different components of the algorithm according to the requirements laid out by the PHM system that adopts this prognostics algorithm. Finally, using simplified examples, the systematic process for verifying a prognostics algorithm is demonstrated as the prognostics algorithm moves up TRLs.

verification↗

NASA NTRS1991

A new system model for radar polarimeters

The validity of the 2 x 2 receive R and transmit T model for radar polarimeter systems, first proposed by Zebker et al. (1987), is questioned. The model is found to be invalid for many practical realizations of radar polarimeters, which can lead to significant errors in the calibration of polarimetric radar images. A more general model is put forward, which addresses the system defects which cause the 2 x 2 model to break down. By measuring one simple parameter from a polarimetric active radar calibration (PARC), it is possible to transform the scattering matrix measurements made by a radar polarimeter to a format compatible with a 2 x 2 R and T matrix model. Alternatively, the PARC can be used to verify the validity of the 2 x 2 model for any polarimetric radar system. Recommendations for the use of PARCs in polarimetric calibration and to measure the orientation angle of the horizontal (H) and vertical (V) coordinate system are also presented.

Freeman, Anthony↗

NASA NTRSDate not supplied

An Overview of the CERES Radiation and Validation Experiment (CRAVE)

The Clouds and the Earth’s Radiant Energy System (CERES) experiment is one of the highest priority scientific satellite instruments developed for NASA’s Earth Observing System. The CERES Radiation and Validation Experiment (CRAVE) provides continuous world-class surface longwave and shortwave radiation measurements and validation of CERES and other satellite products. CRAVE consists of three sites (two active and one legacy). The legacy site was the CERES Ocean Validation Experiment, or COVE, located at Chesapeake Light Station (36.90 N, 75.71 W), 25 km off the coast of southeastern Virginia, USA. COVE was active from 2000-2016 but was deactivated due to structural concerns. The deactivation of COVE interrupted a rare long-term ocean/water scene dataset in the Baseline Surface Radiation Network (BSRN), the gold standard for surface radiation measurements. Shortly after the closure of COVE, Granite Island (46.72 N, 87.41 W), a new water site, was discovered and has been active since the summer of 2018. Granite Island is privately owned, located in Lake Superior, approximately 20 km north of Marquette, Michigan, USA, and 10 km to the nearest land point. The other CRAVE site is in Hampton, Virginia, USA, at Nasa Langley Research Center (37.10 N, 76.38 W), a land scene, and has been operating since December 2014. We will describe CRAVE and its importance, the uniqueness of each site and site logistics, participating networks and measurements made for radiometric, aerosol, meteorological and water skin temperature analysis. CRAVE measurements detecting smoke originating from Canadian wildfires with aerosol optical depths well over 2 and first results from a shortwave calibration round robin experiment will also be presented.

Bryan Fabbri↗

NASA NTRS1977

Computer transformation of partial differential equations into any coordinate system

The use of tensors to provide a compact way of writing partial differential equations in a form valid in all coordinate systems is discussed. In order to find solutions to the equations with their boundary conditions they must be expressed in terms of the coordinate system under consideration. The process of arriving at these expressions from the tensor formulation was automated by a software system, TENSR. An allied system that analyzes the resulting expressions term by term and drops those that are negligible is also described.

Sullivan, R. D.↗

NASA NTRS2006

Validation (not just verification) of Deep Space Missions

ion & Validation (V&V) is a widely recognized and critical systems engineering function. However, the often used definition 'Verification proves the design is right; validation proves it is the right design' is rather vague. And while Verification is a reasonably well standardized systems engineering process, Validation is a far more abstract concept and the rigor and scope applied to it varies widely between organizations and individuals. This is reflected in the findings in recent Mishap Reports for several NASA missions, in which shortfalls in Validation (not just Verification) were cited as root- or contributing-factors in catastrophic mission loss. Furthermore, although there is strong agreement in the community that Test is the preferred method for V&V, many people equate 'V&V' with 'Test', such that Analysis and Modeling aren't given comparable attention. Another strong motivator is a realization that the rapid growth in complexity of deep-space missions (particularly Planetary Landers and Space Observatories given their inherent unknowns) is placing greater demands on systems engineers to 'get it right' with Validation.

verification↗

NASA NTRS1974

Experimental investigation of the landing dynamics of three-legged spacecraft models

An experimental investigation was conducted to obtain accurate data from two three-legged spacecraft landing systems for purposes of validating current and future computer programs for legged landers. Two landing-gear systems were investigated: an inverted tripod system and a cantilever system. Tests were conducted for eight landing conditions chosen to obtain stability data, maximum loads, and maximum strokes for correlation with analytical results. Data from the two models are not directly comparable because of geometry and mass differences, but both are considered to be typical models of the respective landing-gear systems. Results indicate that maximum accelerations for both models, which occurred during a nearly flat landing on a horizontal surface, were nearly the same (approximately minus 18g). Maximum primary strut forces occurred for landings into a 20 deg slope and were 40 kiloNewtons (9000 lbf) and 47 kiloNewtons (10 500 lbf) for the inverted tripod and cantilever models, respectively; and maximum primary strut strokes for both models were 19 cm (7.4 in.). The inverted tripod model was slightly more stable than the cantilever model because of a larger ratio of footpad radius to center-of-gravity height.

Stubbs, S. M.↗

NASA NTRS1986

A description of the active and passive sidewall-boundary-layer removal systems of the 0.3-meter transonic cryogenic tunnel

Results are presented for an operational checkout and shakedown of the active sidewall-boundary-layer removal system newly installed in the Langley 0.3-meter Transonic Cryogenic Tunnel (0.3-m TCT). Prior to the installation of this active removal system, the sidewall-boundary layer was removed passively by exhausting directly to the atmosphere (i.e., no reinjection). With the active removal system using the reinjection compressor, the removal capability is greatly expanded to cover the entire operating envelope of the 0.3-m TCT. Details of the active removal system are presented including the compressor reinjection circuit, the compressor pressure ratio/surge control, and the compressor recirculation loop. The control logic and features of the compressor surge control are explained. Initial tests covering critical operating conditions show mass flow removal rates of about 5 percent at lower Mach numbers can be obtained with the active system. Measured performance characteristics of the compressor are presented. As part of the validation of the active system, limited airfoil tests were made using the new system.

Johnson, C. B.↗

NASA NTRS2001

Spacecraft Formation Flying Maneuvers Using Linear Quadratic Regulation With No Radial Axis Inputs

Regarding multiple spacecraft formation flying, the observation has been made that control thrust need only be applied coplanar to the local horizon to achieve complete controllability of a two-satellite (leader-follower) formation. A formulation of orbital dynamics using the state of one satellite relative to another is used. Without the need for thrust along the radial (zenith-nadir) axis of the relative reference frame, propulsion system simplifications and weight reduction may be accomplished. This work focuses on the validation of this control system on its own merits, and in comparison to a related system which does provide thrust along the radial axis of the relative frame. Maneuver simulations are performed using commercial ODE solvers to propagate the Keplerian dynamics of a controlled satellite relative to an uncontrolled leader. These short maneuver simulations demonstrate the capacity of the controller to perform changes from one formation geometry to another. Control algorithm performance is evaluated based on measures such as the fuel required to complete a maneuver and the maximum acceleration required by the controller. Based on this evaluation, the exclusion of the radial axis of control still allows enough control authority to use Linear Quadratic Regulator (LQR) techniques to design a gain matrix of adequate performance over finite maneuvers. Additional simulations are conducted including perturbations and using no radial control inputs. A major conclusion presented is that control inputs along the three axes have significantly different relationships to the governing orbital dynamics that may be exploited using LQR.

Starin, Scott R.↗

NASA NTRS2012

NASA GSFC CCMC Recent Model Validation Activities

The Community Coordinated Modeling Center (CCMC) holds the largest assembly of state-of-the-art physics-based space weather models developed by the international space physics community. In addition to providing the community easy access to these modern space research models to support science research, its another primary goal is to test and validate models for transition from research to operations. In this presentation, we provide an overview of the space science models available at CCMC. Then we will focus on the community-wide model validation efforts led by CCMC in all domains of the Sun-Earth system and the internal validation efforts at CCMC to support space weather servicesjoperations provided its sibling organization - NASA GSFC Space Weather Center (http://swc.gsfc.nasa.gov). We will also discuss our efforts in operational model validation in collaboration with NOAA/SWPC.

Rastaetter, L.↗

NASA NTRSDate not supplied

Liquefaction of Cryogenic Fluids for Production and Storage of Commodities on Extra-Terrestrial Surfaces

The Cryogenic Fluid In-situ Liquefaction for Landers (CryoFILL) activity has been investigating concepts for the liquefaction of cryogenic fluids produced on the surface of the Moon and Mars. CryoFILL consists of four activities: liquid nitrogen liquefaction testing that was completed in 2019, liquid oxygen liquefaction testing planned to begin in early 2022, fiber-optic sensor temperature measurement system development, and lightweight vacuum jacketed systems designed for the atmosphere of Mars. Additional activities include the development and validation of numerical modeling systems at three stages: thermodynamic models of the liquefaction process for basic energy/mass estimates, nodal models to predict liquefaction tank system level performance, and computational fluid dynamics to assess fluid phenomena occurring within the tank, specifically condensation and stratification within the ullage. These developments form a combined liquefaction and storage solution to support Lunar and Martian exploration. Key testing results from liquid nitrogen testing as well as plans for liquid oxygen testing will be discussed including key sensitivities from analytical evaluation of completed test results. Progress on the development and improvements on fiber optic sensor testing and developments will also be provided. Finally, work on the progress of the lightweight vacuum jacketed systems being co-developed by NASA and industry will be summarized.

Liquefaction↗

NASA NTRS2003

A Physical Validation Program for the GPM Mission

The GPM mission is currently planned for start in the late 2007 - early 2008 time frame. Its main scientific goal is to help answer pressing scientific problems arising within the context of global and regional water cycling. These problems cut across a hierarchy of scales and include climate-water cycle interactions, techniques for improving weather and climate predictions, and better methods for combining observed precipitation with hydrometeorological prediction models for applications to hazardous flood-producing storms, seasonal flood draught conditions, and fresh water resource assessments. The GPM mission will expand the scope of precipitation measurement through the use of a constellation of some 9 satellites, one of which will be an advanced TRMM-like core satellite carrying a dual-frequency Ku-Ka band precipitation radar and an advanced, multifrequency passive microwave radiometer with vertical-horizontal polarization discrimination. The other constellation members will include new dedicated satellites and co-existing operational/research satellites carrying similar (but not identical) passive microwave radiometers. The goal of the constellation is to achieve approximately 3-hour sampling at any spot on the globe -- continuously. The constellation's orbit architecture will consist of a mix of sun-synchronous and non-sun-synchronous satellites with the core satellite providing measurements of cloud-precipitation microphysical processes plus calibration-quality rainrate retrievals to be used with the other retrieval information to ensure bias-free constellation coverage. A major requirement before the retrieved rainfall information generated by the GPM mission can be used effectively by prognostic models to improve weather forecasts, hydrometeorological forecasts, and climate model reanalysis simulations is a capability to quantify the error characteristics of the retrievals. A solution for this problem has been upheld in past precipitation missions because of the lack of suitable error modeling systems incorporated into the validation programs and data distribution systems. An overview of how NASA intends to overcome this problem for the GPM mission using a physically-based error modeling approach within a multi-faceted validation program is described. The solution is to first identify specific user requirements and then determine the most stringent of these requirements that embodies all essential error characterization information needed by the entire user community. In the context of NASA s scientific agenda for the GPM mission, the most stringent user requirement is found within the data assimilation community. The fundamental theory of data assimilation vis-a-vis ingesting satellite precipitation information into the pre-forecast initializations is based on quantifying the conditional bias and precision errors of individual rain retrievals, and the space-time structure of the precision error (i.e., the spatial-temporal error covariance). By generating the hardware and software capability to produce this information in a near real-time fashion, and to couple the derived quantitative error properties to the actual retrieved rainrates, all key validation users can be satisfied. The talk will describe the essential components of the hardware and software systems needed to generate such near real-time error properties, as well as the various paradigm shifts needed within the validation community to produce a validation program relevant to the precipitation user community.

Smith, Eric A.↗