Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “third-party validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

Validation for Global Solar Wind Prediction Using Ulysses Comparison: Multiple Coronal and Heliospheric Models Installed at the Community Coordinated Modeling Center

The prediction of the background global solar wind is a necessary part of space weather forecasting. Several coronal and heliospheric models have been installed and/or recently upgraded at the Community Coordinated Modeling Center (CCMC), including the Wang-Sheely-Arge (WSA)-Enlil model, MHD-Around-a-Sphere (MAS)-Enlil model, Space Weather Modeling Framework (SWMF), and Heliospheric tomography using interplanetary scintillation data. Ulysses recorded the last fast latitudinal scan from southern to northern poles in 2007. By comparing the modeling results with Ulysses observations over seven Carrington rotations, we have extended our third-party validation from the previous near-Earth solar wind to middle to high latitudes, in the same late declining phase of solar cycle 23. Besides visual comparison, wehave quantitatively assessed the models capabilities in reproducing the time series, statistics, and latitudinal variations of solar wind parameters for a specific range of model parameter settings, inputs, and grid configurations available at CCMC. The WSA-Enlil model results vary with three different magnetogram inputs.The MAS-Enlil model captures the solar wind parameters well, despite its underestimation of the speed at middle to high latitudes. The new version of SWMF misses many solar wind variations probably because it uses lower grid resolution than other models. The interplanetary scintillation-tomography cannot capture the latitudinal variations of solar wind well yet. Because the model performance varies with parameter settings which are optimized for different epochs or flow states, the performance metric study provided here can serve as a template that researchers can use to validate the models for the time periods and conditions of interest to them.

Jian, L. K.↗

Can We Use the QA4ECV Black-sky Fraction of Absorbed Photosynthetically Active Radiation (FAPAR) using AVHRR Surface Reflectance to Assess Terrestrial Global Change?

NOAA platforms provide the longest period of terrestrial observation since the 1980s. The progress in calibration, atmospheric corrections and physically based land retrieval offers the opportunity to reprocess these data for extending terrestrial product time series. Within the Quality Assurance for Essential Climate Variables (QA4ECV) project, the black-sky Joint Research Centre (JRC)-fraction of absorbed photosynthetically active radiation (FAPAR) algorithm was developed for the AVHRR sensors on-board NOAA-07 to -16 using the Land Surface Reflectance Climate Data Record. The retrieval algorithm was based on the radiative transfer theory, and uncertainties were included in the products. We proposed a time and spatial composite for providing both 10-day and monthly products at 0.05º × 0.05º. Quality control and validation were achieved through benchmarking against third-party products, including Sea-Viewing Wide Field-of-View Sensor (SeaWiFS) datasets produced with the same retrieval algorithm. Past ground-based measurements, providing a proxy of FAPAR, showed good agreement of seasonality values over short homogeneous canopies and mixed vegetation. The average difference between SeaWiFS and QA4ECV monthly products over 2002–2005 is about 0.075 with a standard deviation of 0.091. We proposed a monthly linear bias correction that reduced these statistics to 0.02 and 0.001. The complete harmonized long-term time series was then used to address its fitness for the purpose of analysis of global terrestrial change.

AVHRR↗

QuEST: Qualifying Environmentally Sustainable Technologies

TEERM focuses its validation efforts on technologies that have shown promise in laboratory testing, but lack testing under realistic or field environment. Mature technologies have advantages over those that are still in the developmental stage such as being more likely to be transitioned into a working environment. One way TEERM begins to evaluate the suitability of technologies is through Technology Readiness Levels (TRLs). TRLs are a systematic metric/measurement system that supports assessments of the maturity of a particular technology and the consistent comparison of maturity between different types of technology. TEERM generally works on demonstrating/validating alternatives that fall within TRLs 5-9. In instances where a mature technology does not exist for a particular Agency application, TEERM works with technology development groups and programs such as NASA's Innovative Partnerships Program (IPP). The IPP's purpose is to identify and document available technologies in light of NASA's needs, evaluate and prioritize those technologies, and reach out to find new partners. All TEERM projects involve multiple partners. Partnering reduces duplication of effort that otherwise might occur if individuals worked their problems alone. Partnering also helps reduce individual contributors' shares of the total cost of technology validation. Through collaboration and financial commitment from project stakeholders and third-party sources, it is possible to fully fund expensive demonstration/validation efforts.

Brown, Christina↗

Excavator Design Validation

The Excavator Design Validation tool verifies excavator designs by automatically generating control systems and modeling their performance in an accurate simulation of their expected environment. Part of this software design includes interfacing with human operations that can be included in simulation-based studies and validation. This is essential for assessing productivity, versatility, and reliability. This software combines automatic control system generation from CAD (computer-aided design) models, rapid validation of complex mechanism designs, and detailed models of the environment including soil, dust, temperature, remote supervision, and communication latency to create a system of high value. Unique algorithms have been created for controlling and simulating complex robotic mechanisms automatically from just a CAD description. These algorithms are implemented as a commercial cross-platform C++ software toolkit that is configurable using the Extensible Markup Language (XML). The algorithms work with virtually any mobile robotic mechanisms using module descriptions that adhere to the XML standard. In addition, high-fidelity, real-time physics-based simulation algorithms have also been developed that include models of internal forces and the forces produced when a mechanism interacts with the outside world. This capability is combined with an innovative organization for simulation algorithms, new regolith simulation methods, and a unique control and study architecture to make powerful tools with the potential to transform the way NASA verifies and compares excavator designs. Energid's Actin software has been leveraged for this design validation. The architecture includes parametric and Monte Carlo studies tailored for validation of excavator designs and their control by remote human operators. It also includes the ability to interface with third-party software and human-input devices. Two types of simulation models have been adapted: high-fidelity discrete element models and fast analytical models. By using the first to establish parameters for the second, a system has been created that can be executed in real time, or faster than real time, on a desktop PC. This allows Monte Carlo simulations to be performed on a computer platform available to all researchers, and it allows human interaction to be included in a real-time simulation process. Metrics on excavator performance are established that work with the simulation architecture. Both static and dynamic metrics are included.

Pholsiri, Chalongrath↗

Testing of Advanced Capabilities to Enable In-time Safety Management and Assurance for Future Flight Operations

In order to refine an initial Concept of Operations, explore Concepts of Use, and expose/validate requirements for future In-Time Aviation Safety Management Systems (IASMS), testing architectures were created, along with a set of capabilities and underlying information exchange protocols. These systems were conceived and developed based on hazards associated with two envisioned urban area flight domains: (1) highly autonomous small uncrewed aerial systems (sUAS) operating at low altitudes, and (2) highly autonomous air taxis. The initial scope of this development is described in [1]; this report provides an update, focusing on the subsequent developments and test activities. As stated in [1], it is important to note that there are many capabilities already in use by the industry (or soon to be in use) that will play critical roles in future IASMS designs. Those reported here were developed to address a gap in the current state-of-the-art regarding specific hazards/risks, and/or to allow for investigation of the interplay between and across hazard types — particularly regarding how overall safety risk can be reduced or managed effectively. Results of testing and development activities are organized by the operational phase wherein a particular capability would be employed (i.e., preflight, in-flight, and post-flight/off-line). Pre-flight: A set of capabilities were developed to help mitigate safety risk prior to flight (e.g., during flight and mission planning). Results of testing summarize (1) validation activities to raise the Technology Readiness Level (TRL) and (2) evaluation activities where the capabilities were applied to flight/mission planning procedures and used by operators/pilots. For the latter, flight plans were automatically assessed, and operators/pilots were notified of hazardous flight segments so as to enable adjustment of the flight plan and re-evaluation, and/or to better inform go/no-go decisions. Capabilities addressed hazards associated with power consumption, third-party risk, wind, navigation system performance, radiofrequency interference, and proximity to geo-spatial threats (e.g., buildings, trees, and no-fly zones). In-flight: Flight experiments tested capabilities that detect and respond to hazards encountered during flight. In the first series, safety hazards were monitored and assessed onboard, and system-generated mitigation maneuvers were recorded (but not acted upon by the vehicle). In the second series, mitigation maneuver commands directed the aircraft in response to safety hazards (i.e., auto-mitigation). The sUAS used for testing is described in full, as is the test architecture, which included commercial avionics, research avionics, and onboard software designed to detect, assess, and respond to hazards. The onboard system was designed as a run-time assurance framework, consistent with [2] and supportive of both supervisory and automated modes. The primary functions included: real-time risk assessment (RTRA), auto-pilot monitoring, constraint monitoring, and contingency select/triggering. RTRA performs integrated risk assessment considering data from several hazard-related monitors (e.g., battery, motors, navigation, communications, population density, and loss-of-control). Post-flight/off-line: Data monitored and recorded during flights can enable IASMS capabilities that execute after flights have completed (or “off-line”). These include: (1) the ability to identify anomalies and trends that may only be observable when comparing data spanning a number of similar flights; (2) the ability to update and validate pre-flight and in-flight capabilities and any underlying models to improve their performance; (3) the ability to report anomalies/off-nominals that may indicate design changes or maintenance actions are needed; and (4) the ability for humans involved in operations to report safety-relevant observations to help in understanding the flight data and/or the operational context of a flight. Progress on three such capabilities is summarized; the first investigates anomaly detection given a limited set of flight logs and applies an approach previously used for space operations. The second explores what could be identified using a larger set of flight logs, including from web-based forums where flight logs are posted by sUAS autopilot users. The third creates a new means of collecting information on UAS incidents and accidents via the Aviation Safety Reporting System (ASRS).

sUAS↗