Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “QA”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

119 records · Page 7

Mission Operations and Command Assurance: Automating an Operations TQM Task

A long-term program is in progress at JPL to reduce cost and risk of mission operations through defect prevention and error management. A major element of this program, Mission Operations and Command Assurance (MO&QA), provides a system level function on flight projects to instill quality in mission operations. MO&CA embodies the Total Quality Management (TQM) principle of Continuous Process Inprovement (CPI) and uses CPI in applying automation to mission operations to reduce risk and costs. MO&CA has led efforts to apply and has implemented automation in areas that impact the daily flight project environment including Incident Surprise Anomoly tracking and reporting; command data verification, tracking, and reporting; and command support data usage. MO&CA's future work in automation will take into account that future missions systems must be designed to avoid increasing error through the introduction of automation, while adapting to the demands of smaller flight teams.

automation↗

Quantifying Uncertainties in Nighttime Light Retrievals From Suomi-NPP and NOAA-20 VIIRS Day/Night Band Data

Satellite observations of nighttime lights (NTL) from Suomi-NPP and NOAA-20 VIIRS Day/Night Band data have been widely used to estimate human activities. Long-term changes such as urban development and abrupt short-term changes such as power outages have been monitored from temporal NTL acquired by satellites. While high temporal NTL variation has been found across NTL data of varying temporal scale (e.g., daily, monthly, and annual composites), the sources of measurement error and uncertainty are poorly understood. This paper quantifies the sources of VIIRS-derived NTL uncertainty due to view-illumination geometry, surface Bidirectional Reflectance Distribution Function (BRDF)/albedo, and the effects of snow cover, lunar irradiance, aerosol loading, cloud mask, vegetation, geometry, and ephemeral artifacts (e.g., the Aurora Borealis). Based on this current assessment of NASA Black Marble retrievals (VNP46, Collection V001), we found that angular and atmospheric effects dominate retrieval uncertainty. Errors introduced by upstream data inputs (e.g., a coarser nighttime snow cover flag and misclassification errors in the existing VIIRS nighttime cloud mask) were also found to impact retrieval quality. Despite these challenges, a consistent daily NTL time series record can be routinely generated from top-of-atmosphere VNP46 radiances. Key recommendations include: (1) the use of lunar-BRDF adjusted and atmospherically corrected NTL (i.e., as identified as high-quality retrievals in the VNP46 QA fields), (2) development and improvement to the VIIRS snow cover and cloud masks algorithms to accurately reflect NTL retrieval conditions, (3) characterizing seasonal variations in NTL due to vegetation and snow, (4) reducing geometric effects due to the spatial mismatch of gridded pixel and observation footprint, (5) employing angularly-consistent NTL observations from multiple VIIRS instruments (i.e., Suomi-NPP and NOAA-20) to reduce pixel-based uncertainties and address persistent data gaps, and (6) being mindful of surface-reflected radiance from aurora events at mid-to-high latitudes.

Zhuosen Wang↗

Genesis Solar Wind Sample Curation Documentation

Introduction: A scientist with experience as a sample science analyst, provider of flight hardware for multiple missions, and senior engineer in an ISO 2000-rated manufacturing plant has described the timeline of key participants in any PI-led sample return mission, the breadth of the organizations involved [1,2], and, of interest to this meeting, choosing the types of data to preserve and issues of future data accessibility. This work broadens that perspective by giving similar lessons from Genesis sample curation point-of-view. Curation participation regarding data gathering was part of the mission review process from the beginning. Genesis’ story illustrates outcome of several choices about types of data to record and preserve. Precision analysis of solar wind atoms captured in pure, ultraclean substrates is the driving science goal; therefore, detailed documentation was captured from all mission and curation phases and from investigator laboratories because these processes affect the final analytical results [3]. Pre-flight: Design and fabrication of the spacecraft. Like many modern small sample return missions, Genesis was a tightly managed team integrated across science, engineering and curation. Communication across the team was excellent, and, for the most part, the hands-on engineering technicians understood the impacts of “small choices” they routinely make, and the eyes-on oversight of manufacturing processes by scientists was mindful of details. The payload was designed by the Jet Propulsion Laboratory and the spacecraft by Lockheed Martin. Solar wind collectors and instruments were fabricated by multiple vendors and laboratories. The main portion of the payload was assembled at JSC. Fabrication procedures and contamination-control data (with witness coupons) were stored primarily at JSC. The original composition, dimensions and configuration of components, results of thermal testing, etc. are still needed for interpretation of analytical data. At times, these must be estimated from secondary information acquired pre-flight. Moreover, some files (e.g., original 3-D models and early Powerpoint) cannot be opened using software. Archived curation data includes 2-D drawings, material usage lists, QA documentation and analyses of consumables used during fabrication. Important chemical information still resides in archived hardware, paints and lubricants, material coupons, cleaning coupons, environmental witness plates and reference materials from manufacturing facilities. Purity and cleanliness of collector substrates. Semi-conductor vendors provided surface cleanliness data and some purity data. Purity for specific elements of interest was verified by science team members in their laboratories [4]. Curation archived procurement and shipping records, analysis reports, and non-proprietary fabrication data. A physical archive of flight collector reference materials is maintained for future use so additional data can be collected as analytical techniques improve. These are of increased value due to the hard landing upon re-entry. Cleaning and cleanliness assessments of flight hardware. Cleaning of the science canister payload was performed at JSC in a dedicated ISO 4 cleanroom using ultrapure water (UPW). The cleanliness of this UPW was monitored throughout processing. The archive for the clean lab also includes airborne particle counts, airborne molecular and inorganic contamination measurements as well as cleanroom construction material coupons and witness coupons. Hardware cleanliness was assessed by particle counts in rinse water batches. This information is recorded in batch cleaning forms and logbooks, and are, perhaps, of decreased value due to the hard landing. Post-flight: Curation-generated data. The curation handling history of each Genesis sample is documented in a typical astromaterials sample database which captures sample location, physical description and characterization data. Samples have a “shelf life”. Crucial to the preservation of samples is ongoing documentation of the sample environment, initially under curatorial control but is now a separate facility function with requires coordination. PI-generated data. Data on sample characterization and cleaning techniques continues to be generated by sample users [5]. These are often captured in LPSC abstracts, but these “engineering” results often are not publishable as stand-alone papers. We are actively looking for ways to make this information more accessible to users. Ion implants into samples have aided science return and can be shared among investigators. These (and similar) materials should be added to the curatorial collection with appropriate process and characterization data generated externally. Summary: Complete data archives for returned astromaterial samples must be broad in types and formats, and inclusive of environmental monitoring.

Genesis↗

NASA Quality Assurance in an MBSE world

Over the past decade or so, the emergence of Model Based Systems Engineering (MBSE) has demonstrated its desirability and value in terms of 1) being a single source of truth, 2) unambiguous definitions and relationships, and 3) after representation, the ability to explore/extract any sets of data on demand. While much work has been done in showing the value to the system engineering discipline in these areas, how does that value translate to the Safety and Mission Assurance (S&MA) world? This paper provides a vision of a very desirable future of NASA S&MA after it is fully integrated into the MBSE framework. We explore the impact and consequences of the MBSE Value items discussed above and how they impact the disciplines of quality assurance, reliability and maintainability, system safety, and software assurance. We provide insight into how the MBSE modeling tools can be used to define S&MA processes (ideally as a result of Use Case [1] elaboration of processes represented in MagicDraw®), produce S&MA products (ViewEditor output of various items), and represent S&MA disciplines (S&MA inside of MagicDraw). We also provide insight into the degree to which some elements can be directly integrated into a SysML® model and when, as often happens, an interface to some external source must be provided. The desirability of this future is part of the reason for the NASA Office of Safety and Mission Assurance’s (OSMA) recent creation of a Model Based Mission Assurance (MBMA) Program [2] and the MBMA annual workshops. We briefly summarize the efforts to date to generate S&MA Use Cases for eventual deployment into pilot and project efforts. Even simple use of the SysML modeling tools can be used to capture quality assurance tasks and integrate them with the systems engineering and produce products that are easy to use by quality practitioners that are unfamiliar with these methods. We anticipate finding opportunities to pilot and implement various Quality Assurance (QA) Use Cases in FY20. The MBMA Program is focused on implementation; the NASA Office of the Chief Engineer's Community of Practice, as well as the SmallSat communities, are very interested in the integration of S&MA. Finally, as projects move forward utilizing whatever efficiency increases they can find in a cost-constrained environment, the S&MA community cannot be caught unawares and needs to continue preparing for the ever-growing implementation of MBSE across NASA and our government and commercial partners.

Evans, John W↗

Optimization Algorithms as Quantum Performance Benchmarks

Combinatorial optimization is anticipated to be one of the primary use cases for quantum computation in the coming years. The Quantum Approximate Optimization Algorithm (QAOA) and Quantum Annealing (QA) have the potential to demonstrate significant run-time performance benefits over current state-of-the-art solutions. Using existing methods for characterizing classical optimization algorithms, we analyze solution quality obtained by solving Max-Cut problems using a quantum annealing device and gate-model quantum simulators and devices. This is used to guide the development of an advanced benchmarking framework for quantum computers designed to evaluate the trade-off between run-time execution performance and the solution quality for iterative hybrid quantum-classical applications. The framework generates performance profiles through effective visualizations that show performance progression as a function of time for various problem sizes and illustrates algorithm limitations uncovered by the benchmarking approach. The framework is an enhancement to the existing open-source QED-C Application-Oriented Benchmark suite and can connect to the open-source analysis libraries. The suite can be executed on various quantum simulators and quantum hardware systems.

benchmarking↗

SAGE III/ISS v5.3 Level 2 Data Product Changes and Improvements

The Stratospheric Aerosol and Gas Experiment on the International Space Station (SAGE III/ISS) is an occultation instrument that acquires measurements of aerosols and gases within the Earth’s stratosphere and upper troposphere. SAGE III/ISS provides level 2 solar species products for aerosol extinction (9 channels), nitrogen dioxide (NO2), ozone (O3), and water vapor (H2O). The level 2 products currently provide three O3 profiles based on differing retrievals. The first O3 profile is based on measurements at short wavelengths within the Hartley-Huggins band (MesO3), the second O3 profile is based on measurements made at visible wavelengths within the Chappius band (MLR O3), and the final profile is found using a more SAGE II like approach (AO3). The SAGE III/ISS also provides level 2 lunar species products for ozone (O3), nitrogen dioxide (NO2), and nitrogen trioxide (NO3). Version 5.3 of the SAGE III/ISS retrieval algorithm introduces improvements that affect the level 2 data products. The largest change to the solar algorithm is the implementation of disturbance monitoring package (DMP) corrections to improve pointing accuracy. The DMP is comprised of a miniature inertial measurement unit that measures rotation in inertial space using ring laser gyroscopes oriented about three orthogonal axes which can be used to correct pointing errors caused by mechanical disturbances. A major change common to solar and lunar algorithms is meteorological input from the coarser 42 level Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2) model data to the 72 level MERRA-2 model data. The final major change involves improving the automated quality assurance (QA) algorithm to recover events that were withheld from the public release because differences between the AO3 and MLR O3 for some events after the eruption of Tonga–Hunga Haʻapai. Other changes for v5.3 include minor bug fixes as well as restoration of some data quality flags that were removed in v5.21. This presentation presents the impacts of these changes as well as overall observations of interest from v5.3 level 2 data products.

Carter Hulsey↗

SAGE III/ISS v5.3 Level 2 Data Product Changes and Improvements

The Stratospheric Aerosol and Gas Experiment on the International Space Station (SAGE III/ISS) is an occultation instrument that acquires measurements of aerosols and gases within the Earth’s stratosphere and upper troposphere. SAGE III/ISS provides level 2 solar species products for aerosol extinction (9 channels), nitrogen dioxide (NO2), ozone (O3), and water vapor (H2O). The level 2 products currently provide three O3 profiles based on differing retrievals. The first O3 profile is based on measurements at short wavelengths within the Hartley-Huggins band (MesO3), the second O3 profile is based on measurements made at visible wavelengths within the Chappius band (MLR O3), and the final profile is found using a more SAGE II like approach (AO3). The SAGE III/ISS also provides level 2 lunar species products for ozone (O3), nitrogen dioxide (NO2), and nitrogen trioxide (NO3). Version 5.3 of the SAGE III/ISS retrieval algorithm introduces improvements that affect the level 2 data products. The largest change to the solar algorithm is the implementation of disturbance monitoring package (DMP) corrections to improve pointing accuracy. The DMP is comprised of a miniature inertial measurement unit that measures rotation in inertial space using ring laser gyroscopes oriented about three orthogonal axes which can be used to correct pointing errors caused by mechanical disturbances. A major change common to solar and lunar algorithms is meteorological input from the coarser 42 level Modern-Era Retrospective analysis for Research and Applications, Version 2 (MERRA-2) model data to the 72 level MERRA-2 model data. The final major change involves improving the automated quality assurance (QA) algorithm to recover events that were withheld from the public release because differences between the AO3 and MLR O3 for some events after the eruption of Tonga–Hunga Haʻapai. Other changes for v5.3 include minor bug fixes as well as restoration of some data quality flags that were removed in v5.21. This presentation presents the impacts of these changes as well as overall observations of interest from v5.3 level 2 data products.

Carter Hulsey↗

Assurance Equations: A Cost and Criticality Model for Optimizing Quality Assurance Surveillance

The cost of quality vs cost of failure correction has been a long-running topic of discussion within the Aerospace community. It leads directly to concepts of “risk tolerance”, and risk-based decision-making. It would be valuable if there was a way to compute the optimal investment in customer-executed quality assurance activities using defect significance with respect to performance objectives, the activities’ defect detection effectiveness, and the cost-penalty for late discovery of impactful defects. This optimization is particularly of interest to projects whose budget constraints significantly limit their risk management options.The cost to fix defects (i.e., failure correction) escalates as the project matures. There have been studies attempting to determine the relative cost of fixing defects discovered during various phases of a project life cycle with important implications, all of which suggest growth factors are large. The commonly referred to 1:10:100 rule represents a cost multiplier for repair/rework across the Design to Fab to Test hardware development phases. Cost premiums for QA activities also accumulate when they are treated as mandatory (due to schedule drag) or are performed later than their assigned phase.This paper describes the modeling of development phase -dependencies in the conduct of typical customer-executed quality assurance activities. Our initial modeling encompasses:• Distinct phases of the production lifecycle• Multiple kinds of Defects, each with some a-priori likelihood of being present• Each defect’s impact on performance Objectives for a type of hardware• The cost and efficacy of assurance techniques at detecting such Defects• The costs of fixing those Defects detected in a given phase of the production lifecycleThe model captures assurance activities’ abilities to Detect defects. Upon detection it is assumed that the Defect is immediately fixed. Defects that “escape” detection by some activity may thereafter be detected by a later activity, but by then the cost of fixing the Defect may have escalated. Defects are related to the performance Objectives they would detract from, were those Defects to remain present in the operating system.We have constructed and are exploring, a model that relates the importance of hardware system elements to mission objectives, the impact of types of Defects on those hardware types, the cost of customer-executed assurance activities (i.e., supplier controls) and their effectiveness towards reducing an impactful quality escape, and the cost of Defect correction across production phase. We describe the approach taken to select the key model aspects, why they are relevant to our NASA mission, and our efforts to populate it with relevant and contemporary data. We use a notional example to illustrate model design and function.

Plante, Jeannette↗

Jefferson County Ecological Conservation: Quantifying the Effects of Hydrologic Restoration in the Camas National Wildlife Refuge and Mud Lake Wildlife Management Area

Wetlands in the Camas National Wildlife Refuge and Mud Lake Wildlife Management Area, a rare landscape feature in the Intermountain West, are crucial for migratory birds along the Pacific Flyway, yet these wetlands have experienced a noticeable decline in extent and inundation over the last 40+ years. Partners at the U.S. Fish and Wildlife Service and Idaho Department of Fish and Game have ongoing restoration efforts that are yet to be quantified. To assist these partners, we used NASA Earth observations to quantify change in wetland extent and determine if the restoration projects met the intended impact. The QA_PIXEL band from Landsat 5 Thematic Mapper (TM) and Landsat 8 Operational Land Imager (OLI) was used to identify water. Landsat 8 OLI imagery was also used for classifying landcover in ArcGIS Pro for 2016 and 2020 and to forecast landcover in 2060 using TerrSet’s Land Change Modeler. We also compared lidar data from 2011 and 2019 to detect changes in vegetation height, ground elevation, and surface water levels, and Sentinel-2 Multispectral Instrument (MSI) imagery to detect changes in vegetation health. Classification results indicated overall accuracies over 84% and Kappa indices above 0.80, well surpassing random classification performance. From 2016 to 2020 our classifications showed an 825 acre increase in wetland extent, and our forecasted 2060 model showed a 2795 acre decrease. Combining our partners’ knowledge of the area with our analyses, we created a remote sensing workflow to enhance future monitoring and decision-making. This study demonstrated the feasibility of NASA Earth observations in quantifying wetlands, informing restoration projects, and supporting avian conservation.

Cassidy Bromka↗

Science Quality Assessment in Support of EOS MISR Data

The Multi-angle Imaging SpectroRadiometer (MISR) instrument, scheduled to fly on board the first Earth Observing System (EOS) spacecraft, will have a data stream that produces over 100 GB of data per day, once in full operation, of information about the Earth's atmosphere, clouds, and surface.

remote sensing Multi-angle Imaging SpectroRadiomet↗

JPL's End Item Data Packages (EIDPs)

A viewgraph presentation on JPL's End Item Data Packages (EIDPs) is shown. The topics include: 1) Background and past efforts; 2) Current effort; and 3) First look; 4) Final look; and 5) Next steps.

End Item Data Package (EIDP)↗