Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Verification of data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,603 records · Page 89

Human Reliability Assessments: Using the Past (Shuttle) to Predict the Future (Orion)

NASA (National Aeronautics and Space Administration) Johnson Space Center (JSC) Safety and Mission Assurance (S&MA) uses two human reliability analysis (HRA) methodologies. The first is a simplified method which is based on how much time is available to complete the action, with consideration included for environmental and personal factors that could influence the human's reliability. This method is expected to provide a conservative value or placeholder as a preliminary estimate. This preliminary estimate or screening value is used to determine which placeholder needs a more detailed assessment. The second methodology is used to develop a more detailed human reliability assessment on the performance of critical human actions. This assessment needs to consider more than the time available, this would include factors such as: the importance of the action, the context, environmental factors, potential human stresses, previous experience, training, physical design interfaces, available procedures/checklists and internal human stresses. The more detailed assessment is expected to be more realistic than that based primarily on time available. When performing an HRA on a system or process that has an operational history, we have information specific to the task based on this history and experience. In the case of a Probabilistic Risk Assessment (PRA) that is based on a new design and has no operational history, providing a "reasonable" assessment of potential crew actions becomes more challenging. In order to determine what is expected of future operational parameters, the experience from individuals who had relevant experience and were familiar with the system and process previously implemented by NASA was used to provide the "best" available data. Personnel from Flight Operations, Flight Directors, Launch Test Directors, Control Room Console Operators and Astronauts were all interviewed to provide a comprehensive picture of previous NASA operations. Verification of the assumptions and expectations expressed in the assessments will be needed when the procedures, flight rules and operational requirements are developed and then finalized.

DeMott, Diana↗

Human Reliability Assessments: Using the Past (Shuttle) to Predict the Future (Orion)

NASA (National Aeronautics and Space Administration) Johnson Space Center (JSC) Safety and Mission Assurance (S&MA) uses two human reliability analysis (HRA) methodologies. The first is a simplified method which is based on how much time is available to complete the action, with consideration included for environmental and personal factors that could influence the human's reliability. This method is expected to provide a conservative value or placeholder as a preliminary estimate. This preliminary estimate or screening value is used to determine which placeholder needs a more detailed assessment. The second methodology is used to develop a more detailed human reliability assessment on the performance of critical human actions. This assessment needs to consider more than the time available, this would include factors such as: the importance of the action, the context, environmental factors, potential human stresses, previous experience, training, physical design interfaces, available procedures/checklists and internal human stresses. The more detailed assessment is expected to be more realistic than that based primarily on time available. When performing an HRA on a system or process that has an operational history, we have information specific to the task based on this history and experience. In the case of a Probabilistic Risk Assessment (PRA) that is based on a new design and has no operational history, providing a "reasonable" assessment of potential crew actions becomes more challenging. To determine what is expected of future operational parameters, the experience from individuals who had relevant experience and were familiar with the system and process previously implemented by NASA was used to provide the "best" available data. Personnel from Flight Operations, Flight Directors, Launch Test Directors, Control Room Console Operators, and Astronauts were all interviewed to provide a comprehensive picture of previous NASA operations. Verification of the assumptions and expectations expressed in the assessments will be needed when the procedures, flight rules, and operational requirements are developed and then finalized.

DeMott, Diana L.↗

Benchmarking the Collocation Stand-Alone Library and Toolkit (CSALT)

This paper describes the processes and results of Verification and Validation (V&V) efforts for the Collocation Stand Alone Library and Toolkit (CSALT). We describe the test program and environments, the tools used for independent test data, and comparison results. The V&V effort employs classical problems with known analytic solutions, solutions from other available software tools, and comparisons to benchmarking data available in the public literature. Presenting all test results are beyond the scope of a single paper. Here we present high-level test results for a broad range of problems, and detailed comparisons for selected problems.

optimal control↗

Towards Verification and Validation for Increased Autonomy

This presentation goes over the work we have performed over the last few years on verification and validation of the next generation onboard collision avoidance system, ACAS X, for commercial aircraft. It describes our work on probabilistic verification and synthesis of the model that ACAS X is based on, and goes on to the validation of that model with respect to actual simulation and flight data. The presentation then moves on to identify the characteristics of ACAS X that are related to autonomy and to discuss the challenges that autonomy pauses on VV. All work presented has already been published.

Giannakopoulou, Dimitra↗

Towards Verification and Validation for Increased Autonomy

This presentation goes over the work we have performed over the last few years on verification and validation of the next generation onboard collision avoidance system, ACAS X, for commercial aircraft. It describes our work on probabilistic verification and synthesis of the model that ACAS X is based on, and goes on to the validation of that model with respect to actual simulation and flight data. The presentation then moves on to identify the characteristics of ACAS X that are related to autonomy and to discuss the challenges that autonomy pauses on VV. All work presented has already been published.

Giannakopoulou, Dimitra↗

Short Arc Orbit Determination for Altimeter Calibration and Validation on TOPEX/Poseidon

TOPEX/Poseidon (T/P) is a joint mission of US NASA and French CNES design launched August 10, 1992. It carries two radar altimeters which alternately share a common antenna. There are two project designated verification sites, a NASA site off the coast at Pt. Conception, CA, and a CNES site near Lampedusa Island in the Mediterranean Sea. Altimeter calibration and validation for T/P is performed over these highly instrumented sites by comparing the spacecraft's altimeter radar range to computed range based on in situ measurements which include the estimated orbit position. This paper presents selected results of orbit determination over each of these sites to support altimeter verification. A short arc orbit determination technique is used to estimate a locally accurate position determination of T/P from less than one revolution of satellite laser ranging (SLR) data. This technique is relatively insensitive to gravitational and nongravitational force modeling errors and is therefore essentially geometrical. The quality of these short arc orbits is demonstrated by covariance analysis and by comparison to orbits determined from longer arcs of data and other tracking data types, such as DORIS and Global Positioning System Demonstration Receiver (GPSDR) data.

radar↗

Column Optical Depth (COD) Derived from CALIOP Ocean Surface Returns

The Cloud Aerosol Lidar with Orthogonal Polarization (CALIOP) on board the Cloud Aerosol Lidar Infrared Pathfinder Satellite Observations (CALIPSO) satellite has been making near-global measurements of clouds and aerosols since mid-June 2006. Among the properties reported in CALIPSO data products are estimates of total column optical depth (COD) obtained by integrating the retrieved extinction coefficients of detected particulates (i.e., aerosols and clouds) from 36 km to the surface. However, due to algorithm detection limits and instrument sensitivities, particulates can go undetected when the particulate loading is especially diffuse or in regions where the signal is attenuated by over-lying layers. Because extinction is not calculated where features are not detected, these undetected particulates can introduce low biases into the reported COD. Lidar ratio assumptions used in the extinction retrievals can also introduce errors. To minimize these biases, future releases of the CALIPSO lidar data products will implement an ocean-derived column optical depth (ODCOD) retrieval. The following paper briefly describes the algorithm, the current status of the algorithm development and verification efforts, and preliminary comparisons to col-located COD estimates derived using other retrieval schemes and obtained from other sensors.

Robert A Ryan↗

Novel Test and Analysis Methodology for the Assessment of Joint under Re-entry Environment

Spacecraft thermal protection systems require unique approaches for design verification due to the extreme environments seen during space travel and atmospheric re-entry. In this work, unique considerations for design, analysis, and verification of a tiled heatshield design with material nonlinearity, material property temperature-dependency, material property rate-dependency, and geometric stress concentrations are presented. Such designs require a thorough application of the building block approach to derive appropriate material property datasets, and a novel application of analysis models to aid in the structural verification with testing. The custom application of material rate-dependency in finite element analysis software is presented as a means of leveraging the rate-dependent test data for the material property dataset. Because bonded tile heatshields contain so many design considerations and nonlinearities which often preclude the ability to easily achieve the requisite test factors in qualification testing, a novel sub-element test was developed that amplifies the stress singularity beyond flight conditions. The test achieves an over-test over flight stress conditions. The approach for sub-element coupon design can be extended to other bonded joint designs and can be used in aiding a qualification program.

Pavel Babuska↗

Designing a Flight Test of a Flight Path Management System for Advanced Air Mobility Research

The National Aeronautics and Space Administration (NASA) has completed a flight test to evaluate the performance of an onboard prototype automation system operating in future high density urban airspace. The test was part of a research investigation of the Urban Air Mobility (UAM) concept, with a focus on a future environment having hundreds of simultaneous operations over a metropolitan area. The complexity of this future UAM airspace may require automation capable of replanning an aircraft’s path in the presence of traffic and other changing constraints. A live-virtual-constructive (LVC) approach was used to conduct the test. Prototype automation technology was integrated into one of the two live aircraft, which were combined with virtual traffic to create a mixed reality environment at the target airspace density. In-flight evaluation enabled verification of the automation’s functions and discovery of any unexpected behaviors resulting from its operation in an actual flight environment. The in-flight evaluation also provided data for validation of air traffic simulations. This paper discusses the design, methodology, and challenges overcome to conduct a successful flight test. Remaining challenges, future work, and recommendations to improve the flight test capability are also discussed.

Advanced Air Mobility↗

Integrated Modeling for Payload Test of the Roman Space Telescope

The Nancy Grace Roman Telescope (RST) is a NASA observatory designed to unravel the secrets of dark energy and dark matter, search for and image exoplanets, and explore many topics in infrared optics. Scheduled to launch in no earlier than October 2026, this 2.4 meter aperture telescope has a field of view 100 times greater than the Hubble Space Telescope. The mission is currently in its construction phase, where integrated modeling between thermal, structural, and optical models of the observatory is necessary to demonstrate science quality images over the range of operational parameters. This presentation discusses the most recent integrated modeling analysis cycle for Roman, including model correlation with our instrument level testing. We include a discussion on improved processes of the handling of the various flows of data between the modeling disciplines and discipline specific monte-carlo analysis predictions. We will finish with the predicted uncertainties and expected performance for our upcoming observatory alignment verification test using machine learning algorithms.

telescope↗

A REDUCED ORDER MODELING APPROACH TO PROBABILISTIC CREEP-DAMAGE PREDICTIONS IN FINITE ELEMENT ANALYSIS

This paper introduces a computationally efficient Reduced Order Modeling (ROM) approach for the probabilistic prediction of creep-damage failure. Component-level probabilistic simulations are needed to assess the reliability and safety of high-temperature components. Full-scale probabilistic creep-damage modeling in finite element (FE) approach is computationally expensive requiring many hundreds of simulations to replicate the uncertainty of component failure. To that end, ROM is proposed to minimize the elevated computational cost while controlling the loss of accuracy. It is proposed that full-scale probabilistic simulations can be completed in 1D at a reduced cost, the extremum conditions extracted, and those conditions applied for lower-cost 2D/3D probabilistic simulations of components that capture the mean and uncertainty of failure. The probabilistic Sine-hyperbolic (Sinh) model is selected which in previous work was calibrated to alloy 304 stainless steel. The Sinh model includes probability density functions (pdfs) for test condition (stress and temperature), initial damage (i.e. microstructure), and material properties uncertainty. The Sinh model is programmed into ANSYS finite element software using the USERCREEP.F material subroutine. First, the Sinh model and FE code are subject to verification and validation to ensure the accuracy of the simulations. Numerous Monte Carlo simulations are executed in a 1D model to generate probabilistic creep deformation, damage, and rupture data. This data is analyzed and the probabilistic parameters corresponding to extreme creep response are extracted. The ROM concept is applied where only the extreme conditions are applied in the 2D probabilistic prediction of a component. The probabilistic predictions between the 1D and 2D geometry is compared to assess ROM for creep. The accuracy of the probabilistic prediction employing the ROM approach will potentially reduce the time and cost of simulating complex engineering systems. Future studies will introduce multi-stage Sinh, stochasticity, and spatial uncertainty for improved prediction.

36 MATERIALS SCIENCE↗

Verifying an interactive consistency circuit: A case study in the reuse of a verification technology

The work done at ORA for NASA-LRC in the design and formal verification of a hardware implementation of a scheme for attaining interactive consistency (byzantine agreement) among four microprocessors is presented in view graph form. The microprocessors used in the design are an updated version of a formally verified 32-bit, instruction-pipelined, RISC processor, MiniCayuga. The 4-processor system, which is designed under the assumption that the clocks of all the processors are synchronized, provides software control over the interactive consistency operation. Interactive consistency computation is supported as an explicit instruction on each of the microprocessors. An identical user program executing on each of the processors decides when and on what data interactive consistency must be performed. This exercise also served as a case study to investigate the effectiveness of reusing the technology which was developed during the MiniCayuga effort for verifying synchronous hardware designs. MiniCayuga was verified using the verification system Clio which was also developed at ORA. To assist in reusing this technology, a computer-aided specification and verification tool was developed. This tool specializes Clio to synchronous hardware designs and significantly reduces the tedium involved in verifying such designs. The tool is presented and how it was used to specify and verify the interactive consistency circuit is described.

Bickford, Mark↗

Time-Lapse Integration at FWU: Fluids, Rock Physics, Numerical Model Integration, and Field Data Comparison

We present the current status of time-lapse seismic integration at the Farnsworth (FWU) CO2 WAG (water-alternating-gas) EOR (Enhanced Oil Recovery) project at Ochiltree County, northwest Texas. As a potential carbon sequestration mechanism, CO2 WAG projects will be subject to some degree of monitoring and verification, either as a regulatory requirement or to qualify for economic incentives. In order to evaluate the viability of time-lapse seismic as a monitoring method the Southwest Partnership (SWP) has conducted time-lapse seismic monitoring at FWU using the 3D Vertical Seismic Profiling (VSP) method. The efficacy of seismic time-lapse depends on a number of key factors, which vary widely from one application to another. Most important among these are the thermophysical properties of the original fluid in place and the displacing fluid, followed by the petrophysical properties of the rock matrix, which together determine the effective elastic properties of the rock fluid system. We present systematic analysis of fluid thermodynamics and resulting thermophysical properties, petrophysics and rock frame elastic properties, and elastic property modeling through fluid substitution using data collected at FWU. These analyses will be framed in realistic scenarios presented by the FWU CO2 WAG development. The resulting fluid/rock physics models will be applied to output from the calibrated FWU compositional reservoir simulation model to forward model the time-lapse seismic response. Modeled results are compared with field time-lapse seismic measurements and strategies for numerical model feedback/update are discussed. While mechanical effects are neglected in the work presented here, complementary parallel studies are underway in which laboratory measurements are introduced to introduce stress dependence of matrix elastic moduli.

4D↗

Subtask 1.3 – Integrated Carbon Capture and Storage for North Dakota Ethanol Production

The Energy & Environmental Research Center (EERC), in partnership with Red Trail Energy, LLC (RTE), a North Dakota ethanol producer; the North Dakota Industrial Commission (NDIC) Renewable Energy Program (REP); and the U.S. Department of Energy (DOE), conducted a feasibility and implementation study for a commercial carbon capture and storage (CCS) effort. This subtask provided technical support and developed recommended practices of how small-scale industrial CO2 emitters (<1,000,000 tonnes of CO2 emitted annually) may economically deploy CCS. The 64-million-gallon dry mill RTE ethanol facility, emitting an average 180,000 tonnes of CO2 annually, was the subject of the case study. The positive outcome of this research, which shows technical and economic potential for ethanol–CCS in North Dakota, has resulted in RTE acquiring an approved Permit to Drill on December 2, 2019, for a stratigraphic test well in early 2020, a necessary step toward a North Dakota CO2 Storage Facility Permit (SFP) for the RTE CCS effort. Outcomes include 1) validating the Broom Creek Formation as a regional target for CCS, 2) determining the full carbon life cycle of an industrial fuel production facility with CCS, 3) developing a field implementation plan (FIP) for small-scale CCS, and 4) determining the validity and pathway for using CCS to meet low-carbon fuel (LCF) standards. The technical team included the EERC, RTE, Trimeric Corporation, Schlumberger Carbon Services, and Computer Modelling Group (CMG). Findings from activities conducted November 2016 – May 2020 are summarized as follows. Several key steps have been accomplished toward validating the Broom Creek Formation as a CO2 injection and storage target at the RTE CCS site, including verification of the presence and structure of sandstone layers that may comprise the potential CO2 storage reservoir and the several thousand feet of overlying confining zone. Work included site characterization using existing data, interpretation of seismic data within the study area, and geologic modeling simulation of CO2 injection. Interpreted results estimate 3000 feet of confining zone between the Broom Creek Formation (storage target) and the lowermost underground source of drinking water (e.g., the Fox Hills Formation). The thickness of the Broom Creek injection target varies 230–420 ft within the survey area. Results were used to inform the location of a stratigraphic test well and associated characterization test program. No impediments were identified within the targeted CO2 storage complex that would prevent the project from moving forward. A stratigraphic test well is the next step to validate these results and to acquire remaining data necessary to develop a North Dakota CO2 SFP application. The EERC generated full carbon life cycle estimates for CCS integration with the RTE ethanol facility. Results indicated that an average 40% reduction in CO2 emissions is possible through CCS implementation. Approximately half of the carbon in the overall life cycle for a dry mill ethanol plant is generated through the fermentation process and emitted to the atmosphere; this is the CO2 stream targeted for CCS. The remaining carbon is attributed to corn feedstock farming (diesel, fertilizer), energy for fuel processing (natural gas, electricity), and transportation (diesel) of the fuel product. Life cycle carbon estimates are also affected by the anticipated energy consumption of a potential capture facility, which depend on the type of CO2 product generated. For example, a 30%–40% net CO2 emission reduction is estimated if a liquefied CO2 facility were incorporated compared to a 40%–50% net CO2 reduction if a supercritical “injection-grade” CO2 product is generated; i.e., more energy is required to further refine the CO2 stream, affecting the full carbon life cycle estimates. A CCS FIP was developed and initiated at the RTE CCS site, resulting in the development of several guidance documents: a CO2 Capture Process Design Package, a North Dakota CO2 geologic Storage Permits Template, and a Public Outreach Package for CCS in North Dakota. General FIP components include CO2 capture system, pipeline and well designs; monitoring, verification, and accounting (MVA) plans; geologic characterization and testing programs; and permitting and outreach plans. Vendor bids were also acquired for the CO2 liquefaction facility. Near-surface characterization (groundwater and soil gas sampling) and geologic characterization (seismic survey) were initiated to inform development of a UIC Class VI-compliant MVA plan compliant with a North Dakota CO2 SFP. Designs (well and geologic testing) were completed for a stratigraphic test well compliant with a North Dakota CO2 SFP. In addition, the outreach plan was executed, including community open houses, meetings with city/county/state officials, and development of public materials. Although other entities continued to mature incentive programs in 2019–2020, California and the Internal Revenue Service (IRS) currently provide the most advanced economic opportunities for CCS integrated with fuel production. The California Low-Carbon Fuel Standard (LCFS) adopted a CCS Protocol in January 2019, allowing submittal of a design-based pathway (DBP) application for an approved temporary (not certified) carbon intensity value for a fully engineered facility. An ethanol–CCS DBP application to the California LCFS Program (officially approved February 28, 2020) was developed to show that the RTE CCS effort meets LCFS requirements. The approved DBP provides confidence to advance the project and supports potential investments. Other entities continue to mature incentive programs. The IRS issued guidance in February 2020 that addresses the definition of beginning of construction and revenue procedure on partnerships for the Enhancement of Carbon Dioxide Sequestration Credit (a.k.a. Section 45Q) CCS tax credit program; the IRS anticipates issuing further guidance on issues such as secure geologic storage, utilization qualifications, and recapture of claimed credits. Maturing incentive programs coupled with workable permitting regulations provide confidence to advance CCS projects in North Dakota and support financial investment to proceed with designing, constructing, and implementing CCS projects at small-scale fuel production facilities. The largest hurdles for CCS implementation at small-scale industrial systems are often business/economic-related (i.e., not technical). Market uncertainty already exists for agriculture-based alternative fuels such as corn ethanol, for which production has increased by ~33%, and prices have correspondingly lowered since 2015. Passing the Section 45Q tax credit program improves economic feasibility for CCS but may require external investors for a small business to achieve maximum benefits. Public–private partnerships with NDIC and DOE have resulted in foundational technical and regulatory knowledge, growing stakeholder confidence, and a pathway to implementation that enables similar industrial CCS projects in the region to advance. This subtask was funded through the EERC–DOE Joint Program on Research and Development for Fossil Energy-Related Resources Cooperative Agreement No. DE-FE0024233. Nonfederal funding was provided by NDIC and RTE. The authors would also like to thank CMG, ESRI, IHS, Neuralog, and Schlumberger for allowing the use of their software packages in support of this work.

42 ENGINEERING↗

Optical Testing and Verification Methods for the James Webb Space Telescope Integrated Science Instrument Module Element

NASA's James Webb Space Telescope (JWST) is a 6.6m diameter, segmented, deployable telescope for cryogenic IR space astronomy (~40K). The JWST Observatory includes the Optical Telescope Element (OTE) and the Integrated Science Instrument Module (ISIM) that contains four science instruments (SI) and the fine guider. The SIs are mounted to a composite metering structure. The SI and guider units were integrated to the ISIM structure and optically tested at the NASA Goddard Space Flight Center as a suite using the Optical Telescope Element SIMulator (OSIM). OSIM is a full field, cryogenic JWST telescope simulator. SI performance, including alignment and wave front error, were evaluated using OSIM. We describe test and analysis methods for optical performance verification of the ISIM Element, with an emphasis on the processes used to plan and execute the test. The complexity of ISIM and OSIM drove us to develop a software tool for test planning that allows for configuration control of observations, associated scripts, and management of hardware and software limits and constraints, as well as tools for rapid data evaluation, and flexible re-planning in response to the unexpected. As examples of our test and analysis approach, we discuss how factors such as the ground test thermal environment are compensated in alignment. We describe how these innovative methods for test planning and execution and post-test analysis were instrumental in the verification program for the ISIM element, with enough information to allow the reader to consider these innovations and lessons learned in this successful effort in their future testing for other programs.

spaceborne telescopes↗

Assessment of Sensor Data Accuracy within Gazebo/ROS for High-Precision Autonomous In-Space Robotic Operations

Modeling high-precision in-space servicing, assembly, and manufacturing operations in a simulated environment is a critical step in the development of robotic systems that will be used to autonomously assemble large-scale structures in space. Limited facility size and high costs for manufacturing prototypes make it challenging to conduct full-scale operational testing under appropriate environmental conditions; therefore, testing in a modular, high-fidelity simulation environment is necessary for verification and validation of technology and architecture designs prior to launch. Several modeling and simulation environments exist both within NASA and industry that can be used to test robotic system design and operations, including the widely used commercial tool Gazebo integrated with Robotic Operating System software. Because the performance of autonomous robotic systems relies heavily on the quality of sensor input data, this paper focuses on assessing the accuracy of pose data from an optical sensor model in the Gazebo environment against the behavior of real hardware. The results of the tests will help developers using Gazebo for large-scale, high-precision simulation to account for modeling inaccuracies within their robotic control system algorithms.

simulation↗

Advanced Inversion Algorithms for Scientific Data Analysis [Slides]

Accurate subsurface characterization is crucial for all subsurface energy exploration. Accurate characterization of uncertain subsurface properties is also critical for monitoring storage of CO 2 , estimating pathways of subsurface contaminant transport, and monitoring potential nuclear explosions for treaty verification. This research will advance our world-leading subsurface sensing capabilities that are crucial for LANL missions in energy security (geothermal energy, oil/gas resource exploration, geologic carbon storage) and nuclear security (facility monitoring, detonation detection).

47 OTHER INSTRUMENTATION↗

Verification and Validation of Performance with Dissemination of Best Practices in District Energy and CHP for Enhanced Resiliency, Energy Efficiency, and Cybersecurity

This report contains the results of the International District Energy Association’s work to analyze, validate, and verify performance data of existing district energy systems and identify industy best practices for the purpose of improving system reliability, resiliency, and efficiency, and to accellerate decarbonization. In addition to a technical evaluation of the surveyed systems and identification of a series of technical performance metrics, the report illustrates the accompanying operations and financial best practices employed by surveyed systems to fully serve their customer base. Additionally, the third chapter of the report describes the current landscape of cybersecurity threats and counteracting measures, and recommends a series of steps for effectively guarding highly networked district energy systems against cybersecurity attacks.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗