Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
NASA NTRS1983

A methodology for collecting valid software engineering data

An effective data collection method for evaluating software development methodologies and for studying the software development process is described. The method uses goal-directed data collection to evaluate methodologies with respect to the claims made for them. Such claims are used as a basis for defining the goals of the data collection, establishing a list of questions of interest to be answered by data analysis, defining a set of data categorization schemes, and designing a data collection form. The data to be collected are based on the changes made to the software during development, and are obtained when the changes are made. To insure accuracy of the data, validation is performed concurrently with software development and data collection. Validation is based on interviews with those people supplying the data. Results from using the methodology show that data validation is a necessary part of change data collection. Without it, as much as 50% of the data may be erroneous. Feasibility of the data collection methodology was demonstrated by applying it to five different projects in two different environments. The application showed that the methodology was both feasible and useful.

Basili, Victor R.↗
DOE OSTI2021

Discovery of AMPX Thermal Scattering Law Processing Issue for Solid Moderators

The 2020 edition of The International Criticality Safety Benchmark Evaluation Project (ICSBEP) Handbook includes a newly produced plastic-moderated evaluation, identified as PU-MET-MIXED-002 and referred to as PMM002. The 2021 edition should include another new plastic-moderated experiment, HEU-MET-THERM-004, referred to as HMT-004. These evaluations are of particular interest, as PMM-002 is moderated with polyethylene, and HMT-004 is moderated with polymethyl methacrylate (Lucite), allowing for investigation of the differences in the thermal scattering laws (TSLs) available for these materials. ENDF/B-VIII.0 includes newly produced data for 1 H-based scattering materials, including Lucite and yttrium hydride. Other available 1 H TSLs include polyethylene, light water, zirconium hydride. The unbound 1 H (free-gas) cross sections were also used. ENDF/B-VIII.0 also includes revisions made to the ENDF/B-VII.1 light water and polyethylene evaluations. Zirconium hydride was unchanged from ENDF/B-VII.1. As of this writing, there are no benchmarks included in the Verified, Archived Library of Inputs and Data (VALID) that are primarily moderated with any solid moderator, so validation that includes the PMM-002 and HMT-004 evaluations would expand the coverage to two new moderators. A study was undertaken at Oak Ridge National Laboratory with two purposes: to provide validation data based on systems that are primarily moderated with polyethylene and Lucite, and to demonstrate the reactivity changes that can result from use of different 1 H TSLs in these plastic-moderated systems. The first objective was mainly to test data and code for SCALE and AMPX, whereas the second objective was to provide useful data for practitioners on the potential variability of predicted $k_{eff}$ based on TSL changes. The use of an exactly correct TSL is often not possible given the materials involved (e.g., lubricants), but this study was intended to provide some indication of the magnitude of the changes among similar materials that can manifest through application of different TSLs in hydrogenous systems over a range of different neutron energy spectra. Unfortunately, the nominal $k_{eff}$ results exposed deviations between SCALE and Monte Carlo N-Particle (MCNP). Some of the results using different TSLs were also unexpected and difficult to explain. An investigation revealed a processing issue in AMPX that was caused by an ambiguous description of the data for incoherent elastic scattering in the ENDF manual. This issue is discussed in detail in a SCALE User Notification, and it affects some solid moderators, but it does not apply to 1 H bound in water. This issue highlights the importance of validating all TSLs used in safety analysis calculations. The remainder of this paper provides a more detailed examination of the results that triggered the investigation and a summary of the findings of that investigation.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
NASA NTRS2004

User-Friendly Interface Developed for a Web-Based Service for SpaceCAL Emulations

A team at the NASA Glenn Research Center is developing a Space Communications Architecture Laboratory (SpaceCAL) for protocol development activities for coordinated satellite missions. SpaceCAL will provide a multiuser, distributed system to emulate space-based Internet architectures, backbone networks, formation clusters, and constellations. As part of a new effort in 2003, building blocks are being defined for an open distributed system to make the satellite emulation test bed accessible through an Internet connection. The first step in creating a Web-based service to control the emulation remotely is providing a user-friendly interface for encoding the data into a well-formed and complete Extensible Markup Language (XML) document. XML provides coding that allows data to be transferred between dissimilar systems. Scenario specifications include control parameters, network routes, interface bandwidths, delay, and bit error rate. Specifications for all satellite, instruments, and ground stations in a given scenario are also included in the XML document. For the SpaceCAL emulation, the XML document can be created using XForms, a Webbased forms language for data collection. Contrary to older forms technology, the interactive user interface makes the science prevalent, not the data representation. Required versus optional input fields, default values, automatic calculations, data validation, and reuse will help researchers quickly and accurately define missions. XForms can apply any XML schema defined for the test mission to validate data before forwarding it to the emulation facility. New instrument definitions, facilities, and mission types can be added to the existing schema. The first prototype user interface incorporates components for interactive input and form processing. Internet address, data rate, and the location of the facility are implemented with basic form controls with default values provided for convenience and efficiency using basic XForms operations. Because different emulation scenarios will vary widely in their component structure, more complex operations are used to add and delete facilities.

Liszka, Kathy J.↗
NASA NTRSDate not supplied

Application of the BSRN Data in Validating the GEWEX SRB Data: Quality-Control and Resulting Available Monthly Means

The BSRN data have been used in validation of the satellite-based GEWEX SRB data, and the recent Release 4.0-IP data cover the 34-year period from July 1983 to June 2017. Here we focus on the validation of the shortwave monthly means from 1992 to 2017 against the BSRN Global 1. We first perform the BSRN-recommended quality-control on the original data. We then calculate the monthly-hourly means from which we calculate the monthly means. Twenty-four (24) monthly-hourly means are required to calculate a monthly mean. To guarantee the quality of the monthly mean, we also require that 95% or more hourly means be available in order for a monthly-hourly mean to be calculated. Since the above quality-control significantly reduces the number of available monthly means, we are curious how many monthly means we would get if we relax the quality-control, and how the resulting monthly means would compare with the GEWEX SRB data. In a version we experimented, only constant upper and lower bounds are imposed on various fluxes, and the original data are largely retained. We found that many more monthly means would be available, and the overall bias and RMS would not be significantly different from that of the strict quality-control. With the strict quality-control, if we let Avail (minimum percentage of available hourly means required for a monthly-hourly mean to be computed) change from 3.2% to 95%, the Bias/RMS/N change from -2.77/20.56/9756 to -0.35/13.41/4581, where N is the number of available monthly means, and Bias and RMS are in the unit of W/sq. m. On the other hand, with the lax quality-control, when Avail changes from 3.2% to 95%, the Bias/RMS/N change from -0.82/19.53/9819 to -0.21/14.41/5875. South Pole (SPO) is one of the sites we are particularly interested in. With the strict quality-control, and when Avail changes from 3.2% to 95%, the Bias/RMS/N change from 5.89/15.85/283 to 1.23/5.52/133; with the lax quality-control, the Bias/RMS/N change from 1.17/11.21/284 to 1.88/7.9/177.

Taiping Zhang↗
DOE OSTI2023

Synthesizing pseudo-Kossel lines from neutron transmission data. II. Validation with neutron diffraction data

A new method to recover the orientation matrix of a single crystal with a known unit cell by analyzing synthesized pseudo-Kossel lines from time-of-flight neutron transmission data has been outlined in a companion article [Dessieux et al. (2023). J. Appl. Cryst. 56, https://dx.doi.org/10.1107/S1600576723001346; referred to here as Article I]. In this work, validation of this new technique is presented by employing experimental neutron transmission and diffraction measurements performed on two copper single-crystal specimens. Time-of-flight spectra were recorded during rotation (ω) of the single crystals about a vertical axis perpendicular to the incident neutron beam. The λ–ω maps recorded in transmission are utilized to determine the crystal orientation with respect to the neutron beam, following the procedure presented in Article I. Further, to validate the indexing procedure, the crystal orientations are compared with those obtained via conventional methods using the diffraction data. The resulting pseudo-Kossel lines across the 2D detectors are also observed for the first time.

36 MATERIALS SCIENCE↗
NASA NTRS2019

Space Communications and Navigation Validation: Extracting Data for the Strategic Center for Networking, Integration, and Communications Scheduling Algorithms

Efficiency in communication system architecture performance between Space Communications and Navigation (SCaN) assets and missions is crucial, as space communication is varied, complex, and often not utilized to its full potential. The SCaN Strategic Center for Networking, Integration, and Communications (SCENIC) new scheduling algorithms, which are designed to simulate the allocation of resources between SCaN assets and missions, have the potential to simulate an increase of this efficiency; however, they require real-world data to be validated against. The purpose of this project was to extract said validation data, which details the frequency and duration of utilized contact windows between missions and assets in the Near Earth Network (NEN), Space Network (SN), and Deep Space Network (DSN). Stored as images in daily operations summaries (DOSs), the tabular data existed in a variety of file formats such as.pdf, .docx, and .doc. Since the tables were stored as images, ABBYY® FineReader® (ABBYY Software Ltd.) optical character recognition (OCR) was implemented, which is a proprietary software that reads images from text. The comma separated value (CSV) output was utilized as input to a series of MATLAB® (The MathWorks, Inc.) methods for reformatting, at which point it was ready to be machine-read. Finally, the results were converted to a Microsoft Excel format for human readability. Along with being used for validation purposes, the data will also be used to map equipment degradation as a function of time to analyze the reliability of network assets.

Kontur, Noah P.↗
DOE OSTI2021

ARC Software Validation Work for the FFTF Reactor

Extensive efforts have been carried out at ANL for the verification and validation of the Argonne Reactor Codes (ARC) software package currently used for the design of Versatile Test Reactor (VTR). The ARC software package consists of steady state neutronics and thermal hydraulics modeling capabilities which are being used by the VTR program to develop most of the VTR reactor design details which will be part of the licensing application. It is anticipated that this software will continue to be used for the design work and for initial operations although additional software may be introduced at a later time. The validation work was focused primarily on obtaining validation data consistent with VTR and usable for the ARC software. Because no critical facilities or operating fast spectrum reactors are available to do experiments for the VTR, the next best option is to identify historical experimental data that can be used as validation data. Early on in VTR, the ZPPR-15 set of experiments was identified as good validation data because of 1) the availability and quality of the data, 2) existing staff that are already familiar with the experimental machine and measurements, 3) most of the ZPPR-15 loadings of interest have already been processed into ARC models, and 4) a full uncertainty quantification has already been done for several loadings of ZPPR-15. The FFTF startup and operations data was identified as the most consistent reactor type that has validation data usable for VTR. Finally, the EBR-II fuel depletion measurements were identified as the best available validation data for VTR. It is important to note that both the FFTF and EBR-II reactors typically come with higher uncertainties than the ZPPR. In the frame of the discussed verification and validation efforts, the present document discusses the analysis of selected FFTF measurements included in the benchmark specifications of the International Reactor Physics Experiment (IRPhE) handbook. The FFTF reactor core configurations from the benchmark specification are presented in Section 2. The analysis is performed with the use of the ARC code suite available at ANL for fast reactor studies and is discussed in Section 3. The reactor parameters from the benchmark include criticality, neutron spectra, effective delayed neutron spectra, control rod worth, isothermal temperature coefficient and low energy gamma-ray spectra. The calculated values and the comparison with the experimental data is discussed in Sections 4 to 9 for each considered reactor parameter. Finally, conclusions are presented in Section 10.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

Why Lady Godiva Should be Replaced as the Default Validation Experiment for U-235 Nuclear Data

For the past seven decades the Lady Godiva benchmark (HEU-MET-FAST-001) has been the primary experiment used for 235 U nuclear data validation. The papers associated with large nuclear data library releases such as ENDF/B-VII.1 and ENDF/B-VIII.0 refer to it frequently. The reasons why HEU-MET-FAST-001 has been used as the primary validation experiment for 235 U nuclear data validation will be discussed in the following section. However, as discussed among the benchmark community, the standards associated with International Criticality Safety Benchmark Evaluation Project (ICSBEP) evaluations have changed throughout time; this is subject of the OECD/NEA WPEC (Working Party on Nuclear Criticality Safety) Subgroup 8. HEU-MET-FAST-001 is an older benchmark (issued during the inaugural year of ICSBEP in 1996 with only minor revisions occurring since then) and (along with many of the other benchmarks from this era) does not meet the standards for a modern benchmark. This work explores why HEU-MET-FAST-001 is useful for 235 U nuclear data validation and discusses other alternative validation experiments.

07 ISOTOPE AND RADIATION SOURCES↗
DOE OSTI2026

Uncertainty quantification of fireball features extracted from nuclear test films using computer vision

Films from the US’s historic nuclear testing era comprise the only extensive collection of imagery depicting high-yield detonations. These films offer unique insights into the characteristics of flows occurring on scales that are difficult to replicate experimentally, and they are a valuable source of data for the validation of models used to describe nuclear detonations. In recent work, we implemented modern computer vision and machine learning techniques to extract features of the fireball following nuclear detonation. With a training dataset of fireball films, we fine-tuned a You Only Look Once 11 (YOLO11) model to detect and track the fireball. Applied to a video, the outer bounding box produced in each frame by YOLO11 is used as an input prompt to Meta’s Segment Anything Model 2 (SAM2), which is shown to accurately predict the boundary of the fireball over time with high resolution. These state-of-the-art computer vision foundation models exhibit impressive visual accuracy in their results but lack an output of values that robustly quantify uncertainty in scientific applications. In this paper, we develop procedures for uncertainty quantification of extracted fireball features. We outline the application of a parallel attention mechanism to calculate uncertainty ranges that complement and better pose model validation data. This higher quality fireball validation data may serve to improve prognostic models describing nuclear detonations in support of nuclear forensic and emergency response activities.

Khristy, Joel [ORNL] (ORCID:0000000209963060)↗
NASA NTRS2000

Reliability Modeling of Microelectromechanical Systems Using Neural Networks

Microelectromechanical systems (MEMS) are a broad and rapidly expanding field that is currently receiving a great deal of attention because of the potential to significantly improve the ability to sense, analyze, and control a variety of processes, such as heating and ventilation systems, automobiles, medicine, aeronautical flight, military surveillance, weather forecasting, and space exploration. MEMS are very small and are a blend of electrical and mechanical components, with electrical and mechanical systems on one chip. This research establishes reliability estimation and prediction for MEMS devices at the conceptual design phase using neural networks. At the conceptual design phase, before devices are built and tested, traditional methods of quantifying reliability are inadequate because the device is not in existence and cannot be tested to establish the reliability distributions. A novel approach using neural networks is created to predict the overall reliability of a MEMS device based on its components and each component's attributes. The methodology begins with collecting attribute data (fabrication process, physical specifications, operating environment, property characteristics, packaging, etc.) and reliability data for many types of microengines. The data are partitioned into training data (the majority) and validation data (the remainder). A neural network is applied to the training data (both attribute and reliability); the attributes become the system inputs and reliability data (cycles to failure), the system output. After the neural network is trained with sufficient data. the validation data are used to verify the neural networks provided accurate reliability estimates. Now, the reliability of a new proposed MEMS device can be estimated by using the appropriate trained neural networks developed in this work.

Perera. J. Sebastian↗
NASA NTRS2001

SIMBIOS Data Product and Algorithm Validation with Emphasis on the Biogeochemical and Inherent Optical Properties

The purpose of our component of the Sensor Intercomparison and Merger for Biological and Interdisciplinary Oceanic Studies (SIMBIOS) program is to address and quantify the relative accuracy of the various ocean color remote sensing products by means of product and algorithm validation. In order to accomplish these goals, we have been collaborating in an international research program in the Gulf of California designed to examine the spatial and temporal bio-optical variability in this region. In addition, we participated in various cruises of opportunity to supplement our validation data set. Our long-term objectives are to: (1) collect optical and biochemical data in oceanic and coastal regions of interest; (2) provide collected data to the Sea-Viewing Wide Field-of-view Sensor (SeaWiFS) Bio-Optical Archive and Storage System (SeaBASS) database, the SIMBIOS project office, and other interested users; (3) maintain and update instrumentation and data archiving and dissemination systems; and (4) determine spatial and temporal error fields for the biological and geophysical data products from the various ocean color missions.

Zaneveld, J. Ronald V.↗
DOE OSTI2023

Summary report of the 4th IAEA Technical Meeting on Fusion Data Processing, Validation and Analysis (FDPVA)

The objective of the Fourth Technical Meeting on Fusion Data Processing, Validation and Analysis was to provide a platform during which a set of topics relevant to fusion data processing, validation and analysis are discussed with the view of extrapolating needs to next step fusion devices such as ITER. The validation and analysis of experimental data obtained from diagnostics used to characterize fusion plasmas are crucial for a knowledge-based understanding of the physical processes governing the dynamics of these plasmas. This paper presents the recent progress and achievements in the domain of plasma diagnostics and synthetic diagnostics data analysis (including image processing, regression analysis, inverse problems, deep learning, machine learning, big data and physics-based models for control) reported at the meeting. The progress in these areas highlight trends observed in current major fusion confinement devices. A special focus is dedicated on data analysis requirements for ITER and DEMO with a particular attention paid to artificial intelligence for automatization and improving reliability of control processes.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
NASA NTRS1986

SAGE 2 satellite data set validation

The results of a validation study of data obtained by the Stratospheric Aerosol and Gas Experiment 2 satellite experiment (SAGE 2) are given. Preliminary SAGE 2 data have been available for the period October, 1984 to May, 1985. In addition, the results of two correlative experimental measurement series have been studied in detail, as well as climatological data obtained by other techniques, including ground-based and airborne lidar. The study shows the SAGE 2 data to be of great potential value to studies of the microphyiscs of stratospheric aerosols, the chemistry of trace gases and stratospheric dynamics. A small number of unidentified errors in the current preliminary data set are described. These will be removed from the next version of the data set which is anticipated to be of archival quality.

Kent, G. S.↗
NASA NTRS1993

Experimental evaluation of certification trails using abstract data type validation

Certification trails are a recently introduced and promising approach to fault-detection and fault-tolerance. Recent experimental work reveals many cases in which a certification-trail approach allows for significantly faster program execution time than a basic time-redundancy approach. Algorithms for answer-validation of abstract data types allow a certification trail approach to be used for a wide variety of problems. An attempt to assess the performance of algorithms utilizing certification trails on abstract data types is reported. Specifically, this method was applied to the following problems: heapsort, Hullman tree, shortest path, and skyline. Previous results used certification trails specific to a particular problem and implementation. The approach allows certification trails to be localized to 'data structure modules,' making the use of this technique transparent to the user of such modules.

Wilson, Dwight S.↗
NASA NTRS2006

Analysis of Raman Lidar and Radiosonde Measurements from the AWEX-G Field Campaign and Its Relation to Aqua Validation

Early work within the Aqua validation activity revealed there to be large differences in water vapor measurement accuracy among the various technologies in use for providing validation data. The validation measurements were made at globally distributed sites making it difficult to isolate the sources of the apparent measurement differences among the various sensors, which included both Raman lidar and radiosonde. Because of this, the AIRS Water Vapor Experiment-Ground (AWEX-G) was held in October-November 2003 with the goal of bringing validation technologies to a common site for intercomparison and resolving the measurement discrepancies. Using the University of Colorado Cryogenic Frostpoint Hygrometer (CFH) as the water vapor reference, the AWEX-G field campaign permitted correction techniques to be validated for Raman lidar, Vaisala RS80-H and RS90/92 that significantly improve the absolute accuracy of water vapor measurements from these systems particularly in the upper troposphere. Mean comparisons of radiosondes and lidar are performed demonstrating agreement between corrected sensors and the CFH to generally within 5% thereby providing data of sufficient accuracy for Aqua validation purposes. Examples of the use of the correction techniques in radiance and retrieval comparisons are provided and discussed.

Whiteman, D. N.↗