Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “validation method”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

A SIMPLIFIED ENERGY MODELING APPROACH FOR BUILDINGS

This paper introduces simplified energy modeling (SEM) approaches using DOE2 and EnergyPlus and proposes a validation method for simplified energy modeling. Simplified energy modeling can allow more design teams and owners to leverage the power of energy modeling to inform their designs and lead to greater energy savings and a lower first cost because the technologies and design strategies can be thoroughly tested against models before the decisions are made. By starting a discussion on how to develop simplified energy modeling approaches and methods to validate results, we hope to increase innovation in energy modeling software while decreasing variation between modelers. Simplified modeling is not yet widely accepted because it is unclear if the simplifications reduce the accuracy of the results. By starting an industry dialogue on a testing method for these modeling approaches, we hope to aid adoption of more streamlined tools when they can be shown to provide similar accuracy to more detailed tools. This paper proposes a method to test simplified energy modeling programs against the DOE prototype buildings. It builds on the testing procedure established for simulation engines by ASHRAE Standard 140. By comparing whole building and end use results from the accepted prototype buildings to any simplified modeling approach across multiple building types and climate zones we can determine if the simplifications create unwarranted error. We will present the SEM approach and results for a case study validation for a sample building types in 16 US climate zones from two such simplified energy modeling tools.

Energy Modeling, 90.1 Appendix G, Asset Score Tool↗
DOE OSTI2025

Lilith: An Enduring Plutonium Critical Assembly [Slides]

A design of an enduring Pu critical assembly has been produced. This is very important to DOE/NNSA missions. This assembly would result in advances/improvements in nuclear data validation, analytical methods validation, dosimetry, reactor kinetics, materials, and more.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
NASA NTRS2011

An Investigation Into HPLC Data Quality Problems

This report summarizes the analyses and results produced by a five-member investigative team of Government, university, and industry experts, established by NASA HQ. The team examined data quality problems associated with high performance liquid chromatography (HPLC) analyses of pigment concentrations in seawater samples produced by the San Diego State University (SDSU) Center for Hydro-Optics and Remote Sensing (CHORS). This report shows CHORS did not validate the methods used before placing them into service to analyze field samples for NASA principal investigators (PIs), even though the HPLC literature contained easily accessible method validation procedures, and the importance of implementing them, more than a decade ago. In addition, there were so many sources of significant variance in the CHORS methodologies, that the HPLC system rarely operated within performance criteria capable of producing the requisite data quality. It is the recommendation of the investigative team to a) not correct the data, b) make all the data that was temporarily sequestered available for scientific use, and c) label the affected data with an appropriate warning, e.g., "These data are not validated and should not be used as the sole basis for a scientific result, conclusion, or hypothesis--independent corroborating evidence is required."

Hooker, Stanford B.↗
DOE OSTI2024

Why the US Needs an Enduring Plutonium Critical Assembly

Jezebel (operated from 1954-1977) has been the primary experiment for fast 239 Pu nuclear data validation for the last 70 years. Validation has included not only k eff , but also spectral indicies, Rossi-α, reactivity coefficients, and neutron leakage spectra. While this has been incredibly valuable to the community, there are three major issues. The first issue is that documentation on many of these experiments were lacking, leading to large uncertainties or (even worse) incorrect assumptions. The second issue is that while there have been many advances in research, there is no way to test those new advances today. The last issue is that since the assembly only operated for 23 years (and at time when 30 other critical assemblies were operating at the same facility and nuclear weapons testing was occurring), there were limited opportunities to observe how any system parameters changed as a function of time. Note that this work is not suggesting that a "Jezebel re placement" used for a limited experiment campaign would have great value. A new enduring (100 year target) plutonium (Pu) assembly with simple geometry and low uncertainties, however, would be extremely valuable. Such a capability would have a transformative impact on many research areas including nuclear data validation, analytical methods validation, dosimetry, reactor kinetics, and materials. The need for a new plutonium assembly is not new: it has been in the DOE Nuclear Criticality Safety Program (NCSP) Mission and Vision for over 10 years and was also the top priority established at the 2022 National Criticality Experiments Research Center (NCERC) Futures meeting. This work will discuss how such a new capability would help ensure that the US retains international leadership in plutonium research. Last, a brief overview of Lilith, a project aimed to design a new enduring plutonium assembly for operation at the NCERC will be given.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2020

Resource for the Development of Biomedical Accelerator Mass Spectrometry (AMS) Final Report

The NIH Research Resource for Biomedical AMS was originally funded at Lawrence Livermore National Laboratory in 1999 to develop and apply the technology of accelerator mass spectrometry (AMS) in broad- based biomedical research. The Resource’s niche is to fill needs for ultra high sensitivity quantitation when isotope-labeled agents are used. The Research Resource’s Technology Research and Development (TR&D) efforts will focus on the needs of the biomedical research community in the context of seven Driving Biomedical Projects (DBPs) that will drive the Center’s technical capabilities through three core TR&Ds. We will expand our present capabilities by developing a fully integrated HPLC AMS to increase our capabilities for metabolic measurements, we will develop methods to understand cellular processes and we will develop and validate methods for the application of AMS in human studies, which is a growing area of demand by collaborators and service users. In addition, we will continue to support new and ongoing collaborative and service projects that require the capabilities of the Resource. The Center will continue to train researchers in the use of the AMS capabilities being developed, and the results of all efforts will be widely disseminated to advance progress in biomedical research. Towards these goals, our specific aims are to:1.) Increase the value and information content of AMS measurements by combining molecular speciation with quantitation of defined macromolecular isolates. Specifically, develop and validate methods for macromolecule labeling, characterization and quantitation.2.) Develop and validate methods and strategies to enable AMS to become more broadly used in human studies. Specifically, demonstrate robust methods for conducting pharmacokinetic/pharmacodynamics studies in humans and model systems.3.) Increase the accessibility of AMS to the Biomedical research community and the throughput of AMS through direct coupling to separatory instruments.4.) Provide high throughput 14C BioAMS analysis for collaborative and service clients.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS2020

Spatial validation reveals poor predictive performance of large-scale ecological mapping models

Mapping aboveground forest biomass is central for assessing the global carbon balance. However, current large-scale maps show strong disparities, despite good validation statistics of their underlying models. Here, we attribute this contradiction to a flaw in the validation methods, which ignore spatial autocorrelation (SAC) in data, leading to overoptimistic assessment of model predictive power. To illustrate this issue, we reproduce the approach of large-scale mapping studies using a massive forest inventory dataset of 11.8 million trees in central Africa to train and validate a random forest model based on multispectral and environmental variables. A standard nonspatial validation method suggests that the model predicts more than half of the forest biomass variation, while spatial validation methods accounting for SAC reveal quasi-null predictive power. This study underscores how a common practice in big data mapping studies shows an apparent high predictive power, even when predictors have poor relationships with the ecological variable of interest, thus possibly leading to erroneous maps and interpretations.

biomass↗
DOE OSTI2024

To have value, comparisons of high-throughput phenotyping methods need statistical tests of bias and variance

The gap between genomics and phenomics is narrowing. The rate at which it is narrowing, however, is being slowed by improper statistical comparison of methods. Quantification using Pearson’s correlation coefficient ( r ) is commonly used to assess method quality, but it is an often misleading statistic for this purpose as it is unable to provide information about the relative quality of two methods. Using r can both erroneously discount methods that are inherently more precise and validate methods that are less accurate. These errors occur because of logical flaws inherent in the use of r when comparing methods, not as a problem of limited sample size or the unavoidable possibility of a type I error. A popular alternative to using r is to measure the limits of agreement (LOA). However both r and LOA fail to identify which instrument is more or less variable than the other and can lead to incorrect conclusions about method quality. An alternative approach, comparing variances of methods, requires repeated measurements of the same subject, but avoids incorrect conclusions. Variance comparison is arguably the most important component of method validation and, thus, when repeated measurements are possible, variance comparison provides considerable value to these studies. Statistical tests to compare variances presented here are well established, easy to interpret and ubiquitously available. The widespread use of r has potentially led to numerous incorrect conclusions about method quality, hampering development, and the approach described here would be useful to advance high throughput phenotyping methods but can also extend into any branch of science. The adoption of the statistical techniques outlined in this paper will help speed the adoption of new high throughput phenotyping techniques by indicating when one should reject a new method, outright replace an old method or conditionally use a new method.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRSDate not supplied

A Method for Validating Causal Diagrams of Human Health Risk in Space Flight

The complexity of cause-and-effect relationships between spaceflight hazards and resulting health conditions clouds understanding of the totality of human system risk in space. In response, NASA has introduced Directed Acyclic Graphs (causal diagrams) into the human systems risk management process. These diagrams allow for a common understanding of the mechanisms that lead from unique hazards of spaceflight to the health outcomes important to agencies and astronauts. However, the paucity of available biomedical data from spaceflight creates a need for methods of validating causal models that can accommodate data from spaceflight model analogs. Here we outline one approach utilizing open-access rodent bone datasets from the Ames Life Sciences Data Archive. The properties of directed acyclic graphs themselves can provide an epistemological and statistical framework for validation of a priori causal representations of human system risk in space flight. The assumed causal connections on the graph creates sets of logical implications: variables that – if the causal diagram is correct – should be correlated, as well as sets that should be conditionally independent. By testing these implied correlations and conditional independencies both statistically and heuristically, we can provide evidence for or against specific causal pathways on the causal diagram. In addition to validation of expert-generated causal diagrams, machine learning techniques can learn the most likely structure of a causal diagram from a given dataset. Comparison with and reconciliation between machine-learned causal diagrams and expert-generated diagrams is another technique for challenging assumptions and improving our understanding of causal mechanisms. Accurately representing complex causation is essential to systemic understanding of human health risks in space travel. Having a robust system of validating causal diagrams helps us arrive at more accurate representations of causal systems. This process will be integral to developing the countermeasures necessary for extended exploration of the moon and Mars.

Robert Reynolds↗
NASA NTRS2026

Methods for Validation of Random Uncertainty Estimates and Their Applications to Ozone Profiles From Limb-Viewing Satellite Instruments

For satellite measurements of atmospheric composition, the random uncertainty estimates provided by retrieval algorithms might be imperfect due to various approximations used in the retrievals or the presence of unknown error sources. This paper presents an overview of the methods used for the validation of random uncertainty estimates. All methods discussed in this study are categorized, and assumptions and limitations of each method are discussed. This overview evaluates these methods in application to ozone profile measurements from limb and occultation satellite instruments and provides practical illustrations of random uncertainty validation.

Viktoria F Sofieva↗
DOE OSTI2024

Development of Nontargeted Workflow of Occupational Exposure by Infrared Ion Spectroscopy and Silicone Wristbands’ Passive Sampling

Monitoring chemical exposure has well-established protocols and procedures for occupational health and industrial hygiene across various career fields and in a variety of occupational and environmental conditions. Iterative analytical development has led to validated methods to provide employers and employees with safe working conditions in terms of industrial hygiene. These validated methods are commonly supported by commercial laboratories. Here, this workflow is explicitly a targeted approach in which sampling media, sample preparation, chemical separation, quantitative target detection, and post processing are well established. This workflow also relies on commercially available chemical surrogates, isotopically labeled internal standards, and certified/standard reference materials from reputable vendors. An untargeted workflow lacks many key ingredients of validated analytical methods to include the known chemical target.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS1984

A validation methodology for fault-tolerant clock synchronization

A validation method for the synchronization subsystem of a fault-tolerant computer system is presented. The high reliability requirement of flight crucial systems precludes the use of most traditional validation methods. The method presented utilizes formal design proof to uncover design and coding errors and experimentation to validate the assumptions of the design proof. The experimental method is described and illustrated by validating an experimental implementation of the Software Implemented Fault Tolerance (SIFT) clock synchronization algorithm. The design proof of the algorithm defines the maximum skew between any two nonfaulty clocks in the system in terms of theoretical upper bounds on certain system parameters. The quantile to which each parameter must be estimated is determined by a combinatorial analysis of the system reliability. The parameters are measured by direct and indirect means, and upper bounds are estimated. A nonparametric method based on an asymptotic property of the tail of a distribution is used to estimate the upper bound of a critical system parameter. Although the proof process is very costly, it is extremely valuable when validating the crucial synchronization subsystem.

Johnson, S. C.↗
DOE OSTI2019

A systematic review of validated suicide outcome classification in observational studies

Abstract Background Suicidal outcomes, including ideation, attempt, and completed suicide, are an important drug safety issue, though few epidemiological studies address the accuracy of suicidal outcome ascertainment. Our primary objective was to evaluate validated methods for suicidal outcome classification in electronic health care database studies. Methods We performed a systematic review of PubMed and EMBASE to identify studies that validated methods for suicidal outcome classification published 1 January 1990 to 15 March 2016. Abstracts and full texts were screened by two reviewers using prespecified criteria. Sensitivity, specificity, and predictive value for suicidal outcomes were extracted by two reviewers. Methods followed PRISMA-P guidelines, PROSPERO Protocol: 2016: CRD42016042794. Results We identified 2202 citations, of which 34 validated the accuracy of measuring suicidal outcomes using International Classification of Diseases (ICD) codes or algorithms, chart review or vital records. ICD E-codes (E950-9) for suicide attempt had 2–19% sensitivity, and 83–100% positive predictive value (PPV). ICD algorithms that included events with ‘uncertain’ intent had 4–70% PPV. The three best-performing algorithms had 74–92% PPV, with improved sensitivity compared with E-codes. Read code algorithms had 14–68% sensitivity and 0–56% PPV. Studies estimated 19–80% sensitivity for chart review, and 41–97% sensitivity and 100% PPV for vital records. Conclusions Pharmacoepidemiological studies measuring suicidal outcomes often use methodologies with poor sensitivity or predictive value or both, which may result in underestimation of associations between drugs and suicidal behaviour. Studies should validate outcomes or use a previously validated algorithm with high PPV and acceptable sensitivity in an appropriate population and data source.

Swain, Richard S.↗
NASA NTRSDate not supplied

Validated Reduced Computational Methods for Realistic RDC Injection Modeling

This paper outlines a simplified computational method without chemistry for single and multi-injector performance for rotating detonation combustion (RDC) environment as well as relevant performance metrics. For non-reactive cases, an unsteady pressure profile from a Method of Characteristics (MOC) simulation was used to simulate an RDC combustion wave travelling around the annulus. In reactive cases, a pre-mixed stoichiometric mixture of hydrogen and air is injected into a 2D unwrapped combustion chamber with a specified injector area. The performance is evaluated based on several performance metrics including unsteady discharge coefficient, average mass flux coming into the combustor, and percentage time obstructed. Finally, the computational method was validated through an experimental RDC test setup with a single element injector in a hydrogen-air detonation driver environment.

Propulsion↗
NASA NTRS1987

Fault-tolerant clock synchronization validation methodology

A validation method for the synchronization subsystem of a fault-tolerant computer system is presented. The high reliability requirement of flight-crucial systems precludes the use of most traditional validation methods. The method presented utilizes formal design proof to uncover design and coding errors and experimentation to validate the assumptions of the design proof. The experimental method is described and illustrated by validating the clock synchronization system of the Software Implemented Fault Tolerance computer. The design proof of the algorithm includes a theorem that defines the maximum skew between any two nonfaulty clocks in the system in terms of specific system parameters. Most of these parameters are deterministic. One crucial parameter is the upper bound on the clock read error, which is stochastic. The probability that this upper bound is exceeded is calculated from data obtained by the measurement of system parameters. This probability is then included in a detailed reliability analysis of the system.

Computer systems↗
NASA NTRS2011

Validation of Methods to Predict Vibration of a Panel in the Near Field of a Hot Supersonic Rocket Plume

This paper describes the measurement and analysis of surface fluctuating pressure level (FPL) data and vibration data from a plume impingement aero-acoustic and vibration (PIAAV) test to validate NASA s physics-based modeling methods for prediction of panel vibration in the near field of a hot supersonic rocket plume. For this test - reported more fully in a companion paper by Osterholt & Knox at 26th Aerospace Testing Seminar, 2011 - the flexible panel was located 2.4 nozzle diameters from the plume centerline and 4.3 nozzle diameters downstream from the nozzle exit. The FPL loading is analyzed in terms of its auto spectrum, its cross spectrum, its spatial correlation parameters and its statistical properties. The panel vibration data is used to estimate the in-situ damping under plume FPL loading conditions and to validate both finite element analysis (FEA) and statistical energy analysis (SEA) methods for prediction of panel response. An assessment is also made of the effects of non-linearity in the panel elasticity.

Bremner, P. G.↗
DOE OSTI2024

Summary Report of the FY24 DOE Contributions to the GIF VHTR CMVB

The Generation-IV Forum (GIF) Very-High-Temperature Reactor-Computational Methods Validation and Benchmark (VHTR-CMVB) initiative, involving organizations from Korea Atomic Energy Research Institute (KAERI) (South Korea), Institute of Nuclear and New Energy Technology of Tsinghua University (INET) (China), U.S. Department of Energy (DOE) (U.S.), Joint Research Centre (JRC) (Europe), and Japan Atomic Energy Agency (JAEA) (Japan), is dedicated to the verification and validation of tools for High-Temperature Gas-Cooled Reactors (HTGRs) analysis, using data shared by Computational Methods Validation and Benchmark (CMVB) signatories. For FY24, the US DOE CMVB has committed to several critical activities. Under WP1, led by the US, the integration of the High Temperature Gas Cooled Reactor - Pebble-Bed Module (HTR-PM) Phenomena Identification and Ranking Table (PIRT) into the comparative PIRT is progressing, with a new draft of the comparison tables issued earlier this year and currently being utilized by INET for their contribution. Neutronic validation efforts under WP3 include the preparation of the burnup analysis benchmark, preliminary calculations, and the development of reference models and results. In WP2, a validation exercise for hot gas mixing in the lower plenum of HTR-PM is in progress, using experimental data from INET (China) to validate modeling approaches. A model of the experimental facility has been developed using StarCCM+, with initial calculations slated for presentation at the GIF CMVB meeting this fall. Another WP2 activity focuses on validating numerical models for air-cooled Reactor Cavity Cooling System (RCCS) with experimental data from the Wisconsin Madison RCCS facility. A high-fidelity model, developed using NEK-RS, is currently being validated with available data from a low power forced convection test. These efforts are aimed at enhancing and confirming the accuracy of HTGR analysis tools, ensuring their alignment with experimental data and regulatory requirements.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
NASA NTRS1991

Meteorological radar methods for validating space observations of precipitation

Meteorological approaches to verification of space measurements of rainfall are examined; validation of Tropical Rainfall Measuring Mission (TRMM) observations is expected to depend significantly on ground-based radars. Two methods of comparison are initially contemplated. TRMM rainfall data over time periods of a month for large areas (500 x 500 km) are averaged and compared with similarly averaged ground truth measurements. Both the rainfall and height distribution data from TRMM are compared with the instantaneous values observed at one or more 'ground truth' stations and from airborne radar and radiometers as available.

Thiele, Otto W.↗
DOE OSTI2018

Topographic Effects on Radiation in the WRF Model with the Immersed Boundary Method: Implementation, Validation, and Application to Complex Terrain

Abstract Topographic effects on radiation, including both topographic shading and slope effects, are included in the Weather Research and Forecasting (WRF) Model, and here they are made compatible with the immersed boundary method (IBM). IBM is an alternative method for representing complex terrain that reduces numerical errors over sloped terrain, thus extending the range of slopes that can be represented in WRF simulations. The implementation of topographic effects on radiation is validated by comparing land surface fluxes, as well as temperature and velocity fields, between idealized WRF simulations both with and without IBM. Following validation, the topographic shading implementation is tested in a semirealistic simulation of flow over Granite Mountain, Utah, where topographic shading is known to affect downslope flow development in the evening. The horizontal grid spacing is 50 m and the vertical grid spacing is approximately 8–27 m near the surface. Such a case would fail to run in WRF with its native terrain-following coordinates because of large local slope values reaching up to 55°. Good agreement is found between modeled surface energy budget components and observations from the Mountain Terrain Atmospheric Modeling and Observations (MATERHORN) program at a location on the east slope of Granite Mountain. In addition, the model captures large spatiotemporal inhomogeneities in the surface sensible heat flux that are important for the development of thermally driven flows over complex terrain.

54 ENVIRONMENTAL SCIENCES↗