Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Verification of data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Verification of Upcoming MCNP Features For Estimating Nuclear Data Sensitivities in Fixed Source Simulations [Abstract]

Predictive simulation codes, like the Monte Carlo N-Particle (MCNP) transport code, are used throughout the nuclear community. These simulations are based on nuclear data. Maximizing the accuracy and precision of nuclear data maximizes the accuracy and precision of the overall simulation. This is imperative to applications that rely on simulations. For example, improving nuclear data for special nuclear material improves simulation accuracy in stockpile stewardship applications, which results in larger safety margins and decreased operational costs. The improvement and validation of nuclear data is completed through integral benchmark experiments. Past benchmarks have primarily been limited to focus on the effective multiplication factor ($\kappa$ eff ); broadening the purview of benchmarks beyond $\kappa$ eff -dependent nuclear data addresses nuclear data deficiencies. Different response types depend on different areas of nuclear data. This dependence is quantified as nuclear data sensitivity: the change in response due to perturbation of a contributing parameter. The larger the nuclear data sensitivity of a response, the more the experiment is influenced by the uncertainties of the nuclear data. The optimization of nuclear data sensitivities in future benchmarks would result in more detailed validation of lesser studied areas of nuclear data. Currently, direct sensitivity capabilities are not easily found for all experiment types and parameters. An MCNP tool to directly estimate the cross section sensitivities of tallied values is under development. Additionally, updates have been made to the perturbation feature of MCNP, which can be used in a less direct approach to estimating sensitivities. This work verifies these features to estimate nuclear data sensitivities in fixed source simulations of a 4.5-kg sphere of alpha- phase weapons-grade plutonium surrounded by differing amounts of copper and polyethylene. Integrated estimates made using MCNP’s tools were found to statistically agree with integrated estimates made from manual perturbation of nuclear data proving the validity of the MCNP tools.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Verification, Validation, and Calibration Through a Causal Lens

While typical validation and verification approaches focus on identifying the associations between data elements using statistical and machine learning methods, the novel methods in this paper focus instead on identifying causal relationships between data elements. Statistical and machine-learning-based approaches are strictly data-driven, meaning that they provide quantitative comparison measures between data sets without explicitly considering the hypotheses behind them. This can lead to the erroneous conclusion that, if two data sets are close enough, the models that generated them are similar. In addition, when experimental and simulated data differ to an extent that fails to meet the acceptance criteria, calibration techniques are used to tweak simulation model parameters to reduce the gap between the two types of data. This produces the false expectation that a simulation model will match reality. The methods presented in this paper move away from these strictly data-driven methods for validation and calibration toward more robust, model-driven methods based on causal inference. Causal inference aims to identify the possible mechanisms that might have generated data. Thus, this analysis targets the prediction of the effects when one (or more) of the identified mechanisms are altered. There are many approaches to identify, quantify, and illustrate causal relationships. For the scope of this paper, directed graphs are employed as causal models. If the directed graph lacks cycles, it is known as a directed acyclic graph. A node in such a graph represents an observed data element while a directed edge connecting two nodes represents a causal relationship between two variables. The developed causal methods are designed to extract causal models from simulation models and experimental data. Causal models capture the causal relationships between data elements (e.g., simulated and experimental data). In this context, validation and verification are performed by comparing causal models. The proposed approach does not only inform system analysts on how a simulation model matches real-world data, but also identifies elements of the simulation model that should be revised when discrepancies between simulation and experimental data are observed. Through these causal methods, analysts can identify the portion of the model equation(s) that are behind an edge connecting two variables. Hence, once the structural differences between causal models have been determined, model calibration can occur by changing only those model parameters that impact the identified causal relationships.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Vexcel Imaging's Suitability for Automatic Verification

Accurate, independently verified geospatial data is essential for automated calibration, validation, and operational decision-making. This study evaluated the positional accuracy of Vexcel Imaging™’s UltraCam® Osprey imagery (7.5 cm GSD) using globally distributed Continuously Operating Reference Stations (CORS) as independent control. Despite manufacturer claims of 15 cm horizontal accuracy, residual errors were consistently one to two orders of magnitude larger, with no subset of imagery meeting precision thresholds. These discrepancies cannot be explained by normal photogrammetric or environmental factors and raise concerns about the reliability of the imagery for high-precision tasks. The results demonstrate that Vexcel imagery, in its current form, is unsuitable for workflows requiring rigorous spatial accuracy or automated verification. At the same time, the reproducible validation framework developed in this study establishes a scalable method for assessing commercial imagery, ensuring that future products can be independently and objectively verified before operational adoption.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

VALIDATION, VERIFICATION, AND CALIBRATION THROUGH A CAUSAL LENS

This paper presents an alternative method based on causal inference to perform validation, verification, and calibration of simulation models. While classical validation and verification approaches focus on the identification of the associations between data elements using statistical and machine learning methods, the novel methods in this paper focus instead on the identification of causal relationships between data elements. Statistical and machine learning-based approaches are strictly data-driven, meaning that they provide quantitative comparison measures between datasets without explicitly considering the hypotheses behind them. This can lead to the erroneous conclusion that, if two data sets are close enough, then the models that generated them are similar. In addition, when experimental and simulated data differ to an extent that fails to meet the acceptance criteria, calibration techniques are used to tweak simulation model parameters to reduce the gap between the two types of data. This produces the false expectation that a simulation model will match reality. The methods presented in this paper move away from these strictly data-driven methods for validation and calibration toward more robust, model-driven methods based on causal inference. Causal inference aims to identify the possible mechanisms that might have generated data. Thus, this analysis targets the prediction of the effects when one (or more) of the identified mechanisms are altered. There are many approaches to identify, quantify and illustrate causal relationships. For the scope of this paper, directed graphs are employed as causal models. If the directed graph lacks cycles it is known as a directed acyclic graph (DAG). A node in such a graph represents an observed data element while a directed edge connecting two nodes represents a causal relationship between two variables. The developed causal methods are designed to extract causal models from simulation models and from experimental data. Causal models capture the causal relationships between data elements (e.g., simulated and experimental data). In this context, validation and verification are performed by comparing causal models. The proposed approach does not only inform system analysts on how a simulation model matches real-world data, but also identifies elements of the simulation model that should be revised when discrepancies between simulation and experimental data are observed. Through these causal methods, analysts have a means to identify the portion of the model equation(s) that are behind an edge connecting two variables. Hence, once the structural differences between causal models have been determined, model calibration can occur by changing only those model parameters that impact the identified causal relationships.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Procedure for locating oil and gas wells in the Appalachian Basin

Locating undocumented (or poorly documented) oil and gas wells for environmental assessment is often difficult. Remnant features that confirm the presence of a well (intact casing/wellhead, well bore, etc.) are typically less than a meter in size and often are obscured from direct observation on the ground or from the air (by dense vegetation, for example). To efficiently find such features, it is useful to first systematically compile publicly available digital data at progressively smaller scales prior to embarking on field campaigns. Further, the information presented here describes the procedure developed and used by the U.S. Department of Energy's National Energy Technology Laboratory to locate potential oil and gas well sites for follow-up field verification and characterization. Digital data are first compiled from national and state resources such as well location/production databases, historical topographic maps, historical aerial photographs, and LiDAR data. Although each data set is likely to be incomplete or inaccurate to some extent, combining the data resources using geographic information system technology can generate potential well site targets with a higher degree of confidence, which improves the efficiency of fieldwork activities. This workflow was developed in the Appalachian Basin region, and although certain aspects may be unique, the general process would be applicable to locating undocumented wells in other regions.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

A Knowledge-based Framework for Building Energy Model Performance Verification

Building energy modeling (BEM) has been widely used by researchers, regulators, and engineers to quantify building energy performance. Quality assurance (QA) and quality control (QC) of the model's performance are essential parts of such analysis. Currently, QA/QC is done in a manual and ad-hoc manner, which is tedious, error-prone, and time-consuming when QA/QC a large number of models. To solve these challenges, we propose a a dAta-driveN buIlding perforMance verificATion framEwork (ANIMATE), which conducts automated output-based verification of building operations requirements (especially for time-series output-based verification of control requirements). While this framework was developed for verifying energy model performance, it can be extended for other applications such as BEM software testing and performance verification of real buildings in the field.

Chen, Yan↗
DOE OSTI2021

Prognostics and Health Management in Nuclear Power Plants: An Updated Method-Centric Review With Special Focus on Data-Driven Methods

In a carbon-constrained world, future uses of nuclear power technologies can contribute to climate change mitigation as the installed electricity generating capacity and range of applications could be much greater and more diverse than with the current plants. To preserve the nuclear industry competitiveness in the global energy market, prognostics and health management (PHM) of plant assets is expected to be important for supporting and sustaining improvements in the economics associated with operating nuclear power plants (NPPs) while maintaining their high availability. Of interest are long-term operation of the legacy fleet to 80 years through subsequent license renewals and economic operation of new builds of either light water reactors or advanced reactor designs. Recent advances in data-driven analysis methods—largely represented by those in artificial intelligence and machine learning—have enhanced applications ranging from robust anomaly detection to automated control and autonomous operation of complex systems. The NPP equipment PHM is one area where the application of these algorithmic advances can significantly improve the ability to perform asset management. This paper provides an updated method-centric review of the full PHM suite in NPPs focusing on data-driven methods and advances since the last major survey article was published in 2015. The main approaches and the state of practice are described, including those for the tasks of data acquisition, condition monitoring, diagnostics, prognostics, and planning and decision-making. Research advances in non-nuclear power applications are also included to assess findings that may be applicable to the nuclear industry, along with the opportunities and challenges when adapting these developments to NPPs. Finally, this paper identifies key research needs in regard to data availability and quality, verification and validation, and uncertainty quantification.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

Data assimilation for burnup distribution of PWR with three-dimensional variational algorithm and artificial neutral network

In this paper, a data-assimilation method has been proposed and applied for the burnup distribution of PWR. The burnup distribution is significant to the safety and economy of the reactor, as it is essential for the fuel-reloading design and optimization. Due to the burnup distribution cannot be measured directly during the reactor operation, the numerical simulation is widely applied to determine the burnup distribution. However, there is a deviation between the numerical simulation and the actual core due to some unavoidable factors, such as component manufacturing deviation, uneven flow distribution and so on. These differences would induce the errors to the simulation values of power distributions and hence to the burnup distribution. To address this problem, a data-assimilation method for the burnup distribution has been proposed with the application of power-distribution measurements. In our research, the three-dimensional variational (3DVAR) algorithm was applied for burnup-distribution calibration and the artificial neutral network (ANN) was applied to establish the relation between power distribution and corresponding burnup distribution. As engineering verification, the proposed data-assimilation method has been applied to the CNP1000 PWR operated in China. The numerical results indicated that the burnup-distribution errors can be reduced notably, as the maximum value of relative errors for power distribution can be reduced from 5.35% to 3.96%. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2020

Putting Our Industry's Data to Work: A Case Study of Large-Scale Data Aggregation: Preprint

With increasing deployment of Advanced Metering Infrastructure (AMI), Building Automation System (BAS) controls, Internet of Things (IoT) network devices, and data-driven evaluation, measurement, and verification studies, the building sector is currently generating a staggering amount of energy-related data. In the right hands, these data sets can contribute to increased comfort and energy savings for building occupants and a more reliable electrical grid; however, due to a combination of factors, including significant privacy concerns, much of the data that are presently generated and stored are not used outside of basic operational applications. In the past year, our team has dedicated over 2,000 person-hours to accessing building energy data for a project funded by the U.S. Department of Energy (DOE) Building Technologies Office. We sought whole-building or end-use (e.g., lighting) timeseries data at the individual-building or equipment level where possible or aggregated information, such as timeseries averages and quartiles by building type (e.g., office, retail, hospital), where sharing individual building information was not an option. We are additionally working with IoT and BAS data sets to derive information important to the project. We have assembled an extensive data set that will enable the development of publicly available end-use load profiles to benefit the U.S. building and electricity industries. Here we present an overview of the data set that we have assembled to date, the motivators and approaches that got us here, and the lessons we learned through our efforts. We also discuss work underway that presents additional options for future data access.

building energy data↗
DOE OSTI2025

Dynamic in-context learning with conversational models for data extraction and materials property prediction

The advent of natural language processing and large language models (LLMs) has revolutionized the extraction of data from unstructured scholarly papers. However, ensuring data trustworthiness remains a significant challenge. In this paper, we introduce PropertyExtractor, an open-source tool that leverages advanced conversational LLMs such as Google gemini-pro and OpenAI gpt-4, blends zero-shot with few-shot in-context learning, and employs engineered prompts for the dynamic refinement of structured information hierarchies—enabling autonomous, efficient, scalable, and accurate identification, extraction, and verification of material property data. Our tests on material data demonstrate precision and recall that exceed 95% with an error rate of ∼9%, highlighting the effectiveness and versatility of the toolkit. Finally, databases for 2D material thicknesses, a critical parameter for device integration, and energy bandgap values are developed using PropertyExtractor. In particular, for the thickness database, the rapid evolution of the field has outpaced both experimental measurements and computational methods, creating a significant data gap. Our work addresses this gap and showcases the potential of PropertyExtractor as a reliable and efficient tool for the autonomous generation of various material property databases, advancing the field.

Ekuma, Chinedu E. (ORCID:0000000258527556)↗
DOE OSTI2021

Thermodynamic properties of the Nd-Bi system via emf measurements, $\mathrm{DFT}$ calculations, machine learning, and $\mathrm{CALPHAD}$ modeling

Thermodynamic properties of the Nd-Bi system were investigated using a combination of experimental measurements, first-principles calculations based on density functional theory (DFT), data mining and machine learning (DM + ML) predictions, and calculation of phase diagrams (CALPHAD) modeling. The electromotive force (emf) of Nd-Bi alloys in molten LiCl-KCl-NdCl 3 at 773–973 K was measured via coulometric titration of Nd into Bi for the determination of thermochemical properties such as activity coefficients and solubilities of Nd in Bi. A new peritectic reaction of [liquid + NdBi 2 = Nd 3 Bi 7 ] at 774 K was confirmed using differential scanning calorimetry, structural (X-ray diffraction), and microstructural (scanning electron microscopy) analyses. The unknown crystal structure of NdBi2 was suggested to be a mixture of the anti-La 2 Sb configuration and the La 2 Te-type configuration based on ML predictions for over 26,000 data-mined AB 2 -type configurations together with DFT-based verifications. Using the newly acquired experimental data and DFT-based calculations, the thermodynamic description of the Nd-Bi system was remodeled, and a more complete Nd-Bi phase diagram was calculated, including the Nd 3 Bi 7 compound, invariant transition reactions, and liquidus temperatures.

36 MATERIALS SCIENCE↗
DOE OSTI2026

XRISM high-resolution X-ray spectroscopy of Cygnus X-1: Orbital and short-term variability of iron absorption

We present the first high-resolution spectroscopy of the black hole high-mass X-ray binary Cygnus X-1 with XRISM, including orbital-phase-resolved analyses and tentative evidence of short-term variability in the Fe K band on second timescales. Using data from the Performance Verification phase in 2024 April, we analyzed spectral variability across orbital phases with the Resolve microcalorimeter and the Xtend CCD imager. The unprecedented resolution of Resolve reveals variability in highly ionized Fe absorption lines. The absorption features show orbital-phase-dependent variability in column density, ionization state, and blueshifted velocity, suggesting structural variations in the focused stellar wind along the line of sight. We also find indications of subtle broadening of the neutral Fe emission profile. In addition, intensity-sorted spectroscopy during dip phases suggests possible variability on timescales of a few seconds in the absorption features, consistent with cooler, denser, and lower-ionized gas clumps. Although the statistical significance is limited, these results hint that the stellar wind and the X-rays from the accretion disk around the black hole may interact on timescales as short as a few seconds. These XRISM results constrain wind-fed accretion in Cyg X-1 and highlight Resolve’s capability to probe plasma environments in high-mass X-ray binaries.

Astronomy and AstroPhysics↗
DOE OSTI2023

Defect Recognition for Eddy Current Testing of Spent Nuclear Fuel Canister using Convolutional Neural Network

This paper proposes an accurate and robust defect detection solution for 304L and 306L stainless steel (SS) weld. In the proposed solution, Eddy current testing (ECT) is employed to generate 2-dimensional (2D) data for samples under test with defects. The 2D data can be treated as images for deep learning-based defect detection. Since convolutional neural networks (CNNs) are powerful in processing images, CNN is employed in this study for defect detection. Experiments are conducted on a submerged arc welding (SAW) 304L SS weld sample with an artificial crack generated by waterjet cutting. The ECT data on this seeded fault sample is utilized to verify the proposed solution. For this purpose, the ECT measurement are separated as from Fault area and Normal area, which are used for CNN training. After training, the testing data is used for verification. Experimental results demonstrate the feasibility and effectiveness of the proposed solution.

Niu, Guangxing↗
DOE OSTI2025

Convection-Permitting Ensembles of an Isolated Mountain Thunderstorm during RELAMPAGO/CACTI

Abstract The north–south-oriented Sierras de Córdoba (SDC) ridge in central Argentina is noted for initiating thunderstorms that may grow into intense mesoscale convective systems (MCSs). It also initiates more isolated, shorter-lived cells under weaker synoptic forcing. These cells are less impactful than MCSs but may be difficult to predict in convective-scale numerical weather prediction (NWP) due to their strong sensitivities to subgrid and partially resolved processes. To study the mechanisms and predictability of such cells, convection-permitting ensemble simulations were conducted of an isolated, diurnally forced SDC thunderstorm during Cloud, Aerosol, and Complex Terrain Interactions (CACTI)/Remote Sensing of Electrification, Lightning, and Mesoscale/Microscale Processes with Adaptive Ground Observations (RELAMPAGO). The rich observational data facilitated detailed ensemble verification, where dry biases in the surface energy balance and soil moisture were identified. These biases promoted rapid removal of convective inhibition and an early onset of precipitating cells over the SDC that were shallower and weaker than the observed cell. Correction, and then overcorrection, of the soil moisture bias in two successive ensembles was required to rectify the surface energy balance and improve the representation of the SDC cell. Nevertheless, substantial ensemble variability in convective precipitation was found, with some members producing more widespread convection than observed and others producing no deep convection at all. This variability was largely explained by a combination of thermodynamic and dynamic mechanisms, dominated by a positive sensitivity of convective precipitation to preconvective moist instability over the ridge. Secondary sensitivities were found to low-level upward mass flux and midlevel cross-barrier winds, the latter of which caused gravity waves with elevated downdrafts that tended to suppress incipient clouds.

Lopez, Andres [Department of Atmospheric and Ocean↗
DOE OSTI2022

Isolated Building Wake Experiment At Texas Tech University’s Wind Engineering Research Field Lab

Texas Tech University’s (TTU’s) Wind Engineering Research Field Lab (WERFL) building is an idealized rectangular building on the scale of a typical suburban house. It is surrounded almost entirely by unobstructed planes and is built on a turn table. WERFL is ideal for providing measurements of an idealized building under real-world conditions including atmospheric stability and variable relative wind angles. The intent of this data set is for verification and validation of computational fluid dynamics model simulations of flow around buildings. The primary focus of this dataset features 24 3D sonic anemometers (20 downwind and 4 upwind) and 10 2D sonic anemometers (9 downwind and 1 upwind) located in the immediate vicinity of the WERFL building. It also includes a variety of instruments intended to measure the ambient meteorological parameters and the undisturbed wind profile.

17 WIND ENERGY↗
DOE OSTI2022

Real-Time Drilling Optimization System for Improved Overall Rate of Penetration and Reduced Cost Per Foot in Geothermal Drilling

The key to success in geothermal drilling is economic feasibility, and a major cost in the development of geothermal resources is the actual drilling of the wells. In this project, a real-time drilling optimization system for geothermal drilling was developed. The system couples three individual components while drilling. The first component is a drill stem vibration analysis model, the second is Mechanical Specific Energy (MSE) analyses, and the third is a detailed PDC Rate of Penetration (ROP) drill bit model for optimum RPM and WOB combinations. The benefit of the coupled system is that the range of WOB and RPM could be selected to avoid drill stem vibrations. Secondly, MSE is used as an efficiency measure and the detailed PDC drill bit model ensures the drill bit does not endure temperatures that exceed the temperature at which the PDC cutters experience accelerated wear. The new detailed PDC bit model is based on rock/bit interaction that physically tracks the PDC cutter wear flats as the bit drills ahead giving the capability to calculate the temperature being generated underneath the worn cutters to better advise on operational parameters to avoid accelerated cutter wear and failure and to ensure that operational parameters are applied so that overall ROP is maximized. By combining the drill stem vibrations and the detailed PDC bit cutter wear and “safe” non-accelerated cutter wear temperature and optimum ranges of operating parameters, it results in higher ROP and lower cost drilling. Single cutter PDC testing performed in different lithologies at Sandia was utilized to verify the PDC cutter forces and depth of cut for new and worn cutters. Based on single cutter PDC temperature modeling, verification using single cutter data from the testing done by National Oilwell Varco (NOV) was performed. Sandia’s Hard-Rock Drilling Facility (HRDF) was utilized to test different drill bit configurations with different cutter designs and wear status with different induced modes of vibration to obtain the critical bit RPM/WOB ranges resulting in ineffective drilling and low ROP. The collected test data were further used to verify and calibrate the full hole PDC ROP model that was developed based on single cutter interaction data. A full coupled drill stem vibration model was formulated and verified with geothermal field data from the Chocolate Mountain Aerial Gunnery Range (CMAGR). A graphical user interface (GUI) was developed using Tkinter library in the computer programming language Python, which integrates all the developed models in one system. The developed system consists mainly of the PDC ROP model, PDC bit wear model, PDC cutter temperature model, Mechanical Specific Energy (MSE) model, and drillstring vibration model integrated into one system. The developed system can be used for both, post well analysis and real-time optimization using different criteria such as ROP maximization or MSE minimization. The software uses Differential Evolution Algorithm (DEA) to find optimum values for operational parameters based on last foot drilled while avoiding the drillstring vibration and cutter temperature critical operating parameters.

15 GEOTHERMAL ENERGY↗
DOE OSTI2022

Verification and validation testing and tools: comparison between MCNP code versions and nuclear data libraries [Slides]

This presentation discusses the primary goal of software testing which is to test the code for correctness. It also discusses the results for individual suites and the role of validation and verification also referred to in the presentation as V&V. In summation, the V&V framework enables easy comparison between calculations performed with different code versions and/or nuclear data libraries. This entire framework will be distributed with the upcoming MCNP6.3 release. V&V test suites shown and several that were not (Criticality, LAQGSM, Lockwood) will be distributed in the new framework.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2025

Robust Solution Verification Experiments on Nonuniform Meshes

The activities of verification, validation, and uncertainty quantification (VVUQ) provide a comprehensive means to assess the credibility of computational models. Within VVUQ, solution verification assesses numerical errors and evaluates whether the simulation is sufficiently accurate for its intended applications. As computational modeling gains traction in the development of complex, high-consequence systems, the need for robust solution verification intensifies, particularly because experimental data for these systems are often limited. This work examines improvements in the robustness of Richardson extrapolation (RE), a method commonly used in solution verification to study the discretization error of computational models using a power law. Nonuniform mesh refinement is discussed alongside other pollutants that affect the robustness of the power law model. Maximum likelihood estimation (MLE) is proposed as a robust strategy to address the uncertainty generated by nonuniform mesh refinement. An exploratory computational fluid dynamics (CFD) study of a 2D planar Poiseuille flow is conducted to determine if nonuniform mesh noise can be modeled with this MLE approach for more robust RE.

Weinmeister, Justin [ORNL] (ORCID:0000000160090237↗