Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “validation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24

Monte Carlo toolkit for designing and validating step-range-filter spectrometer designs

Here, we present a Monte Carlo toolkit for validating step range filter (SRF) spectrometer designs. Geant4 is used to transport charged particles through the SRF filters to generate synthetic SRF data that include realistic CR-39 effects. Synthetic SRF spectra generated by this method inherently account for instrument response and allow for the quantification of SRF performance before shots. The usefulness of this toolkit is demonstrated through its application to a number of problems. A new broadband SRF for the ∼10 MeV wide 3He3He proton spectrum is validated, and an analysis method for analyzing 3He3He-p SRF data that accounts for instrument response is put forth. In addition, an SRF design for the compact recoil-proton spectrometer (CRS) on the Z-machine is validated. Finally, a new calibration technique for the DD-p SRF is proposed and validated.

Johnson, T. M. (ORCID:0000000193032949)↗

A GLOBAL TWO-TEMPERATURE CORONA AND INNER HELIOSPHERE MODEL: A COMPREHENSIVE VALIDATION STUDY

The recent solar minimum with very low activity provides us a unique opportunity for validating solar wind models. During CR2077 (2008 November 20 through December 17), the number of sunspots was near the absolute minimum of solar cycle 23. For this solar rotation, we perform a multi-spacecraft validation study for the recently developed three-dimensional, two-temperature, Alfven-wave-driven global solar wind model (a component within the Space Weather Modeling Framework). By using in situ observations from the Solar Terrestrial Relations Observatory (STEREO) A and B, Advanced Composition Explorer (ACE), and Venus Express, we compare the observed proton state (density, temperature, and velocity) and magnetic field of the heliosphere with that predicted by the model. Near the Sun, we validate the numerical model with the electron density obtained from the solar rotational tomography of Solar and Heliospheric Observatory/Large Angle and Spectrometric Coronagraph C2 data in the range of 2.4 to 6 solar radii. Electron temperature and density are determined from differential emission measure tomography (DEMT) of STEREO A and B Extreme Ultraviolet Imager data in the range of 1.035 to 1.225 solar radii. The electron density and temperature derived from the Hinode/Extreme Ultraviolet Imaging Spectrometer data are also used to compare with the DEMT as well as the model output. Moreover, for the first time, we compare ionic charge states of carbon, oxygen, silicon, and iron observed in situ with the ACE/Solar Wind Ion Composition Spectrometer with those predicted by our model. The validation results suggest that most of the model outputs for CR2077 can fit the observations very well. Based on this encouraging result, we therefore expect great improvement for the future modeling of coronal mass ejections (CMEs) and CME-driven shocks.

79 ASTRONOMY AND ASTROPHYSICS↗

Validation of Geant4 Hadronic Physics Models at Intermediate Energies

GEANT4 provides a number of physics models at intermediate energies (corresponding to incident momenta in the range 1-20 GeV/c). Recently, these models have been validated with existing data from a number of experiments: (a) inclusive proton and n eutron production with a variety of beams (I (-'), I (+), p) at different energies between 1 and 9 GeV/c on a number of nuclear targets (from beryllium to uranium), (2) inclusive pion/kaon/proton production from 14.6 GeV/c proton beams on nuclear targets (from beryllium to gold), (3) inclusive pion production from pion beams between 3-13 GeV/c on a number of nuclear targets (from beryllium to lead). The results of simulation/data comparison for different GEANT4 models are discussed in the context of valid ating the models and determining their usage in physics lists for high energy application. Due to the increasing number of validations becoming available, and the requirement that they be done at regular intervals corresponding to the GEANT4 release sched ule, automated methods of validation are being developed.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Validation of Geant4 hadronic generators versus thin target data

The GEANT4 toolkit is widely used for simulation of high energy physics (HEP) experiments, in particular, those at the Large Hadron Collider (LHC). The requirements of robustness, stability and quality of simulation for the LHC are demanding. This requires an accurate description of hadronic interactions for a wide range of targets over a large energy range, from stopped particle reactions to low energy nuclear interactions to interactions at the TeV energy scale. This is achieved within the Geant4 toolkit by combining a number of models, each of which are valid within a certain energy domain. Comparison of these models to thin target data over a large energy range indicates the strengths and weaknesses of the model descriptions and the energy range over which each model is valid. Software has been developed to handle the large number of validation tests required to provide the feedback needed to improve the models. An automated process for carrying out the validation and storing/displaying the results is being developed and will be discussed.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Recent developments and validation of Geant4 hadronic physics

In the past year several improvements in Geant4 hadronic physics code have been made, both for HEP and nuclear physics applications. We discuss the implications of these changes for physics simulation performance and user code. In this context several of the most-used codes will be covered briefly. These include the Fritiof (FTF) parton string model which has been extended to include antinucleon and antinucleus interactions with nuclei, the Bertinistyle cascade with its improved CPU performance and extension to include photon interactions, and the precompound and deexcitation models. We have recently released new models and databases for low energy neutrons, and the radioactive decay process has been improved with the addition of forbidden beta decays and better gamma spectra following internal conversion. As new and improved models become available, the number of tests and comparisons to data has increased. One of these is a validation of the parton string models against data from the MIPP experiment, which covers the largely untested range of 50 to 100 GeV. At the other extreme, a new stopped hadron validation will cover pions, kaons and antiprotons. These, and the ongoing simplified calorimeter studies, will be discussed briefly. We also discuss the increasing number of regularly performed validations, the demands they place on both software and users, and the automated validation system being developed to address them.

Yarba, Julia↗

Automated annotation of scientific texts for ML-based keyphrase extraction and validation

Advanced omics technologies and facilities generate a wealth of valuable data daily; however, the data often lack the essential metadata required for researchers to find, curate, and search them effectively. The lack of metadata poses a significant challenge in the utilization of these data sets. Machine learning (ML)–based metadata extraction techniques have emerged as a potentially viable approach to automatically annotating scientific data sets with the metadata necessary for enabling effective search. Text labeling, usually performed manually, plays a crucial role in validating machine-extracted metadata. However, manual labeling is time-consuming and not always feasible; thus, there is a need to develop automated text labeling techniques in order to accelerate the process of scientific innovation. This need is particularly urgent in fields such as environmental genomics and microbiome science, which have historically received less attention in terms of metadata curation and creation of gold-standard text mining data sets. In this paper, we present two novel automated text labeling approaches for the validation of ML-generated metadata for unlabeled texts, with specific applications in environmental genomics. Our techniques show the potential of two new ways to leverage existing information that is only available for select documents within a corpus to validate ML models, which can then be used to describe the remaining documents in the corpus. The first technique exploits relationships between different types of data sources related to the same research study, such as publications and proposals. The second technique takes advantage of domain-specific controlled vocabularies or ontologies. In this paper, we detail applying these approaches in the context of environmental genomics research for ML-generated metadata validation. Our results show that the proposed label assignment approaches can generate both generic and highly specific text labels for the unlabeled texts, with up to 44% of the labels matching with those suggested by a ML keyword extraction algorithm.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DES Y1 Results: validating cosmological parameter estimation using simulated Dark Energy Surveys

We use mock galaxy survey simulations designed to resemble the Dark Energy Survey Year 1 (DES Y1) data to validate and inform cosmological parameter estimation. When similar analysis tools are applied to both simulations and real survey data, they provide powerful validation tests of the DES Y1 cosmological analyses presented in companion papers. We use two suites of galaxy simulations produced using different methods, which therefore provide independent tests of our cosmological parameter inference. The cosmological analysis we aim to validate is presented in DES Collaboration et al. (2017) and uses angular two-point correlation functions of galaxy number counts and weak lensing shear, as well as their cross-correlation, in multiple redshift bins. While our constraints depend on the specific set of simulated realisations available, for both suites of simulations we find that the input cosmology is consistent with the combined constraints from multiple simulated DES Y1 realizations in the $\Omega_m-\sigma_8$ plane. For one of the suites, we are able to show with high confidence that any biases in the inferred $S_8=\sigma_8(\Omega_m/0.3)^{0.5}$ and $\Omega_m$ are smaller than the DES Y1 $1-\sigma$ uncertainties. For the other suite, for which we have fewer realizations, we are unable to be this conclusive; we infer a roughly 70% probability that systematic biases in the recovered $\Omega_m$ and $S_8$ are sub-dominant to the DES Y1 uncertainty. Furthermore, as cosmological analyses of this kind become increasingly more precise, validation of parameter inference using survey simulations will be essential to demonstrate robustness.

79 ASTRONOMY AND ASTROPHYSICS↗

Data-driven model validation for neutrino-nucleus cross section measurements

Neutrino-nucleus cross section measurements are needed to improve interaction modeling to meet the precision needs of neutrino experiments in efforts to measure oscillation parameters and search for physics beyond the Standard Model. We review the difficulties associated with modeling neutrino-nucleus interactions that lead to a dependence on event generators in oscillation analyses and cross section measurements alike. We then describe data-driven model validation techniques intended to address this model dependence. The method relies on utilizing various goodness-of-fit tests and the correlations between different observables and channels to probe the model for defects in the phase space relevant for the desired analysis. These techniques shed light on relevant mismodeling, allowing it to be detected before it begins to bias the cross section results. We compare more commonly used model validation methods which directly validate the model against alternative ones to these data-driven techniques and show their efficacy with fake data studies. These studies demonstrate that employing data-driven model validation in cross section measurements represents a reliable strategy to produce robust results that will stimulate the desired improvements to interaction modeling.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Validating first-principles molecular dynamics calculations of oxide/water interfaces with x-ray reflectivity data

Metal oxide/water interfaces play a crucial role in many electrochemical and photocatalytic processes, such as photoelectrochemical water splitting, the creation of fuel from sunlight, and electrochemical CO 2 reduction. First-principles electronic structure calculations can reveal unique insights into these processes, such as the role of the alignment of the oxide electronic energy levels with those of liquid water. An essential prerequisite for the success of such calculations is the ability to predict accurate structural models of these interfaces, which in turn requires careful experimental validation. Here we report a general, quantitative validation protocol for first-principles molecular dynamics simulations of oxide/aqueous interfaces. The approach makes direct comparisons of interfacial x-ray reflectivity (XR) signals from experimental measurements and those obtained from ab initio simulations with semilocal and van der Waals functionals. The protocol is demonstrated here for the case of the Al 2 O 3 (001)/water interface, one of the simplest oxide/water interfaces. We discuss the technical requirements needed for validation, including the choice of the density functional, the simulation cell size, and the optimal choice of the thermodynamic ensemble. Our results establish a general paradigm for the validation of structural models and interactions at solid/water interfaces derived from first-principles simulations. Furthermore, while there is qualitative agreement between the simulated structures and the experimental best-fit structure, direct comparisons of simulated and measured XR intensities show quantitative discrepancies that derive from both bulk regions (i.e., alumina and water) as well as the interfacial region, highlighting the need for accurate density functionals to properly describe interfacial interactions. Our results show that XR data are sensitive not only to the atomic structure (i.e., the atom locations) but also to the electron-density distributions in both the substrate and at the interface.

36 MATERIALS SCIENCE↗

An Innovative Software Tool Suite for Power Plant Model Validation and Parameter Calibration using PMU Measurements

Maintaining good quality of power plant stability models is of critical importance to ensure the secure and economic operation and planning of today’s power grid with its increasing stochastic and dynamic behavior. According to North American Electric Reliability (NERC) standards, all generators in North America with capacities larger than 10 MVA are required to validate their models every five years. Validation is quite costly and can significantly affect the revenue of generator owners, because the traditional staged testing requires generators to be taken offline. Over the past few years, validating and calibrating parameters using online measurements including phasor measurement units (PMUs) and digital fault recorders (DFRs) has been proven to be a cost-effective approach. In this paper, an innovative open-source tool suite is presented for validating power plant models using PPMV tool, identifying bad parameters with trajectory sensitivity analysis, and finally calibrating parameters using an ensemble Kalman filter (EnKF) based algorithm. The architectural design and the detailed procedures to run the tool suite are presented, with results of test on a realistic hydro power plant using PMU measurements for 12 different events. The calibrated parameters of machine, exciter, governor and PSS models demonstrate much better performance than the original models for all the events and show the robustness of the proposed calibration algorithm.

Li, Yuanyuan↗

Experimental validation benchmark data for CFD of transient convection from forced to natural with flow reversal on a vertical flat plate

Transient convection has been investigated experimentally for the purpose of providing Computational Fluid Dynamics (CFD) validation benchmark data. A specialized facility for validation benchmark experiments called the Rotatable Buoyancy Tunnel was used to acquire thermal and velocity measurements of flow over a smooth, vertical heated plate. The initial condition was forced convection downward with subsequent transition to mixed convection, ending with natural convection upward after a flow reversal. Data acquisition through the transient was repeated for ensemble-averaged results. With simple flow geometry, validation data were acquired at the benchmark level. All boundary conditions (BCs) were measured and their uncertainties quantified. Temperature profiles on all four walls and the inlet were measured, as well as as-built test section geometry. Inlet velocity profiles and turbulence levels were quantified using Particle Image Velocimetry. System Response Quantities (SRQs) were measured for comparison with CFD outputs and include velocity profiles, wall heat flux, and wall shear stress. Extra effort was invested in documenting and preserving the validation data. Details about the experimental facility, instrumentation, experimental procedure, materials, BCs, and SRQs are made available through this paper. As a result, the latter two are available for download and the other details are included in this work.

42 ENGINEERING↗

Validation Metrics for Deterministic and Probabilistic Data

Computational modeling and simulation are paramount to modern science. Computational models often replace physical experiments that are prohibitively expensive, dangerous, or occur at extreme scales. Thus, it is critical that these models accurately represent and can be used as replacements for reality. This paper provides an analysis of metrics that may be used to determine the validity of a computational model. While some metrics have a direct physical meaning and a long history of use, others, especially those that compare probabilistic data, are more difficult to interpret. Furthermore, the process of model validation is often application-specific, making the procedure itself challenging and the results difficult to defend. We therefore provide guidance and recommendations as to which validation metric to use, as well as how to use and decipher the results. Furthermore an example is included that compares interpretations of various metrics and demonstrates the impact of model and experimental uncertainty on validation processes.

97 MATHEMATICS AND COMPUTING↗

Verification and Validation of the FLAG Hydrocode for Impact Cratering Simulations

Verification and validation (V&V) are necessary processes to ensure accuracy of the computational methods used to solve problems key to vast numbers of applications and industries. Simulations are essential for addressing impact cratering problems, because these problems often exceed experimental capabilities. We show that the free Lagrange (FLAG) hydrocode, developed at Los Alamos National Laboratory (Los Alamos, NM), can be used for impact cratering simulations by verifying FLAG against two analytical models of aluminum-on-aluminum impacts at different impact velocities and validating FLAG against a glass-into-water laboratory impact experiment. Our verification results show good agreement with the theoretical maximum pressures, with relative errors as low in magnitude as 1.00%. Our validation results demonstrate FLAG's ability to model various stages of impact cratering, with crater radius relative errors as low as 3.48% and crater depth relative errors as low as 0.79%. Our mesh resolution study shows that FLAG converges at resolutions low enough to reduce the required computation time from about 28 h to about 25 min. We anticipate that FLAG can be used to model larger impact cratering problems with increased accuracy and decreased computational cost on current systems relative to other hydrocodes tested by Pierazzo et al. (2008, “Validation of Numerical Codes for Impact and Explosion Cratering: Impacts on Strengthless and Metal Targets,” MAPS, 43(12), pp. 1917–1938).

97 MATHEMATICS AND COMPUTING↗

Validation Metrics for Fixed Effects and Mixed-Effects Calibration

The modern scientific process often involves the development of a predictive computational model. To improve its accuracy, a computational model can be calibrated to a set of experimental data. A variety of validation metrics can be used to quantify this process. Some of these metrics have direct physical interpretations and a history of use, while others, especially those for probabilistic data, are more difficult to interpret. In this work, a variety of validation metrics are used to quantify the accuracy of different calibration methods. Frequentist and Bayesian perspectives are used with both fixed effects and mixed-effects statistical models. Through a quantitative comparison of the resulting distributions, the most accurate calibration method can be selected. Two examples are included which compare the results of various validation metrics for different calibration methods. It is quantitatively shown that, in the presence of significant laboratory biases, a fixed effects calibration is significantly less accurate than a mixed-effects calibration. This is because the mixed-effects statistical model better characterizes the underlying parameter distributions than the fixed effects model. The results suggest that validation metrics can be used to select the most accurate calibration model for a particular empirical model with corresponding experimental data.

97 MATHEMATICS AND COMPUTING↗

Experimental Validation of Subsystem Models for a Novel Variable Displacement Hydraulic Motor

A novel, variable displacement, low-speed high-torque hydraulic motor is being developed that is expected to be highly efficient across a broad operating range. To ensure the final hardware achieves the expected performance, the models used in the development of the motor must be experimentally validated and revised, as necessary. The specific focus in this work is on mechanical energy loss models that were used to guide the design of a single-cylinder motor prototype and on experimental tests used for model validation. Ideally each model, whether friction loss in a piston/cylinder interface or energy loss due to leakage in a valve , would be individually validated by an independent test. This granular approach would remove any question of where the error lies and result in highly accurate models. However, many motor components have multiple forms of energy loss, creating difficulty in validating individual losses. Additionally, it is not physically realizable to divide many components into an individual model equivalent test. Testing the motor as a full assembly is possible, but pinpointing the source of discrepancies between the model and the hardware becomes difficult with dozens of models potentially being partially responsible. A compromise was found by separating the motor into functional component groups that are characterized by the type of loss and ability to test each subcomponent independently. By checking for correlation between test observations and model predictions, revisions could be implemented into the models. This allows future solutions to be more accurately predicted in the design phase to drive the design of better machines.

24 POWER TRANSMISSION AND DISTRIBUTION↗

SYSTEMATIC MULTI-INDEX VALIDATIONS OF SIDI ENGINE FLOW FIELD LES COMPUTATIONS USING CRANK ANGLE-RESOLVED PIV MEASUREMENTS

In-cylinder flow fields make significant impacts on the fuel atomization, fuel mixture formation, and combustion process in spark ignition direct injection (SIDI) engines. In recent years, model-based simulation approaches are preferred in regard to investigating the transient in-cylinder flow field characteristics. Most commonly, the simulation models are validated using single representative flow field at a typical crank angle measured by particle image velocimetry (PIV). However, it provides only limited knowledge about the flow field which is highly three-dimensional and of transient nature. In this study, crank angle-resolved Ply measurements are conducted on three distinct planes inside the cylinder to capture the transient process of flow field characteristics which vary with the crank angle. These three planes consist of one tumble plane through the spark plug tip, one tumble plane along two intake ports, and one swirl plane at 30 mm below the cylinder head. Large eddy simulation (LES) is employed for the numerical computations using the CONVERGE codes. On the basis of large datasets for both temporal and spatial domains, a multi-index systematic validation approach is conducted. Crank angle-resolved calculations of global indices and local indices are implemented using the flow fields velocity data obtained from both PIV and LES on select planes. Global indices reveal the trends in similarities of different crank angle degrees and locations, while local indices give the detail comparison results. In summary, with the systematic multi-index validation approach, the crucial crank angle degrees and locations for model verification will be detected. Furthermore, the corresponding critical flow features are analyzed. Practical guideline of flow field validation is proposed.

Crank angle-resolved PIV↗

Importance of Second-Order Difference-Frequency Wave-Diffraction Forces in the Validation of a Fast Semi-Submersible Floating Wind Turbine Model: Preprint

To better access the abundant offshore wind resource, efforts across the world are being undertaken to develop and improve floating offshore wind turbine technologies. A critical aspect of creating reliable, mature floating wind turbine technology is the development, verification, and validation of efficient computer-aided-engineering (CAE) tools that can be relied upon in the design process. The National Renewable Energy Laboratory (NREL) has created a comprehensive, coupled analysis CAE tool for floating wind turbines, FAST, which has been verified and utilized in numerous floating wind turbine studies. Several efforts are currently underway that leverage the extensive 1/50th-scale DeepCwind wind/wave basin model test dataset, obtained at the Maritime Research Institute Netherlands (MARIN) in 2011, to validate the floating platform functionality of FAST to complement its already validated aerodynamic and structural simulation capabilities. In this paper, further work is undertaken to continue this validation. In particular, the ability of FAST to replicate global response behaviors associated with dynamic wind forces, second-order difference-frequency wave-diffraction forces and their interaction with one another are investigated.

17 WIND ENERGY↗

EXPERIMENTAL VALIDATION OF THEORETICAL BURST STRENGTH SOLUTION FOR DEFECT-FREE THICK-WALLED PIPES

The burst pressure of line pipes is an important strength property required in pipeline design and integrity management. Historically, the Barlow formula in conjunction with the ultimate tensile stress (UTS) of pipeline steels were utilized to estimate the burst strength of line pipes. However, the Barlow formula did not consider the plastic flow effect for ductile steels and is applicable only to thin-walled pipes. In 2006, the present author proposed a new multiaxial plastic yield theory and obtained a theoretical Zhu-Leis solution of burst strength for defect-free thin-walled pipes in term of UTS and strain hardening exponent n of pipeline steels. The Zhu-Leis solution has been validated by various burst test data for thin-walled pipelines for a wide range of steel grades from Grade B to X120. Recently, the present author extended the Zhu-Leis theory of plasticity to thick-walled pipes and obtained the Zhu-Leis solution of burst pressure for thick-walled pipes. The proposed burst pressure solution is applicable to both thin and thick-walled pipes. To experimentally validate the proposed theoretical burst pressure solution, this paper obtains a set of burst test data for three thick-walled pipes in Grade B carbon steel with a nominal diameter of 2.375 inches and three nominal wall thicknesses, resulting in D/t = 15.4, 10.9, 6.9. Through comparisons, these burst data validate the theoretical burst pressure solution for thick-walled pipes. Moreover, two additional burst test datasets collected from literature for thin and thick-walled pipes further validate the proposed burst pressure solution for both thin and thick-walled pipes.

Zhu, Xiankui↗