Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

Trustworthiness modeling and evaluation for a nearly autonomous management and control system

The Nearly Autonomous Management and Control (NAMAC) system supports the advanced reactor operation by recommending control actions to operators based on real-time measurements and digital twins (DTs) learning from the knowledge base. To enable the safe and reliable use of autonomous technologies, NAMAC and its recommendations should be trustworthy to operators and regulators at both the design and operation stages. This study proposes a NAMAC trustworthiness modeling and evaluation framework supported by trustworthiness ontologies and evidence-based approaches. The development-time and run-time ontologies are separately constructed and then converted to Bayesian networks to quantitatively evaluate the NAMAC trustworthiness. This evaluation is demonstrated by collecting and characterizing evidence from NAMAC practices, such as the development and assessment of the NAMAC system, data coverage assessment, and the training and optimizations of neural-network-based DTs. Our proposed approach can aggregate various trustworthiness attributes of complex artificial-intelligence-supported systems for safety-critical applications. It also considers the interaction between different DTs and extends beyond the trustworthiness evaluation of a single DT. In conclusion, the evidence-based method enhances the transparency of the trustworthiness modeling and evaluation processes and helps identify uncertainties and subjectivity involved in the processes.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Evaluation of CMIP6 GCMs Over the CONUS for Downscaling Studies

Despite the necessity of Global Climate Models (GCMs) sub-selection in downscaling studies, an objective approach for their selection is currently lacking. Building on the previously established concepts in GCMs evaluation frameworks, we develop a weighted averaging technique to remove the redundancy in the evaluation criteria and rank 37 GCMs from the sixth phase of the Coupled Models Intercomparison Project over the contiguous United States. GCMs are rated based on their average performance across 66 evaluation measures in the historical period (1981–2014) after each metric is weighted between zero and one, depending on its uniqueness. The robustness of the outcome is tested by repeating the process with the empirical orthogonal function analysis in which each GCM is ranked based on its sum of distances from the reference in the principal component space. The two methodologies work in contrasting ways to remove the metrics redundancy but eventually develop similar GCMs rankings. A disparity in GCMs' behavior related to their sensitivity to the size of the evaluation suite is observed, highlighting the need for comprehensive multi-variable GCMs evaluation at varying timescales for determining their skillfulness over a region. The sub-selection goal is to use a representative set of skillful models over the region of interest without substantial overlap in their future climate responses and modeling errors in representing historical climate. Additional analyses of GCMs' independence and spread in their future projections provide the necessary information to objectively select GCMs while keeping all aspects of necessity in view.

54 ENVIRONMENTAL SCIENCES↗

Informing Robust Functional Relationship Benchmarks: An Evaluation of the Temperature Sensitivity of Ecosystem Respiration Across the Arctic-Boreal Region

During land model development, simulated carbon dynamics are often benchmarked against observational data sets to evaluate model performance. Functional relationship benchmarks are the relationship between a driving variable (e.g., temperature) and a response variable (e.g., ecosystem respiration) and are a promising tool for assessing model performance by evaluating modeled sensitivities to changing environmental conditions. However, observed functional relationships can be influenced by choices made during data collection and throughout the benchmarking process, impacting the inferred skill of land models. To avoid misrepresenting a model's true performance, it is necessary to systematically evaluate best practices when constructing functional relationship benchmarks. We developed a set of guidelines for constructing functional relationship benchmarks, considering the choice of data set, number of daily observations, temporal extent, and temporal resolution across Alaska and Canada over a 20-year period from 2001 to 2020. The temperature sensitivity of ecosystem respiration from observations, evaluated through an apparent Q 10 , is highly variable both spatially and as a result of the data processing approach applied in the benchmark formation. When benchmarking 13 models from the Warming Permafrost Model Intercomparison Project (WrPMIP), the range in inferred model skill is substantially impacted by the choices applied in constructing functional relationship benchmarks. The inferred performance of a given model is most sensitive to the number of daily observations and temporal extent, followed by choice of benchmark data set and temporal averaging. Results from this analysis can guide the development of consistent and robust functional relationships for future model evaluation studies.

Poe, Jeralyn [Northern Arizona University, Flagsta↗

Evaluation method of beam instability in laser ion source using solenoid

In a laser ion source using solenoid field confinement, it is known that an ion beam becomes unstable in the certain range of a magnetic field. Although it is essential to quantify the instability when discussing the unstable region of the beam, it is difficult to evaluate the beam instability by peak current or the amount of charge because an irregular change of temporal profile occurs in addition to the shot-by-shot fluctuation of amplitude. In this study, I propose the most appropriate method to evaluate the beam instability using the difference from an average waveform. The validity of the new method was evaluated by comparing three evaluation methods (variation of maximum value, variation of integral of waveforms, and the proposed evaluation method) with the experimentally obtained waveforms with the stable and unstable regions of a solenoid field. The proposed method was verified to best represent the beam instability by laser-induced plasma.

43 PARTICLE ACCELERATORS↗

A New Approach to Evaluate and Reduce Uncertainty of Model-Based Biodiversity Projections for Conservation Policy Formulation

Biodiversity projections with uncertainty estimates under different climate, land-use, and policy scenarios are essential to setting and achieving international targets to mitigate biodiversity loss. Evaluating and improving biodiversity predictions to better inform policy decisions remains a central conservation goal and challenge. A comprehensive strategy to evaluate and reduce uncertainty of model outputs against observed measurements and multiple models would help to produce more robust biodiversity predictions. We propose an approach that integrates biodiversity models and emerging remote sensing and in-situ data streams to evaluate and reduce uncertainty with the goal of improving policy-relevant biodiversity predictions. In this work, we describe a multivariate approach to directly and indirectly evaluate and constrain model uncertainty, demonstrate a proof of concept of this approach, embed the concept within the broader context of model evaluation and scenario analysis for conservation policy, and highlight lessons from other modeling communities.

essential biodiversity variables↗

Crash Risks Evaluation of Urban Expressways: A Case Study in Shanghai

We report that proactive traffic safety management systems can reduce crashes by identifying crash precursors, evaluating real-time crash risks, and implementing suitable interventions. The basic prerequisite for developing such a system is to propose a reliable crash risk evaluation model that takes real-time traffic flow data as input. Previous studies have primarily focused on real-time crash prediction using some statistical or machine-learning methods. However, further quantitative evaluation and classification of crash risks have been ignored. In this study, we conduct a systematic crash risk evaluation workflow, including crash risk prediction, crash risk quantification, and crash risk classification. Specifically, the crash risk prediction using an extended logit model is proposed, from which CAS, CSD, UAS, DAS, DTV are identified to be contributing factors of crash risks. Then a crash risk quantification model based on the parameter evaluation of the extended logit model is developed. The crash risks of urban expressways and their spatial-temporal evolution trends are quantified. Finally, the crash risks are classified into high crash risk level, moderate crash risk level, and low crash risk level by the k-means cluster algorithm. Then the threshold boundaries of different crash risk levels are determined. The research results provide a proactive guidance for traffic safety management of urban expressways.

33 ADVANCED PROPULSION SYSTEMS↗

Object-Based Evaluation of Dynamical and Statistical Downscaled Precipitation Products over CONUS

High-resolution precipitation data, generated through dynamical downscaling (DD) or statistical downscaling (SD) of global climate model output, provide critical information for regional climate assessment and adaptation planning. Most downscaling development and validation have focused on accurate gridscale precipitation construction and ignored the spatial structure of precipitation across model grids and at the event scale. However, many applications, e.g., hydrologic modeling and the analysis using the downscaled precipitation, require a reasonable representation of the spatial structure of precipitation within watersheds. Therefore, a set of standard metrics to evaluate the representation of the spatial structure of individual storms across diverse downscaled precipitation products is desired. To address this need, we conducted an object-based evaluation of precipitation in decades-long DD and SD products over the contiguous United States (CONUS). Specifically, we evaluate their ability to reproduce various features of precipitation objects in the observations: total volume, precipitation area, peak intensity, and spatial structure. Multiple metrics (bias, Perkins score, and nonparametric statistical tests) are used to quantify model performance. Our evaluation reveals notable variations in performance among individual products across different climate zones and seasons, as well as between extreme and nonextreme events. In general, most DD products exhibit balanced performance across the four precipitation object features, while SD products vary more significantly in their performance across products. Based on this comprehensive evaluation, we provide guidance on choosing downscaled products for specific regions, seasons, and precipitation object features. These findings and recommendations can inform precipitation-relevant modeling and analysis over CONUS, guide future downscaling technique developments, and provide actionable information for climate impact assessment and adaptation.

Downscaling↗

Object-Based Evaluation of Dynamical and Statistical Downscaled Precipitation Products over CONUS

High-resolution precipitation data, generated through dynamical downscaling (DD) or statistical downscaling (SD) of global climate model output, provide critical information for regional climate assessment and adaptation planning. Most downscaling development and validation have focused on accurate gridscale precipitation construction and ignored the spatial structure of precipitation across model grids and at the event scale. However, many applications, e.g., hydrologic modeling and the analysis using the downscaled precipitation, require a reasonable representation of the spatial structure of precipitation within watersheds. Therefore, a set of standard metrics to evaluate the representation of the spatial structure of individual storms across diverse downscaled precipitation products is desired. To address this need, we conducted an object-based evaluation of precipitation in decades-long DD and SD products over the contiguous United States (CONUS). Specifically, we evaluate their ability to reproduce various features of precipitation objects in the observations: total volume, precipitation area, peak intensity, and spatial structure. Multiple metrics (bias, Perkins score, and nonparametric statistical tests) are used to quantify model performance. Our evaluation reveals notable variations in performance among individual products across different climate zones and seasons, as well as between extreme and nonextreme events. In general, most DD products exhibit balanced performance across the four precipitation object features, while SD products vary more significantly in their performance across products. Based on this comprehensive evaluation, we provide guidance on choosing downscaled products for specific regions, seasons, and precipitation object features. These findings and recommendations can inform precipitation-relevant modeling and analysis over CONUS, guide future downscaling technique developments, and provide actionable information for climate impact assessment and adaptation.

Environmental sciences↗

Observational Data for Next-Generation Climate Model Evaluation: Requirements, Considerations, and Best Practices

Climate model simulations are an important source of information about our planet’s climate system and also enable informed decision-making under different future scenarios. As a new archive of results from the next generation of climate models is anticipated to become available with the Coupled Model Intercomparison Project phase 7 (CMIP7), the need to develop efficient and robust methods to evaluate models is paramount. Observations are an integral part of model evaluation, providing a means to quantify and understand the degree to which climate models can faithfully reproduce Earth system processes. Such analysis is critical for constraining climate projections, identifying areas of focus for model development, and assisting analysts in deciphering the utility of models for specific applications. Observations of Earth system come from a diversity of sources, span different space–time domains, and are produced by different communities, and each dataset features different data structures and formats, metadata standards, and its own unique uncertainties. Uncertainties in an observational dataset may stem from gaps in temporal and spatial coverage, instrumentation errors, or assumptions in retrieval and processing methods. How then does one ensure that observational data are ready for use and utilized in the most appropriate way for robust, rapid, and routine climate model evaluation? The CMIP7 Model Benchmarking Task Team with input from the broader climate modeling, model evaluation, and observational data communities present a vision and considerations for best practices toward the optimal and appropriate use of observational data to support next-generation climate model evaluation.

Climate models↗

Multicriteria screening evaluation of geothermal resources on mine lands for direct use heating

Abstract Direct use of geothermal energy is the oldest and most versatile form of utilizing geothermal energy. In the last decade, this utilization has significantly increased, especially with the installation of geothermal (ground-source) heat pumps. Many current and inactive mine land sites across the U.S. could be redeveloped with clean energy technologies such as direct use geothermal, which would revitalize former mining communities, help with reducing greenhouse gas emissions, and accelerate the transition to a clean energy economy. We present a multicriteria screening framework to evaluate various aspects of direct-use geothermal projects on mine lands. The criteria are divided into three categories: (1) technical potential, (2) demand and benefits, and (3) regulatory and permitting. We demonstrate the framework using publicly available data on a national scale (continental U.S.). Then, using an example of abandoned coal mines in Illinois and focusing on resource potential, we illustrate how this evaluation can be applied at the state or more local scales when a region’s characteristics drive spatial variability estimates. The strength of this approach is the ability to combine seemingly disparate parameters and inputs from numerous sources. The framework is very flexible—additional criteria can be easily incorporated and weights modified if input data support them. Vice versa, the framework can also help identify additional data needed for evaluating those criteria. The multicriteria screening evaluation methodology provides a framework for identifying potential candidates for detailed site evaluation and characterization.

15 GEOTHERMAL ENERGY↗

An iron evaluation story: from TALYS model parameters to validation on the ASPIS benchmark with the Monte-Carlo code TRIPOLI-4

Iron is a widespread structural material in nuclear applications. However some benchmark studies show that for many libraries, including TENDL-2019, producing good iron evaluations is still an open challenge. This work aims at improving TENDL-2019 {sup 56}Fe and {sup 54}Fe evaluations through feedbacks from experimental data and Monte Carlo results of the iron benchmark ASPIS. Part of the T6 code infrastructure was used to produce the evaluations. In the new evaluations, particular attention to resonance parameters and formalism was paid to address the importance of elastic scattering. The total inelastic scattering and inelastic continuum cross sections were shaped by modifying the input parameters of TALYS, one of the T6 codes. Additionally, it was necessary to normalise some reaction channels to those of JENDL-4.0. With the new files ASPIS results, obtained with the Monte Carlo code TRIPOLI-4, improved noticeably compared to TENDL 2019. The evaluations and parameter setting modifications are to be included into TENDL-2021. (authors)

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Bayesian Monte Carlo Evaluation Framework for Cross Sections Nuclear Data and Integral Benchmark Experiments

The new Bayesian Monte Carlo (MC) evaluation framework described in this abstract has been conceived as an attempt to improve nuclear data evaluations of differential crosssection data by removing the following two approximations conventionally employed for nuclear data evaluations: all probability density functions (PDFs) of all data and model parameters, both prior and posterior, are assumed to be normal (i.e., Gaussian) PDFs, and all uncertainties and covariances are propagated using a linear approximation. With these approximations removed, the Bayesian MC (BMC) framework could be used to account for nonlinear effects and would enable improved evaluations of differential cross sections and IBE data that are presently performed based on the assumptions itemized above. The BMC would also improve upon the uniform sampling of IBE parameters from within ranges defined by their evaluated uncertainties.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Bayesian Monte Carlo Evaluation Framework for Imperfect Nuclear Data

Bayesian evaluation of resolved resonance region (RRR) nuclear data has historically been carried out using the generalized least squares (GLS) formalism, as implemented in, e.g., SAMMY. We have recently developed a prototype of Bayesian Monte Carlo (BMC) evaluation framework, implemented using a Markov Chain Monte Carlo (MCMC) method with a Metropolis-Hastings (MH) acceptance criterion. This was done in order to remove the approximations underlying the conventional GLS evaluations, namely, the linear approximation, and the approximation that all probability density functions (PDFs) are of the normal kind. Recent works by others have used similar stochastic approaches to quantify cross section uncertainties from ENDF evaluated co-variances, and/or, from integral benchmark data, but those have not been conceived as an evaluation framework like the one presented here.

97 MATHEMATICS AND COMPUTING↗

Dataset for "Evaluating Deep Learning Approaches for Predictions in Unmonitored Basins with Continental-scale Stream Temperature Models" Willard et al. (2024)

This data release provides all data and code used in the paper " "Evaluating Deep Learning Approaches for Predictions in Unmonitored Basins with Continental-scale Stream Temperature Models" Willard et al. (2024)" to model stream temperature, evaluate, and assess results. The associated manuscript explores current open questions in prediction in ungauged and unmonitored basins concerning top-down versus bottom-up approaches, tradeoffs between data available and input requirements, and the appropriate representation of catchment attributes as inputs to deep learning models. Modeling was done primarily with long short-term memory (LSTM) models, and stream site coverage spans 1362 locations across the conterminous United States. The data is organized into these items items:Code repository and data for the paper " "Evaluating Deep Learning Approaches for Predictions in Unmonitored Basins with Continental-scale Stream Temperature Models" Willard et al. (2024)".Code: stream_temp_ml_regionalization.zip contains the code repositoryData to run the code: - data_dir.zip -- contains all files that should be moved to the "DATA_DIR" variable defined in the "set_env_vars.sh" script in the code repository- metadata_dir.zip -- contains all files that should be moved to the "METADATA_DIR" variable defined in the "set_env_vars.sh" script in the code repository- error_analysis_attribute_and_groundwater_dir.zip - workflows for the extended error analysis by stream attribute and groundwater influenceData produced by the code and used in the paper:- outputs_dir.zip - contains model output and results (outputs_dir/results), model weights (outputs_dir/models), and all other outputs used for the paper including feature importances.To cite this code, please use the following BibTeX or MLA entries:bibtex:@misc{willard2024streamdata, author = {Jared Willard and Fabio Ciulla and Helen Weierbach and Vipin Kumar and Charuleka Varadharajan}, title = {Dataset for "Evaluating Deep Learning Approaches for Predictions in Unmonitored Basins with Continental-scale Stream Temperature Models"}, year = {2024}, doi = {10.15485/2448016}, publisher = {ESS-DIVE Repository}, url = {https://doi.org/10.15485/2448016}}MLA: Willard, Jared, et al. Dataset for "Evaluating Deep Learning Approaches for Predictions in Unmonitored Basins with Continental-scale Stream Temperature Models". 2024. ESS-DIVE Repository, doi:10.15485/2448016.

54 ENVIRONMENTAL SCIENCES↗

Selection of Vadose Zone Flow and Transport Properties with Gravel Fraction Corrections for the Hanford Site Composite Analysis and Cumulative Impact Evaluation

This environmental calculation file (ECF) is a compilation of hydrologic and radiological/chemical properties to be used for the updated Hanford Site Composite Analysis (CA) and Cumulative Impact Evaluation (CIE) vadose zone (VZ) modeling. The CA will provide an all-pathways dose projection to a hypothetical future member of the public from all planned low-level radioactive waste disposal facilities and potential contributions from all other projected end-state sources of radioactive material left at the Hanford Site following site closure. Its primary purpose is to support the decision-making process of the U.S. Department of Energy (DOE) under DOE O 435.1 Chg 1, Radioactive Waste Management, related to managing low-level waste disposal facilities at the Hanford Site. The CIE evaluates the effects of cleanup decisions regarding groundwater quality in the Hanford Site Central Plateau (DOE/RL-2018-69, Cumulative Impact Evaluation Technical Approach Document). Due to the complexity and large number of waste sites in source operable units (OUs), the computational tools used for the CIE must be capable of representing a range of site conditions and source terms in the VZ while also efficiently computing the impact that cleanup decisions have on the underlying aquifer. Waste-site proximity between and within source OUs has resulted in contaminants commingling in the vadose and saturated zones in complex ways. Plume commingling requires cleanup decisions to be evaluated considering the surrounding waste sites and existing groundwater contamination, therefore demonstrating the need to evaluate cumulative impacts from the VZ to groundwater.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Evaluation of Hydrogen Generation in High Burnup Demonstration Dry Storage Cask

This report provides a best-estimate evaluation of residual water content (post-dry out) in the High Burnup (HBU) LWR Spent Fuel Demonstration project TN-32 cask, and evaluates the radiolysis of the residual free water, and the physisorbed and chemisorbed waters on the surfaces of the fuel and cask internal contents. The evaluation of radiolytic breakdown of those waters with gamma radiation causing the generation of hydrogen gas (H 2 ) is made using available literature data and models. This evaluation is part of the overall materials performance evaluation of the SNF-in-canister system, and is part of the technical bases for their continued safe dry storage. The TN-32 cask contents included 32 HBU LWR spent fuel assemblies each with 264 fuel rods clad in zirconium alloys, aluminum neutron absorber components, and aluminum and stainless steel structural components. The residual free and surface (physisorbed/chemisorbed) waters are ascribed to water vapor in the free volume and to components’ surfaces, respectively. The total potential radiolytic hydrogen inventory from the water vapor and from waters ascribed to surfaces has been calculated assuming all the water produced molecular H 2 . The residual water that is chemically incorporated into the bulk of a hydrated oxide, i.e., chemisorbed water, and its total potential hydrogen inventory has been calculated. These calculations are at the physical limit of material available and are used for a bounding assessment purpose only.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Data Evaluation of Actinide Cross Sections: 230 Th and 231 Th

A new evaluation of the ENDL cross section set for Thorium (Z=90) is developed using the TALYS statistical model cross section code. The primary goal of this effort is to produce an evaluation that attempts to match as closely as possible fission cross sections developed through surrogate reaction techniques on actinide targets 230 Th and 231 Th. This evaluation effort and the processing needed to render its results into data libraries is a necessary step in making the efforts of nuclear experimentalists useful to the broad community of researchers engaged in simulations of nuclear fusion for basic and applied science. Another aspect, verification and validation against various AGEX experiments, is also presented. The end-product is an updated library that includes the latest measurements of fission cross sections for comparison against those measured via traditional techniques. All the steps in the evaluation, processing, validation and verification, and library release are described in the following sections. For completeness, the appendix contains all the parameters used in the TALYS cross section evaluation for neutrons incident on 230 Th and 231 Th.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Report Series: Evaluation, Finding of Effect, and Mitigation Documentation for the Main Gate (23-GS100), Mercury, Area 23, Nevada National Security Site, Nye County, Nevada

The U.S. Department of Energy (DOE) National Nuclear Security Administration Nevada Field Office (NNSA/NFO) plans to replace the existing guard shack at the Main Gate (23-GS100, Nevada State Historic Preservation Office [SHPO] Resource No. S1758) to the Nevada National Security Site (NNSS) in Nye County, Nevada. The purpose of the project is to improve security. The project is considered an undertaking subject to review under Title 54 of United States Code (USC) § 306108, commonly known as Section 106 of the National Historic Preservation Act, Title 54 USC § 300101, et seq., and its implementing regulations, Title 36 of the Code of Federal Regulations (36 CFR) Part 800. In 2018, Desert Research Institute (DRI) completed an architectural survey of the town of Mercury. This effort resulted in the identification, recordation, and evaluation of the Mercury Historic District (MHD, SHPO Resource No. D230), including the identification of its contributing elements (Reno et al. 2018). The MHD was recommended eligible for listing in the National Register of Historic Places (NRHP, National Register) under the Secretary of the Interior’s (SOI) Significance Criteria A and C, as defined in 36 CFR Part 60.4, as a significant concentration of buildings and structures with a direct and important association with Cold War-era nuclear testing from 1951 through 1992. It has not been evaluated under Criteria B and D to date. As part of a larger modernization program for Mercury, the NNSA/NFO and the SHPO executed the 2018 Programmatic Agreement between the National Nuclear Security Administration Nevada Field Office and the Nevada State Historic Preservation Officer regarding Modernization and Operational Maintenance of the Nevada National Security Site, at Mercury in Nye County, Nevada (Mercury PA). The Mercury PA includes streamlined Section 106 procedures for undertakings in the MHD based on contributing element categories. The Main Gate is identified in Appendix C of the Mercury PA as a Category I contributing element, indicating that it might be individually eligible for the NRHP. It is a historic property for the purposes of Section 106 compliance and subject to the stipulations of the Mercury PA. Per Stipulation VI of the Mercury PA, when the Area of Potential Effect (APE) for an undertaking includes Category I elements, the NNSA/NFO must evaluate the Category I elements for individual NRHP eligibility under all of the SOI Significance Criteria, prior to initiating any activity that may affect the elements. Thus, the purpose of this report is to evaluate the Main Gate as a potential individually eligible historic property in fulfillment of Stipulation VI of the Mercury PA. The evaluation detailed herein concludes that the Main Gate is individually eligible for listing in the NRHP under SOI Significance Criterion A at the national level of significance for its direct, important association with Cold War-era nuclear testing from 1965 (the date the current Main Gate was constructed) through 1992 (when critical nuclear testing on the NNSS ceased).

23-GS100↗