SEARCH · Engineering Papers

Results for “Evaluations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

All sources NASA NTRS DOE OSTI

NASA NTRS2011

Land Surface Verification Toolkit (LVT) - A Generalized Framework for Land Surface Model Evaluation

Model evaluation and verification are key in improving the usage and applicability of simulation models for real-world applications. In this article, the development and capabilities of a formal system for land surface model evaluation called the Land surface Verification Toolkit (LVT) is described. LVT is designed to provide an integrated environment for systematic land model evaluation and facilitates a range of verification approaches and analysis capabilities. LVT operates across multiple temporal and spatial scales and employs a large suite of in-situ, remotely sensed and other model and reanalysis datasets in their native formats. In addition to the traditional accuracy-based measures, LVT also includes uncertainty and ensemble diagnostics, information theory measures, spatial similarity metrics and scale decomposition techniques that provide novel ways for performing diagnostic model evaluations. Though LVT was originally designed to support the land surface modeling and data assimilation framework known as the Land Information System (LIS), it also supports hydrological data products from other, non-LIS environments. In addition, the analysis of diagnostics from various computational subsystems of LIS including data assimilation, optimization and uncertainty estimation are supported within LVT. Together, LIS and LVT provide a robust end-to-end environment for enabling the concepts of model data fusion for hydrological applications. The evolving capabilities of LVT framework are expected to facilitate rapid model evaluation efforts and aid the definition and refinement of formal evaluation procedures for the land surface modeling community.

Kumar, Sujay V.↗

NASA NTRS2012

The Strengths and Limitations of Satellite Data for Evaluating Tropospheric Processes in Chemistry-Climate Models

There is now a wealth of satellite data products available with which to evaluate a model fs simulation of tropospheric composition and other model processes. All of these data products have their strengths and limitations that need to be considered for this purpose. For example, uncertainties are introduced into a data product when 1) converting a slant column to a vertical column and 2) estimating the amount of a total column of a trace gas (e.g., ozone, nitrogen dioxide) that resides in the troposphere. Oftentimes, these uncertainties are not well quantified and the satellite data products are not well evaluated against in situ observations. However, these limitations do not preclude us from using these data products to evaluate our model processes if we understand these strengths and limitations when developing diagnostics. I will show several examples of how satellite data products are being used to evaluate particular model processes with a focus on the strengths and limitations of these data products. In addition, I will introduce the goals of a newly formed team to address issues on the topic of "satellite data for improved model evaluation and process studies" that is established in support of the IGAC/SPARC Global Chemistry ]Climate Modeling and Evaluation Workshop.

Duncan, Bryan↗

NASA NTRS2014

Evaluation of Human and Anthropomorphic Test Device Finite Element Models under Spaceflight Loading Conditions

In an effort to develop occupant protection standards for future multipurpose crew vehicles, the National Aeronautics and Space Administration (NASA) has looked to evaluate the test device for human occupant restraint with the modification kit (THOR‐K) anthropomorphic test device (ATD) in relevant impact test scenarios. With the allowance and support of the National Highway Traffic Safety Administration, NASA has performed a series of sled impact tests on the latest developed THOR‐K ATD. These tests were performed to match test conditions from human volunteer data previously collected by the U.S. Air Force. The objective of this study was to evaluate the THOR‐K finite element (FE) model and the Total HUman Model for Safety (THUMS) FE model with respect to the tests performed. These models were evaluated in spinal and frontal impacts against kinematic and kinetic data recorded in ATD and human testing. Methods: The FE simulations were developed based on recorded pretest ATD/human position and sled acceleration pulses measured during testing. Predicted responses by both human and ATD models were compared to test data recorded under the same impact conditions. The kinematic responses of the models were quantitatively evaluated using the ISO‐metric curve rating system. In addition, ATD injury criteria and human stress/strain data were calculated to evaluate the risk of injury predicted by the ATD and human model, respectively. Results: Preliminary results show well‐correlated response between both FE models and their physical counterparts. In addition, predicted ATD injury criteria and human model stress/strain values are shown to positively relate. Kinematic comparison between human and ATD models indicates promising biofidelic response, although a slightly stiffer response is observed within the ATD. Conclusion: As a compliment to ATD testing, numerical simulation provides efficient means to assess vehicle safety throughout the design process and further improve the design of physical ATDs. The assessment of the THOR‐K and THUMS FE models in a spaceflight testing condition is an essential first step to implementing these models in the computational evaluation of spacecraft occupant safety. Promising results suggest future use of these models in the aerospace field.

Putnam, Jacob P.↗

NASA NTRS2016

Post-Flight Back Pain Following International Space Station Missions: Evaluation of Spaceflight Risk Factors

INTRODUCTION Back pain during spaceflight has often been attributed to the lengthening of the spinal column due to the absence of gravity during both short and long-duration missions. Upon landing and re-adaptation to gravity, the spinal column reverts back to its original length thereby causing some individuals to experience pain and muscular spasms, while others experience no ill effects. With International Space Station (ISS) missions, cases of back pain and injury are more common post-flight, but little is known about the potential risk factors. Thus, the purpose of this project was to perform an initial evaluation of reported post-flight back pain and injury cases to relevant spaceflight risk factors in United States astronauts that have completed an ISS mission. METHODS All US astronauts who completed an ISS mission between Expeditions (EXP) 1 and 41 (2000-2015) were included in this evaluation. Forty-five astronauts (36 males and 9 females) completed 50 ISS missions during the study time period, as 5 astronauts completed 2 ISS missions. Researchers queried medical records of the 45 astronauts for occurrences of back pain and injury. A case was defined as any reported event of back pain or injury to the cervical, thoracic, lumbar, sacral, or coccyx spine regions. Data sources for the cases included the Flight Medicine Clinic's electronic medical record; Astronaut Strength, Conditioning and Rehabilitation electronic documentation; the Private Medical Conference tool; and the Space Medicine Operations Team records. Post-flight cases were classified as an early case if reported within 45 days of landing (R + 45) or a late case if reported from R + 46 to R + 365 days after landing (R + 1y). Risk factors in the astronaut population for back pain include age, sex, prior military service, and prior history of back pain. Additionally, spaceflight specific risk factors such as type of landing vehicle and onboard exercise countermeasures were included to evaluate their contribution to post-flight cases. Prior history of back pain included back pain recorded in the medical record within 3 years prior to launch. Landing vehicle was included in the model to discern if more astronauts experienced back pain or injury following a Shuttle or Soyuz landing. Onboard exercise countermeasures were noted for those astronauts who had a mission following 2009 deployment of the Advanced Resistive Exercise Device (aRED) (EXP 19 to 41). T-test and chi-squared tests were performed to evaluate the association between each individual risk factor and post-flight case. Logistic regression was used to evaluate the combined contribution of all the risk factors on post-flight cases. Separate models were calculated for cases reported by R + 45 and R + 1y. RESULTS During the study time period, there were 13 post-flight cases reported by R + 45 and an additional 5 reported by R + 1y. Most of these cases have been reported since EXP 19 with 10 cases by R + 45 and 4 by R + 1y. Individual risk factors of age, sex, landing vehicle, and prior military service were not significantly associated with post-flight cases identified at R + 45 or R + 1y (p greater than 0.05). Having back pain or injury within 3 years prior to launch significantly increased the likelihood of becoming a case by R + 1y (p = 0.041), but not at R+45 (p=0.204). Additionally, astronauts who experienced onboard exercise countermeasures that included aRED had a significantly increased risk of becoming a case at R + 45 (p = 0.024) and R + 1y (p=0.003). Multiple logistic regression evaluating all the risk factors for cases identified no significant risk factors at either the R + 45 or R + 1y time period (p greater than 0.05). Overall model fit was poor for both the R + 45 (R(exp 2) = 0.132) and R + 1y (R(exp 2) = 0.186) cases showing that there are risk factors not represented in our model. CONCLUSIONS Regardless of cause, post-flight cases are reported more often since aRED was deployed in 2009. This may reflect improved documentation or unidentified risk factors. No spaceflight risk factor explains the data fully. Post-flight cases are probably due to multi-faceted factors that are not easily elucidated in the medical data.

Laughlin, M. S.↗

NASA NTRS2018

2.5/3D Daisy Chain Reliability Evaluation

The objective of this task is to evaluate thermal cycle behavior of advanced 2.5/3D electronics — commercial-off the-shelf (COTS) —packages of different configurations assembled onto printed circuit boards (PCBs). Three key approaches were considered for evaluation. The first approach focused on the through-mold via (TMV™) technology at assembly level. The second approach focused on evaluation of 2.5D (also known as, System in Package (SiP)) in fine pitch ball grid array (FPGA). The third approach focused on evaluation of through-silicone-via (TSV) technologies. This report presents the test results for TMV™ and SiP packaging technologies and reliability, and it also provides lessons learned on quality assurance methods. Specifically, the report presents the test matrix for various 3D TMV™ packaging assembly configurations and reliability characterizations performed under two accelerated thermal cycling (ATC) and accelerated thermal shock cycling (ATSC) conditions. The ATC and ATSC were performed in the range of –55°C to 125°C and –100°C to 100°C, respectively. The report also includes assembly of a SiP— a 60-mm fine-pitch ball-grid array (FPGA) interposer with an IC at the center and six chip-scale package (CSP) daisy chains at the periphery—with package assembly characterizations. After assembly, the daisy chains were subjected to ATCs in the range of –40°C to 125°C for reliability evaluation. Finally, for both TMV™ and SiP packaging configurations, characterizations results by X-ray, optical imaging, and daisy-chain resistance evaluations were also presented.

Ghaffarian, Reza↗

NASA NTRS2019

Scheduling Improvements Following the Phase 1 Field Evaluation of the ATD-2 Integrated Arrival, Departure, and Surface Concept

NASA is conducting the Airspace Technology Demonstration-2 to evaluate an Integrated Arrival, Departure, and Surface (IADS) traffic management system that extends traffic sequencing for the entire life-cycle of a flight from departure gate to arrival gate within multi-airport, metroplex environments. After development and testing in human-in-the-loop simulations, the IADS system was deployed to Charlotte Douglas International Airport for a three-year field evaluation. From the initial IADS concept development through the end of the Phase 1 field evaluation many lessons were learned with regards to the IADS scheduler. In this paper we describe how data from the Phase 1 field evaluation helped identify scheduler improvements and guided the implementation of refinements. The improvements in the IADS scheduler described in this paper are incorporated into the IADS Phase 2 scheduler enabling strategic Surface Metering Programs and will be evaluated during the field evaluation.

Coupe, William J.↗

NASA NTRS2019

In-Flight Evaluation of the Traffic Aware Planner on the NASA HU-25A Guardian Aircraft

NASA’s Traffic Aware Planner (TAP) software is a research-prototype decision support tool that provides pilots with time- and fuel-saving route recommendations that optimize their current trajectory. The software runs on a first-of-a-kind system architecture onboard three aircraft in revenue service conducting operational evaluations with a major domestic airline. Therefore, significant NASA-internal testing is required prior to releasing the software to the partner airline. This paper describes a flight test plan that exercises the functionality of the TAP software in a representative operational environment, describes the system architecture developed and implemented for the NASA Langley HU-25A Guardian aircraft to support the test objectives, presents outcomes of the flight test campaign, and discusses use cases that demonstrate the value of flight testing for this activity.Research into flight path optimization of transport aircraft conducted by the National Aeronautics and SpaceAdministration (NASA) has produced an operational concept known as Traffic Aware Strategic Aircrew Requests(TASAR) [1, 2]. This near-term concept [3] provides the aircrew with a flight deck decision support tool known asthe Traffic Aware Planner (TAP). The TAP software leverages a growing number of information sources on the flightdeck to make time- and fuel-saving route optimization recommendations to the aircrew while en route. The aircrewcan then use the suggestions provided by the tool to make route change requests with a greater likelihood of acceptanceby air traffic control (ATC). Since TASAR is a concept intended for the current operational environment, it isintentionally designed to have no safety-critical impact or require any changes to current Federal AviationAdministration (FAA) rules and procedures [4, 5].The research prototype TAP system [6–8], explained further in Section III.C, continually incorporates up-to-dateaircraft state data from onboard avionics, as well as the latest position of surrounding traffic, the most recent windforecast, and the most recent convective weather forecast, in order to calculate candidate trajectory modifications thatimprove upon the current active route. These trajectories account for user-selectable objective functions [3] of reducedfuel burn, reduced flight time, or an airline-derived combination of factors known as trip cost. Previous analyses andsimulations have estimated substantial savings for airlines employing this technique within the U.S. National AirspaceSystem (NAS) [9–11]. Operational evaluations with Alaska Airlines seek to validate these projected benefits usingmeasured data while simultaneously providing benefits to the airline [12, 13].The TAP software has undergone a number of human-in-the-loop simulations [14] and flight test activities[15–17] in order to validate the operational concept, evaluate human factors considerations (e.g., workload, usability,distraction, etc.), and to assess the ability of the software to function in a representative operational environment (e.g.,connected to live avionics data, using in-flight internet connectivity, etc.). However, these simulations and flight testcampaigns did not account for the hardware architecture implemented on the three aircraft for Alaska Airlines’operational evaluations of the TAP software. Therefore, a need was identified to thoroughly test the functionality ofthe software in a similar hardware architecture to that of the partner airline’s aircraft. Information regarding testapparatus and environments used to evaluate TAP prior to testing on the HU-25A can be found in reference [18].A campaign of flight trials on a NASA aircraft, the HU-25A Guardian, was conducted to ensure that the researchprototype TAP system functions well in a configuration similar to the Alaska Airlines aircraft prior to deployment.This airborne, networked environment enables an assessment of the operational factors unique to the flight environment. Additionally, this activity evaluated the effectiveness and benefit of new TAP functionality andoperation in a relevant flight environment while allowing the rapid prototyping of new concepts and features.This paper is organized as follows: Section II discusses the details of the flight test plan, flight profiles, and theduties of personnel involved with conducting flight operations. Section III describes the test platform, avionicsequipage, and system architecture. Section IV presents a discussion of results, and Section V contains concludingremarks.

Underwood, Matthew C.↗

NASA NTRS2009

Evaluation methodology guidance for stack packages

This report provides evaluation methodology guidance based on previous National Aeronautics and Space Administration (NASA) reports and literature surveys for 3D stack packages and assemblies. Two aspects of technology are covered: the package itself (guidance for functional packages) and package assembly manufacturing and reliability. This work was funded by the NASA Electronic Parts and Packaging (NEPP) Program. The objectives of this NEPP project are to: Perform a literature survey of 3D stack technology; perform a literature survey on the evaluation methodology for 3D package and assembly; combine the two aspects to provide evaluation methodology for both aspects with consideration of interactions between package and assembly; generate guidance on the evaluation methodology for 3D stack package integrity prior to and after assembly; provide recommendations on future experimental activities. The qualification and evaluation methodology guidelines will facilitate NASA projects in effectively evaluating the reliability of very dense and newly available high-density 3D stack packages, allowing more processing power in a smaller board footprint and lower system weight.

Gerke, David↗

NASA NTRSDate not supplied

Towards an Aviation Large Language Model by Fine-tuning and Evaluating Transformers

In the aviation domain, there are many applications for machine learning and artificial intelligence tools that utilize natural language. For example, there is a desire to know the commonalities in written safety reports such as voluntary post incidents reports or aerial wildfire operations reports to better understand the risks present. Another use-case is the possibility of extracting airspace procedures and constraints currently written in documents such as Letters of Agreement. These applications can benefit from the use of state-of-the-art natural language processing techniques when adapted to the language/phraseology specific to the aviation domain. This paper evaluates the viability of adaptation of NLP tools to the aviation domain by fine-tuning transformer based models using aviation data sets. In 2018, a novel language model based on neural units (also called transformers) was created and became known as “Bidirectional Encoder Representations from Transformers” or BERT. This architecture combined with large amounts of English training data and innovative semi-supervised training tasks set the standard for what would later emerge as Large Language Models. The performance of these models was further improved by hyperparameter tuning and refinement of the semi-supervised training task and resulted in “Robustly Optimized BERT Pre-training Approach through hyperparameter tuning” or RoBERTa models. These pre-trained Large Language Models proved to be useful for a wide variety of natural language processing tasks such as text classification and question answering through a process called fine-tuning. The transformer architecture with pre-trained weights served as the basis with the last few layers replaced with layers fine-tuned to perform a new task e.g., a layer that provides a label for the entire input text. This process of fine-tuning can also be used to adapt the models to new domains; e.g., BioBERT started with the pre-trained BERT model and was completed by additional fine-tuning and training on biomedical documents. Transformer-based architectures can also be used to create rich representations of text called embeddings which can serve as the input to other machine learning models. This allows simpler algorithms such as logistic regression to use context-rich representations of the text while still remaining quick to train and evaluate. In the world of aviation, there is a growing demand for natural language processing and understanding but the domain presents unique challenges. Due to the technical content (and specialized language) of most aviation documents, fine-tuning pre-trained Large Language Models to specific tasks has not met the benchmark on natural language processing tasks set by simpler models trained from scratch on the data. To address this deficiency, this paper evaluates the improvements from fine-tuning a Large Language Model on a large set of aviation documents using the original semi-supervised training tasks before performing specific natural language tasks. In fine-tuning, a domain-specific dataset is used on the original training task but with the pre-trained Large Language Model instead of starting from a random initialization. This approach allows the model to be adapted to the specific domain language without discarding the information gained from training on general English data. This paper utilized two major dataset types to train and assess the RoBERTa fine-tuning performance. The first are 7,057 Letters of Agreement which are Federal Aviation Administration (FAA) documents that formalize airspace operations across the national airspace system. They contain many examples of ‘aviation English’ using domain specific terminology and phrasing which serves as a representative basis to perform the semi-supervised fine-tuning. The second type is the 494 document classification labels to be used for evaluation. This down-stream evaluation aims to show the performance of the fine-tuned model, better understand how much data is needed for an effective fine-tuning, and how fine-tuning can be adapted for different applications in-the domain. After semi-supervised training, evaluation begins by encoding the documents for classification using the fine-tuned RoBERTa model. Then a logistic regression classifier is trained to label the document type and compared against our ground truth labels. This currently leads to a 82.8% accuracy on 10-fold cross validation showing improvement over baseline RoBERTa which achieved 81.0%. We plan to measure the improvements on additional tasks and it is expected that these improvements will lead to more robust models that can tackle the natural language processing challenges present in aviation datasets.

ATM↗

DOE OSTI2022

Bayesian Monte Carlo Evaluation Framework for Imperfect Data and Models [Abstract]

Nuclear data evaluation methods conventionally make the following assumptions: prior and posterior probability distribution functions (PDFs) of all model parameters and data are normal (Gaussian); the linear approximation is sufficiently accurate for minimization of a cost function (even for non-linear models); and that both the model (of, e.g., neutron cross section) and experimental data (including their covariance data) are perfect. These assumptions are inherent to the well-known generalized linear least squares (GLLS) minimization method commonly used for evaluations of resolved resonance region (RRR) neutron cross sections. However, these assumptions are often not justified due to the presence of non-normal PDFs, non-linear models (e.g. R -matrix formalism), and inherent imperfections in data and models (e.g. discrepant data sets, discrepancies between the previous evaluation and newly measured data, or imperfect covariance data). We remove the said assumptions in a mathematical framework of Bayes’ theorem, and implement it using the Metropolis-Hastings Monte Carlo method. Parameters of a new kind are introduced to parameterize inherent imperfections, e.g. , any discrepancies between the theoretical model and measured data. These new parameters enable evaluators to quantify their expert judgement about any discrepancies or imperfections in a reproducible manner. We demonstrate the framework with an ongoing evaluation of 233 U in the eV region using the ENDF-B/VIII library and transmission data measured by Guber, et al. , and compare the posterior parameters to those obtained by conventional evaluation methods.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2023

Using High-Resolution NSRDB Data to Evaluate Cloud Mask Forecast from WRF-Solar EPS

Validating spatiotemporal distributions of cloud forecasts using numerical weather prediction (NWP) models is difficult as this requires high-quality cloud-property at significantly high spatial and temporal resolution over extended periods of time. Observations of cloud properties, such as cloud mask, cloud optical thickness, and cloud type, are vital for assessing the capability of NWP models to forecast various types of clouds. Using the National Solar Radiation Database (NSRDB), this research evaluates ensemble cloud-mask predictions from the WRF-Solar ensemble prediction system (WRF-Solar EPS). From the WRF-Solar EPS, day-ahead solar forecasts for the contiguous United States (CONUS) for 2018 are simulated. Given the NSRDB data is accessible at a resolution of 2 km, we can calculate the cloud fraction across the 9-km grid of WRF-Solar EPS. This allows us to spatially assess the cloud-mask forecasts using two methods against the high-resolution NSRDB: (a) considering all 2-km NSRDB clouds in the forecast domain (EMAll), and (b) using a minimum cloud fraction threshold of 50% to designate a pixel as cloudy (EMP50). The low-resolution cloud masks from WRF-Solar EPS are evaluated directly against the cloud-resolving scale gridded observations from NSRDB using EMAll. With EMP50, we presume that scenes with less than 50% cloud cover from the 2-km NSRDB are clear. Thus, this assessment approach allows for a fair comparison with WRF-Solar EPS resolved to a 9-km grid. A method of point-by-point verification is used to evaluate dichotomous (yes/no) cloud mask predictions against the NSRDB. For each pixel of model extent, cloud frequency and traditional metrics (e.g., probability of detection, false alarm rate, and hit rate, etc.) are computed and compared with satellite-derived data sets. Mismatched cloud frequency (MCF) is computed to measure the present capability of WRF-Solar EPS in representing various types of clouds, which are categorized using three levels of cloud top height (CTH) and cloud optical depth (COD) across entire CONUS. Preliminary results show that the WRF-Solar EPS provides MCF values ranging from 9% to 46%, 16% to 33%, and 8% to 27% for low-level, middle-level, and high-level clouds, respectively, for three CTHs. The model produces MCFs ranging from 27% to 46%, 13% to 34%, and 8% to 19% for thin, medium-thickness, and thick clouds, respectively, for three CODs. The presentation will include a detailed description of the current outcomes as well as potential future extensions. The evaluation approach established in this study is readily extensible to the evaluation of cloud predictions from different ensemble NWP models. In addition, the findings of the suggested evaluation technique aid in identifying model weaknesses and will ultimately lead to advances in WRF-Solar EPS's skill in predicting clouds and solar irradiance.

cloud mask forecast↗

DOE OSTI2025

EVALUATION OF HRA METHODOLOGIES FOR APPLICATION IN SDP WORK

This study critically evaluates human reliability analysis (HRA) methodologies applicable to regulatory probabilistic safety assessment (PSA) model, with a particular focus on their role in supporting the significance determination process (SDP) in nuclear safety assessment. Firstly, three widely utilized HRA methods – IDHEAS-ECA, SPAR-H, and ASEP/THERP – were qualitatively and quantitatively assessed. Qualitative assessments were conducted using attributes from the NEA/CSNI/R(2015)1 report, while quantitative evaluations employed regression and correlation analyses to compare predicted human error probabilities (HEPs) against empirical data. Results reveal distinct strengths, for example, IDHEAS-ECA’s robust predictive accuracy and K-HRA’s alignment with operational practices. In addition, dependency analysis and recovery analysis were critically evaluated. For dependency analysis, the methods’ handling of inter-task dependencies and their impact on HEPs were examined, while recovery analysis highlighted strategies for mitigating failure events. Furthermore, strategies were proposed to evaluate performance-shaping factors under conditions of reduced human performance, such as stress, fatigue, or cognitive overload, addressing specific challenges faced in SDP evaluations. Human errors from KINS’s operational performance information system event reports were evaluated as a case study. This study identifies gaps and provides actionable insights to ensure their validity and applicability in SDP HRA applications. This paper is a part of research conducted by KINS, and it should be noted that this result does not represent the regulatory position of KINS.

99 - GENERAL AND MISCELLANEOUS↗

DOE OSTI2024

Validating automated resonance evaluation with synthetic data

The integrity and precision of nuclear data are crucial for a broad spectrum of applications, from national security and nuclear reactor design to medical diagnostics, where the associated uncertainties can significantly impact outcomes. A substantial portion of uncertainty in nuclear data originates from the subjective biases in the evaluation process, a crucial phase in the nuclear data production pipeline. Recent advancements indicate that automation of certain routines can mitigate these biases, thereby standardizing the evaluation process and enhancing reproducibility. This research aims to provide a methodology, framework, and metrics for the validation of automated nuclear data evaluation software leveraging high-quality synthetic data that closely mimic real experimental observables. An introduced error metric provides a scale and intuitive measure of the evaluation quality by quantifying the estimate’s accuracy and performance across the specified energy range. Synthetic data provides access to experimental observables and underlying resonance parameters, enabling comparison of different evaluations. The methodology is demonstrated using Ta-181 isotope data in the resolved resonance region. The Automated Resonance Identification Subroutine (ARIS), which operates without prior resonance information, was used to test and showcase the framework’s capabilities utilizing the proposed error metrics. The results demonstrate the effectiveness of the proposed approach and framework for optimizing software parameters and testing hypotheses through “what-if” controlled experiments, such as modifying assumptions about experimental conditions or average resonance parameters.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2020

Evaluating 239 Pu(n,f) cross sections via machine learning using experimental data, covariances, and measurement features

In this paper, the neutron-induced 239 Pu fission cross section, 239 Pu(n,f), is evaluated from 1–20 MeV using experimental data and associated covariances while also considering information on the measurement, termed features here. For instance, methods to determine the background, sample backing material, or impurities in the sample, are explicitly taken into account in the evaluation process. To this end, outliers in the experimental data are identified with a modified version of the Hybrid Robust Support Vector Machine. In a second step, two machine learning methods (logistic regression with elastic net regularization and random forest regression with SHAP feature importance metric) are used to highlight measurement features that are common among many of the outlying data points. Based on this analysis, penalty uncertainties are added to the experimental covariances of outlying data points that have outlier measurement features and are put through the generalized-least-squares evaluation. The resulting evaluated mean values and covariances differ distinctly from those data evaluated without the penalty uncertainties. These results highlight that certain measurement features should be more closely examined.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2021

Multicriterion benefit evaluation methodology for safety enhancements in nuclear power plants and application for FLEX strategies

Here we propose a multicriterion benefit evaluation (MCBE) methodology for evaluating costs and benefits of implementing safety enhancements in nuclear power plants. The MCBE methodology establishes a comprehensive evaluation scope using multiple evaluation criteria (including public risk, occupational risk, plant revenue, and plant cost) and multiple contexts (including plant normal operations, incidents, and accidents). The MCBE methodology formulates a benefit or cost element as the product of a monetary consequence and its occurrence-likelihood change. The MCBE methodology also incorporates a decision maker’ preferences toward occurrence likelihood and monetary consequence into cost and benefit estimations through a modified usage of cumulative prospect theory. This paper applies the MCBE methodology to the implementation of diverse and flexible coping strategy (FLEX) in a generic nuclear power plant as an illustrative example. The FLEX benefits are evaluated through three types of influence paths: plant safety enhancement, risk-informed activities, and potential additional credit of FLEX in supporting plant operation and maintenance-efficiency improvement. The benefit-to-cost ratio (BCR) of FLEX implementation is calculated. Sensitivity analyses are conducted to examine the impacts of modeling assumptions and input parameter values on the BCR.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

DOE OSTI2022

Evaluating wind speed and power forecasts for wind energy applications using an open-source and systematic validation framework

Building on the verification and validation work developed under the Second Wind Forecast Improvement Project, this work exhibits the value of a consistent procedure to evaluate wind power forecasts. We established an open-source Python code base tailored for wind speed and wind power forecast validation, WE-Validate. The code base can evaluate model forecasts with observations in a coherent manner. To demonstrate the systematic validation framework of WE-Validate, we designed and hosted a forecast evaluation benchmark exercise. We invited forecast providers in industry and academia to participate and submit forecasts for two case studies. We then evaluated the submissions with WE-Validate. Our findings suggest that ensemble means have reasonable skills in time series forecasting, whereas they are often inferior to single ensemble members in wind ramp forecasting. Adopting a voting scheme in ramp forecasting that allows ensemble members to detect ramps independently leads to satisfactory skill scores. Throughout this document, we also emphasize the importance of using statistically robust and resistant metrics as well as equitable skill scores in forecast evaluation.

17 WIND ENERGY↗

DOE OSTI2020

Re-evaluation of the TSL for Yttrium Hydride

Yttrium hydride (YH x ) is of interest as a high-temperature moderator material in advanced nuclear reactor systems because of its superior ability to retain hydrogen at elevated temperatures. Thermal neutron scattering laws (TSL) for hydrogen bound in yttrium hydride (H-YH 2 ) and yttrium bound in yttrium hydride (Y-YH 2 ) were previously evaluated by Naval Nuclear Laboratory using the ab initio approach and released in ENDF/B-VIII.0. In that work, density functional theory, incorporating the generalized gradient approximation (GGA) for the exchange-correlation energy, was used to simulate the face-centered cubic structure of YH 2 and calculate the interatomic Hellmann-Feynman forces for a 2×2×2 supercell containing 96 atoms. Lattice dynamics calculations using PHONON were used to determine the phonon density of states. The calculated phonon density of states for H and Y in YH 2 were then used to prepare H-YH 2 and Y-YH 2 TSL evaluations, in the incoherent approximation, using the LEAPR module of NJOY. In addition, elastic scattering was assumed to be incoherent for both H and Y. While the incoherent elastic scattering approximation is appropriate for H-YH 2 , it introduces an undesirable approximation for Y-YH 2 . In this work, we re-evaluate the TSL for Y-YH 2 using FLASSH (Full Law Analysis Scattering System Hub). Y-YH 2 is evaluated using the FLASSH generalized coherent elastic scattering capability in order to capture the Bragg peaks associated with the YH 2 crystal structure which were neglected in the prior NJOY-based evaluation due to limitations in LEAPR. An experimental approach to validate the Y-YH 2 TSL using neutron transmission measurements is discussed.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2022

Guidelines and Specifications for Enhanced Durability Evaluation of Insulating Glass and Vacuum Insulating Glass Units

The National Renewable Energy Laboratory (NREL) has conducted research into durability of advanced fenestration technologies for almost thirty years. NREL has recently begun to research potential methods to enhance durability evaluation of present market insulating glass units (IGU). This document outlines a proposed process flow for adoption of these methods by interested parties. The proposed evaluation process flow described below is still a topic of active research at NREL as well as with industry partners. NREL will periodically update this document based on research results, both internal and external to NREL that may inform development of improved durability evaluation. This document is also meant to serve as a guideline for interested parties who may wish to engage in enhanced durability evaluation of IGU's both internally or with external testing laboratories. NREL will work to support adoption of the suggestions in this guideline by interested partners in the hope that these practices are more broadly adopted by the industry. Further research into this area will continue to inform both adaptations of this guideline as well as development of improved durability evaluation standards.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗