Course: 17664 TSQP - Evaluation [Slides]
The purpose of this course is to help you understand evaluation concepts, Kirkpatrick’s Evaluation Model, and to be able to apply basic evaluation techniques to training courses, programs, and processes.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
The purpose of this course is to help you understand evaluation concepts, Kirkpatrick’s Evaluation Model, and to be able to apply basic evaluation techniques to training courses, programs, and processes.
The U.S. General Services Administration (GSA) is interested in installing internet-connected, gas submeters to better understand gas consumption in its portfolio of buildings. The GSA in partnership with the National Renewable Energy Laboratory conducted a demonstration to assess a specific submeter technology. This technology was implemented at two GSA separate facilities located in Dallas, Texas. This demonstration evaluated hardware and software installations and integrations, data integrity and accuracy, and included an economic analysis. This demonstration evaluated a gas submeter technology provided by the vendor Vata Verks, who produces a non-invasive strap-on submeter. The company was founded with a mission to conserve water (and eventually gas) cheaply and simply, as explained on its website. The product intends to streamline submeter deployments for gas and water by eliminating most hardware costs while allowing for easy integration of submetered data into other systems such as the Building Automation System and removing tenant and building disruption This demonstration evaluated the product features when used on a gas utility meter.
The presentation discusses the eight new Thermal Scattering Law (TSL) evaluations that have been submitted to the National Nuclear Data Center (NNDC). It explains that evaluations are performed using the NCSU FLASSH code and that NJOY/LEAPR input files are provided to allow for independent testing. The presentation concludes by stating that more evaluations are expected to be submitted before the end of 2020.
This presentation discusses the experimental setup input parameters that are basic quantities for reproducibility. The goal is to increase quality of the evaluated data and decrease time needed for an evaluation. Full reproducibility should be achieved efficiently by including any type of constraints in the optimization procedure and to improve computational time to reach convergence. The goal of an automated reproducibility aligns to the goal of “certified” evaluated data, i.e., defined by a well-defined metric.
The evaluated nuclear data of the 63,65 Cu isotopes have been updated in the resolved and unresolved resonance energy regions with the goal of improving the fidelity between calculated and measured critical benchmarks. Particular focus was devoted to the 63 Cu(n, γ) cross sections because of their large sensitivity to the reactivity in the benchmark calculations. By focusing on these cross sections, the set of parameters in both resolved and unresolved resonance regions were fitted to available experimental data. The newly updated evaluations were compared to the currently available ENDF/B-VIII.0 evaluations in reactivity calculations for copper-sensitive integral benchmarks. An initial study of the sensitivity of the benchmarks to the angular distributions was performed, finding it necessary to describe the angular distributions more accurately above 100 keV. This investigation uncovered the need for more accurate data for the 63 Cu(n, γ) cross section above 100 keV. An increase in the capture cross section was found to improve agreement for the considered benchmarks.
Developing new technologies that enhance our abilities to identify molecular signatures is important for the health of the public, the environment, and national security. Deploying these novel technologies requires multiple stages of validation and evaluation. This project focused on evaluating a technology developed at PNNL named Structures for Lossless Ion Manipulations (SLIM) in the context of a commercially available ion mobility mass spectrometer instrument from Agilent Technologies. There is a benefit to integrating the recently developed SLIM module into a real-world instrument platform. Data from the combined instrument will provide new standards for characterizing samples using ion mobility. These standards, related to both the drift time of ions and their resulting collisional cross sections (CCS) will be of value in characterizing samples from both humans and environmental samples, such as soil microbiomes. Because no previous instrument has been designed and implemented to simultaneously analyze the same sample in both constant and oscillating drift fields, the project will provide a new research capability that will support research in a wide variety of applications. The evaluation approach included measuring the resolving power, mass range, sensitivity, and accuracy of two SLIM variations (constant and oscillating fields). Although SLIM performed similarly to the commercial instrument in terms of sensitivity, mass range, and accuracy the resolving power of SLIM far exceeded that of the commercially-available instrument. This validation will provide critically useful data to our commercial partners which they can use to reach informed decisions on the advantages of the SLIM technology such as the outstanding performance and the low cost of manufacturing.
This presentation covers neutron capture and transmission measurements and evaluation of 54 Fe (Iron) at the Rensselaer Polytechnic Insitute (RPI) Linear Accelerator (LINAC). Presentation conclusions find that 54 Fe is near its conclusion, covariance matrix generation is needed for both experiments before data is made available and released (Completion by April 2023). RPI will be conducting a full Resolved Resonance Region (RRR) evaluation following release of data (Completion by September 2023). Additionally, the neutron beam imager shows promise in providing an easy way to align samples in the most intense part of the neutron beam and there will be a new evaluation that will offer improvements in crit safety, shielding, and stellar applications.
In 2020, PNNL completed a literature search for high-efficiency particulate air (HEPA) filter age information. HEPA filters are used in many operations to remove particulate matter from effluent exhaust streams. They are thought to degrade over time both during proper storage and during normal operational service; however, the rate at which the filters degrade remains unknown. This brings into question if age is an adequate indicator of HEPA filter performance. Data from six previous reports were obtained from the literature search and combined to create a data set of 1600 operating filters. Filter usage was identified from multiple facilities. The various types of filters (e.g., axial flow, self-contained, and standard 24 x 24 x 11.5 inches; and both separator and separatorless) reported were constructed to the requirements Section FC (HEPA Filters) or Section FK (Special HEPA Filters) of the American Society of Mechanical Engineers AG-1 code. Filters were presumed to have continuously met the operational criteria and in particular passed both annual efficiency tests and annual DP measurements. The environmental conditions within the exhaust system were also assumed adequate for long-term filter operation. The data, by the nature of the reports, excludes rejected filters from quality assurance evaluations, intake, or installation testing. The collective data set shows over half of were operating past the current 10-year Department of Energy (DOE) limit. Data was evaluated for age lifetime using a linear trendline, survival function, probability functions, and failure rate; financial impacts were also addressed. This report supports the notion that HEPA filters can operate safely and efficiently under proper maintenance well past the 10-year lifetime established by DOE. Using the results of the four age evaluation approaches, they collectively point to the reasonableness of an operating HEPA filter lifetime of 20 years. The results are not necessarily definitive, but nevertheless, they are promising. The analysis provides reasonable assurance that when implemented using a graded approach with well-defined performance and operational requirements, extending the service life for HEPA filters beyond 10 years is low risk.
The ETF ammonia tolerant grout waste form was developed by the Vitreous State Laboratory (VSL) to stabilize ammonium and thereby prevent emission of ammonia vapor during solidification of an ammonium-rich, concentrated sulfate aqueous waste stream generated at the Hanford Effluent Treatment Facility (ETF). VSL did not observe expansion in samples prepared during the waste form development work. However, Savannah River National Laboratory (SRNL) personnel did encounter expansion, in some cases up to 20+ volume percent expansion in ETF grout samples while preforming work scope to evaluate compositional ranges for ETF brine and of the precipitation and solidification reagent. The work requests can be found in Washington River Protection Services (WRPS) Statement of Work (SOW), Requisitions #: 339922 Revisions 0 and 1, March 11, 2021, and November 2021. Additional evaluation was requested by WRPS SOW Requisitions #: 356730, Revision 1, February 1, 2022, to determine whether differences in the slags or differences in sample preparation protocols used by VSL and SRNL were the cause of the expansion observed in the SRNL samples. G. Chen, WRPS, arranged a materials exchange between VSL and SRNL and coordinated the VSL protocol for grouting and curing 1L of Base Case ETF simulant. Materials were exchanged between VSL and SRNL and both laboratories performed the VSL struvite precipitation-solidification protocol and cured samples for at least 28 days. VSL and SRNL encountered expansion, at varying degrees, in all samples prepared under the material exchange and grouting protocol. Additional material exchange testing between VSL and SRNL and alternative materials evaluation to further investigate the causes of grout expansion and differentiate the effects of various dry materials, grouting procedures, and curing containers was requested by WRPS SOW Requisitions #: 356730, Revision 2, January 10, 2023. Pacific Northwest National Laboratory (PNNL) assembled and distributed materials to both laboratories for testing. Observations from this study indicated variations in MgO additions, water-to-dry mix (dm) ratios and curing containers did not correlate to ETF ammonia grout expansion. In comparison to previous work, SRNL observed the addition rate of NaOH to be a potential factor for the final pH in the struvite precipitation step. In the current testing NaOH was added in bulk (5-10 mL at one time) during the precipitation step. The resulting pH was 7.57-9.15 which is slightly higher than pH observed at a slower addition rate. Additional testing is needed to identify the presence and/or struvite stability in high-pH struvite precipitant ETF ammonia grout samples. Therefore, based on current data, it is unknown if NaOH addition rate is the fundamental cause of grout expansion.
Uranium silicide (U 3 Si 2 ) is a candidate material for the high-density nuclear fuel in commercial light water reactors [1], [2]. Its higher uranium density, 11.3 g-U/cm3, compared to that of uranium dioxide (UO 2 ), 9.7 g-U/cm3, can improve the performance of a nuclear reactor while using low enriched uranium (LEU) and diversify the choice of cladding materials [1]–[3]. It also has a higher thermal conductivity than UO 2 , which can reduce the thermal stress on the material caused by a temperature gradient across the fuel pellet and provide a larger margin for some postulated accidents [1], [2], [4]. Furthermore, compared to U3Si, another high-density fuel candidate, it has better resistance to in-pile swelling due to less irradiation-induced rapid amorphization [1], [3]. Corresponding to its importance in nuclear engineering, many previous studies have reported the properties of U3Si2. Experiments showed that U 3 Si 2 is a paramagnetic (PM) metal, where a slight linear increase in magnetic susceptibility was measured with increasing temperature [5], [6]. In addition, thermodynamic quantities such as thermal expansion coefficient, heat capacity, and thermal conductivity were experimentally determined over a wide temperature range [1], [7], [8]. In several computational studies, ab initio atomistic simulations based on density functional theory (DFT) were performed to calculate various properties including elastic constants, electronic density of states (DOS), and phonon dispersion curves [9]–[12]. Nevertheless, thermal neutron scattering cross sections, which are critical to the prediction of the parameters in reactor physics that are ultimately related to reactor criticality, have not yet been evaluated for U3Si2. The scattering cross section can be calculated from the phonon DOS, or the energy spectrum of lattice vibrations, of the crystalline system [13], [14]. However, there is also no experimental data available for the phonon DOS of U 3 Si 2 . While some computational studies reported the phonon DOS and/or dispersion curves from ab initio simulations [9]–[12], the accuracy cannot be guaranteed because it is unclear whether the spin-polarization behavior of PM U 3 Si 2 was properly described. In the present study, the thermal neutron scattering cross section for U 3 Si 2 is evaluated for the first time by calculating the phonon DOS for U3Si2 from ab initio lattice dynamics (AILD) simulations based on DFT. First, U 3 Si 2 is modeled based on the experimental structure, and AILD simulations are performed on the modeled U3Si2 to optimize the structure. Next, AILD simulations are performed for supercells with atomic displacement to calculate Hellmann-Feynman forces. Based on the calculated forces, partial phonon DOSs for U and Si are obtained, and the thermal neutron scattering law (TSL) for U 3 Si 2 is finally evaluated. To verify the accuracy of the calculations in the present study, the calculation results are compared with experimental data on the structure and heat capacity of U3Si2 [1], [7], [8], [15].
The overall goal of the task was to identify cost effective biological sampling techniques for MHK environmental monitoring. Protected, demersal, and pelagic fish and selected nektonic invertebrates were surveyed using capture and remote sensing techniques at the PacWave sites. The performance of capture and remote sensing monitoring techniques were evaluated and generic nekton monitoring indices were developed for MHK technologies and sites. In parallel to data collections at the PacWave site, the ability of regression models to characterize, detect, and predict change in acoustic data was evaluated using acoustic data collected in Admiralty Inlet, WA, during a biological monitoring study of the proposed SnoPud tidal turbine project site. Expected outcomes of these efforts included an evaluation of instrumentation and techniques used to monitor biological variability; identification of data streams that can be used to detect and quantify change; and sampling requirements to ensure detection of change in monitored variables.
Southwest Research Institute (SwRI) was contracted by Sovereign Hydroseal Energy, Inc to provide evaluation of the NOH2O polymer grout technology as a feasible tank bottom repair in support of Washington River Protection Solutions, LLC (WRPS) Solicitation #358012 Statement of Work (SOW) “Screening Evaluation of Sovereign NOH2O”, which a copy is located in Appendix A. The evaluation focused on screening the mechanical properties of NOH2O under normal, chemical and radiological conditions, which was identified as Task 2 of the SOW. Testing was performed in accordance with the SwRI Test Plan located in Appendix B.
Wind energy is a crucial technology for achieving net-zero emissions by 2050. However, the growth and deployment of wind energy in North America have led to the deaths of many bat species due to operating wind turbines. Hundreds of thousands of bats are estimated to die at wind turbines annually in North America. Operational minimization, which includes feathering turbine blades and curtailment, has been documented to reduce bat fatality effectively. Curtailment refers to altering turbine operation based on wind speed, time of year, temperature, sensors, and activity models. However, when turbines are curtailed, they do not generate power, resulting in energy loss and revenue for wind energy facilities. The Electric Power Research Institute (EPRI) funded the development of Turbine Integrated Mortality Reduction (TIM SM ) Technology, which curtails turbine operation when bats are detected. The initial TIMR system research showed promising results, with an 85% reduction in overall bat fatalities and a 91% reduction for the little brown bat. However, these results were based on a single site during one fall season, and it was unclear if similar results could be replicated at other wind energy facilities. This research aimed to validate the TIMR system results from the prior field study at a second site in the U.S., estimate the power production and reduction in bat mortality at turbines with installed TIMR systems relative to blanket curtailment and fully operational turbines, test the TIMR system in two calendar years and during the summer and fall periods, and evaluate the operational and commercial characteristics of the TIMR system for potential wind industry adoption. The study was conducted at a 500.9-MW wind energy facility in southeast Adair County, Iowa. Three experimental treatments were involved in this randomized block design study: TIMR, Curtailment at 5.0 m/s, and Normal Operation. In 2021, three treatments were used at 18 turbines, expanding to four treatments across 36 turbines in 2022. The TIMR system worked as designed throughout the entire study; however, because of unexpected wind turbine operational challenges in 2021, there was not sufficient sample size to evaluate the treatment differences. In 2022, there were significant differences in fatality levels between treatment types and normal operating turbines. Curtailment at 5.0 m/s reduced fatalities by 30.8% compared to normal operations, and TIMR decreased fatalities by 48.6% compared to normal operations. Two different methods were used to evaluate the differences in energy loss for each treatment. The TIMR system resulted in 1.3% to 1.6 % annual energy loss in 2021 and 1.0% to 1.2 % in 2022. The Curtailment at 5.0 m/s resulted in 0.6% to 0.8 % annual energy loss in 2021 and 0.5% to 0.6 % in 2022. The project achieved all the stated objectives and demonstrated that TIMR is an effective technology that balances bat fatality reduction with energy generation. The results will support the deployment of TIMR and other acoustic sensor-based technologies. The research provides valuable insights into the impact of different treatments on fatality rates and energy outputs, contributing to the ongoing efforts to mitigate the environmental impact of wind energy.
Alkali–silica reaction (ASR) is a type of material degradation in concrete structures that leads to concrete cracking and rebar corrosion, thereby reducing the material’s structural integrity and the overall structure’s lifetime and raising safety concerns. Ultrasonic nondestructive evaluation (NDE) has been proven to be a valuable technique for assessing concrete properties and monitoring ASR progression in concrete. However, the deployment and analysis of ultrasonic NDE and its data requires specialized expertise, often relying on the engineer’s subjective interpretation. With the surge in computational power, artificial intelligence (AI) and machine learning (ML) algorithms have become popular in automating NDE data analysis. Various industrial sectors are increasingly adopting ML algorithms for NDE data analysis with a growing emphasis on AI–assisted automation. Regulatory agencies are also preparing for this technological shift, anticipating corresponding revisions in standards. Thus, there is an urgent need to identify the capabilities and limitations of current ML technologies for the evaluation of concrete material properties and damage status. Furthermore, the effects of various factors on ML model performance must be thoroughly investigated. The study summarized herein evaluated the effectiveness of two ML models (i.e., support vector regression (SVR) and deep neural network (DNN)) in predicting concrete material damage induced by ASR based on the long-term ultrasonic monitoring data. Four distinct concrete specimens were cast with artificially induced ASR, and over a period exceeding 500 days, ultrasonic signals and expansion data were continuously collected. For the SVR model, wave velocity and 12 other wave features were extracted from the ultrasonic signals, with 6 out of 13 features selected as input for the model. Different combinations of training and testing datasets were designed to explore factors influencing prediction performance, including the range of data within training and testing sets, in addition to various signal preprocessing methodologies. These findings suggest the importance of using a training dataset with a broader data range compared with testing datasets for improved model performance alongside consistent signal preprocessing across datasets.
Protocols play an essential role in Advance Reactor systems. A diverse set of protocols are available to these reactors. Advanced Reactors benefit from technologies that can minimize their resource utilization and costs. Evaluation frameworks are often used when assessing protocols and processes related to cryptographic security systems. The following report discusses the various characteristics associated with these protocol evaluation frameworks, and derives a novel evaluative framework.
Evaluation of high temperature (HT) microcontrollers for Geothermal applications. The goal was to evaluate the limited microcontrollers capable of operating above 200 degrees Celsius. The knowledge obtained from this research has been documented for contributing to future development of high temperature geothermal tools. Along with this research, a HT data link for communications was developed and evaluated, also for HT geothermal tools.
The collaboration between the Los Alamos National Laboratory Neutron Science Center (LANSCE) and TerraPower LLC enables the enhanced understanding of fast spectrum critical systems consisting of chlorine. Specifically, TerraPower is interested in updating the nuclear data for the stable isotopes of chlorine, 35 Cl and 37 Cl, because these nuclides are the primary constituents of the chloride fuel salt in the Molten Chloride Reactor Experiment (MCRE), for which TerraPower is leading the design. The Cooperative Research and Development Agreement (CRADA) between the parties is funded by DOE’s Office of Nuclear Energy’s Gateway for Accelerated Innovation in Nuclear (GAIN) initiative to provide the nuclear community with access to the technical, regulatory, and financial support necessary to motivate innovative nuclear reactor technologies toward commercialization. New measurements of 35 Cl(n,p total ) and 35 Cl(n,α total ) were completed at LANSCE to constrain the reaction theory models that are used to generate the updated evaluations. The updated evaluations were then tested across the sensitivities of the MCRE by TerraPower to provide direct feedback to the evaluation for application specific sensitivities.
Sandia National Laboratories has tested and evaluated the Reftek Wrangler digitizer. The Wrangler digitizers are intended to record sensor output for seismic and infrasound monitoring applications. The purpose of this digitizer evaluation is to measure the performance characteristics in such areas as power consumption, input impedance, sensitivity, full scale, self-noise, dynamic range, system noise, response, passband, and timing. The Wrangler digitizers are being evaluated for potential use in the International Monitoring System (IMS) and On-site Inspection (OSI) components of Comprehensive Nuclear-Test-Ban Treaty (CTBT).