Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “EVALUATION”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

Scheduling Improvements Following the Phase 1 Field Evaluation of the ATD-2 Integrated Arrival, Departure, and Surface Concept

NASA is conducting the Airspace Technology Demonstration-2 (ATD-2) to evaluate an Integrated Arrival, Departure, and Surface (IADS) traffic management system that extends traffic sequencing for the entire life-cycle of a flight from departure gate to arrival gate within multi-airport, metroplex environments. After development and testing in human-in-the-loop simulations, the IADS system was deployed to Charlotte Douglas International Airport for a three-year field evaluation. From the initial IADS concept development through the end of the Phase 1 field evaluation many lessons were learned with regards to the IADS scheduler. In this paper we describe how data from the Phase 1 field evaluation helped identify scheduler improvements and guided the implementation of refinements. The improvements in the IADS scheduler described in this paper are incorporated into the IADS Phase 2 scheduler enabling strategic Surface Metering Programs and will be evaluated during the field evaluation.

Airspace Technology Demonstration-2↗

Advancing the Theory of Nuclear Data Evaluations [Abstract]

We present recent advances in the R-matrix formalism as well as the Bayesian evaluation framework for improved nuclear data evaluations. The advances in the R matrix formalism include: 1) direct processes, 2) doorway, as well as multistep, processes, and 3) various forms of the Reich-Moore approximation for eliminated capture channels. Furthermore, to address unreasonably small posterior uncertainties often encountered in nuclear data evaluations of large data sets using the conventional form of the Bayes’ theorem, we introduce imperfections (of the data or the model) as a formal evaluation tool for taming the evaluated uncertainties in harmony with Bayes’ theorem. These theoretical advances were motivated by the nuclear data evaluations of differential resolved resonance cross section data using the code SAMMY, as well as the integral benchmark experiments using the SCALE code system, being performed at Oak Ridge National Laboratory for the Nuclear Criticality Safety Program. Some pedagogical applications of the new formalism, as well as a snapshot of the SAMMY modernization efforts, will be presented.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Evaluation of Saccadic Component Measure on Smooth Pursuit Tests

ABSTRACT Introduction Despite the advancement of eye-tracking technology for smooth pursuit (SP) eye movement evaluation, qualitative observation offers much information that is not captured by computers; hence, both objective and qualitative information should be utilized to evaluate SP. This study examined the consistency among our clinicians when evaluating SP using normal (N), grossly normal (GN), mildly abnormal (MA), and abnormal (AB) as classifications. We then evaluated the effect of combining GN and MA into a single subclinical (SUBC) category. We also evaluated the computerized percent saccade (PS) metric by determining its sensitivity and specificity in classifying SP. Materials and Methods Retrospective horizontal and vertical SP test videos and numerical data for 70 participants were obtained from the Neuro Kinetics Neuro-Otologic Test Center and de-identified. From this, eye-tracking videos, time plots of eye-tracking positional data, and tables of SP eye-tracking performance data were generated for 0.1, 0.3, and 0.5 Hz in both horizontal and vertical planes, totaling 6 tests per subject. Three clinicians rated each subject’s SP performance as N, GN, MA, or AB for a total of 6 ratings (3 frequencies, horizontal and vertical). This process was repeated using N, SUBC, and AB as rating categories. Clinicians also provided an overall SP rating for each plane as follows: AB if the results were abnormal for 2 or more frequencies tested. Alternatively, if fewer than 2 frequencies presented with a rating of AB, then an overall rating of MA, GN, or N was determined at the respective clinician’s discretion. Results When the 3 clinicians were tasked with classifying SP videos using 4 clinical categories, fair overall agreement was demonstrated. However, when MA and GN categories were combined into an SUBC category, the overall agreement for the 3 clinicians improved slightly for both horizontal SP (HSP) and vertical SP (VSP). This pattern of agreement did not differ considerably when comparing HSP versus VSP, and good consistency and reliability was observed across clinicians. Again, inter-rater consistency was smaller for VSP versus HSP despite the reduction in clinical categories. Cut-off values were generated for the PS metric and demonstrated good specificity and sensitivity when they were exceeded for 2 or more frequencies in a particular plane when evaluating a subject’s SP test. Conclusions

General & Internal Medicine↗

Modernization of Technical Requirements for Licensing of Advanced Non-Light Water Reactors: Risk-Informed and Performance-Based Evaluation of Defense-in-Depth Adequacy

This document supports the work contained in Nuclear Energy Institute (NEI) 18-04 “Risk-Informed Performance-Based Technology Inclusive Guidance for Advanced Reactor Licensing Basis Development” Revision 0. NEI 18-04 presents a modern, technology-inclusive, risk-informed, and performance-based (TI-RIPB) process for selection of Licensing Basis Events (LBEs); safety classification of structures, systems, and components (SSCs) and associated risk-informed special treatments; and determination of defense-in-depth (DID) adequacy for non-LWRs. The NEI guidance document provides one acceptable means for addressing the aforementioned topics as part of demonstrating a specific design provides reasonable assurance of adequate radiological protection. This report provides the framework and associated methodology guidelines and discussion for establishing, then evaluating, confirming, and documenting the adequacy of defense-in-depth (DID) for advanced non-light-water reactor technologies. It was developed as part of the Licensing Modernization Project led by Southern Company and cost-shared by the United States Department of Energy and has benefited from considerable NRC formal reviews and public workshops. The methodology converts the DID philosophy into a structured process that is implementable, embraces existing United States and international definitions and philosophies of DID that set the foundation for the process. It builds on the DID framework developed in the Department of Energy Next Generation Nuclear Plant Project and earlier works on this subject. The approach to establishing DID adequacy involves the incorporation of DID attributes into the plant capabilities and programmatic elements of DID. The integrated evaluation of DID adequacy includes both quantitative elements to incorporate risk-informed and performance-based (RIPB) considerations and qualitative elements that address uncertainties and limitations in the quantitative models and supporting data. Demonstration of DID adequacy ensures that there are multiple layers of defense for risk-significant challenges to the design and that the plant capabilities and programs that support each layer are provided in a manner that minimizes dependencies among these layers. The focus of this report is assurance of DID adequacy with respect to protection of the public from radiological exposures resulting from accidental releases of radioactive material. While other hazards are not specifically addressed, this methodology is expected to be beneficial for determining DID adequacy for them as well. Risk-informed evaluation of DID considers the integrated performance of all plant SSCs and associated programs to manage daily operational activities, transients, and accidents, including the evaluation of strategies for accident prevention and mitigation. The RIPB LBE scenario methodology used in this evaluation defines the challenges to the plant safety features included in the plant design basis and beyond, and the scope of all deterministic and probabilistic safety evaluations. By examining event sequences across the whole spectrum of LBEs, a systematic assessment of DID can be accomplished.

42 ENGINEERING↗

An Assessment of Evaluation Practices of Low- And Moderate-Income Solar Programs

As concerns about social equity and clean energy rise, state and local governments, utilities, and non-profit organizations are offering at least 41 active programs in 21 states to promote solar adoption as a way to reduce energy bills for low- and moderate-income (LMI) households while meeting other policy goals such as job creation and clean energy generation. A new study from Berkeley Lab looks at how those programs are being evaluated. The report provides background on how they seek to address LMI household energy burdens and gives a brief discussion of the art and science of program evaluation, drawn from decades of experience in energy efficiency programs. It then turns to how LMI solar programs are currently being evaluated, highlighting trends among evaluation methods, metrics tracked, and best practices employed. A few programs are explored in more depth to illustrate notable practices that could be applied to other program evaluations. These LMI solar programs are mostly young, operate on tight budgets, and vary considerably in their design and stated goals. Consequently, well-designed program evaluation is critical to better understand what works, what could be improved, and how to maximize program impacts under budget constraints.

14 SOLAR ENERGY↗

Well Stimulation Treatment in California: Evaluation of Disclosure Data, May 2015 – October 2019

In this document, disclosure data from 1,228 well stimulations occurring May 2015 to October 2019 in California are evaluated. This evaluation updates a previous study that was based on 618 well stimulations occurring May 2015 to June 2016 (Stringfellow, W.T., Camarillo, M.K., and Jordan, P. 2017, Status of Well Stimulation in California Since Implementation of SB-4 Regulations, Berkeley National Laboratory, Berkeley, CA). While the goal of the previous study was to provide a summary of information obtained since passage of California Senate Bill No. 4—Oil and Gas, the goal of the current evaluation is to provide a basis for studying chemical indicators that could potentially be used to detect aquifer contamination. We evaluate chemical indicators is a separate document using the data contained herein as well as data from monitoring wells located within oil fields. Compared with the data used by Stringfellow et al. 2017, the current evaluation is based on a more diverse data set. While the previous study contained data for only two oil fields with more than 30 well stimulations, the current data set contains four oil fields with more than 30 well stimulations, making these data more ideal for comparisons using statistical tests. More producers and a new oil field, Buena Vista Nose, are represented in the current data set. Some well stimulation practices remain relatively unchanged since the previous study. Well stimulation is mostly occurring in Kern County with the exception of a single well stimulation in Orange County. Almost all well stimulations are hydraulic fracturing. The current data set contains two acid fracturing treatments and one matrix acidizing treatment. The median number of chemicals added per well stimulation—excluding water and proppant—was previously 21; the number in the expanded data set is 20. Median water added per well stimulation was previously 89,000 gallons and is now 98,000 gallons. While this water use represents a 10% increase, water use for hydraulic fracturing in California is still lower than water use in other oil and gas fields. Chemical formulations appear to have shifted since the previous study. This shift may be related to a change in the predominant service company. We identified 26 chemicals for which the frequency of use has changed by more than 30%. The total number of chemicals used has also expanded. Previously, 178 unique chemicals were identified as being added to well stimulation fluids. The total number of unique chemicals is now 205. Analytical data available for base and recovered fluids has expanded. The number of base fluid samples has increased from 12 to 35 and the number of recovered fluid samples has increased from 1,078 to 2,166. The expanded data set is more ideal for an evaluation of indicator chemicals and comparisons across different formations. The analytical data for recovered fluid samples indicates that many ions, radioactive constituents, and organics are consistently observed, making these chemicals ideal for consideration as potential indicators of aquifer contamination.

54 ENVIRONMENTAL SCIENCES↗

Energy Efficiency and Performance Evaluation of Self-Contained, Medium Temperature, Reach-In Refrigerated Display Cases

This project was part of an effort by Commonwealth Edison Company (ComEd) to evaluate the energy and peak demand saving potential of emerging technologies in the Chicago area. This document focuses on the assessment of energy-efficient, medium-temperature, self-contained refrigerated display cases utilizing environmentally friendly refrigerants. The results of this evaluation will be considered by CLEAResult to develop a new energy efficiency rebate measure for ComEd's incentive programs. This rebate measure will become an addition to the Technical Reference Manual (TRM). In 2016, the United Nations passed the Kigali Montreal Protocol Amendment which placed restrictions on certain types of refrigerants. In compliance with this amendment, the US Environmental Protection Agency (EPA) placed a ban on the manufacture of refrigeration systems using hydrofluorocarbons including R134a starting in January 2020. Although the ban has halted manufacture, the EPA continues to allow the use of these refrigerants. Therefore, it is critical to provide incentives for replacing these refrigerants with other environmentally friendly and energy-efficient alternatives. The energy-efficient refrigerator cases evaluated here (referred to as EE Case A and B) contain environmentally-friendly refrigerants in compliance with the EPA hydrofluorocarbon ban. These consist of natural refrigerant propane (R290), and HFC drop-in hydrofluoroolefin R513a, respectively. These cases also contain other energy-efficient components including efficient lighting, more robust evaporator and condenser fans and different-sized heat exchangers. EE Case A is also built with materials that better insulate the case, which improves energy efficiency. To ascertain the energy efficiency contribution of these design components, the consumption of the evaporator and condenser fan motors, compressor, and lighting/controller were evaluated individually. The medium-temperature, self-contained reach-in refrigerated display case was selected due to its widespread use in convenience stores and small supermarkets. Self-contained refrigeration has also seen increased use in restaurants due to curbside pickup during the COVID-19 pandemic. For this study, the refrigerated display cases' performance was evaluated in a controlled environmental chamber at representative indoor dry-bulb and humidity conditions found in supermarkets within ComEd's service territory climate zone. The test method used in this project was foundationally inspired by the ANSI/ASHRAE 72-2018 method of testing. However, modifications to the ANSI/ASHRAE test methodology were implemented to better represent customer operation of the units. In addition to the indoor supermarket conditions, the cases were also evaluated at the "upper target," or environmental conditions used in ANSI/ASHRAE 72-2018. The case total power and case components were metered to obtain their daily power (kW) and energy consumption(kWh). The cases were filled with thermal filler mass to replicate thermal mass of product loading. Additionally, product simulators were used to provide product temperature information. Door actuators were mounted to each of the cases' three doors to replicate regular door openings and effects of shopper traffic.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Nuclear Reaction Data Evaluations for Nuclear Applications Physics and modeling [Slides]

This presentation provides an evaluation workflow scheme for the project. It also discusses nuclear data libraries, including an overview of worldwide nuclear data libraries and current funding agencies. The presentation also discusses the theoretical framework commonly used in the evaluated work, including the R-matrix theory (≈ up to a few keVs for heavy nuclei) as well as pre-equilibrium (PE) and Hauser-Feshbach (HF) theory (⪆1 MeV for medium/heavy nuclei). Inclusion of experimental effects are also discussed. Examples of evaluated work, including theory, experiments, and applications are discussed, specifically n+ 28,29,30 Si evaluation (direct capture component), n+ 53 Cr evaluation (strong multiple scattering effects), n+ 239 Pu(n,γf) reaction (fluctuations in neutron multiplicities), and α+ 17,18 O evaluations (emerging neutron spectra).

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Open Source Evaluation Framework for Solar Forecasting

The Solar Forecast Arbiter is an open-source evaluation framework for solar forecasting. The framework enables evaluations of solar irradiance, solar power, and net-load forecasts that are impartial, repeatable and auditable. The Solar Forecast Arbiter addresses stakeholder-informed use cases including evaluation of forecast skill, comparisons to reference data sets, private forecast trials, and evaluation of probabilistic forecast skill. The framework includes a data validation toolkit, reference data sources, data privacy protocols, and benchmark forecast capabilities for intra-hour and day ahead forecast horizons. Reports and metrics communicate the relative merits of the test and benchmark forecasts. The reports are created from standardized templates and include graphics for qualitatively evaluating deterministic and probabilistic forecasts and standard metrics for quantitatively evaluating forecasts. The Solar Forecast Arbiter is designed to support all solar forecasting stakeholders, including Solar Forecasting 2 Topic Area 2 and Topic Area 3 teams.

14 SOLAR ENERGY↗

Evaluating the 238 U PFNS Including Chi-Nu Experimental Data

This report documents an evaluation of 238 U prompt fission neutron spectra (PFNS) which is a deliverable for a FY2024 Q4 NCSP (Nuclear Criticality Safety Program) milestone. This evaluation is new; its prior input is based on extended Los Alamos and exciton models implemented in the code CoH. Experimental covariances were estimated for five experimental data sets. One of these data sets that was measured by the Chi-Nu team of LANL and LLNL. It covers the 238 U PFNS for continuous incident-neutron energies of 1–20 MeV and outgoing-neutron energies from 10 keV– 10 MeV with high precision. Contrary to Chi-Nu data, previous data sets were measured in a limited energy range. The resulting evaluated data correspond well to the experimental PFNS taken into account for the evaluation. The evaluated PFNS also produce average mean energies in agreement with associated Chi-Nu data. If one uses the new evaluated data to predict the neutron multiplication factor, k eff , of the Flattop, Flattop-Pu and BigTen ICSBEP critical assemblies (which all have thick reflectors with high percentages of 238 U), the differences of simulated values compared to those using ENDF/B-VIII.1β3 is modest (less than 25 pcm). In addition to that, the new PFNS predict on average 238 U LLNL pulsed-sphere neutron-leakage spectra slightly better than ENDF/BVIIII.0 and ENDF/B-VIII.1β3 PFNS. The differences are, however, well within the experimental uncertainties.

238U↗

A computational modeling framework for pre-clinical evaluation of cardiac mapping systems

There are a variety of difficulties in evaluating clinical cardiac mapping systems, most notably the inability to record the transmembrane potential throughout the entire heart during patient procedures which prevents the comparison to a relevant “gold standard”. Cardiac mapping systems are comprised of hardware and software elements including sophisticated mathematical algorithms, both of which continue to undergo rapid innovation. The purpose of this study is to develop a computational modeling framework to evaluate the performance of cardiac mapping systems. The framework enables rigorous evaluation of a mapping system’s ability to localize and characterize (i.e., focal or reentrant) arrhythmogenic sources in the heart. The main component of our tool is a library of computer simulations of various dynamic patterns throughout the entire heart in which the type and location of the arrhythmogenic sources are known. Our framework allows for performance evaluation for various electrode configurations, heart geometries, arrhythmias, and electrogram noise levels and involves blind comparison of mapping systems against a “silver standard” comprised of computer simulations in which the precise transmembrane potential patterns throughout the heart are known. A feasibility study was performed using simulations of patterns in the human left atria and three hypothetical virtual catheter electrode arrays. Activation times (AcT) and patterns (AcP) were computed for three virtual electrode arrays: two basket arrays with good and poor contact and one high-resolution grid with uniform spacing. The average root mean squared difference of AcTs of electrograms and those of the nearest endocardial action potential was less than 1 ms and therefore appears to be a poor performance metric. In an effort to standardize performance evaluation of mapping systems a novel performance metric is introduced based on the number of AcPs identified correctly and those considered spurious as well as misclassifications of arrhythmia type; spatial and temporal localization accuracy of correctly identified patterns was also quantified. This approach provides a rigorous quantitative analysis of cardiac mapping system performance. Proof of concept of this computational evaluation framework suggests that it could help safeguard that mapping systems perform as expected as well as provide estimates of system accuracy.

59 BASIC BIOLOGICAL SCIENCES↗

Shuttle orbiter Ku-band radar/communications system design evaluation

Tasks performed in an examination and critique of a Ku-band radar communications system for the shuttle orbiter are reported. Topics cover: (1) Ku-band high gain antenna/widebeam horn design evaluation; (2) evaluation of the Ku-band SPA and EA-1 LRU software; (3) system test evaluation; (4) critical design review and development test evaluation; (5) Ku-band bent pipe channel performance evaluation; (6) Ku-band LRU interchangeability analysis; and (7) deliverable test equipment evaluation. Where discrepancies were found, modifications and improvements to the Ku-band system and the associated test procedures are suggested.

Dodds, J.↗

The application of flow competence evaluations to the assessment of flood-flow velocities and stresses

The concept of flow competence is generally employed to evaluate the velocities, discharges, and bottom stresses of river floods inferred from the size of the largest sediment particles transported. Flow competence has become an important tool for evaluating the hydraulics of exceptional floods on Earth, including those which eroded the Channeled Scabland of eastern Washington, and has potential for similar evaluations of the floods which carved the outflow channels on Mars. For the most part, flow-competence evaluations were empirical, based on data compiled from a variety of sources including major terrestrial floods caused by natural processes or dam failures. Such flow-competence relationships would appear to provide a straight-forward assessment of flood-flow stresses and velocities based on the maximum size of gravel and boulders transported. However, a re-examination of the data base and comparisons with measurements of selective entrainment and transport of gravel in rivers open to question such evaluations. Analyses of the forces acting on the grain during entrainment by pivoting, rolling, or sliding, an approach which focuses more on the physical processes than the purely empirical relationships can be demonstrated. These derived equations require further testing by flume and field measurements before being applied to flow-competence evaluations. Such tests are now underway.

Komar, Paul D.↗

Preliminary evaluation of space station transmission line in a ring configuration

The results of a preliminary evaluation of a space station type transmission line and commercial transmission lines in a ring configuration, are reported. In a ring configuration, each node has two paths for the return current of each wire. The additional path can create an unbalanced condition, where the magnetic fields created by the forward and return currents do not cancel. This evaluation was to quantify the effects of the unbalanced case upon the external fields. The transmission lines evaluated were standard commercial coaxial cables, RG59 and RG213, and a space station designed flat Litz transmission line. Each was evaluated in a balanced and unbalanced mode of operation. Currents and their harmonic content were recorded and compared. As expected, the harmonic content of the different current (I delta) was substantial for the unbalanced case as compared to the balanced case. For the balanced case, very little difference was noted among the various transmission lines evaluated. The evaluation is discussed, and the test circuit, the measurements, and the resulting data are described.

Roth, Mary Ellen↗

Thiokol/Wasatch installation evaluation of the redesigned field joint protection system (concepts 1 and 3)

The procedures, performance, and results obtained from the Thiokol Corporation/Wasatch Redesigned Field Joint Protection System (FJPS) Installation Evaluation are documented. The purpose of the evaluation was to demonstrate and develop the procedures required to install two different concepts (referred to as Concepts 1 and 3) of the redesigned FJPS. The processing capability of each configuration was then evaluated and compared. The FJPS is installed on redesigned solid rocket motors (RSRM) to protect the field joints from rain intrusion and to maintain the joint temperature sensor measurement between 85 and 122 F while the boosters are on the launch pad. The FJPS is being redesigned to reduce installation timelines at KSC and to simplify or eliminate installation processing problems related to the present design of an EPDM moisture seal/extruded cork combination. Several installation techniques were evaluated, and a preferred method of application was developed for each concept. The installations were performed with the test article in the vertical (flight) position. Comparative timelines between the two concepts were also developed. An additional evaluation of the Concept 3 configuration was performed with the test article in the horizontal position, to simulate an overhead installation on a technical evaluation motor (TEM).

Cook, M.↗

Evaluation of a proposed expert system development methodology: Two case studies

Two expert system development projects were studied to evaluate a proposed Expert Systems Development Methodology (ESDM). The ESDM was developed to provide guidance to managers and technical personnel and serve as a standard in the development of expert systems. It was agreed that the proposed ESDM must be evaluated before it could be adopted; therefore a study was planned for its evaluation. This detailed study is now underway. Before the study began, however, two ongoing projects were selected for a retrospective evaluation. They were the Ranging Equipment Diagnostic Expert System (REDEX) and the Backup Control Mode Analysis and Utility System (BCAUS). Both projects were approximately 1 year into development. Interviews of project personnel were conducted, and the resulting data was used to prepare the retrospective evaluation. Decision models of the two projects were constructed and used to evaluate the completeness and accuracy of key provisions of ESDM. A major conclusion reached from these case studies is that suitability and risk analysis should be required for all AI projects, large and small. Further, the objectives of each stage of development during a project should be selected to reduce the next largest area of risk or uncertainty on the project.

Gilstrap, Lewey↗

Evaluation of components, subsystems, and networks for high rate, high frequency space communications

The development of new space communications technologies by NASA has included both commercial applications and space science requirements. At NASA's Lewis Research Center, methods and facilities have been developed for evaluating these new technologies in the laboratory. NASA's Systems Integration, Test and Evaluation (SITE) Space Communication System Simulator is a hardware-based laboratory simulator for evaluating space communications technologies at the component, subsystem, system, and network level, geared toward high frequency, high data rate systems. The SITE facility is well-suited for evaluation of the new technologies required for the Space Exploration Initiative (SEI) and advanced commercial systems. This paper describes the technology developments and evaluation requirements for current and planned commercial and space science programs. Also examined are the capabilities of SITE, the past, present, and planned future configurations of the SITE facility, and applications of SITE to evaluation of SEI technology.

Kerczewski, Robert J.↗

Training evaluation final report

In the area of management training, 'evaluation' refers both to the specific evaluation instrument used to determine whether a training effort was considered effective, and to the procedures followed to evaluate specific training requests. This report recommends to evaluate new training requests in the same way new procurement or new projects are evaluated. This includes examining training requests from the perspective of KSC goals and objectives, and determining expected ROI of proposed training program (does training result in improved productivity, through savings of time, improved outputs, and/or personnel reduction?). To determine whether a specific training course is effective, a statement of what constitutes 'good performance' is required. The user (NOT the Training Branch) must define what is 'required level of performance'. This 'model' will be the basis for the design and development of an objective, performance-based, training evaluation instrument.

Sepulveda, Jose A.↗