Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

Evaluation of Visual Analytics Environments: The Road to the Visual Analytics Science and Technology Challenge Evaluation Methodology

The evaluation of visual analytics environments was a topic in Illuminating the Path [Thomas 2005] as a critical aspect of moving research into practice. For a thorough understanding of the utility of the systems available, evaluation not only involves assessing the visualizations, interactions or data processing algorithms themselves, but also the complex processes that a tool is meant to support (such as exploratory data analysis and reasoning, communication through visualization, or collaborative data analysis [Lam 2012; Carpendale 2007]). Researchers and practitioners in the field have long identified many of the challenges faced when planning, conducting, and executing an evaluation of a visualization tool or system [Plaisant 2004]. Evaluation is needed to verify that algorithms and software systems work correctly and that they represent improvements over the current infrastructure. Additionally to effectively transfer new software into a working environment, it is necessary to ensure that the software has utility for the end-users and that the software can be incorporated into the end-user’s infrastructure and work practices. Evaluation test beds require datasets, tasks, metrics and evaluation methodologies. As noted in [Thomas 2005] it is difficult and expensive for any one researcher to setup an evaluation test bed so in many cases evaluation is setup for communities of researchers or for various research projects or programs. Examples of successful community evaluations can be found [Chinchor 1993; Voorhees 2007; FRGC 2012]. As visual analytics environments are intended to facilitate the work of human analysts, one aspect of evaluation needs to focus on the utility of the software to the end-user. This requires representative users, representative tasks, and metrics that measure the utility to the end-user. This is even more difficult as now one aspect of the test methodology is access to representative end-users to participate in the evaluation. In many cases the sensitive nature of data and tasks and difficult access to busy analysts puts even more of a burden on researchers to complete this type of evaluation. User-centered design goes beyond evaluation and starts with the user [Beyer 1997, Shneiderman 2009]. Having some knowledge of the type of data, tasks, and work practices helps researchers and developers know the correct paths to pursue in their work. When access to the end-users is problematic at best and impossible at worst, user-centered design becomes difficult. Researchers are unlikely to go to work on the type of problems faced by inaccessible users. Commercial vendors have difficulties evaluating and improving their products when they cannot observe real users working with their products. In well-established fields such as web site design or office software design, user-interface guidelines have been developed based on the results of empirical studies or the experience of experts. Guidelines can speed up the design process and replace some of the need for observation of actual users [heuristics review references]. In 2006 when the visual analytics community was initially getting organized, no such guidelines existed. Therefore, we were faced with the problem of developing an evaluation framework for the field of visual analytics that would provide representative situations and datasets, representative tasks and utility metrics, and finally a test methodology which would include a surrogate for representative users, increase interest in conducting research in the field, and provide sufficient feedback to the researchers so that they could improve their systems.

evaluation↗

Advanced Modeling and Simulation Methods for Evaluation of Thermal Neutron Scattering Materials

With the rise of interest in thermal neutron scattering data for advanced reactor, criticality safety, and shielding applications, new experimental data are required for evaluation of new materials or for re-evaluation (or validations) of previously evaluated materials. New experimental data are evaluated in a three-step process: (1) computing the phonon characteristics, (2) computing the dynamic structure factor (DSF) from the data, and (3) using the experimental setup to simulate the experimental data. All three steps have challenges, ranging from the need for a sufficiently general material simulation code—a processing code that can compute the corresponding DSF—to having a detailed layout of the instrument/beamline/facility where the data were measured. Whereas phonon characteristics of materials can be computed using various methods (molecular dynamics, density functional theory, etc.), a high-fidelity computation of the DSF and the simulation of the experiment based on the DSF is vital to the accuracy of the evaluation. The latter two steps can be achieved by using the two corresponding code systems developed by instrument scientists at the Spallation Neutron Source (SNS) at Oak Ridge National Laboratory: (1) OCLIMAX, a program that calculates the dynamic structure factor from DFT and MD simulation results, and (2) MCViNE, a Monte Carlo neutron ray-tracing program designed to simulate neutron scattering experiments. Recently, polyethylene and yttrium hydride were measured at the Wide Angular-Range Chopper (ARCS) and SEQUOIA instrument stations of the SNS. These experiments are simulated using the density functional theory code, the Cambridge Serial Total Energy Package (CASTEP), to compute its phonon characteristics (eigenvalues/vectors and PDOS), which is then processed using OCLIMAX to yield the DSF, and finally the data at each instrument station are simulated by the MCViNE for comparison to the measured data for evaluation. For comparison to conventional evaluation methods, the scattering data processed from OCLIMAX are compared against those processed from the LEAPR module of NJOY, and the results from MCViNE simulations are compared against previously used simplified beamline models implemented in the Monte Carlo N-Particle (MCNP) code.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Working Party on International Nuclear Data Evaluation Cooperation (WPEC)

The OECD Nuclear Energy Agency (NEA) organizes cooperation between the major nuclear data evaluation projects in the world. The NEA Working Party on International Nuclear Data Evaluation Cooperation (WPEC) was established to promote the exchange of information on nuclear data evaluation, measurement, nuclear model calculation, validation, and related topics, and to provide a framework for cooperative activities between the participating projects. The working party assesses nuclear data improvement needs and addresses these needs by initiating joint activities in the framework of dedicated WPEC subgroups. Studies recently completed comprise a number of works related to nuclear data covariance and associated processing issues, as well as more specific studies related to the resonance parameter representation in the unresolved resonance region, the gamma production from fission product capture reactions, the {sup 235}U capture cross section, the EXFOR database, and the improvement of nuclear data for advanced reactor systems. Ongoing activities focus on the evaluation of {sup 239}Pu in the resonance region, scattering angular distribution in the fast energy range, and reporting/usage of experimental data for evaluation in the resolved resonance region. New activities include two subgroups on improved fission product yield evaluation methodologies and on modern nuclear database structures. Future activities under discussion include a pilot project for a Collaborative International Evaluated Library Organization (CIELO) and methods to provide feedback from nuclear and covariance data adjustment for improvement of nuclear data. In addition to the above mentioned short-term task-oriented subgroups, WPEC also hosts a longer-term subgroup charged with reviewing and compiling the most important nuclear data requirements in a high priority request list (HPRL)

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

A Software Developer’s Guide to Informal Evaluation of Visual Analytics Environments Using VAST Challenge Information

The VAST Challenge has been a popular venue for academic and industry participants for over ten years. Many participants comment that the majority of their time in preparing VAST Challenge entries is discovering elements in their software environments that need to be redesigned in order to solve the given task. Fortunately, there is no need to wait until the VAST Challenge is announced to test out software systems. The Visual Analytics Benchmark Repository contains all past VAST Challenge tasks, data, solutions and submissions. This paper details the various types of evaluations that may be conducted using the Repository information. In this paper we describe how developers can do informal evaluations of various aspects of their visual analytics environments using VAST Challenge information. Aspects that can be evaluated include the appropriateness of the software for various tasks, the various data types and formats that can be accommodated, the effectiveness and efficiency of the process supported by the software, and the intuitiveness of the visualizations and interactions. Researchers can compare their visualizations and interactions to those submitted to determine novelty. In addition, the paper provides pointers to various guidelines that software teams can use to evaluate the usability of their software. While these evaluations are not a replacement for formal evaluation methods, this information can be extremely useful during the development of visual analytics environments.

evaluation↗

Dosimetric evaluation of the interplay effect in respiratory-gated RapidArc radiation therapy

Purpose: Volumetric modulated arc therapy (VMAT) with gating capability has had increasing adoption in many clinics in the United States. In this new technique, dose rate, gantry rotation speed, and the leaf motion speed of multileaf collimators (MLCs) are modulated dynamically during gated beam delivery to achieve highly conformal dose coverage of the target and normal tissue sparing. Compared with the traditional gated intensity-modulated radiation therapy technique, this complicated beam delivery technique may result in larger dose errors due to the intrafraction tumor motion. The purpose of this work is to evaluate the dosimetric influence of the interplay effect for the respiration-gated VMAT technique (RapidArc, Varian Medical Systems, Palo Alto, CA). Our work consisted of two parts: (1) Investigate the interplay effect for different target residual errors during gated RapidArc delivery using a one-dimensional moving phantom capable of producing stable sinusoidal movement; (2) Evaluate the dosimetric influence in ten clinical patients’ treatment plans using a moving phantom driven with a patient-specific respiratory curve. Methods: For the first part of this study, four plans were created with a spherical target for varying residual motion of 0.25, 0.5, 0.75, and 1.0 cm. Appropriate gating windows were applied for each. The dosimetric effect was evaluated using EDR2 film by comparing the gated delivery with static delivery. For the second part of the project, ten gated lung stereotactic body radiotherapy cases were selected and reoptimized to be delivered by the gated RapidArc technique. These plans were delivered to a phantom, and again the gated treatments were compared to static deliveries by the same methods. Results: For regular sinusoidal motion, the dose delivered to the target was not substantially affected by the gating windows when evaluated with the gamma statistics, suggesting the interplay effect has a small role in respiratory-gated RapidArc therapy. Varied results were seen when gated therapy was performed on the patient plans that could only be attributed to differences in patient respiratory patterns. Patients whose plans had the largest percentage of pixels failing the gamma statistics exhibited irregular breathing patterns including substantial interpatient variation in depth of respiration. Conclusions: The interplay effect has a limited impact on gated RapidArc therapy when evaluated with a linear phantom. Variations in patient breathing patterns, however, are of much greater clinical significance. Caution must be taken when evaluating patients’ respiratory efforts for gated arc therapy.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Medium-Duty Plug-in Electric Delivery Truck Fleet Evaluation

In this paper, the authors present an overview of medium-duty electric vehicle (EV) operating behavior based on in-use data collected from Smith Newton electric delivery vehicles and compare their performance and operation to conventional diesel trucks operating in the same fleet. The vehicles' drive cycles and operation are analyzed and compared to demonstrate the importance of matching specific EV technologies to the appropriate operational duty cycle. The results of this analysis show that the Smith Newton EVs demonstrated a 68% reduction in energy consumption over the data reporting period compared to the conventional diesel vehicles, as well as a 46.4% reduction in carbon dioxide equivalent emissions based on the local energy generation source.

33 ADVANCED PROPULSION SYSTEMS↗

Evaluation of Neutron Reactions on Iron Isotopes for CIELO and ENDF/B-VIII.0

A new suite of evaluations for {sup 54,56,57,58}Fe has been developed in the framework of the CIELO international collaboration. New resolved resonance ranges were evaluated for {sup 54}Fe and {sup 57}Fe, while modifications were applied to resonances in {sup 56}Fe. The low energy part of the {sup 56}Fe file is almost totally based on measurements. At higher energies in {sup 56}Fe and in the whole fast neutron range for minor isotopes the evaluation consists of model predictions carefully adjusted to available experimental data. We also make use of the high quality and well experimentally-constrained dosimetry evaluations from the IRDFF library. Special attention was dedicated to the elastic angular distributions, which were found to affect results of the integral benchmarking. The new set of iron evaluations was developed in concert with other CIELO evaluations and they were tested together in the integral experiments before being adopted for the ENDF/B-VIII.0 library.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Automated Resonance Fitting for Nuclear Data Evaluation

Global and national efforts to deliver high-quality nuclear data to users have a wide-ranging impact, affecting applications in national security, reactor operations, basic science, medicine, and more. Cross section evaluation is a major part of this effort, combining theory and experimentation to produce recommended values and uncertainties for reaction probabilities. Resonance region evaluation is a specialized type of nuclear data evaluation that can require significant manual effort and months of time from expert scientists. In this article, non-convex non-linear optimization methods are combined with concepts of inferential statistics to infer a resonance model from experimental data in an automated manner that is not dependent on prior evaluation(s). This methodology aims to enhance the workflow of a resonance evaluator by minimizing time, effort, and the potential for bias from prior assumptions, while enhancing reproducibility and documentation, thereby addressing well-known challenges in the field.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

RIPL - Reference Input Parameter Library for Calculation of Nuclear Reactions and Nuclear Data Evaluations

We describe the physics and data included in the Reference Input Parameter Library, which is devoted to input parameters needed in calculations of nuclear reactions and nuclear data evaluations. Advanced modelling codes require substantial numerical input, therefore the International Atomic Energy Agency (IAEA) has worked extensively since 1993 on a library of validated nuclear-model input parameters, referred to as the Reference Input Parameter Library (RIPL). A final RIPL coordinated research project (RIPL-3) was brought to a successful conclusion in December 2008, after 15 years of challenging work carried out through three consecutive IAEA projects. The RIPL-3 library was released in January 2009, and is available on the Web through (http://www-nds.iaea.org/RIPL-3/). This work and the resulting database are extremely important to theoreticians involved in the development and use of nuclear reaction modelling (ALICE, EMPIRE, GNASH, UNF, TALYS) both for theoretical research and nuclear data evaluations. The numerical data and computer codes included in RIPL-3 are arranged in seven segments: MASSES contains ground-state properties of nuclei for about 9000 nuclei, including three theoretical predictions of masses and the evaluated experimental masses of Audi et al. (2003). DISCRETE LEVELS contains 117 datasets (one for each element) with all known level schemes, electromagnetic and {gamma}-ray decay probabilities available from ENSDF in October 2007. NEUTRON RESONANCES contains average resonance parameters prepared on the basis of the evaluations performed by Ignatyuk and Mughabghab. OPTICAL MODEL contains 495 sets of phenomenological optical model parameters defined in a wide energy range. When there are insufficient experimental data, the evaluator has to resort to either global parameterizations or microscopic approaches. Radial density distributions to be used as input for microscopic calculations are stored in the MASSES segment. LEVEL DENSITIES contains phenomenological parameterizations based on the modified Fermi gas and superfluid models and microscopic calculations which are based on a realistic microscopic single-particle level scheme. Partial level densities formulae are also recommended. All tabulated total level densities are consistent with both the recommended average neutron resonance parameters and discrete levels. GAMMA contains parameters that quantify giant resonances, experimental gamma-ray strength functions and methods for calculating gamma emission in statistical model codes. The experimental GDR parameters are represented by Lorentzian fits to the photo-absorption cross sections for 102 nuclides ranging from {sup 51}V to {sup 239}Pu. FISSION includes global prescriptions for fission barriers and nuclear level densities at fission saddle points based on microscopic HFB calculations constrained by experimental fission cross sections.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Bayesian Monte Carlo Evaluation of Imperfect (n, 233 U) Data and Model

Conventional nuclear data evaluation methods using generalized linear least squares make the following assumptions: prior and posterior probability distribution functions (PDFs) of all model parameters and data are normal (Gaussian); the linear approximation is sufficiently accurate to minimize the cost function (even for nonlinear models); the model (e.g., of neutron cross section) and experimental data (including covariance data) are without defect and prior PDFs of parameters and measured data are known perfectly. Neglect of covariance between model parameters and measured data in conventional evaluations contributes to imperfections. These assumptions are inherent to the generalized linear least squares minimization method commonly used for resolved resonance region neutron cross section evaluations but are often not justified due to the presence of non-normal PDFs, nonlinear models (e.g., R-matrix formalism), and inherent imperfections in data and models (e.g., imperfect covariance data). Here, these assumptions are removed in a mathematical framework of Bayes’ theorem, which is implemented using the Metropolis-Hastings Monte Carlo method. Most importantly, new parameters are introduced to parameterize discrepancies between the theoretical model and measured data to quantify judgement about discrepancies or imperfections in a reproducible manner. An evaluation of 233U in the eV region using the ENDF-B/VIII.0 library and transmission data (Guber et al.) is presented, and posterior parameters are compared to those obtained by conventional evaluation methods. This example illustrates the effects of removing the most harmful assumption: that of model-data perfection.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Disposal systems evaluations and tool development : Engineered Barrier System (EBS) evaluation.

Key components of the nuclear fuel cycle are short-term storage and long-term disposal of nuclear waste. The latter encompasses the immobilization of used nuclear fuel (UNF) and radioactive waste streams generated by various phases of the nuclear fuel cycle, and the safe and permanent disposition of these waste forms in geological repository environments. The engineered barrier system (EBS) plays a very important role in the long-term isolation of nuclear waste in geological repository environments. EBS concepts and their interactions with the natural barrier are inherently important to the long-term performance assessment of the safety case where nuclear waste disposition needs to be evaluated for time periods of up to one million years. Making the safety case needed in the decision-making process for the recommendation and the eventual embracement of a disposal system concept requires a multi-faceted integration of knowledge and evidence-gathering to demonstrate the required confidence level in a deep geological disposal site and to evaluate long-term repository performance. The focus of this report is the following: (1) Evaluation of EBS in long-term disposal systems in deep geologic environments with emphasis on the multi-barrier concept; (2) Evaluation of key parameters in the characterization of EBS performance; (3) Identification of key knowledge gaps and uncertainties; and (4) Evaluation of tools and modeling approaches for EBS processes and performance. The above topics will be evaluated through the analysis of the following: (1) Overview of EBS concepts for various NW disposal systems; (2) Natural and man-made analogs, room chemistry, hydrochemistry of deep subsurface environments, and EBS material stability in near-field environments; (3) Reactive Transport and Coupled Thermal-Hydrological-Mechanical-Chemical (THMC) processes in EBS; and (4) Thermal analysis toolkit, metallic barrier degradation mode survey, and development of a Disposal Systems Evaluation Framework (DSEF). This report will focus on the multi-barrier concept of EBS and variants of this type which in essence is the most adopted concept by various repository programs. Empasis is given mainly to the evaluation of EBS materials and processes through the analysis of published studies in the scientific literature of past and existing repository research programs. Tool evaluations are also emphasized, particularly on THCM processes and chemical equilibria. Although being an increasingly important aspect of NW disposition, short-term or interim storage of NW will be briefly discussed but not to the extent of the EBS issues relevant to disposal systems in deep geologic environments. Interim storage will be discussed in the report Evaluation of Storage Concepts FY10 Final Report (Weiner et al. 2010).

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Comparative Performance Evaluation of High-performance Data Transfer Tools

Data transfer in wide-area networks has been long studied in different contexts, from data sharing among data centers to online access to scientific data. Many software tools and platforms have been developed to facilitate easy, reliable, fast, and secure data transfer over wide area networks, such as GridFTP, FDT, bbcp, mdtmFTP, and XDD. However, few studies have shown the full capabilities of existing data transfer tools from the perspective of whether such tools have fully adopted state-of-the-art techniques through meticulous comparative evaluations. In this paper, we evaluate the performance of the four highperformance data transfer tools (GridFTP, FDT, mdtmFTP, and XDD) in various environments. Our evaluation suggests that each tool has strengths and weaknesses. FDT and GridFTP perform consistently in diverse environments. XDD and mdtmFTP show improved performance in limited environments and datasets during our evaluation. Unlike other studies on data transfer tools, we also evaluate the predictability of the tools' performance, an important factor for scheduling different stages of science workflows. Performance predictability also helps in (auto)tuning the configurable parameters of the data transfer tool. We apply statistical learning techniques such as linear/polynomial regression, and k-nearest neighbors (kNN), to assess the performance predictability of each tool using its control parameters. Our results show that we can achieve good prediction performance for GridFTP and mdtmFTP using linear regression and kNN, respectively.

Nadig, Deepak↗

An Observational Evaluation of RKW Theory over the U.S. Southern Great Plains

The theory of Rotunno et al. (“RKW” theory) addresses the behavior of squall-line cold pools in vertically sheared flows. It predicts that, within a given thermodynamic environment, a balance between baroclinic vorticity generation by the cold pool and low-level environmental vertical wind shear induces an upright updraft along the gust front that maximizes the initiation of new convective cells. Although this theory has been evaluated numerically, its applicability to observed systems remains unclear and is limited by a lack of critical measurements, including high-frequency thermodynamic and wind profiles across the gust front. Herein, observations from the Atmospheric Radiation Measurement Southern Great Plains (ARM-SGP) observatory near Lamont, Oklahoma, are used to evaluate RKW theory for 10 well-observed squall lines over a 11-yr period. For this evaluation, RKW parameters including cold-pool intensity (c), low-level ambient, line-normal vertical shear (ΔV n ), subcloud and cloud-layer updraft tilts, and multiple measures of system intensity are estimated. Furthermore, the c estimates rely on thermodynamic retrievals from the Atmosphere Emitted Radiance Interferometer (AERI), which are uncertain but verify reasonably well against independent observations. As predicted by the theory, for c/ΔV n ≥ 1, c/ΔV n correlates positively with updraft tilt and negatively with system intensity, but these results are not always statistically significant and are also sensitive to the method by which ΔV n is evaluated. Specifically, ΔV n evaluations that extend above the cold-pool top yield greater consistency with RKW predictions. Also, some measures of intensity correlate more strongly with standard moist instability metrics than with RKW parameters.

Cold pools↗

Combinatorial theory of the semiclassical evaluation of transport moments. I. Equivalence with the random matrix approach

To study electronic transport through chaotic quantum dots, there are two main theoretical approaches. One involves substituting the quantum system with a random scattering matrix and performing appropriate ensemble averaging. The other treats the transport in the semiclassical approximation and studies correlations among sets of classical trajectories. There are established evaluation procedures within the semiclassical evaluation that, for several linear and nonlinear transport moments to which they were applied, have always resulted in the agreement with random matrix predictions. We prove that this agreement is universal: any semiclassical evaluation within the accepted procedures is equivalent to the evaluation within random matrix theory. The equivalence is shown by developing a combinatorial interpretation of the trajectory sets as ribbon graphs (maps) with certain properties and exhibiting systematic cancellations among their contributions. Remaining trajectory sets can be identified with primitive (palindromic) factorisations whose number gives the coefficients in the corresponding expansion of the moments of random matrices. The equivalence is proved for systems with and without time reversal symmetry.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Recent Work Leading Towards a New Evaluation of the Neutron Standards

A new version of the ENDF/B library has been planned. The first step in producing this new library is evaluating the neutron standards. An evaluation is now underway with support from a Data Development Project of the IAEA. In addition to the neutron cross section standards, new evaluations are being done for prompt fission neutron spectra and a number of reference data. Efforts have been made to handle uncertainties in a proper way in these evaluations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Resolved resonance region evaluations of n+ 206,207,208 Pb for fast spectrum applications

Resolved resonance region evaluations of the major isotopes of natural lead, 206 Pb, 207 Pb, 208 Pb, have been per formed to support the development of Generation IV reactors. In this study, validation of nuclear data for lead fast reactors was performed with simulations of shielding benchmarks, integral critical benchmarks, and quasi-differential scattering measurements. Sensitivity analyses of these systems showed that elastic scattering reactions above 100 keV were the dominant reactions driving system performance. The resolved resonance regions (RRRs) of the lead isotopes extend past 100 keV, making the RRR an ideal starting point to evaluate lead cross sections. Since the R-matrix requires knowledge of bound, distant, and observed resonances, it was necessary to evaluate from 10 -5 eV up to the respective limit of the RRR. The 208 Pb RRR evaluation was extended to 1.5 MeV in order to obtain resonance parameters used to calculate new elastic scattering angular distributions up to 1.5 MeV. Resonance parameter uncertainties and covariance were generated using the R-matrix code SAMMY. The new RRR parameters show a direct improvement to the scattering kernel below 1.5 MeV which in turn greatly improves fast critical experiments over ENDF/B-VIII.0.

206Pb↗