Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Performance evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 703 records · Page 39

Evaluation Of Performance Of A Telerobot

Paper presents evaluation of performance of six-axis telerobot system with shared control and advanced modes of force feedback. Various tasks performed with and without force feedback. Study found force feedback reduces average time required to complete task by about 30 percent. Also reduces sum of squared forces by factor of 7 and errors by 63 percent.

Hannaford, Blake↗

Evaluating the Performance of Unmanned Ground Vehicle Water Detection

Water detection is a critical perception requirement for unmanned ground vehicle (UGV) autonomous navigation over cross-country terrain. During the Robotics Collaborative Technology Alliances (RCTA) program, the Jet Propulsion Laboratory (JPL) developed a set of water detection algorithms that are used to detect, localize, and avoid water bodies large enough to be a hazard to a UGV. The JPL water detection software performs the detection and localization stages using a forward-looking stereo pair of color cameras. The 3D coordinates of water body surface points are then output to a UGV's autonomous mobility system, which is responsible for planning and executing safe paths. There are three primary methods for evaluating the performance of the water detection software. Evaluations can be performed in image space on the intermediate detection product, in map space on the final localized product, or during autonomous navigation to characterize the avoidance of a variety of water bodies. This paper describes a methodology for performing the first two types of water detection performance evaluations.

stereo vision↗

Evaluating the performance of a high resolution SLIM IM module integrated with an Agilent 6560 IM-QTOF MS

Integrate a combined Structures of Lossless Ion Manipulations (SLIM) module that incorporates both constant (not time-varying) field (CF) and oscillatory (time-varying) field (OF) IM separation stages into an Agilent 6560 IM-QTOF mass spectrometer system. Once the SLIM module has been integrated into the Agilent system, its performance will be evaluated using a variety of different samples.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Fabrication, evaluation, and performance of machined metal grazing incidence telescopes

The fabrication, evaluation, and performance of machined grazing incidence metal optics for applications ranging from telescopes with imaging of minutes of arc to those with imaging in the less-than-10-arcsec range are reviewed. Issues considered in the fabrication process include the choice of machining substrate, the choice of machine tool, the choice of method for removing tool marks, and the choice of a reflective surface. Means of evaluating the mirror performance include the FWHM and determination of noncircularity and surface roughness. Characteristics of previous and current grazing incidence metal mirrors are presented, with emphasis on the Einstein and Rosat optics.

Bowyer, Stuart↗

On evaluating the performability of degradable computing systems

If the performance of a computing system is 'degradable', performance and reliability issues must be dealt with simultaneously in the process of evaluating system effectiveness. For this purpose, a unified measure, called 'performability', is introduced and the foundations of performability modeling and evaluation are established. A critical step in the modeling process is the introduction of a 'capability function' which relates low-level system behavior to user-oriented performance levels. A hierarchical modeling scheme is used to formulate the capability function and capability is used, in turn, to evaluate performability. These techniques are then illustrated for a specific application: the performability evaluation of an aircraft computer in the environment of an air transport mission.

Meyer, J. F.↗

Image Navigation and Registration Performance Assessment Evaluation Tools for GOES-R ABI and GLM

The GOES-R Flight Project has developed an Image Navigation and Registration (INR) Performance Assessment Tool Set (IPATS) for measuring Advanced Baseline Imager (ABI) and Geostationary Lightning Mapper (GLM) INR performance metrics in the post-launch period for performance evaluation and long term monitoring. IPATS utilizes a modular algorithmic design to allow user selection of data processing sequences optimized for generation of each INR metric. This novel modular approach minimizes duplication of common processing elements, thereby maximizing code efficiency and speed. Fast processing is essential given the large number of sub-image registrations required to generate INR metrics for the many images produced over a 24 hour evaluation period. This paper describes the software design and implementation of IPATS and provides preliminary test results.

Image registration↗

Image Navigation and Registration Performance Assessment Evaluation Tools for GOES-R ABI and GLM

The GOES-R Flight Project has developed an Image Navigation and Registration (INR) Performance Assessment Tool Set (IPATS) for measuring Advanced Baseline Imager (ABI) and Geostationary Lightning Mapper (GLM) INR performance metrics in the post-launch period for performance evaluation and long term monitoring. IPATS utilizes a modular algorithmic design to allow user selection of data processing sequences optimized for generation of each INR metric. This novel modular approach minimizes duplication of common processing elements, thereby maximizing code efficiency and speed. Fast processing is essential given the large number of sub-image registrations required to generate INR metrics for the many images produced over a 24 hour evaluation period. This paper describes the software design and implementation of IPATS and provides preliminary test results.

image registration↗

Signal detection theory and methods for evaluating human performance in decision tasks

Signal Detection Theory (SDT) can be used to assess decision making performance in tasks that are not commonly thought of as perceptual. SDT takes into account both the sensitivity and biases in responding when explaining the detection of external events. In the standard SDT tasks, stimuli are selected in order to reveal the sensory capabilities of the observer. SDT can also be used to describe performance when decisions must be made as to the classification of easily and reliably sensed stimuli. Numbers are stimuli that are minimally affected by sensory processing and can belong to meaningful categories that overlap. Multiple studies have shown that the task of categorizing numbers from overlapping normal distributions produces performance predictable by SDT. These findings are particularly interesting in view of the similarity between the task of the categorizing numbers and that of determining the status of a mechanical system based on numerical values that represent sensor readings. Examples of the use of SDT to evaluate performance in decision tasks are reviewed. The methods and assumptions of SDT are shown to be effective in the measurement, evaluation, and prediction of human performance in such tasks.

Obrien, Kevin↗

Collected notes from the Benchmarks and Metrics Workshop

In recent years there has been a proliferation of proposals in the artificial intelligence (AI) literature for integrated agent architectures. Each architecture offers an approach to the general problem of constructing an integrated agent. Unfortunately, the ways in which one architecture might be considered better than another are not always clear. There has been a growing realization that many of the positive and negative aspects of an architecture become apparent only when experimental evaluation is performed and that to progress as a discipline, we must develop rigorous experimental methods. In addition to the intrinsic intellectual interest of experimentation, rigorous performance evaluation of systems is also a crucial practical concern to our research sponsors. DARPA, NASA, and AFOSR (among others) are actively searching for better ways of experimentally evaluating alternative approaches to building intelligent agents. One tool for experimental evaluation involves testing systems on benchmark tasks in order to assess their relative performance. As part of a joint DARPA and NASA funded project, NASA-Ames and Teleos Research are carrying out a research effort to establish a set of benchmark tasks and evaluation metrics by which the performance of agent architectures may be determined. As part of this project, we held a workshop on Benchmarks and Metrics at the NASA Ames Research Center on June 25, 1990. The objective of the workshop was to foster early discussion on this important topic. We did not achieve a consensus, nor did we expect to. Collected here is some of the information that was exchanged at the workshop. Given here is an outline of the workshop, a list of the participants, notes taken on the white-board during open discussions, position papers/notes from some participants, and copies of slides used in the presentations.

Drummond, Mark E.↗

FPGA-based HPC accelerators: An evaluation on performance and energy efficiency

Hardware specialization is a promising direction for the future of digital computing. Reconfigurable technologies enable hardware specialization with modest non-recurring engineering cost, but their performance and energy efficiency compared to state-of-the-art processor architectures remain an open question. In this article, we use FPGAs to evaluate the benefits of building specialized hardware for numerical kernels found in scientific applications. In order to properly evaluate performance, we not only compare Intel Arria 10 and Xilinx U280 performance against Intel Xeon, Intel Xeon Phi, and NVIDIA V100 GPUs, but we also extend the Empirical Roofline Toolkit (ERT) to FPGAs in order to assess our results in terms of the Roofline model. We show design optimization and tuning techniques for peak FPGA performance at reasonable hardware usage and power consumption. As FPGA peak performance is known to be far less than that of a GPU, we also benchmark the energy efficiency of each platform for the scientific kernels comparing against microbenchmark and technological limits. Results show that while FPGAs struggle to compete in absolute terms with GPUs on memory- and compute-intensive kernels, they require far less power and can deliver nearly the same energy efficiency.

97 MATHEMATICS AND COMPUTING↗

Optical-communication systems for deep-space applications

The feasibility of using optical communication systems for data telemetry from deep space vehicles to Earth based receivers is evaluated. Performance analysis shows that practical, photon counting optical systems can transmit data reliably at 30 to 40 dB high rates than existing RF systems, or can be used to extend the communication range by 15 to 20 dB. The advantages of pulse-position modulation (PPM) formats are discussed, and photon counting receiver structures designed for PPM decoding are described. The effects of background interference and weather on receiver performance are evaluated. Some consideration is given to tracking and beam pointing operations, since system performance ultimately depends on the accuracy to which these operations can be carried out. An example of a tracking and pointing system utilizing an optical uplink beacon is presented, and it is shown that microradian beam pointing is within the capabilities of state-of-the-art technology. Recommendations for future theoretical studies and component development programs are presented.

Vilnrotter, V. A.↗

Standardizing Performance Metrics for Building-Level Electrical Distribution Systems

Building-level electrical distribution systems comprise a myriad of current-carrying equipment, conversion devices, and protection devices that deliver power from the utility or local distributed energy resources to end-use building loads. Electric power has traditionally been generated, transmitted, and distributed in alternating current (AC). However, the last decade has seen a significant increase in the integration of native direct current (DC) equipment that has elevated the importance of DC distribution systems. Numerous studies have comparatively examined the performance of various electrical distribution systems in buildings but have failed to achieve uniform conclusions, primarily because of a lack of consistent and analogous performance evaluation methods. This paper aims to fill this gap by providing a standard set of metrics and measurement boundaries to consistently evaluate the performance of AC, DC, or hybrid AC/DC electrical distribution systems. The efficacy of the proposed approach is evaluated on a representative medium-sized commercial office building model with AC distribution and an equivalent hybrid AC/DC and DC distribution model, wherein the AC distribution model is concluded to be the most efficient. The simulation results show variation in computed metrics with different selected boundaries that verify the effectiveness of the proposed approach in ensuring consistent computation of the performance of building-level electrical distribution systems. This paper provides an initial set of guidelines for building energy system stakeholders to adopt appropriate solutions, thus leading to more efficient energy systems.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Method of Test for Evaluating Building Performance Simulation Software

ANSI/ASHRAE Standard 140 Method of Test for Evaluating Building Performance Simulation Software specifies the method to test the core competency of building performance simulation (BPS) software, a broad class of software which includes building energy modeling (BEM) software. Standard 140 has been widely used by BEM software vendors to help diagnose and compare the results from the program with other modeling programs. The Standard is also referenced by codes, standards, government programs, and other incentive programs as a part of minimum requirements for qualifying BEM software. The explanatory information, summary tables and figures in this 140 User’s Manual (referred to as “this Manual”) are provided to help users in implementing various suites of tests specified in Standard 140-2023 (referred to in this manual as “Standard 140” or “the Standard”).

97 MATHEMATICS AND COMPUTING↗

Field validation of isotropic analytical models for simulating fabric shades

Fabric roller shades are common shading materials used in commercial and residential buildings. Accurately characterizing and modeling shades helps practitioners select the appropriate product and its control strategy based on climate and occupants' priorities, such as visual comfort and view to outdoors. Previous studies established a generalized method for modeling complex fenestration systems using data-driven tabulated bidirectional scattering distribution functions. However, deploying such a method at scale to all fabric shading products on the market is too costly and time-consuming. Analytical models that are based on a limited set of measurements (e.g., normal-normal and normal-hemispherical visible transmittance and reflectance, and directional cut-off angles) can be used to model the wide variety of shading products on the market. This study evaluates the performance of two isotropic analytical models, Roos-Wienold and Modified-Kotey, for modeling fabric roller shades, with a focus on the model's ability to predict occupant visual comfort. The performance evaluation was conducted through laboratory and field measurements and simulations. The results showed that both models are sufficient for predicting vertical illuminance at seated eye-level. Roos-Wienold model was able to predict binary visual comfort classification (glare/no-glare) under a wide range of luminance conditions, while Modified-Kotey model did not perform as well under high-contrast low-adaptation conditions. Both models are insufficient in predicting visual comfort at a four-point scale (e.g., imperceptible, perceptible, disturbing, intolerable). The two isotropic models become less accurate when the fabric exhibits high anisotropy.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Notes on SAW Tag Interrogation Techniques

We consider the problem of interrogating a single SAW RFID tag with a known ID and known range in the presence of multiple interfering tags under the following assumptions: (1) The RF propagation environment is well approximated as a simple delay channel with geometric power-decay constant alpha >/= 2. (2) The interfering tag IDs are unknown but well approximated as independent, identically distributed random samples from a probability distribution of tag ID waveforms with known second-order properties, and the tag of interest is drawn independently from the same distribution. (3) The ranges of the interfering tags are unknown but well approximated as independent, identically distributed realizations of a random variable rho with a known probability distribution f(sub rho) , and the tag ranges are independent of the tag ID waveforms. In particular, we model the tag waveforms as random impulse responses from a wide-sense-stationary, uncorrelated-scattering (WSSUS) fading channel with known bandwidth and scattering function. A brief discussion of the properties of such channels and the notation used to describe them in this document is given in the Appendix. Under these assumptions, we derive the expression for the output signal-to-noise ratio (SNR) for an arbitrary combination of transmitted interrogation signal and linear receiver filter. Based on this expression, we derive the optimal interrogator configuration (i.e., transmitted signal/receiver filter combination) in the two extreme noise/interference regimes, i.e., noise-limited and interference-limited, under the additional assumption that the coherence bandwidth of the tags is much smaller than the total tag bandwidth. Finally, we evaluate the performance of both optimal interrogators over a broad range of operating scenarios using both numerical simulation based on the assumed model and Monte Carlo simulation based on a small sample of measured tag waveforms. The performance evaluation results not only provide guidelines for proper interrogator design, but also provide some insight on the validity of the assumed signal model. It should be noted that the assumption that the impulse response of the tag of interest is known precisely implies that the temperature and range of the tag are also known precisely, which is generally not the case in practice. However, analyzing interrogator performance under this simplifying assumption is much more straightforward and still provides a great deal of insight into the nature of the problem.

Barton, Richard J.↗