Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Performance evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

A hybrid numerical and machine learning framework for evaluating the performance of a 780 cm 2 aqueous organic redox flow battery

Aqueous organic redox flow battery (AORFB) is a promising cost-competitive technology for large-scale energy storage. Among existing work, the dihydroxyphenazine (DHP)-based AORFB has demonstrated high energy density and low capacity degradation in 10 cm2 cells during lab tests. However, its commercial-scale performance in more complex environments remains unknown, posing a barrier for commercialization. To address this gap, this work presents a comprehensive performance evaluation of a 780 cm 2 DHP-based AORFB by combining physics-based numerical model, machine learning (ML)-based surrogate models, and ML-derived sensitivity quantification. Specifically, we first select 12 key battery parameters that include 10 physicochemical quantities and 2 operation quantities, then select 6 performance metrics that include energy efficiency (EE), discharging capacity, charging energy, and power losses due to concentration, activation, and ohmic over-potentials. With such selection, 12800 combinations of the 12 parameters are subsequently generated using the Latin Hypercube Sampling method. These combinations, together with 38 pre-defined State of Charge, are then integrated to a validated AORFB model developed in COMSOL to compute the performance metrics. With both input parameters and performance metrics, 60 deep neural network (DNN) surrogate models are then trained to approximate the relationship between the 10 physicochemical quantities and 6 performance metrics at each flow rate and current density. Sensitivity scores are then calculated based on the DNN models. Two additional sensitivity analysis tools, i.e., MARS, and SHAP, are also used to cross-validate the sensitivity scores from the DNN. The results demonstrate that 1) the standard potential ranks the first in controlling EE and charging energy, 2) the membrane conductivity is most critical for power loss and EE, and 3) specific area and reaction rate control activation power loss.

25 ENERGY STORAGE↗
NASA NTRS1990

Development and testing of methodology for evaluating the performance of multi-input/multi-output digital control systems

The development of a controller performance evaluation (CPE) methodology for multiinput/multioutput digital control systems is described. The equations used to obtain the open-loop plant, controller transfer matrices, and return-difference matrices are given. Results of applying the CPE methodology to evaluate MIMO digital flutter suppression systems being tested on an active flexible wing wind-tunnel model are presented to demonstrate the CPE capability.

Polotzky, Anthony S.↗
NASA NTRS2010

Metrics for Offline Evaluation of Prognostic Performance

Prognostic performance evaluation has gained significant attention in the past few years. Currently, prognostics concepts lack standard definitions and suffer from ambiguous and inconsistent interpretations. This lack of standards is in part due to the varied end-user requirements for different applications, time scales, available information, domain dynamics, etc. to name a few. The research community has used a variety of metrics largely based on convenience and their respective requirements. Very little attention has been focused on establishing a standardized approach to compare different efforts. This paper presents several new evaluation metrics tailored for prognostics that were recently introduced and were shown to effectively evaluate various algorithms as compared to other conventional metrics. Specifically, this paper presents a detailed discussion on how these metrics should be interpreted and used. These metrics have the capability of incorporating probabilistic uncertainty estimates from prognostic algorithms. In addition to quantitative assessment they also offer a comprehensive visual perspective that can be used in designing the prognostic system. Several methods are suggested to customize these metrics for different applications. Guidelines are provided to help choose one method over another based on distribution characteristics. Various issues faced by prognostics and its performance evaluation are discussed followed by a formal notational framework to help standardize subsequent developments.

Saxena, Abhinav↗
NASA NTRS1992

Small Business Innovations

The purpose of QASE RT is to enable system analysts and software engineers to evaluate performance and reliability implications of design alternatives. The program resulted from two Small Business Innovation Research (SBIR) projects. After receiving a description of the system architecture and workload from the user, QASE RT translates the system description into simulation models and executes them. Simulation provides detailed performance evaluation. The results of the evaluations are service and response times, offered load and device utilizations and functional availability.

Source record↗
DOE OSTI2024

Evaluation of Converter Performance Considering Static and Dynamic Device Part-to-Part Variability

This paper presents a methodology to incorporate and analyze the impact of semiconductor device part-to-part variation on power converter performance. By integrating extensive static and dynamic device characterization data with an automated compact model generation process that reflects manufacturing variability, device models with inherent variability features are utilized in converter simulations for a comprehensive assessment of performance impacts. The traditional converter performance evaluation process typically yields fixed efficiency values, often dismissing the inherent part-to-part variability caused by the manufacturing process of semiconductor devices. To address this limitation, a large population of devices was characterized to capture variations in static parameters-such as transfer, output, and capacitance characteristics-as well as dynamic behaviors, including switching losses. This data-driven approach enables the development of individual compact models, which were then integrated into converter simulations to evaluate efficiency ranges rather than single point estimated values. The converter simulation results show that part-to-part component variation can lead to significant efficiency deviations, exceeding several percentage points in high-power conversion applications. By offering a more accurate representation of converter behavior under real-world manufacturing conditions, this methodology enables designers to anticipate performance variability, improving the robustness of power converter designs.

device characterization↗
NASA NTRS2011

Grazing Incidence Wavefront Sensing and Verification of X-Ray Optics Performance

Evaluation of interferometrically measured mirror metrology data and characterization of a telescope wavefront can be powerful tools in understanding of image characteristics of an x-ray optical system. In the development of soft x-ray telescope for the International X-Ray Observatory (IXO), we have developed new approaches to support the telescope development process. Interferometrically measuring the optical components over all relevant spatial frequencies can be used to evaluate and predict the performance of an x-ray telescope. Typically, the mirrors are measured using a mount that minimizes the mount and gravity induced errors. In the assembly and mounting process the shape of the mirror segments can dramatically change. We have developed wavefront sensing techniques suitable for the x-ray optical components to aid us in the characterization and evaluation of these changes. Hartmann sensing of a telescope and its components is a simple method that can be used to evaluate low order mirror surface errors and alignment errors. Phase retrieval techniques can also be used to assess and estimate the low order axial errors of the primary and secondary mirror segments. In this paper we describe the mathematical foundation of our Hartmann and phase retrieval sensing techniques. We show how these techniques can be used in the evaluation and performance prediction process of x-ray telescopes.

Saha, Timo T.↗
NASA NTRSDate not supplied

Evaluation of Performance, Fatigue and Workload During Real-time, Reactive Telerobotic Mission Control Operations

Real-time, reactive telerobotic mission control operations require personnel to actively operate and manage remotely controlled vehicles or robots in real time. Due to the physical separation of the vehicle from the operator, such operations present many additional factors that can influence fatigue (i.e., degraded mental performance) and workload (i.e., mental and physical cost of task requirements), making it difficult to assess how long an individual can safely and accurately conduct operations. The upcoming Volatiles Investigating Polar Exploration Rover (VIPER) mission involves remotely controlling a lunar vehicle from an Earth-based mission control station. METHODS: In order to determine how long personnel could successfully maintain alertness and performance while operating a rover, we studied seven trained operators (i.e., 5 drivers, 2 real-time scientists) in a simulated mission control environment. We further assessed workload in order to determine whether perceived workload levels would change relative to time on task and by time of day. Operators completed two five-hour simulations in a randomized order, beginning at noon and at midnight. Performance was evaluated every 30 minutes using the psychomotor vigilance task (PVT), Karolinska Sleepiness Scale (KSS), and NASA Task Load Index (NASA-TLX). RESULTS: On average, participants rated themselves as sleepier (5.06 ± 2.28) on the night drives compared to the day drives (3.12 ± 1.44; P < .0001). Workload was rated higher during the day drives (37.93 ± 20.09) compared to night (32.09 ± 21.74; P < .01). Lastly, participants had no change in average reaction time between the two drives. CONCLUSIONS: From the analysis, we observed performance began to degrade after approximately three hours of driving. Our findings suggest that rotating drivers at least every three hours would be prudent to allow for breaks, and to minimize performance degradation, particularly during night shifts.

fatigue↗
DOE OSTI2025

Performance Portability Evaluation of Fluid-Structure Interaction Simulations on Heterogeneous Platforms

The rapid proliferation of heterogeneous programming languages and multi-vendor hardware has underscored the critical need to evaluate the performance portability of scientific applications. In this work, we present the systematic porting and optimization of a massively parallel fluid-structure interaction code across multiple heterogeneous programming frameworks for deployment on leadership-class supercomputers from major vendors. Our analysis focuses on at-scale performance for simulations involving hundreds of millions of deformable cells, executed on a combination of CPUs and GPUs spanning thousands of nodes on exascale machines. We benchmark the performance of each implementation, highlighting the trade-offs inherent in adopting diverse programming models. Key insights regarding the portability of CUDA on multi-vendor platforms, the superior multi-core CPU performance from SYCL, and architectural considerations on performance optimization are distilled from our experience, offering guidance to other users of high performance computing based on our findings.

Martin, Aristotle [Duke University]↗
DOE OSTI2022

What you get is not always what you see—pitfalls in solar array assessment using overhead imagery

Effective integration planning for small, distributed solar photovoltaic (PV) arrays into electric power grids requires access to high quality data: the location and power capacity of individual solar PV arrays. Unfortunately, national databases of small-scale solar PV do not exist; those that do are limited in their spatial resolution, typically aggregated up to state or national levels. While several promising approaches for solar PV detection have been published, strategies for evaluating the performance of these models are often highly heterogeneous from study to study. The resulting comparison of these methods for practical applications for energy assessments becomes challenging and may imply that the reported performance evaluations overly optimistic. The heterogeneity comes in many forms, each of which we explore in this work: the degree of diversity of the locations and sensors (e.g. different satellites, aerial photography) from which the training and validation data originate, the validation of ground truth (manual annotation of imagery vs known solar PV locations), the level of spatial aggregation (e.g. array-level vs regional estimates), and inconsistencies in the training and validation datasets (e.g. different datasets are used for each study and those data are not always made accessible). For each, we discuss emerging practices from the literature to address them or suggest directions of future research. As part of our investigation, we evaluate solar PV identification performance in two large regions: the entire state of Connecticut and the city of San Diego, CA. In Connecticut, we also use 33,114 known parcel-level solar PV installations from Berkeley Lab’s Tracking the Sun dataset to evaluate parcel-level performance and evaluate capacity estimates using 169 municipalities. We also make our code (which we call SolarMapper), pre-trained models, training data, and predictions publicly available and provide a web portal for interactively inspecting each prediction that was made. Here our findings suggest that traditional performance evaluation of the automated identification of solar PV from satellite imagery may be optimistic due to common limitations in the validation process. The takeaways from this work are intended to inform and catalyze the large-scale practical application of automated solar PV assessment techniques by energy researchers and professionals.

14 SOLAR ENERGY↗
NASA NTRS2003

A Gold Standards Approach to Training Instructors to Evaluate Crew Performance

The Advanced Qualification Program requires that airlines evaluate crew performance in Line Oriented Simulation. For this evaluation to be meaningful, instructors must observe relevant crew behaviors and evaluate those behaviors consistently and accurately against standards established by the airline. The airline industry has largely settled on an approach in which instructors evaluate crew performance on a series of event sets, using standardized grade sheets on which behaviors specific to event set are listed. Typically, new instructors are given a class in which they learn to use the grade sheets and practice evaluating crew performance observed on videotapes. These classes emphasize reliability, providing detailed instruction and practice in scoring so that all instructors within a given class will give similar scores to similar performance. This approach has value but also has important limitations; (1) ratings within one class of new instructors may differ from those of other classes; (2) ratings may not be driven primarily by the specific behaviors on which the company wanted the crews to be scored; and (3) ratings may not be calibrated to company standards for level of performance skill required. In this paper we provide a method to extend the existing method of training instructors to address these three limitations. We call this method the "gold standards" approach because it uses ratings from the company's most experienced instructors as the basis for training rater accuracy. This approach ties the training to the specific behaviors on which the experienced instructors based their ratings.

Baker, David P.↗
NASA NTRS1975

Performance and evaluation of the Viking lander camera performance prediction program

A computer program is described for predicting the performance of the Viking lander cameras. The predictions are primarily concerned with two objectives: (1) the picture quality of a reference test chart (of which there are three on each lander) to aid in diagnosing camera performance; and (2) the picture quality of cones with surface properties of a natural terrain to aid in predicting favorable illumination and viewing geometries and operational camera commands. Predictions made with this program are verified by experimental data obtained with a Viking-like laboratory facsimile camera.

Huck, F. O.↗
NASA NTRS2014

Cascade Distiller System Performance Testing Interim Results

The Cascade Distillation System (CDS) is a rotary distillation system with potential for greater reliability and lower energy costs than existing distillation systems. Based upon the results of the 2009 distillation comparison test (DCT) and recommendations of the expert panel, the Advanced Exploration Systems (AES) Water Recovery Project (WRP) project advanced the technology by increasing reliability of the system through redesign of bearing assemblies and improved rotor dynamics. In addition, the project improved the CDS power efficiency by optimizing the thermoelectric heat pump (TeHP) and heat exchanger design. Testing at the NASA-JSC Advanced Exploration System Water Laboratory (AES Water Lab) using a prototype Cascade Distillation Subsystem (CDS) wastewater processor (Honeywell d International, Torrance, Calif.) with test support equipment and control system developed by Johnson Space Center was performed to evaluate performance of the system with the upgrades as compared to previous system performance. The system was challenged with Solution 1 from the NASA Exploration Life Support (ELS) distillation comparison testing performed in 2009. Solution 1 consisted of a mixed stream containing human-generated urine and humidity condensate. A secondary objective of this testing is to evaluate the performance of the CDS as compared to the state of the art Distillation Assembly (DA) used in the ISS Urine Processor Assembly (UPA). This was done by challenging the system with ISS analog waste streams. This paper details the results of the AES WRP CDS performance testing.

Callahan, Michael R.↗