Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Performance Evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,315 records · Page 73

Evaluation of a Bioabsorbable Self-Expandable Vein Stent-Base Made of Poly(l-lactide) In Vitro and In Vivo

PurposeThis study was designed to evaluate performance and tissue response to a self-expandable bioabsorbable vein stent-base cut from a tube with enhanced stiffness and strength in vitro and in vivo.MethodsA diamond-shaped stent-base was cut from a sequential biaxially strained poly(l-lactide) (PLLA) tube for optimized performance. The performance of the stent-base was evaluated in a finite element analysis model, and validation was attempted in vitro through a cyclic flat-plate compression and radial force measurement. The performance of the stent-base was tested in vivo using 3 sheep with 2 implants each for 2 and 3½ weeks, respectively.ResultsIn vitro the stent-base showed an elliptical deformation but no fractures. In vivo the stent-base showed adequate radial force and no migration. All implanted stent-bases showed multiple fractures not only at the predicted stress zones but at all connecting points. Fragments of the caudal stent-base stayed in the vein wall indicating sufficient tissue coverage to avoid embolization of the fractured stent pieces, whereas fragments from the cranial device remaining were few. Neointima formation was confirmed histologically at 2 and 3½ weeks.ConclusionA bioabsorbable self-expandable stent-base made from PLLA for large veins seems feasible, but over time, the PLLA used in this study appears too stiff and lacks the sufficient flexibility to move with the vena cava, causing multiple fractures.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Evaluation of Optimal Net Load Management in Microgrids Using Hardware-in-the-Loop Simulation

This paper presents the performance evaluation of a net load management (NLM) engine that balances load and generation in an isolated community to power a critical facility after a grid interruption event (e.g., the loss of a large generation unit). This NLM engine is particularly important for microgrid systems because it provides a high-speed, cost-optimal control solution to coordinate grid-forming inverters and to dispatch grid-following inverters and deferrable loads in microgrid systems to enhance grid resilience and reliability. The NLM algorithm cost-optimally dispatches the grid-following inverters and deferrable loads based on the demanded power and load priorities, and the grid-forming inverters use droop control to form system voltages and share active and reactive power. A controller-hardware-in-the-loop platform is developed to evaluate the control performance of the NLM algorithm with two sequential contingency events of lost generation units. The experimental results indicate that the NLM engine can maintain system stability, achieve the targeted system voltage and frequency, and balance load and generation to serve the critical facility with improved system resilience and reliability.

grid-following inverter↗

Evaluation of Optimal Net Load Management in Microgrids Using Hardware-in-the-Loop Simulation

This presentation discusses the performance evaluation of a net load management (NLM) engine that balances load and generation in an isolated community to power a critical facility after a grid interruption event (e.g., the loss of a large generation unit). This NLM engine is particularly important for microgrid systems because it provides a high-speed, cost-optimal control solution to coordinate grid-forming inverters and to dispatch grid-following inverters and deferrable loads in microgrid systems to enhance grid resilience and reliability. The NLM algorithm cost-optimally dispatches the grid-following inverters and deferrable loads based on the demanded power and load priorities, and the grid-forming inverters use droop control to form system voltages and share active and reactive power. A controller-hardware-in-the-loop platform is developed to evaluate the control performance of the NLM algorithm with two sequential contingency events of lost generation units. The experimental results indicate that the NLM engine can maintain system stability, achieve the targeted system voltage and frequency, and balance load and generation to serve the critical facility with improved system resilience and reliability.

droop control↗

Accelerating Random Forest Classification on GPU and FPGA

Random Forests (RFs) are a commonly used machine learning method for classification and regression tasks spanning a variety of application domains, including bioinformatics, business analytics, and software optimization. While prior work has focused primarily on improving performance of the training of RFs, many applications, such as malware identification, cancer prediction, and banking fraud detection, require fast RF classification. In this work, we accelerate RF classification on GPU and FPGA. In order to provide efficient support for large datasets, we propose a hierarchical memory layout suitable to the GPU/FPGA memory hierarchy. We design three RF classification code variants based on that layout, and we investigate GPU- and FPGA-specific considerations for these kernels. Our experimental evaluation, performed on an Nvidia Xp GPU and on a Xilinx Alveo U250 FPGA accelerator card using publicly available datasets on the scale of millions of samples and tens of features, covers various aspects. First, we evaluate the performance benefits of our hierarchical data structure over the standard compressed sparse row (CSR) format. Second, we compare our GPU implementation with cuML, a machine learning library targeting Nvidia GPUs. Third, we explore the performance/accuracy tradeoff resulting from the use of different tree depths in the RF. Finally, we perform a comparative performance analysis of our GPU and FPGA implementations. Our evaluation shows that for high accuracy targets, our GPU implementation yields 5-9x speedup over CSR, and up to a 2x speedup over cuML.

FPGA, Xilinx FPGA, GPU, Random Forest classificati↗

Evaluating Multi-Input/Multi-Output Digital Control Systems

Controller-performance-evaluation (CPE) methodology for multi-input/multi-output (MIMO) digital control systems developed. Procedures identify potentially destabilizing controllers and confirm satisfactory performance of stabilizing ones. Methodology generic and used in many types of multi-loop digital-controller applications, including digital flight-control systems, digitally controlled spacecraft structures, and actively controlled wind-tunnel models. Also applicable to other complex, highly dynamic digital controllers, such as those in high-performance robot systems.

Pototzky, Anthony S.↗

Evaluation of Optimal Net Load Management in Microgrids Using Hardware-in-the-Loop Simulation: Preprint

This paper presents the performance evaluation of a net load management (NLM) engine that balances load and generation in an isolated community to power a critical facility after a grid interruption event (e.g., lost a large generation unit). The NLM engine is a central dispatch control system that provides high-speed cost-optimal coordination of a set of net loads (combination of generation and deferrable loads) connected to the same electrical network. The NLM algorithm cost-optimally dispatches the grid-following inverters and deferrable loads based on the demanded power and load priorities, and the grid-forming inverters use droop control to form system voltages and share active and reactive power. A controller-hardware-in-the-loop platform is developed to evaluate the control performance of the NLM algorithm. The experimental results indicate that the NLM engine can achieve the targeted system voltage and frequency and balance load and generation to serve the critical facility.

droop control↗

High-Performance Computing Optimization for Aladyn – Adaptive Neural Network Molecular Dynamics Mini-Application

This report provides a description and performance evaluation of the optimization techniques for high performance computing (HPC) implementation of the open source Computational Materials mini-application Aladyn (https://github.com/nasa/aladyn). Aladyn is a basic molecular dynamics code written in FORTRAN 2003, which is designed to demonstrate the use of adaptive neural networks (ANNs) in atomistic simulations. The role of ANNs is to efficiently reproduce the very complex energy landscape resulting from the atomic interactions in materials with the accuracy of the more expensive quantum mechanics-based calculations. The ANN is trained on a large set of atomic structures calculated using the density functional theory (DFT) method. While achieving orders of magnitude faster computational performance than DFT, the ANN-based approach was still very computationally demanding compared to the conventional approach of using empirically fitted energy functions. After its initial development, Aladyn was evaluated and optimized by experts at the NASA Advanced Supercomputing (NAS) division to exploit modern supercomputer architectures. The code has been optimized for execution on multicore central processing units (CPUs), including Intel® Skylake microarchitecture, and on graphic accelerators, such as Nvidia® V100 graphic processing units (GPUs), using Open Multi-Processing (OpenMP) and Open Accelerators (OpenACC) programming interfaces. The optimization achieved a speedup of 4.7 times the baseline version on CPU performance and an additional 2.4 times on CPU+GPU performance. Atomistic computer simulations are a fundamental tool in materials research to model material properties form physics-based first principles. Atomic interaction, governed by Quantum Mechanics (QM) require sophisticated and highly computationally demanding mathematical models to calculate [1]. Classical methods use approximate functional forms, empirically fitted through a set of variable parameters to emulate atomic energies as direct functions of atomic coordinates [2]. While empirical potentials are computationally much simpler, allowing simulations of large-scale systems of up to a trillion (1012) atoms [3], they are substantially less accurate compared to quantum calculations and applicable only to very specific atomic configurations or predefined crystallographic phases. A recently suggested approach is to use heuristic machine learning methods [4], such as those based on Adaptive Neural Networks (ANNs) to predict atomic energies, after being trained on a sufficiently large database of QM-calculated structures [5,6]. This approach reduces significantly the computational complexity, allowing for simulations of orders of magnitude larger systems compared to QM-based methods without compromising accuracy. Still, compared to classical methods using empirical energy functions, ANN methods remain two- to three orders of magnitude more computationally demanding. Hence, the computational cost of simulations, together with the need for extensive training of ANNs, still makes the practical implementation of ANN-based methods quite challenging. The purpose of the Aladyn mini-application software [7], available as open source at https://github.com/nasa/aladyn, is to be a testbed for exploring possible optimization strategies to develop highly scalable parallel algorithms for ANN-based atomistic simulations. Aladyn is aimed at utilizing the architecture of the high-end modern highperformance computing (HPC) hardware based on multicore central processing units (CPUs) equipped with graphic processing unit (GPU) accelerators. Specifically, the goal is to optimize the performance on a single HPC compute node, before implementing scaling to multi-node parallelization using message passing interface (MPI). At the same time, the open source code of Aladyn can serve as a training model for students and professors in academia.

Yamakov, Vesselin I.↗

Evaluating iterative reconstruction performance in computed tomography

Purpose: Iterative reconstruction (IR) offers notable advantages in computed tomography (CT). However, its performance characterization is complicated by its potentially nonlinear behavior, impacting performance in terms of specific tasks. This study aimed to evaluate the performance of IR with both task-specific and task-generic strategies. Methods: The performance of IR in CT was mathematically assessed with an observer model that predicted the detection accuracy in terms of the detectability index (d′). d′ was calculated based on the properties of the image noise and resolution, the observer, and the detection task. The characterizations of image noise and resolution were extended to accommodate the nonlinearity of IR. A library of tasks was mathematically modeled at a range of sizes (radius 1–4 mm), contrast levels (10–100 HU), and edge profiles (sharp and soft). Unique d′ values were calculated for each task with respect to five radiation exposure levels (volume CT dose index, CTDI{sub vol}: 3.4–64.8 mGy) and four reconstruction algorithms (filtered backprojection reconstruction, FBP; iterative reconstruction in imaging space, IRIS; and sinogram affirmed iterative reconstruction with strengths of 3 and 5, SAFIRE3 and SAFIRE5; all provided by Siemens Healthcare, Forchheim, Germany). The d′ values were translated into the areas under the receiver operating characteristic curve (AUC) to represent human observer performance. For each task and reconstruction algorithm, a threshold dose was derived as the minimum dose required to achieve a threshold AUC of 0.9. A task-specific dose reduction potential of IR was calculated as the difference between the threshold doses for IR and FBP. A task-generic comparison was further made between IR and FBP in terms of the percent of all tasks yielding an AUC higher than the threshold. Results: IR required less dose than FBP to achieve the threshold AUC. In general, SAFIRE5 showed the most significant dose reduction potentials (11–54 mGy, 77%–84%), followed by SAFIRE3 (7–36 mGy, 50%–61%) and IRIS (6–26 mGy, 37%–50%). The dose reduction potentials highly depended on task size and task contrast, with tasks of lower contrasts and smaller sizes, i.e., more challenging tasks, indicating higher dose reductions. Softer edge profile showed higher dose reduction potentials with SAFIRE3 and SAFIRE5, but not with IRIS. The task-generic comparison between IR and FBP demonstrated the overall superiority of IR performance, as IR allowed a larger percent of tasks to exceed the threshold AUC: IRIS, 8%–12%; SAFIRE3, 10%–16%; and SAFIRE5, 20%–33%. The improvement with IR was generally more pronounced at lower dose levels. Conclusions: Expanding beyond traditional contrast and noise based assessments of IR, we performed both task-specific and task-generic evaluations of IR performance. The task-specific evaluation demonstrated the dependency of IR’s dose reduction potential on task attributes, which can be employed to optimize IR for clinical indications with specific range of size and contrast. The task-generic evaluation demonstrated IR’s overall superiority over FBP in terms of the range of tasks exceeding a threshold performance level, which can be employed for general comparisons between algorithms.

60 APPLIED LIFE SCIENCES↗

Geant4 Computing Performance Benchmarking and Monitoring

Performance evaluation and analysis of large scale computing applications is essential for optimal use of resources. As detector simulation is one of the most compute intensive tasks and Geant4 is the simulation toolkit most widely used in contemporary high energy physics (HEP) experiments, it is important to monitor Geant4 through its development cycle for changes in computing performance and to identify problems and opportunities for code improvements. All Geant4 development and public releases are being profiled with a set of applications that utilize different input event samples, physics parameters, and detector configurations. Results from multiple benchmarking runs are compared to previous public and development reference releases to monitor CPU and memory usage. Observed changes are evaluated and correlated with code modifications. Besides the full summary of call stack and memory footprint, a detailed call graph analysis is available to Geant4 developers for further analysis. The set of software tools used in the performance evaluation procedure, both in sequential and multi-threaded modes, include FAST, IgProf and Open|Speedshop. In conclusion, the scalability of the CPU time and memory performance in multi-threaded application is evaluated by measuring event throughput and memory gain as a function of the number of threads for selected event samples.

97 MATHEMATICS AND COMPUTING↗

Temporal Cyber Attack Detection.

Rigorous characterization of the performance and generalization ability of cyber defense systems is extremely difficult, making it hard to gauge uncertainty, and thus, confidence. This difficulty largely stems from a lack of labeled attack data that fully explores the potential adversarial space. Currently, performance of cyber defense systems is typically evaluated in a qualitative manner by manually inspecting the results of the system on live data and adjusting as needed. Additionally, machine learning has shown promise in deriving models that automatically learn indicators of compromise that are more robust than analyst-derived detectors. However, to generate these models, most algorithms require large amounts of labeled data (i.e., examples of attacks). Algorithms that do not require annotated data to derive models are similarly at a disadvantage, because labeled data is still necessary when evaluating performance. In this work, we explore the use of temporal generative models to learn cyber attack graph representations and automatically generate data for experimentation and evaluation. Training and evaluating cyber systems and machine learning models requires significant, annotated data, which is typically collected and labeled by hand for one-off experiments. Automatically generating such data helps derive/evaluate detection models and ensures reproducibility of results. Experimentally, we demonstrate the efficacy of generative sequence analysis techniques on learning the structure of attack graphs, based on a realistic example. These derived models can then be used to generate more data. Additionally, we provide a roadmap for future research efforts in this area.

97 MATHEMATICS AND COMPUTING↗

Interim Project Results: United Parcel Service's Second-Generation Hybrid-Electric Delivery Vans (Fact Sheet)

This fact sheet describes the performance evaluation of United Parcel Service's second-generation hybrid-electric delivery vans. The Fleet Test and Evaluation Team at the National Renewable Energy Laboratory (NREL) is evaluating the 18-month, in-service performance of 11 of these vans along with 11 comparable conventional diesel vans operating in Minneapolis, Minnesota. As a complement to thefield study, the team recently completed fuel economy and emissions testing at NREL's Renewable Fuels and Lubricants (ReFUEL) laboratory.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

JANNAF boundary layer integral matrix procedure

Accurate predictions of the thrust loss due to boundary layer effects and of the wall heat flux are very important to the design and performance evaluation of rocket nozzles. A method used in analytical procedures for liquid fuel rocket engine performance prediction and evaluation is presented. A computer program is examined that is a fast and accurate procedure for solving the set of boundary layer equation (momentum, energy, and species) for laminar or turbulent, chemically reacting flows with a wide variety of boundary conditions. Results of comparison of the various turbulent models are presented. A summary of the modifications and additions to the program is examined.

Evans, R. M.↗

Strapdown system performance optimization test evaluations (SPOT), volume 1

A three axis inertial system was packaged in an Apollo gimbal fixture for fine grain evaluation of strapdown system performance in dynamic environments. These evaluations have provided information to assess the effectiveness of real-time compensation techniques and to study system performance tradeoffs to factors such as quantization and iteration rate. The strapdown performance and tradeoff studies conducted include: (1) Compensation models and techniques for the inertial instrument first-order error terms were developed and compensation effectivity was demonstrated in four basic environments; single and multi-axis slew, and single and multi-axis oscillatory. (2) The theoretical coning bandwidth for the first-order quaternion algorithm expansion was verified. (3) Gyro loop quantization was identified to affect proportionally the system attitude uncertainty. (4) Land navigation evaluations identified the requirement for accurate initialization alignment in order to pursue fine grain navigation evaluations.

Blaha, R. J.↗

Strapdown system Performance Optimization Test evaluations (SPOT), volume 2

A three axis inertial system is packaged in an Apollo gimbal fixture for fine grain evaluation of strapdown system performance in dynamic environments. These evaluations have provided information to assess the effectiveness of real-time compensation techniques and to study system performance tradeoffs to factors such as quantization iteration rate. The strapdown performance and tradeoff studies conducted in this program are discussed.

Blaha, R. J.↗

Integrated regenerative fuel cell experimental evaluation

An experimental test program was conducted to investigate the performance characteristics of an integrated regenerative fuel cell (IRFC) concept. The IRFC consists of a separate fuel cell unit and electrolysis cell unit in the same structure, with internal storage of fuel cell product water and external storage of electrolysis cell produced hydrogen and oxygen. The fuel cell unit incorporates an enhanced Orbiter-type cell capable of improved performance at reduced weight. The electrolysis cell features a NiCo2O4 catalyst oxygen evolution eletrode with a porous Teflon cover to retard electrolyte loss. Six complete IRFC assemblies were assembled and performance tested at an operating temperature of 200 F (93.3 C) and reactant pressures up to 170 psia (117.2 n/cu cm) on IRFC No. 4. Anomalous pressure charge/discharge characteristics were encountered during performance evaluation. A reversible fuel cell incorporating a proprietary bi-functional oxygen electrode operated satisfactory at 200 F (93.3 C) at reactant pressures up to 50 psia (41.4 n/cu cm) as a regenerative fuel cell for one cycle, before developing an electrical short in the fuel cell mode. Electrolysis cell 300-hour endurance tests demonstrated the electrolyte retention capability of the electrode Teflon cover and the performance stability of the bi-functional oxygen electrode at high potential.

Martin, Ronald E.↗

Evaluation of a performance appraisal framework for radiation therapists in planning and simulation

Constantly evolving technology and techniques within radiation therapy require practitioners to maintain a continuous approach to professional development and training. Systems of performance appraisal and adoption of regular feedback mechanisms are vital to support this development yet frequently lack structure and rely on informal peer support. A Radiation Therapy Performance Appraisal Framework (RT-PAF) for radiation therapists in planning and simulation was developed to define expectations of practice and promote a supportive and objective culture of performance and skills appraisal. Evaluation of the framework was conducted via an anonymous online survey tool. Nine peer reviewers and fourteen recipients provided feedback on its effectiveness and the challenges and limitations of the approach. Findings from the evaluation were positive and suggested that both groups gained benefit from and expressed a strong interest in embedding the approach more routinely. Respondents identified common challenges related to the limited ability to implement suggested development strategies; this was strongly associated with time and rostering issues. This framework successfully defined expectations for practice and provided a fair and objective feedback process that focussed on skills development. It empowered staff to maintain their skills and reach their professional potential. Management support, particularly in regard to provision of protected time was highlighted as critical to the framework's ongoing success. The demonstrated benefits arising in terms of staff satisfaction and development highlight the importance of this commitment to the modern radiation therapy workforce.

07 ISOTOPE AND RADIATION SOURCES↗

Enhanced Component Performance Study: Air-Operated Valves 1998–2013

This report presents a performance evaluation of air-operated valves (AOVs) at U.S. commercial nuclear power plants. The data used in this study are based on the operating experience failure reports from fiscal year 1998 through 2013 for the component reliability as reported in the Institute of Nuclear Power Operations (INPO) Consolidated Events Database (ICES). The AOV failure modes considered are failure-to-open/close, failure to operate or control, and spurious operation. The component reliability estimates and the reliability data are trended for the most recent 10-year period, while yearly estimates for reliability are provided for the entire active period. No statistically significant increasing or decreasing trends were identified in the AOV failure data.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗