SEARCH · Engineering Papers

Results for “Benchmarks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

All sources NASA NTRS DOE OSTI

NASA NTRS2003

NAS Parallel Benchmarks, Multi-Zone Versions

We describe an extension of the NAS Parallel Benchmarks (NPB) suite that involves solving the application benchmarks LU, BT and SP on collections of loosely coupled discretization meshes. The solutions on the meshes are updated independently, but after each time step they exchange boundary value information. This strategy, which is common among structured-mesh production flow solver codes in use at NASA Ames and elsewhere, provides relatively easily exploitable coarse-grain parallelism between meshes. Since the individual application benchmarks also allow fine-grain parallelism themselves, this NPB extension, named NPB Multi-Zone (NPB-MZ), is a good candidate for testing hybrid and multi-level parallelization tools and strategies.

vanderWijngaart, Rob F.↗

NASA NTRS2004

Unstructured Adaptive (UA) NAS Parallel Benchmark

We present a complete specification of a new benchmark for measuring the performance of modern computer systems when solving scientific problems featuring irregular, dynamic memory accesses. It complements the existing NAS Parallel Benchmark suite. The benchmark involves the solution of a stylized heat transfer problem in a cubic domain, discretized on an adaptively refined, unstructured mesh.

Feng, Huiyu↗

NASA NTRS2005

Middleware Evaluation and Benchmarking for Use in Mission Operations Centers

Middleware technologies have been promoted as timesaving, cost-cutting alternatives to the point-to-point communication used in traditional mission operations systems. However, missions have been slow to adopt the new technology. The lack of existing middleware-based missions has given rise to uncertainty about middleware's ability to perform in an operational setting. Most mission architects are also unfamiliar with the technology and do not know the benefits and detriments to architectural choices - or even what choices are available. We will present the findings of a study that evaluated several middleware options specifically for use in a mission operations system. We will address some common misconceptions regarding the applicability of middleware-based architectures, and we will identify the design decisions and tradeoffs that must be made when choosing a middleware solution. The Middleware Comparison and Benchmark Study was conducted at NASA Goddard Space Flight Center to comprehensively evaluate candidate middleware products, compare and contrast the performance of middleware solutions with the traditional point- to-point socket approach, and assess data delivery and reliability strategies. The study focused on requirements of the Global Precipitation Measurement (GPM) mission, validating the potential use of middleware in the GPM mission ground system. The study was jointly funded by GPM and the Goddard Mission Services Evolution Center (GMSEC), a virtual organization for providing mission enabling solutions and promoting the use of appropriate new technologies for mission support. The study was broken into two phases. To perform the generic middleware benchmarking and performance analysis, a network was created with data producers and consumers passing data between themselves. The benchmark monitored the delay, throughput, and reliability of the data as the characteristics were changed. Measurements were taken under a variety of topologies, data demands, and data characteristics, using several middleware products. All results were compared to systems using traditional point-to-point socket communication. By comparing performance results under Merent settings, inferences could be made about each middleware's ability to meet certain requirements of the GPM mission. The second phase simulated a middleware-based mission operations center. Actual mission support tools were either used or simulated to create real world demands on the middleware. Network and computer demands were watched closely to verify that no specific idiosyncrasies of mission operations activities would prove unsupportable by the middleware. In our presentation, we will address some commonly accepted misconceptions concerning middleware in mission support architectures. Specifically, we will focus on the perception that middleware solutions are too slow or impose too much overhead for real-time mission operations, and that middleware solutions are too expensive for small

Antonucci, Rob↗

NASA NTRS2008

Benchmarking Gas Path Diagnostic Methods: A Public Approach

Recent technology reviews have identified the need for objective assessments of engine health management (EHM) technology. The need is two-fold: technology developers require relevant data and problems to design and validate new algorithms and techniques while engine system integrators and operators need practical tools to direct development and then evaluate the effectiveness of proposed solutions. This paper presents a publicly available gas path diagnostic benchmark problem that has been developed by the Propulsion and Power Systems Panel of The Technical Cooperation Program (TTCP) to help address these needs. The problem is coded in MATLAB (The MathWorks, Inc.) and coupled with a non-linear turbofan engine simulation to produce "snap-shot" measurements, with relevant noise levels, as if collected from a fleet of engines over their lifetime of use. Each engine within the fleet will experience unique operating and deterioration profiles, and may encounter randomly occurring relevant gas path faults including sensor, actuator and component faults. The challenge to the EHM community is to develop gas path diagnostic algorithms to reliably perform fault detection and isolation. An example solution to the benchmark problem is provided along with associated evaluation metrics. A plan is presented to disseminate this benchmark problem to the engine health management technical community and invite technology solutions.

Simon, Donald L.↗

NASA NTRS2008

Structural Benchmark Creep Testing for the Advanced Stirling Convertor Heater Head

The National Aeronautics and Space Administration (NASA) has identified the high efficiency Advanced Stirling Radioisotope Generator (ASRG) as a candidate power source for use on long duration Science missions such as lunar applications, Mars rovers, and deep space missions. For the inherent long life times required, a structurally significant design limit for the heater head component of the ASRG Advanced Stirling Convertor (ASC) is creep deformation induced at low stress levels and high temperatures. Demonstrating proof of adequate margins on creep deformation and rupture for the operating conditions and the MarM-247 material of construction is a challenge that the NASA Glenn Research Center is addressing. The combined analytical and experimental program ensures integrity and high reliability of the heater head for its 17-year design life. The life assessment approach starts with an extensive series of uniaxial creep tests on thin MarM-247 specimens that comprise the same chemistry, microstructure, and heat treatment processing as the heater head itself. This effort addresses a scarcity of openly available creep properties for the material as well as for the virtual absence of understanding of the effect on creep properties due to very thin walls, fine grains, low stress levels, and high-temperature fabrication steps. The approach continues with a considerable analytical effort, both deterministically to evaluate the median creep life using nonlinear finite element analysis, and probabilistically to calculate the heater head s reliability to a higher degree. Finally, the approach includes a substantial structural benchmark creep testing activity to calibrate and validate the analytical work. This last element provides high fidelity testing of prototypical heater head test articles; the testing includes the relevant material issues and the essential multiaxial stress state, and applies prototypical and accelerated temperature profiles for timely results in a highly controlled laboratory environment. This paper focuses on the last element and presents a preliminary methodology for creep rate prediction, the experimental methods, test challenges, and results from benchmark testing of a trial MarM-247 heater head test article. The results compare favorably with the analytical strain predictions. A description of other test findings is provided, and recommendations for future test procedures are suggested. The manuscript concludes with describing the potential impact of the heater head creep life assessment and benchmark testing effort on the ASC program.

Krause, David L.↗

NASA NTRS2010

Development of a Benchmark Example for Delamination Fatigue Growth Prediction

The development of a benchmark example for cyclic delamination growth prediction is presented and demonstrated for a commercial code. The example is based on a finite element model of a Double Cantilever Beam (DCB) specimen, which is independent of the analysis software used and allows the assessment of the delamination growth prediction capabilities in commercial finite element codes. First, the benchmark result was created for the specimen. Second, starting from an initially straight front, the delamination was allowed to grow under cyclic loading in a finite element model of a commercial code. The number of cycles to delamination onset and the number of cycles during stable delamination growth for each growth increment were obtained from the analysis. In general, good agreement between the results obtained from the growth analysis and the benchmark results could be achieved by selecting the appropriate input parameters. Overall, the results are encouraging but further assessment for mixed-mode delamination is required

Krueger, Ronald↗

NASA NTRS2010

Development of a Benchmark Example for Delamination Fatigue Growth Prediction

Krueger, Ronald↗

NASA NTRS2013

Development and Applications of Benchmark Examples for Static Delamination Propagation Predictions

The development and application of benchmark examples for the assessment of quasistatic delamination propagation capabilities was demonstrated for ANSYS (TradeMark) and Abaqus/Standard (TradeMark). The examples selected were based on finite element models of Double Cantilever Beam (DCB) and Mixed-Mode Bending (MMB) specimens. First, quasi-static benchmark results were created based on an approach developed previously. Second, the delamination was allowed to propagate under quasi-static loading from its initial location using the automated procedure implemented in ANSYS (TradeMark) and Abaqus/Standard (TradeMark). Input control parameters were varied to study the effect on the computed delamination propagation. Overall, the benchmarking procedure proved valuable by highlighting the issues associated with choosing the appropriate input parameters for the VCCT implementations in ANSYS® and Abaqus/Standard®. However, further assessment for mixed-mode delamination fatigue onset and growth is required. Additionally studies should include the assessment of the propagation capabilities in more complex specimens and on a structural level.

Krueger, Ronald↗

NASA NTRS2013

Aircraft Engine Gas Path Diagnostic Methods: Public Benchmarking Results

Recent technology reviews have identified the need for objective assessments of aircraft engine health management (EHM) technologies. To help address this issue, a gas path diagnostic benchmark problem has been created and made publicly available. This software tool, referred to as the Propulsion Diagnostic Method Evaluation Strategy (ProDiMES), has been constructed based on feedback provided by the aircraft EHM community. It provides a standard benchmark problem enabling users to develop, evaluate and compare diagnostic methods. This paper will present an overview of ProDiMES along with a description of four gas path diagnostic methods developed and applied to the problem. These methods, which include analytical and empirical diagnostic techniques, will be described and associated blind-test-case metric results will be presented and compared. Lessons learned along with recommendations for improving the public benchmarking processes will also be presented and discussed.

Simon, Donald L.↗

NASA NTRS2014

Benchmarking a Soil Moisture Data Assimilation System for Agricultural Drought Monitoring

Despite considerable interest in the application of land surface data assimilation systems (LDAS) for agricultural drought applications, relatively little is known about the large-scale performance of such systems and, thus, the optimal methodological approach for implementing them. To address this need, this paper evaluates an LDAS for agricultural drought monitoring by benchmarking individual components of the system (i.e., a satellite soil moisture retrieval algorithm, a soil water balance model and a sequential data assimilation filter) against a series of linear models which perform the same function (i.e., have the same basic inputoutput structure) as the full system component. Benchmarking is based on the calculation of the lagged rank cross-correlation between the normalized difference vegetation index (NDVI) and soil moisture estimates acquired for various components of the system. Lagged soil moistureNDVI correlations obtained using individual LDAS components versus their linear analogs reveal the degree to which non-linearities andor complexities contained within each component actually contribute to the performance of the LDAS system as a whole. Here, a particular system based on surface soil moisture retrievals from the Land Parameter Retrieval Model (LPRM), a two-layer Palmer soil water balance model and an Ensemble Kalman filter (EnKF) is benchmarked. Results suggest significant room for improvement in each component of the system.

soil moisture↗

NASA NTRS2017

Global Gridded Crop Model Evaluation: Benchmarking, Skills, Deficiencies and Implications.

Crop models are increasingly used to simulate crop yields at the global scale, but so far there is no general framework on how to assess model performance. Here we evaluate the simulation results of 14 global gridded crop modeling groups that have contributed historic crop yield simulations for maize, wheat, rice and soybean to the Global Gridded Crop Model Intercomparison (GGCMI) of the Agricultural Model Intercomparison and Improvement Project (AgMIP). Simulation results are compared to reference data at global, national and grid cell scales and we evaluate model performance with respect to time series correlation, spatial correlation and mean bias. We find that global gridded crop models (GGCMs) show mixed skill in reproducing time series correlations or spatial patterns at the different spatial scales. Generally, maize, wheat and soybean simulations of many GGCMs are capable of reproducing larger parts of observed temporal variability (time series correlation coefficients (r) of up to 0.888 for maize, 0.673 for wheat and 0.643 for soybean at the global scale) but rice yield variability cannot be well reproduced by most models. Yield variability can be well reproduced for most major producing countries by many GGCMs and for all countries by at least some. A comparison with gridded yield data and a statistical analysis of the effects of weather variability on yield variability shows that the ensemble of GGCMs can explain more of the yield variability than an ensemble of regression models for maize and soybean, but not for wheat and rice. We identify future research needs in global gridded crop modeling and for all individual crop modeling groups. In the absence of a purely observation-based benchmark for model evaluation, we propose that the best performing crop model per crop and region establishes the benchmark for all others, and modelers are encouraged to investigate how crop model performance can be increased. We make our evaluation system accessible to all crop modelers so that other modeling groups can also test their model performance against the reference data and the GGCMI benchmark.

wheat↗

NASA NTRS2017

First benchmark of the Unstructured Grid Adaptation Working Group

Unstructured grid adaptation is a technology that holds the potential to improve the automation and accuracy of computational fluid dynamics and other computational disciplines. Difficulty producing the highly anisotropic elements necessary for simulation on complex curved geometries that satisfies a resolution request has limited this technology's widespread adoption. The Unstructured Grid Adaptation Working Group is an open gathering of researchers working on adapting simplicial meshes to conform to a metric field. Current members span a wide range of institutions including academia, industry, and national laboratories. The purpose of this group is to create a common basis for understanding and improving mesh adaptation. We present our first major contribution: a common set of benchmark cases, including input meshes and analytic metric specifications, that are publicly available to be used for evaluating any mesh adaptation code. We also present the results of several existing codes on these benchmark cases, to illustrate their utility in identifying key challenges common to all codes and important differences between available codes. Future directions are defined to expand this benchmark to mature the technology necessary to impact practical simulation workflows.

Ibanez, Daniel↗

NASA NTRS2021

Comparing Commercial and Research Computational Fluid Dynamic Codes Using High-Order Workshop Benchmark Problems

The commercial computational fluid dynamic (CFD) code ANSYS Fluent and multiple research CFD codes (ez4d with uses the conservation element and solution element (CESE) method and codes that use the flux reconstruction (FR) method) were tested using three different benchmark problems from the International Workshop for High-Order CFD Methods. The benchmark problems included the transonic Ringleb flow, vortex transport by uniform flow, and laminar boundary layer on a flat plate. Simulation results from all three benchmark problems showed that the Fluent solutions had less error than the ez4d solutions for a given degree of freedom. As expected, both the Fluent and ez4d solutions had larger errors for a given degree of freedom than the simulations that used the FR method because both Fluent and ez4d utilized a second-order scheme whereas the FR codes utilized a fourth-order scheme.

CFD↗

NASA NTRSDate not supplied

Benchmark Models for Classification of Radiation Type Induced in Immune Cells

NASA Biological and Physical Sciences and the Science Mission Directorate have published a benchmark dataset of mouse immune cells subjected to radiation-induced DNA damage. The dataset comprises ML-ready microscopic imagery of said cells, including labels indicating radiation type and dose. The machine learning team at NASA Interagency Implementation and Advanced Concept Team (IMPACT) created multiple benchmark models. Initially, we conducted a preliminary analysis using thresholding. The algorithm used thresholds on average brightness of the available images to classify them into their respective radiation type. We also tested machine learning approaches. Convolutional Neural Networks (CNN) emerged as the best-performing model. This poster presents the benchmark scores obtained by the models.

Vishal Perekadan↗

NASA NTRS2021

Benchmarking Planning Applications on the Qualcomm Snapdragon

We benchmark several space planning/scheduling applications on the Qualcomm Snapdragon 855 Handheld Development Kit (HDK), a high performance embedded processor used in many mobile phones.We are flying 2 Snapdragon HDKs onboard the International Space Station (ISS) where they are hosted by the the Spaceborne Computer-2 by Hewlett Packard Enterprise linked by USB and 12V power delivery. We run computational benchmarks using three planner/ schedulers that are used for several space missions: Multi- Mission Executive (MEXEC), Compressed Large-scale Activity Scheduling and Planning (CLASP), and M2020 Ground Scheduler (Surrogate). We compare the Snapdragon performance to a performance baseline on Linux workstations. In addition, we are currently working on benchmarking the same applications on other space flight processors, such as the LEON4 Processor on the Sabertooth card, the LEON3 Processor on the Sphinx card, and the RAD750 processor.

Chien, Steve↗

NASA NTRSDate not supplied

Investigations of an Aeroelastic Optimization Benchmark Problem with MPhys and FUN3D

High-fidelity aeroelastic optimization capabilities are becoming more common in recent years. In this field, researchers typically present results on the design problems of greatest interest to them. The lack of consistency with respect to baseline configurations, design variables, objectives and constraints, etc., makes it difficult to compare approaches and creates an obstacle for the community to work together to advance the state of the art. The High-Fidelity Aeroelastic Optimization Benchmark Working Group has been established to create and study a series of benchmark aeroelastic optimization problems that the community can utilize to compare methods on the same optimization problem. This paper presents optimization results from NASA Langley Research Center for the first benchmark case of this series with analysis based on MPhys, a multiphysics library for OpenMDAO, and FUN3D, an unstructured computational fluid dynamics suite of tools. For the optimization case presented, it is demonstrated that a progression of increasing aerodynamic model fidelity reduces the wall time required to complete the optimization.

Aeroelasticity↗

NASA NTRSDate not supplied

Investigations of an Aeroelastic Optimization Benchmark Problem with MPhys and FUN3D

CFD↗

NASA NTRS2026

Benchmarking Bayesian Optimization Frameworks and Acquisition Strategies for Materials Discovery and Autonomous Laboratories

Bayesian optimization (BO) can accelerate materials discovery by guiding expensive experiments toward the most promising processing conditions. We systematically compare five BO surrogate and framework combinations (Gaussian processes in Ax, Gaussian processes and Monte-Carlo neural networks in BayBE, random forests in Lolopy, and tree-structured Parzen (TPE) estimators in Hyperopt) on three benchmarks that mimic common materials design tasks (a discrete solid-electrolyte composition space, a hybrid discrete/continuous laminate-composite design problem solved with micromechanics modeling, and the continuous Ishigami analytic function which is a standard optimization benchmark). Each BO surrogate is paired with posterior mean, probability of improvement, and expected improvement acquisition functions and run for 100 trials from randomized initial samples with uniform random search providing a control. Across five random seeds per setting, BayBE’s Gaussian-process surrogate with expected improvement consistently reached ≥95 % of the known optimum in the fewest evaluations, while Lolopy’s random forest matched or exceeded GP performance on purely categorical or mixed spaces at a higher computational cost. Posterior mean alone often stagnated at local optima, underscoring the need for exploration, whereas probability and expected improvement balanced exploration and exploitation leading to better optimization in fewer trials. Execution times ranged from milliseconds for TPE to minutes for neural-network and random-forest surrogates. These results establish baseline expectations for BO in automated materials laboratories and highlight expected improvement with Gaussian processes as a reliable first choice, with random forests offering a strong alternative when categorical variables dominate. The benchmark suite and code are released to facilitate future surrogate, acquisition, and constraint-handling research in data-driven materials optimization.

Bayesian optimization↗