Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “BENCHMARKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

Simple Benchmark Specifications for Space Radiation Protection

This report defines space radiation benchmark specifications. This specification starts with simple, monoenergetic, mono-directional particles on slabs and progresses to human models in spacecraft. This report specifies the models and sources needed to what the team performing the benchmark needs to produce in a report. Also included are brief descriptions of how OLTARIS, the NASA Langley website for space radiation analysis, performs its analysis.

Singleterry, Robert C. Jr.↗

EVA Health and Human Performance Benchmarking Study

Multiple HRP Risks and Gaps require detailed characterization of human health and performance during exploration extravehicular activity (EVA) tasks; however, a rigorous and comprehensive methodology for characterizing and comparing the health and human performance implications of current and future EVA spacesuit designs does not exist. This study will identify and implement functional tasks and metrics, both objective and subjective, that are relevant to health and human performance, such as metabolic expenditure, suit fit, discomfort, suited postural stability, cognitive performance, and potentially biochemical responses for humans working inside different EVA suits doing functional tasks under the appropriate simulated reduced gravity environments. This study will provide health and human performance benchmark data for humans working in current EVA suits (EMU, Mark III, and Z2) as well as shirtsleeves using a standard set of tasks and metrics with quantified reliability. Results and methodologies developed during this test will provide benchmark data against which future EVA suits, and different suit configurations (eg, varied pressure, mass, CG) may be reliably compared in subsequent tests. Results will also inform fitness for duty standards as well as design requirements and operations concepts for future EVA suits and other exploration systems.

Abercromby, A. F.↗

Reference Solutions for Benchmark Turbulent Flows in Three Dimensions

A grid convergence study is performed to establish benchmark solutions for turbulent flows in three dimensions (3D) in support of turbulence-model verification campaign at the Turbulence Modeling Resource (TMR) website. The three benchmark cases are subsonic flows around a 3D bump and a hemisphere-cylinder configuration and a supersonic internal flow through a square duct. Reference solutions are computed for Reynolds Averaged Navier Stokes equations with the Spalart-Allmaras turbulence model using a linear eddy-viscosity model for the external flows and a nonlinear eddy-viscosity model based on a quadratic constitutive relation for the internal flow. The study involves three widely-used practical computational fluid dynamics codes developed and supported at NASA Langley Research Center: FUN3D, USM3D, and CFL3D. Reference steady-state solutions computed with these three codes on families of consistently refined grids are presented. Grid-to-grid and code-to-code variations are described in detail.

Diskin, Boris↗

Aerodynamic Shape Optimization Benchmarks with Error Control and Automatic Parameterization

Results are presented for four optimization benchmark problems posed by the AIAA Aerodynamic Design Optimization Discussion Group. The benchmarks are intended to exercise optimization frameworks on representative airfoil and wing design problems. All problems involve drag minimization subject to geometric and aerodynamic constraints. Our design approach involves two forms of adaptation. First, the shape parameterization is gradually and automatically enriched from an initially coarse search space. Second, adjoint solutions are used to drive adaptive mesh refinement to control discretization error. The error threshold is tailored so that the nest meshes, with the greatest accuracy, are used only when nearing the optimum. On the inviscid airfoil design problem, while reducing the drag by a factor of 10, we show how the combination of progressive parameterization and tiered discretization error control can dramatically accelerate the optimization. On the viscous airfoil design problem, we use inviscid analysis-driven optimization to reduce the total drag by a factor of two. Next, we improve the span efficiency factor of a wing by performing twist optimization. Finally, we optimize the Common Research Model wing, managing to hold drag roughly fixed, while targeting an initially-violated pitching moment constraint. Our approach aims to introduce greater complexity and accuracy only when necessary to improve the design, and also support a greater degree of automation.

Anderson, George R.↗

Benchmarking Simulations of the Compton Spectrometer and Imager with Calibrations

The Compton Spectrometer and Imager (COSI) is a balloon-borne 𝛾-ray (0.2-5 MeV) telescope designed to study astrophysical sources. COSI employs a compact Compton telescope design utilizing 12 high-purity germanium double-sided strip detectors and is inherently sensitive to polarization. In 2016, COSI was launched from Wanaka, New Zealand and completed a successful 46-day flight on NASA’s new Super Pressure Balloon. In order to perform imaging, spectral, and polarization analysis of the sources observed during the 2016 flight, we compute the detector response from well-benchmarked simulations. As required for accurate simulations of the instrument, we have built a comprehensive mass model of the instrument and developed a detailed detector effects engine which applies the intrinsic detector performance to Monte Carlo simulations. The simulated detector effects include energy, position, and timing resolution, thresholds, dead strips, charge sharing, charge loss, crosstalk, dead time, and detector trigger conditions. After including these effects, the simulations closely resemble the measurements, the standard analysis pipeline used for measurements can also be applied to the simulations, and the responses computed from the simulations are accurate. We have computed the systematic error that we must apply to measured fluxes at certain energies, which is 6.3% on a rage. Here we describe the detector effects engine and the benchmarking tests performed with calibrations.

Sleator, Clio C.↗

Julia Programming Language Benchmark Using a Flight Simulation

Julia’s goal to provide scripting language ease-of-coding with compiled language speed is explored. The runtime speed of the relatively new Julia programming language is assessed against other commonly used languages including Python, Java, and C++. An industry-standard missile and rocket simulation, coded in multiple languages, was used as a test bench for runtime speed. All language versions of the simulation, including Julia, were coded to a highly-developed object-oriented simulation architecture tailored specifically for time-domain flight simulation. A “speed-of-coding” second-dimension is plotted against runtime for each language to portray a space that characterizes Julia’s scripting language efficiencies in the context of the other languages. With caveats, Julia runtime speed was found to be in the class of compiled or semi-compiled languages. However, some factors that affect runtime speed at the cost of ease-of-coding are shown. Julia’s built-in functionality for multi-core processing is briefly examined as a means for obtaining even faster runtime speed. The major contribution of this research to the extensive language benchmarking body-of-work is comparing Julia to other mainstream languages using a complex flight simulation as opposed to benchmarking with single algorithms.

Sells, Ray↗

Improved Benchmarking of Cohesive Elements in Abaqus Standard for Predicting Disbond and Delamination in Composite Structures

Traditional approaches for aircraft certification require the assumption of an initial flaw condition, either represented as barely visible impact damage (BVID) or through inclusion of a Teflon insert to serve as surrogate damage. Based on the initial composite damage state, the structure must be shown to demonstrate structural durability and damage tolerance (DaDT) according to the following criteria: a. Damage displays no detrimental growth under cyclic loading b. The structure is able to sustain design limit load (DLL) Currently, the only available manner for validating structural performance is through test. Since damage can occur over a wide variety of areas within a structure, this approach has proven to be increasingly expensive and time consuming for composite airframes and acreage structure within the design-test-certification building block. A further complicating factor is the requirement to accurately capture the most critical damage morphologies as a starting condition. To understand the severity of the damage, it is either required to experimentally determine the most critical areas at tremendous expense or rely on legacy data of similar structural testing, which limits design space expansion. A preferred solution is to use advanced analysis to provide improved understanding of load margins for critical locations based on a wide variety of potential starting damage conditions. The standard industry approach for DaDT certification adheres to the use of the traditional virtual crack closure technique (VCCT) method. VCCT is generally a preferred method because it conforms to the current certification principles of damage from a known flaw, and when used correctly, can be effective at predicting delamination propagation under static and cyclic loading. The VCCT method requires the inclusion of an initial flaw in the finite element (FE) model requiring a-priori knowledge of the flaw location. This in turn requires a plethora of analysis cases to be examined to cover a reasonable span of potential damage states. Additionally, the VCCT approach requires node-to-node connectivity rendering it incompatible with the best practices and approaches for using continuum damage mechanics (CDM) based progressive damage and failure analysis (PDFA) tools within a typical FE solver. Alternatives to VCCT have emerged in the form of cohesive elements which utilize the cohesive zone model (CZM). Unlike VCCT which models linear elastic fracture mechanics, cohesive elements couples continuum and fracture based responses through the use of bilinear traction separation laws. These laws are defined based on a penalty stiffness, a cohesive strength, and a strain energy release rate. The approach can be mesh regularized with native cohesive elements within many FE solvers such as Abaqus and LS-DYNA. In Phase I of the NASA Advanced Composites Consortium (ACC) post-buckled stiffened panel with BVID, Strength and Life [1], the performance of cohesive elements were benchmarked in comparison to VCCT and LEFM solutions and showed good agreement using Abaqus explicit [2]. To realize savings on current and future programs, it is still necessary to close technical gaps related to the use of cohesive elements with Abaqus Standard. Within a program environment, standard finite element analysis is the preferred analytical capability for quasi-static loading as it eliminates uncertainty due to oscillatory behavior commonly seen with explicit analysis. This oscillatory behavior creates difficulties in writing margins of safety based on the analysis. The use of negative tangent stiffness material models complicates convergence which typically requires the use of numerical controls such as viscous damping to overcome. To date, there has not been a comprehensive study on how to establish best practices for cohesive element convergence for predictive capability within the Abaqus implicit solver. In pursuit of these goals, under the NASA ACC program, several numerical benchmark problems were proposed including pure mode I (double cantilevered beam – DCB), pure mode II (end notch flexure – ENF), and symmetric/unsymmetric evolving mixed mode (single leg bend – SLB). This paper focuses on the use of cohesive elements to model the delamination through the use of CZM. Specifically, finite element models for the DCB, ENF, symmetric SLB, and unsymmetric SLB, are developed and various solution controls for convergence are studied to develop a best practice. Once the best practice has been developed, the predictive capability of the objective CZM model is used to analyze the hat pull-off strength of a standard hat stiffened configuration under various loading conditions.

Abaqus↗

The DejaVu Runtime Verification Benchmark

In this paper we present a benchmark for evaluating runtime verification tools. It was originally created in order to compare the DEJAVU runtime verification tool1 with another similar tool. DEJAVU’s logic is first-order past time temporal logic. In order to monitor such properties efficiently, Binary Decision Diagrams (BDDs) [1] are used for representing the data observed in a trace. The details on the logic and its algorithm are described in e.g. [2, 3, 4]. The benchmark consists of six properties, formulated in English, and formalized in DEJAVU’s logic. For each property is provided (normally) three traces, of sizes varying from 10,000 events to one million events. Traces are represented in CSV format.

Ulus, Dogan↗

Benchmarking Surface Tension Measurement Method using two Oscillation Modes in Levitated Liquid Metals

The Faraday forcing method in levitated liquid droplets has recently been introduced as a method for measuring surface tension using resonance. By subjecting an electrostatically-levitated liquid metal droplet to a continuous, oscillatory, electric field, at a frequency nearing that of the droplet’s first principal mode of oscillation (known as mode 2), the method was previously shown to determine surface tension of materials that would be particularly difficult to process by other means, e.g. liquid metals and alloys. It also offers distinct advantages in future work involving high viscosity samples because of the continuous forcing approach. This work presents 1) a benchmarking experimental method to measure surface tension by excitation of the second principal mode of oscillation (known as mode 3) in a levitated liquid droplet and 2) a more rigorous quantification of droplet excitation using a projection method. Surface tension measurements compare favorably to literature values for Zirconium, Inconel 625, and Rhodium, using both modes 2 and 3. Thus, this new method serves as a credible, self-consistent benchmarking technique for the measurement of surface tension

Levitation↗

Updated Numerical Analysis Benchmarks for Meteoroid Relevant Materials

Proposal for update of numerical analysis benchmark for meteoroid relevant materials. - Two recently performed shots are proposed to be numerical analysis benchmarks for numerical simulations of impacts of high-density meteoroids (Al 2 O 3 surrogate) and low-density meteoroids (Nylon surrogate). - A pair of general Whipple shields have been studied: - Bumper and rear walls are the same material and thickness between shields - Separation is 4.5 cm for Al 2 O 3 and 1.5 cm for Nylon - Information gathered includes high speed (1 MHz) shadowgraphs of debris cloud, bumper hole size and rear wall hole area.

Orbital Debris↗

Can We Model Forest Demography Globally? Benchmarking of State-of-the-Art Demographic DGVMs

Forests in Dynamic Global Vegetation Models (DGVMs) have historically been simulated as area-averaged plant functional types in each gridcell instead of representing communities of trees of different sizes and ages (demography). Just as the behaviour of a tree differs according to its ontogeny, so the behaviour of forests is known to differ depending on their demography. Accurately simulating demography is therefore key in order to address questions on afforestation and management strategies, as well as assessments of resilience of forests to disturbances such as drought and fire or diversity changes after a disturbance. Ultimately, demography determines the overall forest biomass in natural forests and is a key arbiter of growth and mortality rates. DGVMs are now able to simulate size and age structure of the trees in forests. However, these models have so far not been benchmarked alongside each other. We evaluate 6 DGVMs (BiomeE, CABLE-POP, FATES, LPJ-GUESS, JULES-RED, ORCHIDEE) against observations on regrowth dynamics as well as natural forests at boreal, temperate and tropical sites. We examine whether the models capture observed regrowth dynamics after disturbance, well-known stand size structure and well-established processes such as self-thinning. We outline the planned route forward towards a standardised international benchmarking framework for demographic DGVMs.

Dynamic Global Vegetation Models↗

Benchmarking GOCART-2G in the Goddard Earth Observing System (GEOS)

The Goddard Chemistry Aerosol Radiation and Transport (GOCART) model, which controls the sources sinks and chemistry within the Goddard Earth Observing System, recently underwent a major refactoring and update to the representation of physical processes. This paper serves to document code changes that were included in GOCART 2nd Generation (GOCART-2G) and establishes a benchmark simulation that is to be used for future development of the system. The code refactoring increases flexibility such multiple instances of an aerosol species can be run and interact with radiation and cloud microphysics, in addition to the output of multiple wavelength aerosol optical properties in support of data assimilation. From a science perspective, a new radiatively active tracer, brown carbon, was added to distinguish smoke from other sources of organic aerosol thereby improving optical properties entering the radiative calculations. A four-year benchmark simulation was evaluated using in situ and space borne measurements to develop a baseline and prioritize future development. A comparison of simulated aerosol optical depth between GOCART-2G and MODIS retrievals indicates the model captures the overall spatial pattern and seasonal cycle of aerosol optical depth but overestimates aerosol extinction over dusty regions and underestimates aerosol extinction over northern hemisphere boreal forests, requiring further tuning of emissions. This MODIS-based analysis is corroborated by comparisons to MISR and selected AERONET stations. Despite the underestimate of aerosol optical depth in biomass burning regions in GEOS, there is an overestimate in the surface mass of organic carbon in the United States, especially during the summer months.

Allison B Collow↗

NASA Space Observatory Precision Pointing Benchmark Problem Development

An interagency workshop in May 2021 on guidance, navigation, and control (GNC) verification and validation (V&V) methods and techniques identified needs for the formulation and public release of relevant V&V benchmark problems as a practical way forward for the discipline. A decision was made by the NASA Technical Fellow for GNC and the NASA Engineering and Safety Center (NESC) GNC Technical Discipline Team (TDT) to produce a benchmark problem focusing on the problem of space observatory precision pointing. This report contains the results of the NESC assessment.

NASA Engineering and Safety Center↗

Land Model Testbed: Accelerating Development, Benchmarking and Analysis of Land Surface Models

A Land Model Testbed (LMT), designed to provide a computational framework for systematically assessing model fidelity and supporting rapid development of complex multiscale models, offers a general-purpose workflow for conducting large ensemble simulations of multiple land surface models, post-processing large volumes of model output, and evaluating model results. It leverages existing tools for launching model simulations and the International Land Model Benchmarking (ILAMB) package for assessing model fidelity through comparison with best-available observational datasets. Increased complexity and proliferation of uncertain parameters in process representations in land surface models has driven the need for frequent and intensive testing and evaluating of models to quantify uncertainties and optimize parameters such that results are consistent with observations. The LMT described here meets these needs by providing tools to run thousands of ensemble simulations simultaneously and post-process their output files, by automating execution of an enhanced version of ILAMB with site-specific benchmarks and multivariate functional relationships, and by offering ensemble diagnostics and a customizable dashboard for displaying model performance metrics and associated graphics. We envision the LMT capabilities will serve as a foundational computational resource for a proposed user facility focused on terrestrial multiscale model--data integration.

Sreepathi, Sarat↗

Transparent Object Tracking Benchmark

Visual tracking has achieved considerable progress in recent years. However, current research in the field mainly focuses on tracking of opaque objects, while little attention is paid to transparent object tracking. In this paper, we make the first attempt in exploring this problem by proposing a Transparent Object Tracking Benchmark (TOTB). Specifically, TOTB consists of 225 videos (86K frames) from 15 diverse transparent object categories. Each sequence is manually labeled with axis-aligned bounding boxes. To the best of our knowledge, TOTB is the first benchmark dedicated to transparent object tracking. In order to understand how existing trackers perform and to provide comparison for future research on TOTB, we extensively evaluate 25 state-of-the-art tracking algorithms. The evaluation results exhibit that more efforts are needed to improve transparent object tracking. Besides, we observe some nontrivial findings from the evaluation that are discrepant with some common beliefs in opaque object tracking. For example, we find that deeper features are not always good for improvements. Moreover, to encourage future research, we introduce a novel tracker, named TransATOM, which leverages transparency features for tracking and surpasses all 25 evaluated approaches by a large margin. By releasing TOTB, we expect to facilitate future research and application of transparent object tracking in both the academia and industry.

97 MATHEMATICS AND COMPUTING↗

Towards a Benchmark Experiment with the Compact Nuclear Power Source

The Compact Nuclear Power Source (CNPS) was a high-assay low enriched uranium (HALEU) tristructural isotropic (TRISO)-fueled, graphite-moderated microreactor constructed in 1987 at Los Alamos National Laboratory. The reactor, conceived as a power source for short-range radar stations, was designed to be "walk-away safe," and was cooled by heat pipes and ambient air. Though the project was formally cancelled after the fuel and moderator material had been received, its potential to meaningfully advance the body of critical and integral data was evident, and critical experiments with a mock-up of the reactor proceeded at TA-18 until its disassembly in 1991. The reflector and some components of the core would later go on to see service as part of the New Production Reactor Modular High-Temperature Gas-Cooled Reactor (NP-MHTGR) critical experiments. In light of the progress made by groups like Westinghouse and X-Energy towards contemporary graphite-moderated microreactors, the system remains an attractive candidate for the basis of a benchmark experiment even today. Uncertainties remain, however, in some features of the system–inconsistencies in the dimensions and composition of the reactor’s components as described in literature and implemented in current computational models. The identification, quantification, and to the extent possible, the minimization of these uncertainties is a crucial task on the path towards creating a benchmark based on the CNPS system. The present work seeks to initiate this process with a survey of available documentation and improvements to the accuracy of CNPS neutronics models.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Wind Turbine Noise Code Benchmark: A Comparison and Verification Exercise: Preprint

In a number of institutions and companies, researchers and engineers are developing modeling numerical frameworks that are used to predict the aerodynamic noise emission from a wind turbine rotor. The simulation codes vary from empirically-tuned engineering models to high-fidelity computational ones. Their common feature is the fact that they all specifically model the main aerodynamic noise mechanisms occurring at the rotating blades, namely trailing edge noise and turbulent inflow noise. Nevertheless, it can be expected that these different modeling frameworks will produce different results for a same given rotor design, and identical operating conditions. Trailing edge noise is put at the forefront of the study as it is recognized as the main audible source of noise from wind turbines. The present benchmark aims at comparing the results from the different modeling approaches, and drawing some conclusions from these comparisons. This effort, denoted as Wind Turbine Noise Code benchmark, was initiated in 2019 as a joint-activity between the IEA Wind Task 39 (Quiet Wind Turbine Technology) and Task 29 (Detailed Aerodynamics of Wind Turbines, now Task 47). In addition to the investigation of the noise emissions themselves, the rotor aerodynamic characteristics are investigated as they have a significant impact on the noise generation mechanisms. A number of test cases are defined and the aerodynamic and aeroacoustic predictions from the various models are compared. There exist some discrepancies between the different methods, but it is difficult to conclude if one methodology is better than another in order to design a wind turbine with noise as a constraint.

aeroacoustics↗

Investigation of the Impact of TSL Data Libraries and Geometry Variations on the MSRE Benchmark

The increasing demand for green, low-carbon energy solutions has amplified the focus on advanced nuclear reactor technologies. Among these, Molten Salt Reactors (MSRs) have been spotlighted because of their special features. Historically, Oak Ridge National Laboratory (ORNL) started the Molten Salt Reactor Experiment (MSRE) in 1956. Later, ORNL and the University of California, Berkeley (UCB) jointly developed the MSRE benchmark. This was then reviewed by the International Reactor Physics Experiment Evaluation Project (IRPhEP) committee and added to their 2019 handbook. In the current study, the benchmark model was recreated via the Serpent code. The calculated effective multiplication factor was 1.02087 ± 0.00019, which gives a deviation of about 2000 pcm compared to the benchmark/experimental result. At North Carolina State University (NCSU), recent evaluations were conducted on thermal scattering cross sections for molten salt FLiBe and 20% porous nuclear graphite, anticipated for incorporation in the ENDF/B-VIII.1 library. The influences of these thermal scattering law (TSL) data and geometry changes on the multiplication factor were examined using the ENDF/B-VII.1 and ENDF/B-VIII.0 libraries. Introducing the TSL data for FLiBe and 20% porous graphite resulted in an effective multiplication factor of +270 pcm compared to the reference value obtained with ENDF/B-VII.1 in this study. The adjustment of the reactor vessel dimensions resulted in decrease of -448 pcm in the calculated effective multiplication factor.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗