Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Performance benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

Adaptive Learning for Reliability Analysis using Support Vector Machines

A novel algorithm is presented for adaptive learning of an unknown function that separates two regions of a domain.In the context of reliability analysis these two regions represent the failure domain, where a set of constraints or requirements are violated, and a safe domain where they are satisfied. The Limit State Function (LSF) separates these two regions. Evaluating the constraints for a given parameter point requires the evaluation of a computational model that may well be expensive. For this reason we wish to construct a meta-model that can estimate the LSFas accurately as possible, using only a limited amount of training data. This work presents an adaptive strategy employing a Support Vector Machine (SVM) as a meta-model to provide a semi-algebraic approximation of the LSF.We describe an optimization process that is used to select informative parameter points to add to training data at each iteration to improve the accuracy of this approximation. A formulation is introduced for bounding the predictions of the meta-model; in this way we seek to incorporate this aspect of Gaussian Process Models (GPMs) within anSVM meta-model. Finally, we apply our algorithm to two benchmark test cases, demonstrating performance that is comparable with, if not superior, to a standard technique for reliability analysis that employs GPMs

Adaptive learning↗

Microstructure Segmentation with Deep Learning Encoders Pre-Trained on a Large Microscopy Dataset

This study examined the improvement of microscopy segmentation accuracy by transfer learning from a large dataset of microscopy images called MicroNet. Many neural network encoder architectures, including VGG, Inception, and ResNet, were trained on over 100,000 labelled microscopy images from 54 classes. These pre-trained encoders were then embedded into multiple segmentation architectures including U-Net and DeepLabV3+ to evaluate segmentation performance on newly created benchmark microscopy datasets. Compared to ImageNet pre-training, models pre-trained on MicroNet generalized better to out-of-distribution micrographs taken under different imaging and sample conditions and were more accurate with less training data. When training with only a single Ni-superalloy image, pre-training on MicroNet produced a 72.2 percent reduction in relative segmentation error. These results suggest that transfer learning from large in-domain datasets generate models with learned feature representations that are more useful for downstream tasks and will likely improve any microscopy image analysis technique that can leverage pre-trained encoders.

machine learning↗

AssistTaxi: A Comprehensive Dataset for Taxiway Analysis and Autonomous Operations

The availability of high-quality datasets play a crucial role in advancing research and development especially, for safety critical and autonomous systems. This poster presents AssistTaxi, which is a comprehensive novel dataset which is a collection of images for runway and taxiway analysis. The dataset comprises of more than 300,000 frames of diverse and carefully collected data, gathered from Melbourne (MLB) and Grant-Valkaria (X59) general aviation airports. The importance of AssistTaxi lies in its potential to advance autonomous operations, enabling researchers and developers to train and evaluate algorithms for efficient and safe taxiing. Researchers can utilize AssistTaxi to benchmark their algorithms, assess performance, and explore novel approaches for runway and taxiway analysis. Additionally, the dataset serves as a valuable resource for validating and enhancing existing algorithms as well as facilitating innovation in autonomous operations for aviation. We also propose an initial approach to label the dataset using a contour based detection and line extraction technique.

Data Collection↗

The mass storage testing laboratory at GSFC

Industry-wide benchmarks exist for measuring the performance of processors (SPECmarks), and of database systems (Transaction Processing Council). Despite storage having become the dominant item in computing and IT (Information Technology) budgets, no such common benchmark is available in the mass storage field. Vendors and consultants provide services and tools for capacity planning and sizing, but these do not account for the complete set of metrics needed in today's archives. The availability of automated tape libraries, high-capacity RAID systems, and high- bandwidth interconnectivity between processor and peripherals has led to demands for services which traditional file systems cannot provide. File Storage and Management Systems (FSMS), which began to be marketed in the late 80's, have helped to some extent with large tape libraries, but their use has introduced additional parameters affecting performance. The aim of the Mass Storage Test Laboratory (MSTL) at Goddard Space Flight Center is to develop a test suite that includes not only a comprehensive check list to document a mass storage environment but also benchmark code. Benchmark code is being tested which will provide measurements for both baseline systems, i.e. applications interacting with peripherals through the operating system services, and for combinations involving an FSMS. The benchmarks are written in C, and are easily portable. They are initially being aimed at the UNIX Open Systems world. Measurements are being made using a Sun Ultra 170 Sparc with 256MB memory running Solaris 2.5.1 with the following configuration: 4mm tape stacker on SCSI 2 Fast/Wide; 4GB disk device on SCSI 2 Fast/Wide; and Sony Petaserve on Fast/Wide differential SCSI 2.

Venkataraman, Ravi↗

Implementation of NAS Parallel Benchmarks in Java

A number of features make Java an attractive but a debatable choice for High Performance Computing (HPC). In order to gauge the applicability of Java to the Computational Fluid Dynamics (CFD) we have implemented NAS Parallel Benchmarks in Java. The performance and scalability of the benchmarks point out the areas where improvement in Java compiler technology and in Java thread implementation would move Java closer to Fortran in the competition for CFD applications.

Frumkin, Michael↗

On-Orbit Radiometric Characterization of OLI (Landsat 8) for Applications in Aquatic Remote Sensing

Landsat-8 carries two separate sensors, namely the Operational Land Imager (OLI) and the Thermal Infrared Radiometer Suite (TIRS), that image the earth surface throughout the visible and thermal portions of the spectrum. Compared to Landsat heritage sensors, the OLI has enhanced features, which include its 12-bit radiometric resolution and the addition of a band centered at 443 nm. The dramatically improved data quality/quantity expands existing applications of Landsat imagery in aquatic sciences from the retrieval of bio-geochemical properties, such as near-surface concentrations of chlorophyll-a (CHL) and total suspended solids (TSS), to benthic mapping. This study offers analysis of OLI's absolute radiometric performance over bodies ofwater using benchmark observations, namely the top-of-atmosphere (TOA) ocean color observations and marine in situ radiometric measurements. Sensor-to-sensor comparisons are performed to derive gain factors (g1) from near-concurrent observations in TOA radiance and reflectance domains. The gains in the radiance domain were further validated/adjusted by determining a second set of gains (g2) via analysis of OLI-derived water-leaving radiance, i.e., L(sub w) (gamma), against in situ measurements made either at the Ocean Color AErosol RObotic NETwork (AERONETOC) sites or during field campaigns. The analyses yield the OLI calibration uncertainties that need to be accounted forwhen studying aquatic environments. Itwas found that, for the visible and near-infrared channels, the OLI radiometric responses, on average, are well in agreement (b 2 % discrepancies) with the TOA radiances estimated by ocean color satellites or those predicted by models based onmeasurements of aquatic and atmospheric properties. However, the TOA radiance at the new 443-nm band is found to be, on average, 3.4 % larger than the reference observations. The inter-sensor comparisons in the reflectance domain, however, indicated slightly different results with the OLI responses being low in the blue bands. To enhance the retrieval accuracy of aquatic-science products from OLI datasets, sets of temporally averaged gains (radiance and reflectance) are derived and recommended for use prior to the retrieval of in-water products.

Remote Sensing↗

The NAS kernel benchmark program

A collection of benchmark test kernels that measure supercomputer performance has been developed for the use of the NAS (Numerical Aerodynamic Simulation) program at the NASA Ames Research Center. This benchmark program is described in detail and the specific ground rules are given for running the program as a performance test.

Bailey, D. H.↗

Performance Evaluation and Modeling Techniques for Parallel Processors

In practice, the performance evaluation of supercomputers is still substantially driven by singlepoint estimates of metrics (e.g., MFLOPS) obtained by running characteristic benchmarks or workloads. With the rapid increase in the use of time-shared multiprogramming in these systems, such measurements are clearly inadequate. This is because multiprogramming and system overhead, as well as other degradations in performance due to time varying characteristics of workloads, are not taken into account. In multiprogrammed environments, multiple jobs and users can dramatically increase the amount of system overhead and degrade the performance of the machine. Performance techniques, such as benchmarking, which characterize performance on a dedicated machine ignore this major component of true computer performance. Due to the complexity of analysis, there has been little work done in analyzing, modeling, and predicting the performance of applications in multiprogrammed environments. This is especially true for parallel processors, where the costs and benefits of multi-user workloads are exacerbated. While some may claim that the issue of multiprogramming is not a viable one in the supercomputer market, experience shows otherwise. Even in recent massively parallel machines, multiprogramming is a key component. It has even been claimed that a partial cause of the demise of the CM2 was the fact that it did not efficiently support time-sharing. In the same paper, Gordon Bell postulates that, multicomputers will evolve to multiprocessors in order to support efficient multiprogramming. Therefore, it is clear that parallel processors of the future will be required to offer the user a time-shared environment with reasonable response times for the applications. In this type of environment, the most important performance metric is the completion of response time of a given application. However, there are a few evaluation efforts addressing this issue.

Dimpsey, Robert Tod↗

Ada issues in implementing ART-Ada

Due to the Ada mandate of a number of government agencies, interest in deploying expert systems such as Ada has increased. Recently, several Ada-based expert system tools have been developed. According to a recent benchmark report, these tools do not perform as well as similar tools written in C. While poorly implemented Ada compilers contribute to the poor benchmark result, some fundamental problems of the Ada language itself have been uncovered. Here, the authors describe Ada language issues encountered during the deployment of ART-Ada, an expert system tool for Ada deployment. ART-Ada is being used to implement several prototype expert systems for the Space Station Freedom and the U.S. Air Force.

Lee, S. Daniel↗

Understanding the Cray X1 System

This paper helps the reader understand the characteristics of the Cray X1 vector supercomputer system, and provides hints and information to enable the reader to port codes to the system. It provides a comparison between the basic performance of the X1 platform and other platforms that are available at NASA Ames Research Center. A set of codes, solving the Laplacian equation with different parallel paradigms, is used to understand some features of the X1 compiler. An example code from the NAS Parallel Benchmarks is used to demonstrate performance optimization on the X1 platform.

Cheung, Samson↗

Toward Scalable Benchmarks for Mass Storage Systems

This paper presents guidelines for the design of a mass storage system benchmark suite, along with preliminary suggestions for programs to be included. The benchmarks will measure both peak and sustained performance of the system as well as predicting both short- and long-term behavior. These benchmarks should be both portable and scalable so they may be used on storage systems from tens of gigabytes to petabytes or more. By developing a standard set of benchmarks that reflect real user workload, we hope to encourage system designers and users to publish performance figures that can be compared with those of other systems. This will allow users to choose the system that best meets their needs and give designers a tool with which they can measure the performance effects of improvements to their systems.

Miller, Ethan L.↗

Hover and Forward Flight Performance Modeling of the Ingenuity Mars Helicopter

In 2015, NASA’s Jet Propulsion Laboratory partnered with Ames Research Center, Langley Research Center, and AeroVironment to develop Ingenuity, a small coaxial helicopter capable of flying within Mars’ unique atmospheric conditions. Ingenuity was successfully deployed from its protective shroud on the underside of the Mars 2020 Perseverance Rover and has flown 17 flights on Mars as of December 2021. A number of rotorcraft analysis tools were utilized, and a series of experimental tests were performed to ready Ingenuity for its launch with the Perseverance Rover in July 2020. In this paper, RotCFD, a Reynolds-averaged Navier-Stokes flow solver, is used to model Ingenuity in hover and forward flight for the purposes of validating tools to aid in the development of a future generation of Mars rotorcraft. The results from the RotCFD modeling are benchmarked against results from hover performance tests of the Ingenuity prototype in the 25-Foot Space Simulator at the Jet Propulsion Laboratory and are also compared to hover and forward flight predictions made by CAMRAD II, a well-known comprehensive rotorcraft analysis code. Surrogate performance models are trained to obtain a set of trimmed rotor settings for Ingenuity at different forward flight speeds, which are then used as inputs for the RotCFD forward flight simulations. Additionally, a study of the airframe-rotor interaction and a study of the aerodynamics of the individual airframe components of Ingenuity in forward flight are performed. Finally, to better understand performance predictions by RotCFD and CAMRAD II, a study is conducted on how sectional angles of attack in each code vary with radial station and azimuth.

Hover↗

Experiences using OpenMP based on Computer Directed Software DSM on a PC Cluster

In this work we report on our experiences running OpenMP programs on a commodity cluster of PCs running a software distributed shared memory (DSM) system. We describe our test environment and report on the performance of a subset of the NAS Parallel Benchmarks that have been automaticaly parallelized for OpenMP. We compare the performance of the OpenMP implementations with that of their message passing counterparts and discuss performance differences.

Hess, Matthias↗

Improved Benchmarking of Cohesive Elements in Abaqus Standard for Predicting Disbond and Delamination in Composite Structures

Traditional approaches for aircraft certification require the assumption of an initial flaw condition, either represented as barely visible impact damage (BVID) or through inclusion of a Teflon insert to serve as surrogate damage. Based on the initial composite damage state, the structure must be shown to demonstrate structural durability and damage tolerance (DaDT) according to the following criteria: a. Damage displays no detrimental growth under cyclic loading b. The structure is able to sustain design limit load (DLL) Currently, the only available manner for validating structural performance is through test. Since damage can occur over a wide variety of areas within a structure, this approach has proven to be increasingly expensive and time consuming for composite airframes and acreage structure within the design-test-certification building block. A further complicating factor is the requirement to accurately capture the most critical damage morphologies as a starting condition. To understand the severity of the damage, it is either required to experimentally determine the most critical areas at tremendous expense or rely on legacy data of similar structural testing, which limits design space expansion. A preferred solution is to use advanced analysis to provide improved understanding of load margins for critical locations based on a wide variety of potential starting damage conditions. The standard industry approach for DaDT certification adheres to the use of the traditional virtual crack closure technique (VCCT) method. VCCT is generally a preferred method because it conforms to the current certification principles of damage from a known flaw, and when used correctly, can be effective at predicting delamination propagation under static and cyclic loading. The VCCT method requires the inclusion of an initial flaw in the finite element (FE) model requiring a-priori knowledge of the flaw location. This in turn requires a plethora of analysis cases to be examined to cover a reasonable span of potential damage states. Additionally, the VCCT approach requires node-to-node connectivity rendering it incompatible with the best practices and approaches for using continuum damage mechanics (CDM) based progressive damage and failure analysis (PDFA) tools within a typical FE solver. Alternatives to VCCT have emerged in the form of cohesive elements which utilize the cohesive zone model (CZM). Unlike VCCT which models linear elastic fracture mechanics, cohesive elements couples continuum and fracture based responses through the use of bilinear traction separation laws. These laws are defined based on a penalty stiffness, a cohesive strength, and a strain energy release rate. The approach can be mesh regularized with native cohesive elements within many FE solvers such as Abaqus and LS-DYNA. In Phase I of the NASA Advanced Composites Consortium (ACC) post-buckled stiffened panel with BVID, Strength and Life [1], the performance of cohesive elements were benchmarked in comparison to VCCT and LEFM solutions and showed good agreement using Abaqus explicit [2]. To realize savings on current and future programs, it is still necessary to close technical gaps related to the use of cohesive elements with Abaqus Standard. Within a program environment, standard finite element analysis is the preferred analytical capability for quasi-static loading as it eliminates uncertainty due to oscillatory behavior commonly seen with explicit analysis. This oscillatory behavior creates difficulties in writing margins of safety based on the analysis. The use of negative tangent stiffness material models complicates convergence which typically requires the use of numerical controls such as viscous damping to overcome. To date, there has not been a comprehensive study on how to establish best practices for cohesive element convergence for predictive capability within the Abaqus implicit solver. In pursuit of these goals, under the NASA ACC program, several numerical benchmark problems were proposed including pure mode I (double cantilevered beam – DCB), pure mode II (end notch flexure – ENF), and symmetric/unsymmetric evolving mixed mode (single leg bend – SLB). This paper focuses on the use of cohesive elements to model the delamination through the use of CZM. Specifically, finite element models for the DCB, ENF, symmetric SLB, and unsymmetric SLB, are developed and various solution controls for convergence are studied to develop a best practice. Once the best practice has been developed, the predictive capability of the objective CZM model is used to analyze the hat pull-off strength of a standard hat stiffened configuration under various loading conditions.

Abaqus↗

Reconstruction of Thermal Protection System Aeroheating using a Green’s Function Approach

Inverse heat transfer (IHT) techniques are often used to reconstruct the surface heating conditions on spacecraft thermal protection systems (TPS) during atmospheric entry. Current IHT techniques for entry spacecraft applications, however, demand substantial computational resources, and are impractical for analyses such as uncertainty quantification and real-time health monitoring. In this paper, a Green’s function sensor fusion approach is used to reconstruct the TPS surface aeroheating conditions on experimental spaceflight and ground test systems from collocated temperature and heat flux sensors embedded in the TPS. The algorithm leverages Green’s functions to model the heat conduction within the spacecraft TPS and stabilizes the recovery of the surface heating condition using the direct heat flux sensor measurement. The algorithm is validated using arc-jet ground test data and applied to the reconstruction of the Mars 2020 backshell heating during Martian atmospheric entry. The performance of the algorithm is benchmarked against a current state-of-the-art IHT framework, FIAT_Opt. The Green’s function-based reconstruction algorithm recovers the net hot-wall heat flux absorbed by the TPS and the incident heat flux from the atmospheric entry environment in close agreement with FIAT_Opt. Notably, computation of the surface heating condition is completed in three orders of magnitude less time with the Green’s function sensor fusion approach using a consumer-grade PC, versus with FIAT_Opt running on a high performance computer cluster. The efficiency of the algorithm is leveraged to compute the uncertainty contributions of input parameters to the total uncertainty in reconstructed Mars 2020 backshell heating for the full atmospheric entry heat pulse. The sensitivity analysis uncovers that, at different times throughout the entry heat pulse, uncertainties in the TPS specific heat, thermal conductivity, and emissivity are all dominant drivers of the reconstruction uncertainty. These results demonstrate Green’s functions and sensor-fusion techniques as promising IHT approaches to reconstruct atmospheric entry environments from TPS-embedded measurements, and highlight how these techniques may give access to post-flight analyses previously hindered by the prohibitive cost of current methods.

Kenneth McAfee↗

Reconstruction of Thermal Protection System Aeroheating using a Green’s Function Approach

Inverse heat transfer (IHT) techniques are often used to reconstruct the surface heating conditions on spacecraft thermal protection systems (TPS) during atmospheric entry. Current IHT techniques for entry spacecraft applications, however, demand substantial computational resources, and are impractical for analyses such as uncertainty quantification and real-time health monitoring. In this paper, a Green’s function sensor fusion approach is used to reconstruct the TPS surface aeroheating conditions on experimental spaceflight and ground test systems from collocated temperature and heat flux sensors embedded in the TPS. The algorithm leverages Green’s functions to model the heat conduction within the spacecraft TPS and stabilizes the recovery of the surface heating condition using the direct heat flux sensor measurement. The algorithm is validated using arc-jet ground test data and applied to the reconstruction of the Mars 2020 backshell heating during Martian atmospheric entry. The performance of the algorithm is benchmarked against a current state-of-the-art IHT framework, FIAT_Opt. The Green’s function-based reconstruction algorithm recovers the net hot-wall heat flux absorbed by the TPS and the incident heat flux from the atmospheric entry environment in close agreement with FIAT_Opt. Notably, computation of the surface heating condition is completed in three orders of magnitude less time with the Green’s function sensor fusion approach using a consumer-grade PC, versus with FIAT_Opt running on a high performance computer cluster. The efficiency of the algorithm is leveraged to compute the uncertainty contributions of input parameters to the total uncertainty in reconstructed Mars 2020 backshell heating for the full atmospheric entry heat pulse. The sensitivity analysis uncovers that, at different times throughout the entry heat pulse, uncertainties in the TPS specific heat, thermal conductivity, and emissivity are all dominant drivers of the reconstruction uncertainty. These results demonstrate Green’s functions and sensor-fusion techniques as promising IHT approaches to reconstruct atmospheric entry environments from TPS-embedded measurements, and highlight how these techniques may give access to post-flight analyses previously hindered by the prohibitive cost of current methods.

Kenneth McAfee↗

Multi-Core Processor Memory Contention Benchmark Analysis Case Study

Multi-core processors dominate current mainframe, server, and high performance computing (HPC) systems. This paper provides synthetic kernel and natural benchmark results from an HPC system at the NASA Goddard Space Flight Center that illustrate the performance impacts of multi-core (dual- and quad-core) vs. single core processor systems. Analysis of processor design, application source code, and synthetic and natural test results all indicate that multi-core processors can suffer from significant memory subsystem contention compared to similar single-core processors.

Simon, Tyler↗

The NAS parallel benchmarks

A new set of benchmarks has been developed for the performance evaluation of highly parallel supercomputers in the framework of the NASA Ames Numerical Aerodynamic Simulation (NAS) Program. These consist of five 'parallel kernel' benchmarks and three 'simulated application' benchmarks. Together they mimic the computation and data movement characteristics of large-scale computational fluid dynamics applications. The principal distinguishing feature of these benchmarks is their 'pencil and paper' specification-all details of these benchmarks are specified only algorithmically. In this way many of the difficulties associated with conventional benchmarking approaches on highly parallel systems are avoided.

Bailey, D. H.↗