Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “reliability metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
NASA NTRS2008

Space Transportation Systems Life Cycle Cost Assessment and Control

Civil and military applications of space transportation have been pursued for just over 50 years and there has been, and still is, a need for safe, dependable, affordable, and sustainable space transportation systems. Fully expendable and partially reusable space transportation systems have been developed and put in operation that have not adequately achieved this need. Access to space is technically achievable, but presently very expensive and will remain so until there is a breakthrough in the way we do business. Since 1991 the national Space Propulsion Synergy Team (SPST) has reviewed and assessed the lessons learned from the major U.S. space programs of the past decades focusing on what has been learned from the assessment and control of Life Cycle Cost (LCC) from these systems. This paper presents the results of a selected number of studies and analyses that have been conducted by the SPST addressing the need, as well as the solutions, for improvement in LCC. The major emphasis of the SPST processes is on developing the space transportation system requirements first (up front). These requirements must include both the usual system flight performance requirements and also the system functional requirements, including the infrastructure on Earth's surface, in-space and on the Moon and Mars surfaces to determine LCC. This paper describes the development of specific innovative engineering and management approaches and processes. This includes a focus on flight hardware maturity for reliability, ground operations approaches, and business processes between contractor and government organizations. A major change in program/project cost control is being proposed by the SPST to achieve a sustainable space transportation system LCC - controlling cost as a program metric in addition to the existing practice of controlling performance and weight. Without a firm requirement and methodically structured cost control, it is unlikely that an affordable and sustainable space transportation system LCC will ever be achieved. '

Robinson, John W.↗
NASA NTRS2010

Development of Supersonic Retro-Propulsion for Future Mars Entry, Descent, and Landing Systems

Recent studies have concluded that Viking-era entry system technologies are reaching their practical limits and must be succeeded by new methods capable of delivering large payloads (greater than 10 metric tons) required for human exploration of Mars. One such technology, termed Supersonic Retro-Propulsion, has been proposed as an enabling deceleration technique. However, in order to be considered for future NASA flight projects, this technology will require significant maturation beyond its current state. This paper proposes a roadmap for advancing the component technologies to a point where Supersonic Retro-Propulsion can be reliably used on future Mars missions to land much larger payloads than are currently possible using Viking-based systems. The development roadmap includes technology gates that are achieved through testing and/or analysis, culminating with subscale flight tests in Earth atmosphere that demonstrate stable and controlled flight. The component technologies requiring advancement include large engines capable of throttling, computational models for entry vehicle aerodynamic/propulsive force and moment interactions, aerothermodynamic environments modeling, entry vehicle stability and control methods, integrated systems engineering and analyses, and high-fidelity six degree-of-freedom trajectory simulations. Quantifiable metrics are also proposed as a means to gage the technical progress of Supersonic Retro-Propulsion. Finally, an aggressive schedule is proposed for advancing the technology through sub-scale flight tests at Earth by 2016.

Edquist, Karl T.↗
NASA NTRS2024

m:N Working Group Meeting Summary November 2023

From November 28th to 30th, 2023 the m:N UAS working group and its subgroups [small Unmanned Aircraft Systems (sUAS), Large UAS, High Altitude Platform Systems (HAPS), and Urban Air Mobility (UAM)] met at the NASA Langley Research Center in Hampton, VA for an in person meeting. The subgroups meet multiple times throughout the year, virtually. Twice a year however, participants from all the subgroups come together in person to further identify and discuss challenges, and path forward ideas for incorporating UAS into the airspace. The m:N UAS working group is run by Jay Shively (Adaptive Aerospace) and Andy Thurling (Thurling Aero Consulting) and is comprised of members from government, industry, and academia in an effort to identify and reduce barriers to m:N operations. This effort also includes identifying requirements, use cases, and metrics to support organizations and groups including the FAA, RTCA, and ASTM. Each subgroup is run by a government/industry team (see below). sUAS Subgroup Garrett Sadler (NASA) Scott Scheff (HF Designworks) Large UAS Subgroup Conrad Rory (NASA) Brandon Suarez (Reliable Robotics) HAPS Subgroup Andy Thurling (Thurling Aero Consulting) Jeff Homola (NASA) UAM Subgroup Mike Politowicz (NASA) Scott Scheff (HF Designworks), member-at-large

m:N operations↗
NASA NTRS1986

Assessment of pilot workload - Converging measures from performance based, subjective and psychophysiological techniques

This study explores the relationship between the P300 component of the event-related brain potential (ERP) and the processing demands of a complex real-world task. Seven male volunteers enrolled in an Instrument Flight Rule (IFR) aviation course flew a series of missions in a single engine fixed-based simulator. In dual task conditions subjects were also required to discriminate between two tones differing in frequency. ERPs time-locked to the tones, subjective effort ratings and overt performance measures were collected during two 45 min flights differing in difficulty (manipulated by varying both atmospheric conditions and instrument reliability). The more difficult flight was associated with poorer performance, increased subjective effort ratings, and smaller secondary task P300s. Within each flight, P300 amplitude was negatively correlated with deviations from command headings indicating that P300 amplitude was a sensitive workload metric both between and within the flight missions.

Kramer, Arthur F.↗
NASA NTRSDate not supplied

Open-Source Data Engineering at NASA: CCMC's Approach to Managing Petabyte-Scale Heliophysics Data

The Community Coordinated Modeling Center (CCMC) at NASA Goddard Space Flight Center (GSFC) leads heliophysics research by providing open access to numerous models and their outputs. Our resources are available on-demand and continuously updated with real-time data, covering sun-earth interactions across multiple domains. These domains include coronal, heliosphere, inner and global magnetosphere, ionosphere, thermosphere, and lower atmosphere interactions. Operating in a hybrid environment, CCMC utilizes both self-owned hardware and Amazon Web Services (AWS) cloud infrastructure. Managing petabytes of data across multiple locations necessitates robust data engineering solutions. To address this challenge, CCMC has adopted industry-standard and open-source tools. We use Apache Airflow as our primary data engineering platform, Python for scripting and data processing, and GitLab for version control and CI/CD. Additionally, we employ Kubernetes for containerized services, Grafana and Prometheus for metrics and monitoring, and Terraform and Puppet for reproducible infrastructure as code. This presentation will discuss lessons learned from our data engineering experiences, platforms evaluated but found unsuitable for our scientific data requirements, and specific techniques developed to enhance data transfer speed and reliability. By using these technologies effectively, CCMC continues to advance heliophysics research through efficient data management and open-access modeling.

space weather↗
NASA NTRS2011

Comparative Assessment and Decision Support System for Strategic Military Airlift Capability

The Lockheed Martin Aeronautics Company has been awarded several programs to modernize the aging C-5 military transport fleet. In order to ensure its continuation amidst budget cuts, it was important to engage the decision makers by providing an environment to analyze the benefits of the modernization program. This paper describes an interface that allows the user to change inputs such as the scenario airfields, take-off conditions, and reliability characteristics. The underlying logistics surrogate model was generated using data from a discrete-event simulation. Various visualizations such as intercontinental flight paths illustrated in 3D, have been created to aid the user in analyzing scenarios and performing comparative assessments for various output logistics metrics. The capability to rapidly and dynamically evaluate and compare scenarios was developed enabling real time strategy exploration and trade-offs.

Salmon, John↗
NASA NTRS2019

Airborne Radar for sUAS Sense and Avoid

A primary challenge for the safe integration of small UAS operations into the National Airspace System (NAS) is traffic deconfliction, both from manned and unmanned aircraft. The UAS Traffic Management (UTM) project being conducted at the National Aeronautics and Space Administration (NASA) considers a layered approach to separation provision, ranging from segregation of operations through airspace volumes (geofences) to autonomous sense and avoid (SAA) technologies for higher risk, densely occupied airspace. Cooperative SAA systems, such as Automatic Dependent Surveillance-Broadcast (ADS-B) and/or vehicle-to-vehicle communication systems provide significant additional risk mitigation but they fail to adequately mitigate collision risks for non-cooperative (non-transponder equipped) airborne aircraft. The RAAVIN (Radar on Autonomous Aircraft to Verify ICAROUS Navigation) flight test being conducted by NASA and the Mid-Atlantic Aviation Partnership (MAAP) was designed to investigate the applicability and performance of a prototype, commercially available sUAS radar to detect and track non-cooperative airborne traffic, both manned and unmanned. The radar selected for this research was a Frequency Modulated Continuous Wave (FMCW) radar with 120 degree azimuth and 80 degree elevation field of view operating at 24.55GHz center frequency with a 200 MHz bandwidth. The radar transmits 2 watts of power thru a Metamaterial Electronically Scanning Array antenna in horizontal polarization. When the radar is transmitting, personnel must be at least 1 meter away from the active array to limit nonionizing radiation exposure. The radar physical dimensions are 18.7cm by 12.1cm by 4.1cm and it weighs less than 820 grams making it well suited for installation on small UASs. The onboard, SAA capability, known as ICAROUS, (Independent Configurable Architecture for Reliable Operations of Unmanned Systems), developed by NASA to support sUAS operations, will provide autonomous guidance using the traffic radar tracks from the onboard radar. The RAAVIN set of studies will be conducted in three phases. The first phase included outdoor, ground-based radar evaluations performed at the Virginia Tech’s Kentland Farm testing range in Blacksburg, VA. The test was designed to measure how well the radar could detect and track a small UAS flying in the radar’s field of view. The radar was used to monitor 5 test flights consisting of outbound, inbound and crossing routes at different ranges and altitudes. The UAS flown during the ground test was the Inspire 2, a quad copter weighing less than 4250 grams (10 pounds) at maximum payload. The radar was set up to scan and track targets over its full azimuthal field of view from 0 to 40 degrees in elevation. The radar was configured to eliminate tracks generated from any targets located beyond 2000 meters from the radar and moving at velocities under 1.45 meters per second. For subsequent phases of the study the radar will be integrated with a sUAS platform to evaluate its performance in flight for SAA applications ranging from sUAS to manned GA aircraft detections and tracking. Preliminary data analysis from the first outdoor ground tests showed the radar performed well at tracking the vehicle as it flew outbound and repeatedly maintained a track out to 1000 meters (maximum 1387 meters) until the vehicle slowed to a stop to reverse direction to fly inbound. As the Inspire flew inbound tracks from beyond 800 meters, a reacquisition time delay was consistently observed between when the Inspire exceeds a speed of 1.45 meters per second and when the radar indicated an inbound target was present and maintained its track. The time delay varied between 6 seconds to over 37 seconds for the inbound flights examined, and typically resulted in about a 200 meter closure distance before the Inspire track was maintained. The radar performed well at both acquiring and tracking the vehicle as it flew crossing routes out past 400 meters across the azimuthal field of view. The radar and ICAROUS software will be integrated and flown on a BFD-1400-SE8-E UAS during the next phase of the RAAVIN project. The main goal at the conclusion of this effort is to determine if this radar technology can reliably support minimum requirements for SAA applications of sUAS. In particular, the study will measure the range of vehicle detections, lateral and vertical angular errors, false and missed/late detections, and estimated distance at closest point of approach after an avoidance maneuver is executed. This last metric is directly impacted by sensor performance and indicates its suitability for the task.

Szatkowski, George N.↗
NASA NTRSDate not supplied

A Physics of Failure Approach to Evaluate Printed Circuit Board Reliability

Risk based technology infusion is a deliberate and systematic process which defines the analysis and communication methodology by which new technology is applied and integrated into existing and new designs, identifies technology development needs based on trends analysis and facilitates the identification of shortfalls against performance objectives. This presentation at IPC Works Asia Aerospace 2019 Events provides the audience a snapshot of quality variations in printed wiring board quality, as assessed, using experiences in processing and risk analysis of PWB structural integrity coupons. The presentation will focus on printed wiring board quality metrics used, the relative type and number of non-conformances observed and trend analysis using statistical methods. Trend analysis shows the top five non-conformances observed across PWB suppliers, the root cause(s) behind these non-conformance and suggestions of mitigation plans. The trends will then be matched with the current state of the PWB supplier base and its challenges and opportunities. The presentation further discusses the risk based SMA approaches and methods being applied at GSFC for evaluating candidate printed wiring board technologies which promote the adoption of higher throughput and faster processing technology for GSFC missions.

Printed Circuit Boards↗
NASA NTRS1989

Microcomputer-based tests for repeated-measures: Metric properties and predictive validities

A menu of psychomotor and mental acuity tests were refined. Field applications of such a battery are, for example, a study of the effects of toxic agents or exotic environments on performance readiness, or the determination of fitness for duty. The key requirement of these tasks is that they be suitable for repeated-measures applications, and so questions of stability and reliability are a continuing, central focus of this work. After the initial (practice) session, seven replications of 14 microcomputer-based performance tests (32 measures) were completed by 37 subjects. Each test in the battery had previously been shown to stabilize in less than five 90-second administrations and to possess retest reliabilities greater than r = 0.707 for three minutes of testing. However, all the tests had never been administered together as a battery and they had never been self-administered. In order to provide predictive validity for intelligence measurement, the Wechsler Adult Intelligence Scale-Revised and the Wonderlic Personnel Test were obtained on the same subjects.

Kennedy, Robert S.↗
NASA NTRS2006

NASA Crew Launch Vehicle Overview

The US. Vision for Space Exploration, announced January 2004, outlines the National Aeronautics and Space Administration s (NASA) strategic goals and objectives. These include: 1) Flying the Shuttle as safely as possible until its retirement, not later than 2010. 2) Bringing a new Crew Exploration Vehicle (CEV) into service as soon as possible after Shuttle retirement. 3) Developing a balanced overall program of science, exploration, and aeronautics at NASA, consistent with the redirection of the human spaceflight program to focus on exploration. 4) Completing the International Space Station (ISS) in a manner consistent with international partner commitments and the needs of human exploration. 5) Encouraging the pursuit of appropriate partnerships with the emerging commercial space sector. 6) Establishing a lunar return program having the maximum possible utility for later missions to Mars and other destinations. Following the confirmation of the new NASA Administrator in April 2005, the Agency commissioned a team of aerospace subject matter experts from government and industry to perform the Exploration Systems Architecture Study (ESAS), which provided in-depth information for selecting the follow-on launch vehicle designs to enable these goals, The ESAS team analyzed a number of potential launch systems, with a focus on: (1) a human-rated launch vehicle for crew transport and (2) a heavy lift launch vehicle (HLLV) to carry cargo. After several months of intense study utilizing technical performance, budget, and schedule objectives, the results showed that the optimum architecture to meet the challenge of safe, reliable crew transport is a two-stage variant of the Space Shuttle propulsion system - utilizing the reusable Solid Rocket Booster (SRB) as the first stage, along with a new upper stage that uses a derivative of the RS-25 Space Shuttle Main Engine to deliver 25 metric tons to low-Earth orbit. The CEV that this new Crew Launch Vehicle (CLV) lofts into space early next decade will initially ferry astronauts to the Space Station and be capable of carrying crews back to lunar orbit and of supporting missions to Mars orbit. The HLLV system will utilize the Shuttle External Tank combined with SRBs. The focus of this paper is on the CLV system, giving an overview of plans and progress to date.

Dumbacher, Daniel L.↗
NASA NTRS2016

Long-Term Reliability of a Hard-Switched Boost Power Processing Unit Utilizing SiC Power MOSFETs

Silicon carbide (SiC) power devices have demonstrated many performance advantages over their silicon (Si) counterparts. As the inherent material limitations of Si devices are being swiftly realized, wide-band-gap (WBG) materials such as SiC have become increasingly attractive for high power applications. In particular, SiC power metal oxide semiconductor field effect transistors' (MOSFETs) high breakdown field tolerance, superior thermal conductivity and low-resistivity drift regions make these devices an excellent candidate for power dense, low loss, high frequency switching applications in extreme environment conditions. In this paper, a novel power processing unit (PPU) architecture is proposed utilizing commercially available 4H-SiC power MOSFETs from CREE Inc. A multiphase straight boost converter topology is implemented to supply up to 10 kilowatts full-scale. High Temperature Gate Bias (HTGB) and High Temperature Reverse Bias (HTRB) characterization is performed to evaluate the long-term reliability of both the gate oxide and the body diode of the SiC components. Finally, susceptibility of the CREE SiC MOSFETs to damaging effects from heavy-ion radiation representative of the on-orbit galactic cosmic ray environment are explored. The results provide the baseline performance metrics of operation as well as demonstrate the feasibility of a hard-switched PPU in harsh environments.

Ikpe, Stanley A.↗
NASA NTRS1994

John F. Kennedy Space Center, Safety, Reliability, Maintainability and Quality Assurance, Survey and Audit Program

This document is the product of the KSC Survey and Audit Working Group composed of civil service and contractor Safety, Reliability, and Quality Assurance (SR&QA) personnel. The program described herein provides standardized terminology, uniformity of survey and audit operations, and emphasizes process assessments rather than a program based solely on compliance. The program establishes minimum training requirements, adopts an auditor certification methodology, and includes survey and audit metrics for the audited organizations as well as the auditing organization.

Source record↗
NASA NTRS2022

Formal Methods for Trusted Space Autonomy, Boon or Bane?

Trusted Space Autonomy is challenging in that space systems are complex artifacts deployed in a high stakes environment with complicated operational settings. Thus far these challenges have been met using the full arsenal of tools: formal methods, informal methods, testing, runtime techniques, and operations processes. Using examples from previous deployments of autonomy to the Remote Agent on DS-1, Autonomous Sciencecraft on EO-1, WATCH on MER, IPEX, AEGIS on MER, MSL, and M2020, and the M2020 Onboard planner, we discuss how each of these approaches have been used to enable successful deployment of autonomy. We next focus on relatively limited use of formal methods (both prior to deployment and runtime methods). From the needs perspective, formal methods represent the best chance for reliable autonomy as testing, informal methods, and operations accommodations do not scale well with increasing complexity of the autonomous system. However from the practice perspective, formal methods have been limited in their application due to difficulty in eliciting formal specifications and challenges in representing complex constraints such as metric time and resources. We discuss some of these challenges as well as the opportunity to extend formal and informal methods into runtime validation systems.

Chien, Steve↗
NASA NTRS1994

Understanding software faults and their role in software reliability modeling

This study is a direct result of an on-going project to model the reliability of a large real-time control avionics system. In previous modeling efforts with this system, hardware reliability models were applied in modeling the reliability behavior of this system. In an attempt to enhance the performance of the adapted reliability models, certain software attributes were introduced in these models to control for differences between programs and also sequential executions of the same program. As the basic nature of the software attributes that affect software reliability become better understood in the modeling process, this information begins to have important implications on the software development process. A significant problem arises when raw attribute measures are to be used in statistical models as predictors, for example, of measures of software quality. This is because many of the metrics are highly correlated. Consider the two attributes: lines of code, LOC, and number of program statements, Stmts. In this case, it is quite obvious that a program with a high value of LOC probably will also have a relatively high value of Stmts. In the case of low level languages, such as assembly language programs, there might be a one-to-one relationship between the statement count and the lines of code. When there is a complete absence of linear relationship among the metrics, they are said to be orthogonal or uncorrelated. Usually the lack of orthogonality is not serious enough to affect a statistical analysis. However, for the purposes of some statistical analysis such as multiple regression, the software metrics are so strongly interrelated that the regression results may be ambiguous and possibly even misleading. Typically, it is difficult to estimate the unique effects of individual software metrics in the regression equation. The estimated values of the coefficients are very sensitive to slight changes in the data and to the addition or deletion of variables in the regression equation. Since most of the existing metrics have common elements and are linear combinations of these common elements, it seems reasonable to investigate the structure of the underlying common factors or components that make up the raw metrics. The technique we have chosen to use to explore this structure is a procedure called principal components analysis. Principal components analysis is a decomposition technique that may be used to detect and analyze collinearity in software metrics. When confronted with a large number of metrics measuring a single construct, it may be desirable to represent the set by some smaller number of variables that convey all, or most, of the information in the original set. Principal components are linear transformations of a set of random variables that summarize the information contained in the variables. The transformations are chosen so that the first component accounts for the maximal amount of variation of the measures of any possible linear transform; the second component accounts for the maximal amount of residual variation; and so on. The principal components are constructed so that they represent transformed scores on dimensions that are orthogonal. Through the use of principal components analysis, it is possible to have a set of highly related software attributes mapped into a small number of uncorrelated attribute domains. This definitively solves the problem of multi-collinearity in subsequent regression analysis. There are many software metrics in the literature, but principal component analysis reveals that there are few distinct sources of variation, i.e. dimensions, in this set of metrics. It would appear perfectly reasonable to characterize the measurable attributes of a program with a simple function of a small number of orthogonal metrics each of which represents a distinct software attribute domain.

Munson, John C.↗
NASA NTRS2013

NASA Tech Briefs, Februrary 2013

Topics covered include: Measurements of Ultra-Stable Oscillator (USO) Allan Deviations in Space; Gaseous Nitrogen Orifice Mass Flow Calculator; Validation of Proposed Metrics for Two-Body Abrasion Scratch Test Analysis Standards; Rover Low Gain Antenna Qualification for Deep Space Thermal Environments; Automated, Ultra-Sterile Solid Sample Handling and Analysis on a Chip; Measuring and Estimating Normalized Contrast in Infrared Flash Thermography; Spectrally and Radiometrically Stable, Wideband, Onboard Calibration Source; High-Reliability Waveguide Vacuum/Pressure Window; Methods of Fabricating Scintillators With Radioisotopes for Beta Battery Applications; Magnetic Shield for Adiabatic Demagnetization Refrigerators (ADR); CMOS-Compatible SOI MESFETS for Radiation-Hardened DC-to-DC Converters; Silicon Heat Pipe Array; Adaptive Phase Delay Generator; High-Temperature, Lightweight, Self-Healing Ceramic Composites for Aircraft Engine Applications; Treatment to Control Adhesion of Silicone-Based Elastomers; High-Temperature Adhesives for Thermally Stable Aero-Assist Technologies; Rockballer Sample Acquisition Tool; Rock Gripper for Sampling, Mobility, Anchoring, and Manipulation; Advanced Magnetic Materials Methods and Numerical Models for Fluidization in Microgravity and Hypogravity; Data Transfer for Multiple Sensor Networks Over a Broad Temperature Range; Using Combustion Synthesis to Reinforce Berms and Other Regolith Structures; Visible-Infrared Hyperspectral Image Projector; Three-Axis Attitude Estimation With a High-Bandwidth Angular Rate Sensor Change_Detection.m; AGATE: Adversarial Game Analysis for Tactical Evaluation; Ionospheric Simulation System for Satellite Observations and Global Assimilative; Modeling Experiments (ISOGAME); An Extensible, User- Modifiable Framework for Planning Activities; Mission Operations Center (MOC) - Precipitation Processing System (PPS) Interface Software System (MPISS); Automated 3D Damaged Cavity Model Builder for Lower Surface Acreage Tile on Orbiter; Mixed Linear/Square-Root Encoded Single-Slope Ramp Provides Low-Noise ADC with High Linearity for Focal Plane Arrays; RUSHMAPS: Real-Time Uploadable Spherical Harmonic Moment Analysis for Particle Spectrometers; Powered Descent Guidance with General Thrust-Pointing Constraints; X-Ray Detection and Processing Models for Spacecraft Navigation and Timing; and Extreme Ionizing-Radiation-Resistant Bacterium

Source record↗
NASA NTRSDate not supplied

Goal-Oriented Discretization Error Control in Coupled Nearfield-Farfield Low-Boom Simulations

The method of adjoint weighted residuals is used to determine the level of discretization error in loudness predictions of sonic booms on the ground. We analyze the standard nearfied-farfield domain decomposition approach. In the nearfield domain, the three-dimensional Euler equations are solved to obtain a pressure signature generated by the aircraft. In the farfield, this waveform is propagated through the atmosphere to the ground by solving the augmented Burgers’ equation. Loudness is characterized using weighted sound-exposure-level metrics. We formulate discretization error estimates for the ground signature and loudness metrics for this one-way coupled system. Although the nearfield solution is independent of the farfield, the adjoint formulation for the coupled system provides feedback from the farfield to identify high-error regions in the nearfield. The results demonstrate that the discrete adjoint implementation is asymptotically consistent and provides reliable error estimates. Furthermore, we show how the error can be controlled through adaptive refinement of the nearfield mesh. The approach is evaluated on two- and three-dimensional problems, including the X-59 flight demonstration aircraft.

CST↗
NASA NTRS1995

Towards Principled Experimental Study of Autonomous Mobile Robots

We review the current state of research in autonomous mobile robots and conclude that there is an inadequate basis for predicting the reliability and behavior of robots operating in unengineered environments. We present a new approach to the study of autonomous mobile robot performance based on formal statistical analysis of independently reproducible experiments conducted on real robots. Simulators serve as models rather than experimental surrogates. We demonstrate three new results: 1) Two commonly used performance metrics (time and distance) are not as well correlated as is often tacitly assumed. 2) The probability distributions of these performance metrics are exponential rather than normal, and 3) a modular, object-oriented simulation accurately predicts the behavior of the real robot in a statistically significant manner.

Object-Oriented Simulation↗