Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Evaluation Framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
NASA NTRS2023

Robustness of Observing System Simulation Experiments

Observing System Simulation Experiments (OSSEs) are used to investigate the potential performance of proposed new instruments on numerical weather prediction (NWP). As OSSEs involve a framework in which the atmosphere and observations are all completely simulated, it is necessary to perform validation of the OSSE to ensure that it is sufficiently realistic to provide useful experimental results. A common issue that affects the forecast skill and observation impacts is the tendency of OSSEs to have insufficient model error compared to NWP in the real world. In this work, two versions of the National Aeronautics and Space Administration Global Modeling and Assimilation Office (NASA/GMAO) NWP OSSE framework are compared, with the newer framework having more simulated forecast model error than the older framework due to changes to the NWP system. The performance of the updated OSSE is validated against corresponding behavior of the same NWP system in the real world in terms of the simulated observations, the analysis increments, forecast error, and observation impacts. OSSE results of analysis and forecast impacts for three proposed new observation systems are also compared between the older and newer frameworks to evaluate the robustness of the OSSE experiments and the role of model error in observation impacts. These three new instruments are the Geostationary eXtended Observations (GeoXO) hyperspectral infrared sounder (GXS), the Midwave Infrared Sounding of Temperature and humidity in a Constellation for Winds (MISTiC Winds) atmospheric motion vectors, and additional Global Navigation Satellite System Radio Occultations (GNSS-RO).

Observing System Simulation Experiments↗
NASA NTRSDate not supplied

Investigating Risks Due to Artemis EVA Tempo Via Probabilistic Risk Assessment

Spaceflight operations pose unique challenges to crew health, safety, and resource management. As space agencies and private companies continue to push the boundaries of human exploration, it is essential to understand the risks associated with Extravehicular Activities (EVAs) and develop strategies to mitigate them. The tempo at which EVAs are conducted – the total number and frequency of these activities – can have a profound impact on medical risks, resource consumption, and overall mission success. Probabilistic risk assessment (PRA) provides a powerful framework for evaluating complex systems and identifying potential hazards. Our work employs the Medical Extensible Dynamic Probabilistic Risk Assessment Tool (MEDPRAT) [1] to simulate mission events, occurrence and treatment of medical conditions, and track the utilization of resources. Coupled with the Evidence Library [2], a medical evidence base for exploration-class missions developed by the Exploration Medical Capability within NASA’s Human Research Program, we can estimate these risks with increased fidelity and optimize medical kit contents to meet specific mission requirements. This presentation provides a detailed examination of how EVA tempo influences medical risk estimates for a lunar surface design reference mission. A comprehensive analysis is conducted to assess the additional mass and volume burden imposed on medical kits required to maintain adequate levels of risk mitigation. Furthermore, we estimate the distribution of the number of successful EVAs completed based on the level of task impairment imposed by medical events and flight rules related to specific medical events, such as decompression sickness.

Modeling↗
NASA NTRS2016

Deductive Evaluation: Implicit Code Verification With Low User Burden

We describe a framework for symbolically evaluating C code using a deductive approach that discovers and proves program properties. The framework applies Floyd-Hoare verification principles in its treatment of loops, with a library of iteration schemes serving to derive loop invariants. During evaluation, theorem proving is performed on-the-fly, obviating the generation of verification conditions normally needed to establish loop properties. A PVS-based prototype is presented along with results for sample C functions.

Di Vito, Ben L.↗
NASA NTRSDate not supplied

A Systems Approach to AI Model Integration and Performance Evaluation for the Generic UAM Simulation Framework

This paper introduces py-guam, an open-source experimentation framework developed for the NASA Generic Urban Air Mobility simulation (GUAM) environment, facilitating the integration and evaluation of advanced artificial intelligence (AI) algorithms. We present a systems approach which enables the seamless incorporation of data-driven models, including off-nominal and failure state detection, into the GUAM’s Cognitive Architecture (CA). The framework supports customizable experimentation parameters, derives Safety Performance Indicators (SPIs) from UL 4600 safety case analyses, and employs rapid UAM simulations to assess AI impacts on flight performance across diverse scenarios. Through comprehensive testing and validation experiments, we demonstrate GUAM’s capability to enhance safety and efficiency in urban air mobility operations. Additionally, the open-source nature of py-guam fosters community collaboration, ensuring continuous improvement and adaptability to evolving technological advancements. This work establishes a robust tool for developing and testing AI-driven urban air mobility (UAM) systems, advancing the safety and reliability of autonomous urban air vehicles.

Artificial Intelligence↗
NASA NTRS2020

Unified Analysis of Aerospace Structures through Implementation of Rapid Tools into a Stress Framework

Rapid structural analysis tools have become an important part of the design cycle for aerospace companies for the last several decades. As these tools have been developed over time, there is often little consideration for shared software infrastructure between the tools, which makes design and analysis cumbersome due to a lack of commonality in input and output data and reporting, as well as poor traceability of results. Under the Rapid Tools task of the Advanced Composites Consortium (ACC), four aerospace tools were recently implemented or enhanced in the HyperSizer stress framework and then evaluated by this consortium of industry and government organizations. The framework provides an automated software environment for executing the rapid tools for analysis and sizing, quantifying margins of safety for thousands of load cases, and generating reports in support of FAA certification. This paper describes the process by which the tools were enhanced or implemented in the stress framework under the ACC project, and the evaluation conducted by industry consortium members.

Craig S Collier↗
NASA NTRS2016

Deductive Evaluation: Formal Code Analysis With Low User Burden

We describe a framework for symbolically evaluating iterative C code using a deductive approach that automatically discovers and proves program properties. Although verification is not performed, the method can infer detailed program behavior. Software engineering work flows could be enhanced by this type of analysis. Floyd-Hoare verification principles are applied to synthesize loop invariants, using a library of iteration-specific deductive knowledge. When needed, theorem proving is interleaved with evaluation and performed on the fly. Evaluation results take the form of inferred expressions and type constraints for values of program variables. An implementation using PVS (Prototype Verification System) is presented along with results for sample C functions.

Di Vito, Ben. L↗
NASA NTRS2022

Evaluating Network Performance of Containerized Test Framework for Distributed Space Systems

Distributed space systems are a mission architecture consisting of multiple spacecraft as a cohesive system which provide multipoint sampling, increased mission coverage, or improved sample resolution, while reducing mission risk through redundancy. To fully realize the potential of these systems, eventually scaling to hundreds or thousands of spacecraft, distributed space systems need to be operated as a single entity, which will enable a variety of novel scientific space missions. The Distributed Spacecraft Autonomy (DSA) project is a software project which aims to mature the technology needed for those systems, namely autonomous decision-making and swarm networking. The DSA project leverages a containerized swarm test framework to simulate spacecraft software, which can identify emergent behavior early in development. Container virtualization allows distributed spacecraft systems to be simulated entirely in software on a single computer, avoiding the overhead associated with conventional approaches like hardware facsimiles and virtual machines. For this approach to be effective, the simulated system behavior must not be artificially influenced by the swarm test framework itself. To address this, we present a series of benchmarks to quantify virtual network bandwidth available on a single-host computer and contextualize this against the network and application behavior of the DSA swarm test framework.

Distributed Spacecraft Autonomy↗
NASA NTRS2015

The Agricultural Model Intercomparison and Improvement Project: Phase I Activities by a Global Community of Science

The Agricultural Model Intercomparison and Improvement Project (AgMIP) was founded in 2010. Its mission is to improve substantially the characterization of world food security as affected by climate variability and change, and to enhance adaptation capacity in both developing and developed countries. The objectives of AgMIP are to: Incorporate state-of-the-art climate, crop/livestock, and agricultural economic model improvements into coordinated multi-model regional and global assessments of future climate impacts and adaptation and other key aspects of the food system. Utilize multiple models, scenarios, locations, crops/livestock, and participants to explore uncertainty and the impact of data and methodological choices. Collaborate with regional experts in agronomy, animal sciences, economics, and climate to build a strong basis for model applications, addressing key climate related questions and sustainable intensification farming systems. Improve scientific and adaptive capacity in modeling for major agricultural regions in the developing and developed world, with a focus on vulnerable regions. Improve agricultural data and enhance data-sharing based on their intercomparison and evaluation using best scientific practices. Develop modeling frameworks to identify and evaluate promising adaptation technologies and policies and to prioritize strategies.

farm crops↗
NASA NTRSDate not supplied

Automated End-to-End Spacecraft Connectivity Across Diverse Links

An increasing variety of communications services are available to space missions. Yet varying standards between providers hinder adoption due to the complexity of managing many configurations. We present a software framework to automatically establish end-to-end communications during contacts with a provider by configuring spacecraft protocols at the physical, link, and network layers. Underlying protocols and routes are abstracted, allowing the user to simply send data to a destination with the framework ensuring its delivery. We evaluate a full implementation of the framework in laboratory experiments conducted on an emulated communications testbed. These tests demonstrate data delivery across three different services with rapid (<20s) reconfiguration as the spacecraft transitions between providers.

satellite communication↗
NASA NTRS2020

Description and Evaluation of the Specified-Dynamics Experiment in the Chemistry-Climate Model Initiative

We provide an overview of the REF-C1SD specified-dynamics experiment that was conducted as part of phase 1 of the Chemistry-Climate Model Initiative (CCMI). The REF-C1SD experiment, which consisted of mainly nudged general circulation models (GCMs) constrained with (re)analysis fields, was designed to examine the influence of the large-scale circulation on past trends in atmospheric composition. The REF-C1SD simulations were produced across various model frameworks and are evaluated in terms of how well they represent different measures of the dynamical and transport circulations. In the troposphere there are large (~40 %) differences in the climatological mean distributions, seasonal cycle amplitude, and trends of the meridional and vertical winds. In the stratosphere there are similarly large (~50 %) differences in the magnitude, trends and seasonal cycle amplitude of the transformed Eulerian mean circulation and among various chemical and idealized tracers. At the same time, interannual variations in nearly all quantities are very well represented, compared to the underlying reanalyses. We show that the differences in magnitude, trends and seasonal cycle are not related to the use of different reanalysis products; rather, we show they are associated with how the simulations were implemented, by which we refer both to how the large-scale flow was prescribed and to biases in the underlying free-running models. In most cases these differences are shown to be as large or even larger than the differences exhibited by free-running simulations produced using the exact same models, which are also shown to be more dynamically consistent. Overall, our results suggest that care must be taken when using specified-dynamics simulations to examine the influence of large-scale dynamics on composition.

Clara Orbe↗
NASA NTRS2012

Desert Research and Technology Studies (DRATS) 2010 Science Operations: Operational Approaches and Lessons Learned for Managing Science during Human Planetary Surface Missions

Desert Research and Technology Studies (Desert RATS) is a multi-year series of hardware and operations tests carried out annually in the high desert of Arizona on the San Francisco Volcanic Field. These activities are designed to exercise planetary surface hardware and operations in conditions where long-distance, multi-day roving is achievable, and they allow NASA to evaluate different mission concepts and approaches in an environment less costly and more forgiving than space.The results from the RATS tests allows election of potential operational approaches to planetary surface exploration prior to making commitments to specific flight and mission hardware development. In previous RATS operations, the Science Support Room has operated largely in an advisory role, an approach that was driven by the need to provide a loose science mission framework that would underpin the engineering tests. However, the extensive nature of the traverse operations for 2010 expanded the role of the science operations and tested specific operational approaches. Science mission operations approaches from the Apollo and Mars-Phoenix missions were merged to become the baseline for this test. Six days of traverse operations were conducted during each week of the 2-week test, with three traverse days each week conducted with voice and data communications continuously available, and three traverse days conducted with only two 1-hour communications periods per day. Within this framework, the team evaluated integrated science operations management using real-time, tactical science operations to oversee daily crew activities, and strategic level evaluations of science data and daily traverse results during a post-traverse planning shift. During continuous communications, both tactical and strategic teams were employed. On days when communications were reduced to only two communications periods per day, only a strategic team was employed. The Science Operations Team found that, if communications are good and down-linking of science data is ensured, high quality science returns is possible regardless of communications. What is absent from reduced communications is the scientific interaction between the crew on the planet and the scientists on the ground. These scientific interactions were a critical part of the science process and significantly improved mission science return over reduced communications conditions. The test also showed that the quality of science return is not measurable by simple numerical quantities but is, in fact, based on strongly non-quantifiable factors, such as the interactions between the crew and the Science Operations Teams. Although the metric evaluation data suggested some trends, there was not sufficient granularity in the data or specificity in the metrics to allow those trends to be understood on numerical data alone.

Eppler, Dean↗
NASA NTRS2024

Envisioning an Optimal Network of Space-Based Lasers for Orbital Debris Remediation

The rapid increase in resident space objects, including satellites and orbital debris, poses a significant threat to the safety and sustainability of space missions. This paper explores orbital debris remediation using a network of collaborative space-based lasers, leveraging laser ablation for momentum transfer on debris. A novel delta-v vector analysis framework quantifies the e↵ects of multiple simultaneous laser-to-debris (L2D) engagements by using vector composition of the imparted delta-v vectors. The paper introduces the Concurrent LocationScheduling Problem (CLSP), which optimizes the placement of laser platforms and the scheduling of L2D engagements to maximize debris remediation capacity. Due to the computational complexity of the CLSP, it is decomposed into two sequential subproblems: (1) optimal laser platform locations are determined using the Maximal Covering Location Problem, and (2) a novel integer linear programming-based approach schedules L2D engagements within the network configuration to maximize remediation capacity. Computational experiments are conducted to evaluate the proposed framework’s e↵ectiveness under various mission scenarios, demonstrating key network functions such as collaborative nudging, deorbiting, and just-in-time collision avoidance. A sensitivity analysis further examines how varying the number and distribution of laser platforms a↵ects debris remediation capacity, providing insights into optimizing the performance of space-based laser networks.

David O Williams Rogers↗
NASA NTRS2020

Urban Air Mobility System Testbed Using CAVE Virtual Reality Environment

Urban Air Mobility (UAM) refers to a system of air passenger and small cargo transportation within an urban area. The UAM framework also includes other urban Unmanned Aerial Systems (UAS) services that will be supported by a mix of onboard, ground, piloted, and autonomous operations. Over the past few years UAM research has gained wide interest from companies and federal agencies as an on-demand innovative transportation option that can help reduce traffic congestion and pollution as well as increase mobility in metropolitan areas. The concepts of UAM/UAS operation in the National Airspace System (NAS) remains an active area of research to ensure safe and efficient operations. With new developments in smart vehicle design and infrastructure for air traffic management, there is a need for methods to integrate and test various components of the UAM framework. In this work, we report on the development of a virtual reality (VR) testbed using the Cave Automatic Virtual Environment (CAVE) technology for human-automation teaming and airspace operation research of UAM. Using a four-wall projection system with motion capture, the CAVE provides an immersive virtual environment with real-time full body tracking capability. We created a virtual environment consisting of San Francisco city and a vertical take-off-and-landing passenger aircraft that can fly between a downtown location and the San Francisco International Airport. The aircraft can be operated autonomously or manually by a single pilot who maneuvers the aircraft using a flight control joystick. The interior of the aircraft includes a virtual cockpit display with vehicle heading, location, and speed information. The system can record simulation events and flight data for post-processing. The system parameters are customizable for different flight scenarios; hence, the CAVE VR testbed provides a flexible method for development and evaluation of UAM framework.

Marayong, Panadda↗
NASA NTRS1977

Rotation histories of the natural satellites

Recent advances in the theory of rotation are combined with traditional approaches to study the rotational evolution of the 33 known natural satellites. A calculation similar to that reported by Burns and Safronov (1973) is applied to each satellite to obtain the characteristic time of decay of any wobble motion to smooth rotation about the principal axis of maximum moment of inertia. Stability criteria and capture probabilities are calculated for the 3/2 spin resonance. Results show that only the regular satellites and Iapetus, Hyperion, Triton, and the moon are tidally evolved. Of these, 13 have confirmed synchronous rotation periods; capture probabilities into the 3/2 resonance indicate that none of the remaining 10 should be captured in nonsynchronous, commensurate spin states. For the most part, the irregular satellites retain their original spins except for a relaxation to principal axis rotation. Tidal evolution of the obliquities of the satellites is evaluated in the framework of the generalization of Cassini's laws for the moon. Nearly resonant, forced librations in longitude of 4.8 and 0.5 deg are calculated on the basis of the observed shapes of Phobos and Deimos, respectively.

Peale, S. J.↗
NASA NTRS1978

Planning for airport access: An analysis of the San Francisco Bay area. Existing studies

The transportation systems studies completed, which have a bearing on the area's airport access system, are surveyed. A brief description of some of the selected studies, a framework for their evaluation, and an assessment of their proposals are provided. Areas of concentration presented include: (1) San Francisco International Airport, (2) Metropolitan Oakland International Airport, (3) San Jose Municipal Airport, and (4) Regional and Local transportation systems.

Source record↗
NASA NTRS1979

Light scattering properties of spheroidal particles

In the present paper, the light scattering characteristics of spheroidal particles are evaluated within the framework of a scattering theory developed for a homogeneous isotropic spheroid. This approach is shown to be well suited for computing the scattering quantities of spheroidal particles of fairly large sizes (up to a size parameter of 30). The effects of particle size, shape, index of refraction, and orientation on the scattering efficiency factors and the scattering intensity functions are studied and interpreted physically. It is shown that, in the case of oblique incidence, the scattering properties of a long slender prolate spheroid resemble those of an infinitely long circular cylinder.

Asano, S.↗
NASA NTRS1985

A theoretical basis for the analysis of redundant software subject to coincident errors

Fundamental to the development of redundant software techniques fault-tolerant software, is an understanding of the impact of multiple-joint occurrences of coincident errors. A theoretical basis for the study of redundant software is developed which provides a probabilistic framework for empirically evaluating the effectiveness of the general (N-Version) strategy when component versions are subject to coincident errors, and permits an analytical study of the effects of these errors. The basic assumptions of the model are: (1) independently designed software components are chosen in a random sample; and (2) in the user environment, the system is required to execute on a stationary input series. The intensity of coincident errors, has a central role in the model. This function describes the propensity to introduce design faults in such a way that software components fail together when executing in the user environment. The model is used to give conditions under which an N-Version system is a better strategy for reducing system failure probability than relying on a single version of software. A condition which limits the effectiveness of a fault-tolerant strategy is studied, and it is posted whether system failure probability varies monotonically with increasing N or whether an optimal choice of N exists.

Eckhardt, D. E., Jr.↗
NASA NTRS1985

A theoretical basis for the analysis of multiversion software subject to coincident errors

Fundamental to the development of redundant software techniques (known as fault-tolerant software) is an understanding of the impact of multiple joint occurrences of errors, referred to here as coincident errors. A theoretical basis for the study of redundant software is developed which: (1) provides a probabilistic framework for empirically evaluating the effectiveness of a general multiversion strategy when component versions are subject to coincident errors, and (2) permits an analytical study of the effects of these errors. An intensity function, called the intensity of coincident errors, has a central role in this analysis. This function describes the propensity of programmers to introduce design faults in such a way that software components fail together when executing in the application environment. A condition under which a multiversion system is a better strategy than relying on a single version is given.

Eckhardt, D. E., Jr.↗