Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
NASA NTRS2011

Integration Process for the Habitat Demonstration Unit

The Habitat Demonstration Unit (HDU) is an experimental exploration habitat technology and architecture test platform designed for analog demonstration activities. The HDU previously served as a test bed for testing technologies and sub-systems in a terrestrial surface environment. in 2010 in the Pressurized Excursion Module (PEM) configuration. Due to the amount of work involved to make the HDU project successful, the HDU project has required a team to integrate a variety of contributions from NASA centers and outside collaborators The size of the team and number of systems involved With the HDU makes Integration a complicated process. However, because the HDU shell manufacturing is complete, the team has a head start on FY--11 integration activities and can focus on integrating upgrades to existing systems as well as integrating new additions. To complete the development of the FY-11 HDU from conception to rollout for operations in July 2011, a cohesive integration strategy has been developed to integrate the various systems of HDU and the payloads. The highlighted HDU work for FY-11 will focus on performing upgrades to the PEM configuration, adding the X-Hab as a second level, adding a new porch providing the astronauts a larger work area outside the HDU for EVA preparations, and adding a Hygiene module. Together these upgrades result in a prototype configuration of the Deep Space Habitat (DSH), an element under evaluation by NASA's Human Exploration Framework Team (HEFT) Scheduled activates include early fit-checks and the utilization of a Habitat avionics test bed prior to installation into HDU. A coordinated effort to utilize modeling and simulation systems has aided in design and integration concept development. Modeling tools have been effective in hardware systems layout, cable routing, sub-system interface length estimation and human factors analysis. Decision processes on integration and use of all new subsystems will be defined early in the project to maximize the efficiency of both integration and field operations. In addition a series of tailored design reviews are utilized to quickly define the systems and their integration into the DSH configuration. These processes are necessary to ensure activities, such as partially reversing integration of the X-Hab second story of the HDU and deploying and stowing the new work porch for transportation to the JSC Rock Yard and to the Arizona Black Point Lava Flow Site are performed with minimal or no complications. In addition, incremental test operations leading up to an Integrated systems test allows for an orderly systems test program. For FY-11 activities, the HDU DSH will act as a laboratory utilizing a new X-Hab inflatable second floor with crew habitation features. In addition to the day to day operations involving maintenance of the HDU and exploring the surrounding terrain, testing and optimizing the use of the new X-Hab, work porch, Hygiene Module, and other sub-system enhancements will be the focus of the FY-11 test objectives. The HDU team requires a successful integration strategy using a variety of tools and approaches to prepare the DSH for these test objectives. In a challenging environment where the prototyping influences the system design, as well as Vice versa, results of the HDU DSH field tests will influence future designs of habitat systems.

Gill, Tracy↗
NASA NTRS2020

Integrated assessment of climate change impacts on crop productivity and income of commercial maize farms in northeast South Africa

Agriculture in South Africa sustains about 70% of the region’s population for food, income and employment, playing an important role for food security and the local economy. The focus of the study was the commercial maize farms of the Free State Province given their importance in the National economy. The Regional Integrated Assessment (phase I) was implemented to assess climate change and adaptation that links climate, crops, economic data and tools developed by the Agricultural Model Intercomparison and Improvement Project (AgMIP). In this context, the“system”is defined as a whole of agronomic and socio-economic factors. Within that framework three core questions were being evaluated: (i) Impacts of climate change under current system; (ii) Impacts of climate change under future system; (iii) The role of adaptation under climate change and the future system. Maize production will decrease between 10% to 16% as a result of projected climate impacts. Also, current agricultural production systems are negatively affected by climate change with an increase in poverty rates between 2% to 3%. The projected adoption of the adapted technology would result in positive increased net returns and a decrease in poverty rate of between 12%and 22%. The results of this study show that implementing adaptation measures and other strategies as indicated by the local stakeholders will have positive impacts on the agricultural production systems and can contribute to support and inform climate change policy decision making such as the development of National Adaptation Plans.

Integrated assessment↗
NASA NTRS2019

The Terrestrial Organism and Biogeochemistry Spatial Sampling Design for the National Ecological Observatory Network

The National Ecological Observatory Network (NEON) seeks to facilitate ecological prediction at a continental scale by measuring processes that drive change and responses at sites across the United States for thirty years. The spatial distribution of observations of terrestrial organisms and soil within NEON sites is determined according to a “design‐based” sample design that relies on the randomization of sampling locations. Development of the sample design was guided by high‐level NEON objectives and the multitude of data products that will be subjected to numerous analytical approaches to address the causes and consequences of ecological change. A requirement framework permeates the NEON design, ensuring traceability from each facet of the design to the high‐level requirements that make the NEON mission statement actionable. Requirements were developed for the terrestrial sample design to guide the key components of the design: Randomizing the sample locations ensures the unbiased collection of data, is appropriate for organisms and soil, and provides data suitable for a variety of analyses. Stratification increases efficiency and allows sampling to focus on those parts of the landscape measured by other NEON observation platforms. Attention to the sample size and spatial plot allocation ensures that data products will be sufficient to inform questions asked of the data and the NEON objectives. Establishing a framework with the capacity for re‐evaluate and design iteration allows for adaption to unexpected challenges and optimization of the sample design based on early data returns. The utility of the NEON sampling design is highlighted by its application across terrestrial systems. The data generated from this unique design will be used to quantify patterns in: the abundance and diversity of small mammals, breeding birds, insects, and soil microbes; vegetation structure, biomass, productivity, and diversity; and soil biogeochemistry.

National Ecological Observatory Network↗
NASA NTRS2019

Chemical Ecosystem Selection on Mineral Surfaces Reveals Long-Term Dynamics Consistent with the Spontaneous Emergence of Mutual Catalysis

How did chemicals first become organized into systems capable of self-propagation and adaptive evolution? One possibility is that the first evolvers were chemical ecosystems localized on mineral surfaces and composed of sets of molecular species that could catalyze each other’s formation. We used a bottom-up experimental framework; chemical ecosystem selection (CES) to evaluate this perspective and search for surface-associated; and mutually catalytic chemical systems based on the changes in chemistry they are expected to induce. Here, we report the results of preliminary CES experiments conducted using a synthetic “prebiotic soup” and pyrite grains, which yield dynamical patterns that are suggestive of the emergence of mutual catalysis. While more research is needed to better understand the specific patterns observed here and determine whether they are reflective of self-propagation, these results illustrate the potential power of CES to test competing hypotheses for the emergence of protobiological chemical systems.

autocatalysis↗
NASA NTRSDate not supplied

NASA’s Space Sustainability Strategy: An Overview and a Conceptual Framework for Implementation

Space sustainability is crucial for the continuation of humanity's exploration and utilization of space. NASA recognizes the imperative of maintaining a sustainable space operating environment to safeguard the future of space activities. On April 9, 2024, NASA released its first part of an integrated Space Sustainability Strategy focused on the domain of earth orbit, with future volumes to address impacts of space operations on earth itself; cislunar space, including the Lagrange points and the lunar surface; and deep space, including other celestial bodies. This commitment extends to ensuring the safety of astronauts conducting operations in Low Earth Orbit (LEO), as well as enabling missions to the Moon and Mars. Moreover, sustainable space practices are essential for the uninterrupted operation of NASA's robotic science missions, which monitor Earth's changes and probe the mysteries of the universe. The strategy sets out a set of goals, the first of which is to establish a framework for assessing space sustainability at NASA, addressing the absence of a universally accepted framework with key metrics that promotes understanding of how metrics are interconnected. This framework will provide a basis for evaluating the sustainability of orbital activities, with metrics informed by input from the domestic and international space community. Objectives under this goal include defining key parameters affecting space sustainability, determining tolerable levels of risk, and annually publishing NASA's effect on space sustainability, including compliance with orbital debris mitigation standards. The subsequent goals build upon this foundation, identifying effective actions for enhancing space sustainability, incentivizing adoption of these actions through technology development and policy changes, and ensuring coordination with the global space community. An organizational change within NASA is also proposed to facilitate day-to-day coordination and accountability for space sustainability efforts. By pursuing these goals, NASA aims to strengthen the technical foundations of and continue to set an example for sustainable space operations. Regular evaluation of NASA mission performance in context with meeting sustainability objectives will ensure alignment with evolving challenges in the space operating environment.

Matt Forsbacka↗
NASA NTRS2017

Global Gridded Crop Model Evaluation: Benchmarking, Skills, Deficiencies and Implications.

Crop models are increasingly used to simulate crop yields at the global scale, but so far there is no general framework on how to assess model performance. Here we evaluate the simulation results of 14 global gridded crop modeling groups that have contributed historic crop yield simulations for maize, wheat, rice and soybean to the Global Gridded Crop Model Intercomparison (GGCMI) of the Agricultural Model Intercomparison and Improvement Project (AgMIP). Simulation results are compared to reference data at global, national and grid cell scales and we evaluate model performance with respect to time series correlation, spatial correlation and mean bias. We find that global gridded crop models (GGCMs) show mixed skill in reproducing time series correlations or spatial patterns at the different spatial scales. Generally, maize, wheat and soybean simulations of many GGCMs are capable of reproducing larger parts of observed temporal variability (time series correlation coefficients (r) of up to 0.888 for maize, 0.673 for wheat and 0.643 for soybean at the global scale) but rice yield variability cannot be well reproduced by most models. Yield variability can be well reproduced for most major producing countries by many GGCMs and for all countries by at least some. A comparison with gridded yield data and a statistical analysis of the effects of weather variability on yield variability shows that the ensemble of GGCMs can explain more of the yield variability than an ensemble of regression models for maize and soybean, but not for wheat and rice. We identify future research needs in global gridded crop modeling and for all individual crop modeling groups. In the absence of a purely observation-based benchmark for model evaluation, we propose that the best performing crop model per crop and region establishes the benchmark for all others, and modelers are encouraged to investigate how crop model performance can be increased. We make our evaluation system accessible to all crop modelers so that other modeling groups can also test their model performance against the reference data and the GGCMI benchmark.

wheat↗
NASA NTRS2017

Aeroelastic Uncertainty Quantification Studies Using the S4T Wind Tunnel Model

This paper originates from the joint efforts of an aeroelastic study team in the Applied Vehicle Technology Panel from NATO Science and Technology Organization, with the Task Group number AVT-191, titled "Application of Sensitivity Analysis and Uncertainty Quantification to Military Vehicle Design." We present aeroelastic uncertainty quantification studies using the SemiSpan Supersonic Transport wind tunnel model at the NASA Langley Research Center. The aeroelastic study team decided treat both structural and aerodynamic input parameters as uncertain and represent them as samples drawn from statistical distributions, propagating them through aeroelastic analysis frameworks. Uncertainty quantification processes require many function evaluations to asses the impact of variations in numerous parameters on the vehicle characteristics, rapidly increasing the computational time requirement relative to that required to assess a system deterministically. The increased computational time is particularly prohibitive if high-fidelity analyses are employed. As a remedy, the Istanbul Technical University team employed an Euler solver in an aeroelastic analysis framework, and implemented reduced order modeling with Polynomial Chaos Expansion and Proper Orthogonal Decomposition to perform the uncertainty propagation. The NASA team chose to reduce the prohibitive computational time by employing linear solution processes. The NASA team also focused on determining input sample distributions.

Nikbay, Melike↗
NASA NTRS1995

Excursions in technology policy

This technical report presents a summary of three distinct projects: (1) Measuring economic benefits; (2) Evaluating the SBIR program; and (3) A model for evaluating changes in support for science and technology. the first project deals with the Technology Applications Group (TAG) at NASA Langley Research Center. The mission of TAG is to assist firms interested in commercializing technologies. TAG is a relatively new group as is the emphasis on technology commercialization for NASA. One problem faced by TAG and similar groups at other centers is measuring their effectiveness. The first project this summer, a paper entitled, 'Measuring the Economic Benefits of Technology Transfer from a National Laboratory: A Primer,' focused on this measurement problem. We found that the existing studies of the impact of technology transfer on the economy were conceptually flawed. The 'primer' outlines the appropriate theoretical framework for measuring the economic benefits of technology transfer. The second project discusses, one of the programs of TAG, the Small Business Innovation Research (SBIR) program. This program has led to over 400 contracts with Small Business since its inception in 1985. The program has never been evaluated. Crucial questions such as those about the extent of commercial successes from the contracts need to be answered. This summer we designed and implemented a performance evaluation survey instrument. The analysis of the data will take place in the fall. The discussion of the third project focuses on a model for evaluating changes in support for science and technology. At present several powerful forces are combining to change the environment for science and technology policy. The end of the cold war eliminated the rationale for federal support for many projects. The new- found Congressional conviction to balance the budget without tax increases combined with demographic changes which automatically increase spending for some politically popular programs will make it difficult to find funding for science and technology. Also, the two political parties have very different conceptions of the appropriate future for research and development spending. All these changes create the potential for serious, perhaps unintended, consequences for the economic future of the country. In a paper entitled, 'A Conceptual Framework for Evaluating the Impact of Changes in Federal Support for Science and Technology,' we introduce a model to evaluate the effects of changes in federal spending for science and technology. This paper both provides a way of organizing informed discussions and points out important research topics for science and technology policy.

Archibald, Robert B.↗
NASA NTRSDate not supplied

Analysis of Alternative Architectures for Cargo Lunar Landers

NASA’s Human Landing System (HLS) program has been working with commercial partners to develop human-class lunar landers to return the first American woman and next American man to the lunar surface in the mid 2020’s. In an effort to expand human presence beyond low Earth orbit, NASA’s Artemis program aims to facilitate a sustainable, long-term human presence in cis-lunar space. A component of this will require significant infrastructure to be delivered to the lunar surface. Delivering this infrastructure will require a significant lander capability that has yet to be developed. A thorough understanding of cargo lunar lander architectures is required such that select alternatives can be identified that best support the Artemis program’s objective of sustainability. The goal of this study is to aid NASA and its partners in the understanding of the cargo lunar lander trades space, as well as identify potential robust alternatives. The results will support NASA as it moves forward with key activities such as requirements formulation, agency strategic planning, and potential cargo lunar lander procurements. The study builds off of recent work performed by the Human Landing System program’s Architecture and Systems Analysis group to encompass a broad trade space of cargo lunar lander architecture alternatives. The current trade space as depicted by the morphological matrix and mission graph in Fig. 1 and Fig. 2, respectively, includes key alternative options that have become highly relevant due to current HLS activities and include on-orbit refueling, active cryogenic fluid management, Earth orbit aggregation, and global lunar access. The authors believe that there is also a statistically relevant impact of lander-payload configuration on the primary structure of the vehicle that could greatly impact alternative selection. Because of this, several conceptual lander-payload configurations will be evaluated to determine the level of impact. The current set of conceptual configurations are shown in Fig. 3 and Fig. 4. To aid the conceptual evaluation of these configurations, a catalogue of notional payloads has been developed that represent a wide range of masses and volumes that are expected to be delivered in support of a sustained human lunar presence, including pressurized and unpressurized rovers, surface habitats, power systems, and other support infrastructure. In order to execute this study in a timely fashion, a similar approach to that utilized in a similar 2019 study focused on 2024 human lunar sorties will be employed [1]. The team utilized a novel architecture synthesis framework currently being developed by NASA/MSFC to evaluate over 600,000 lunar lander architectures over a two month time frame [2]. From this large data set, varying ground rules and assumptions were applied as filters to explore the trade space to identify alternatives which exhibited robustness, as measured by launch vehicle payload margin, to absorb the natural growth that occurs during design maturation. The set of Earth-Moon system Delta-Vs assumed from the 2019 study, shown in Fig. 5, will be repurposed to accelerate model formulation for this effort. Additionally, current efforts in collaboration with the Georgia Institute of Technology’s Aerospace System Design Lab will be integrated to provide probabilistic modeling of the cargo lunar lander architectures to aid in identifying robust design alternatives [3]. The approach will help minimize potential impacts due to large levels of uncertainty inherent to pre phase-A conceptual design. By leveraging these past and present studies and partnerships, a highly detailed set of data can be generated in a short time period to aid NASA in the coming years to support the goal of a sustained human lunar presence.

Architectures↗
NASA NTRS2023

Surface Shortwave Radiative Fluxes Derived from the U.S. Air Force Cloud Depiction Forecast System World-Wide Merged Cloud Analysis

We present a global-scale evaluation of surface shortwave (SW↓) radiative fluxes as derived with cloud amount information from the U.S. Air Force (USAF) Cloud Depiction Forecast System (CDFS) II World-Wide Merged Cloud Analysis (WWMCA) and implemented in the framework of the NASA Land Information System (LIS). Evaluation of this product is done against ground observations, a satellite-based product from the Moderate Resolution Imaging Spectroradiometer (MODIS), and several reanalysis outputs. While the LIS/USAF product tends to overestimate the SW↓ fluxes when compared to ground observations and satellite estimates, its performance is comparable or better than the following reanalysis products: ERA5, CFSR, and MERRA-2. Results are presented using all available observations over the globe and independently for several regional domains of interest. When evaluated against ground observations over the globe, the bias in the LIS/USAF product at daily time scale was about 9.34 W m −2 and the RMS was 29.20 W m −2 while over the United States the bias was about 10.65 W m −2 and the RMS was 35.31 W m −2 . The sample sizes used were not uniform over the different regions, and the quality of both ground truth and the outputs of the other products may vary regionally. It is important to note that the LIS/USAF is a near-real-time (NRT) product of interest for potential users and as such fills a need that is not met by most products. Due to latency issues, the level of observational inputs in the NRT product is less than in the reanalysis data.

Cloud radiative effects↗
NASA NTRS1986

NASA PC software evaluation project

The USL NASA PC software evaluation project is intended to provide a structured framework for facilitating the development of quality NASA PC software products. The project will assist NASA PC development staff to understand the characteristics and functions of NASA PC software products. Based on the results of the project teams' evaluations and recommendations, users can judge the reliability, usability, acceptability, maintainability and customizability of all the PC software products. The objective here is to provide initial, high-level specifications and guidelines for NASA PC software evaluation. The primary tasks to be addressed in this project are as follows: to gain a strong understanding of what software evaluation entails and how to organize a structured software evaluation process; to define a structured methodology for conducting the software evaluation process; to develop a set of PC software evaluation criteria and evaluation rating scales; and to conduct PC software evaluations in accordance with the identified methodology. Communication Packages, Network System Software, Graphics Support Software, Environment Management Software, General Utilities. This report represents one of the 72 attachment reports to the University of Southwestern Louisiana's Final Report on NASA Grant NGT-19-010-900. Accordingly, appropriate care should be taken in using this report out of context of the full Final Report.

Dominick, Wayne D.↗
NASA NTRS1992

DEPEND: A simulation-based environment for system level dependability analysis

The design and evaluation of highly reliable computer systems is a complex issue. Designers mostly develop such systems based on prior knowledge and experience and occasionally from analytical evaluations of simplified designs. A simulation-based environment called DEPEND which is especially geared for the design and evaluation of fault-tolerant architectures is presented. DEPEND is unique in that it exploits the properties of object-oriented programming to provide a flexible framework with which a user can rapidly model and evaluate various fault-tolerant systems. The key features of the DEPEND environment are described, and its capabilities are illustrated with a detailed analysis of a real design. In particular, DEPEND is used to simulate the Unix based Tandem Integrity fault-tolerance and evaluate how well it handles near-coincident errors caused by correlated and latent faults. Issues such as memory scrubbing, re-integration policies, and workload dependent repair times which affect how the system handles near-coincident errors are also evaluated. Issues such as the method used by DEPEND to simulate error latency and the time acceleration technique that provides enormous simulation speed up are also discussed. Unlike any other simulation-based dependability studies, the use of these approaches and the accuracy of the simulation model are validated by comparing the results of the simulations, with measurements obtained from fault injection experiments conducted on a production Tandem Integrity machine.

Goswami, Kumar↗
NASA NTRS1996

The smart highway project: Smart highways, smart vehicles, smart engineering

The Smart Highway project is a six mile, limited access roadway being built between Interstate 81 and Blacksburg, Virginia. The initial construction segment will be two miles long and is designed to serve as a test bed and test track for Intelligent Transportation Systems (ITS) research. The Center for Transportation Research (CTR) at Virginia Tech is developing three evaluation tools for its ITS research including DYNAVIMTS (a software framework), and the FLASH Lab (a 1/15th scale model highway and vehicle system). The Smart Highway rounds out the Center's evaluation methodology by allowing full scale operational tests, evaluations, and research under both experimental and conventional traffic conditions. Currently under development is a concept for a fully automated highway using a 'Cooperative Infrastructure Managed System' which involves ultra wide band communication beacons installed in the infrastructure with appropriate sensors, receivers and processors on board the vehicles. The project is part of the research program funded by the National Automated Highway System Consortium. The CTR hopes to develop the automated concept to prototype status by 1997. Other smart transportation and smart engineering concepts are proposed. This presentation will address the goals and objectives of the Smart Highway project, overview its status and importance to the region, and identify some of the transportation technology now under development and planned in the future.

Pethtel, Ray D.↗
NASA NTRSDate not supplied

A Framework for Assessing Earth Observation Metadata Quality: Implications for Data Discovery and Open Science

The Common Metadata Repository (CMR) contains metadata records describing NASA’s collection of over 8,000 Earth observation data products. The Analysis and Review of CMR (ARC) Team at Marshall Space Flight Center assesses the quality of these metadata records. Metadata, rather than the data itself, is indexed for search in both discipline-specific datacenters and global or aggregated catalogs (such as Earth data Search), making it essential for determining whether a data product is appropriate for a given research question or application need. Since metadata connects users to data, it should be as accurate and complete as possible in addition to meeting minimum database requirements. The ARC team has developed a metadata quality framework by which to assess quality. The framework consists of a set of quality criteria that converge around the dimensions of correctness, completeness, and consistency, with the goal of improving the discoverability, accessibility, and usability of NASA’s Earth Observation data. The application of the framework has resulted in a measurable improvement in NASA’s metadata quality. Key aspects of the framework’s success are the ability to systematically evaluate metadata and provide actionable quality improvement recommendations. Lessons learned from the project will be shared along with implementation details which may be relevant to other science disciplines. By aiming to make data more discoverable and accessible to a broad user community, the ARC metadata quality framework helps contribute to NASA’s commitment to open science.

Jeanne Le Roux↗
NASA NTRS2004

Comparing Methods for UAV-Based Autonomous Surveillance

We describe an approach to evaluating algorithmic and human performance in directing UAV-based surveillance. Its key elements are a decision-theoretic framework for measuring the utility of a surveillance schedule and an evaluation testbed consisting of 243 scenarios covering a well-defined space of possible missions. We apply this approach to two example UAV-based surveillance methods, a TSP-based algorithm and a human-directed approach, then compare them to identify general strengths, and weaknesses of each method.

Freed, Michael↗
NASA NTRS1979

Performability modeling with continuous accomplishment sets

A general modeling framework that permits the definition, formulation, and evaluation of performability is described. It is shown that performability relates directly to system effectiveness, and is a proper generalization of both performance and reliability. A hierarchical modeling scheme is used to formulate the capability function used to evaluate performability. The case in which performance variables take values in a continuous accomplishment set is treated explicitly.

Meyer, J. F.↗
NASA NTRS1989

Spray automated balancing of rotors - How process parameters influence performance

This paper addresses the application of spray-automated balancing of rotors, and the influence that various operating parameters will have on balancing performance. Spray-automated balancing uses the fuel-air repetitive explosion process to imbed short, discrete bursts of high velocity, high temperature powder into a rotating part at an angle selected to reduce unbalance of the part. The shortness of the burst, the delay in firing of the gun, the speed of the disk and the variability in speed all influence the accuracy and effectiveness of the automated balancing process. The paper evaluates this influence by developing an analytical framework and supplementing the analysis with empirical data obtained while firing the gun at a rotating disk. Encouraging results are obtained, and it is shown that the process should perform satisfactorily over a wide range of operating parameters. Further experimental results demonstrate the ability of the method to reduce vibration levels induced by mass unbalance in a rotating disk.

Smalley, A. J.↗