SEARCH · Engineering Papers

Results for “evaluation framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

All sources NASA NTRS DOE OSTI

NASA NTRS2023

Can Remotely Sensed Snow Disappearance Explain Seasonal Water Supply?

Understanding the relationship between remotely sensed snow disappearance and seasonal water supply may become vital in coming years to supplement limited ground based, in situ measurements of snow in a changing climate. For the period 2001–2019, we investigated the relationship between satellite derived Day of Snow Disappearance (DSD)—the date at which snow has completely disappeared—and the seasonal water supply, i.e., the April—July total streamflow volume, for 15 snow dominated basins across the western U.S. A Monte Carlo framework was applied, using linear regression models to evaluate the predictive skill—defined here as a model’s ability to accurately predict seasonal flow volumes—of varied predictors, including DSD and in situ snow water equivalent (SWE), across a range of spring forecast dates. In all basins there is a statistically significant relationship between mean DSD and seasonal water supply (p ≤ 0.05), with mean DSD explaining roughly half of the variance. Satellite-based model skill improves later in the forecast season, surpassing the skill of in-situ-based (SWE) models in skill in 10 of the 15 basins by the latest forecast date. We found little to no correlation between model error and basin characteristics such as elevation and the ratio of snow water equivalent to total precipitation. Despite a relatively short data record, this exploratory analysis shows promise for improving seasonal water supply prediction, in particular for snow dominated basins lacking in situ observations.

snow remote sensing↗

NASA NTRS2022

Remote Sensing of Aerosol Water Fraction, Dry Size Distribution and Soluble Fraction Using Multi-Angle, Multi-Spectral Polarimetry

A framework to infer volume water fraction, soluble fraction and dry size distributions of fine-mode aerosol from multi-angle, multi-spectral polarimetry retrievals of column-averaged ambient aerosol properties is presented. The method is applied to observations of the Research Scanning Polarimeter (RSP) obtained during two NASA aircraft campaigns, namely the Aerosol Cloud meTeorology Interactions oVer the western ATlantic Experiment (ACTIVATE) and the Cloud, Aerosol, and Monsoon Processes Philippines Experiment (CAMP2Ex). All aerosol retrievals are statistically evaluated using in situ data. Volume water fraction is inferred from the retrieved ambient real part of the refractive index, assuming a dry refractive index of 1.54 and by applying a volume mixing rule to obtain the effective ambient refractive index. The uncertainties in inferred volume water fraction resulting from this simplified model are discussed and estimated to be lower than 0.2 and decreasing with increasing volume water fraction. The daily mean retrieved volume water fractions correlate well with the in situ values with a mean absolute difference of 0.09. Polarimeter-retrieved ambient effective radius for daily data is shown to increase as a function of volume water fraction as expected. Furthermore, the effective variance of the size distributions also increases with increasing effective radius, which we show is consistent with an external mixture of soluble and insoluble aerosol. The relative variations of effective radius and variance over an observation period are then used to estimate the soluble fraction of the aerosol. Daily results of soluble fraction correlate well with in situ-observed sulfate mass fraction with a correlation coefficient of 0.79. Subsequently, inferred water and soluble fractions are used to derive dry fine-mode size distributions from their ambient counterparts. While dry effective radii obtained in situ and from RSP show similar ranges, in situ values are generally substantially smaller during the ACTIVATE deployments, which may be due to biases in RSP retrievals or in the in situ observations, or both. Both RSP and in situ observations indicate the dominance of aerosol with low hygroscopicity during the ACTIVATE and CAMP2Ex campaigns. Furthermore, RSP indicates a high degree of external mixing of particles with low and high hygroscopicity. These retrievals of fine-mode water volume fraction and soluble fraction may be used for the evaluation of water uptake in atmospheric models. Furthermore, the framework allows us to estimate the variation in the concentration of fine-mode aerosol larger than a specific dry radius limit, which can be used as a proxy for the variation in cloud condensation nucleus concentrations. This framework may be applied to multi-angle, multi-spectral satellite data expected to be available in the near future.

Amount of water in particulate matter↗

NASA NTRS2022

Remote Sensing of Aerosol Water Fraction, Dry Size Distribution and Soluble Fraction Using Multi-Angle, Multi-Spectral Polarimetry

A framework to infer volume water fraction, soluble fraction and dry size distributions of fine mode aerosol from multi-angle, multi-spectral polarimetry retrievals of column-averaged ambient aerosol properties is presented. The method is applied to observations of the Research Scanning Polarimeter (RSP) obtained during two NASA aircraft campaigns, namely the Aerosol Cloud meTeorology Interactions oVer the western ATlantic Experiment (ACTIVATE) and the Cloud, Aerosol, and Monsoon Processes-Philippines Experiment (CAMP2Ex). All aerosol retrievals are statistically evaluated using in situ data. Volume water fraction is inferred from the retrieved ambient real part of the refractive index, assuming a dry refractive index of 1.54 and by applying a volume mixing rule to obtain the effective ambient refractive index. The uncertainties in inferred volume water fraction resulting from this simplified model are discussed and estimated to be lower than 0.2 and decreasing with increasing volume water fraction. The daily mean retrieved volume water fractions correlate well with the in situ values with a mean absolute difference of 0.09. Polarimeter-retrieved ambient effective radius for daily data is shown to increase as a function of volume water fraction as expected. Furthermore, the effective variance of the size distributions also increases with increasing effective radius, which we show is consistent with an external mixture of soluble and insoluble aerosol. The relative variations of effective radius and variance over an observation period are then used to estimate the soluble fraction of the aerosol. Daily results of soluble fraction correlate well with in situ observed sulfate mass fraction with a correlation coefficient of 0.79. Subsequently, inferred water and soluble fractions are used to derive dry fine-mode size distributions from their ambient counterparts. While dry effective radii obtained in situ and from RSP show similar ranges, in situ values are generally substantially smaller during the ACTIVATE deployments, which may be due to biases in RSP retrievals or in the in situ observations, or both. Both RSP and in situ observations indicate the dominance of aerosol with low hygroscopicity during the ACTIVATE and CAMP2Ex campaigns. Furthermore, RSP indicates a high degree of external mixing of particles with low and high hygroscopicity. These retrievals of fine mode water volume fraction and soluble fraction may be used for the evaluation of water uptake in atmospheric models. Furthermore, the framework allows to estimate the variation in the concentration of fine-mode aerosol larger than a specific dry radius limit, which can be used as a proxy for the variation in cloud condensation nucleus concentrations. This framework may be applied to multi-angle, multi-spectral satellite data expected to be available in the near future.

Bastiaan Van Diedenhoven↗

NASA NTRS2019

Characteristics of Total Column CO2 Retrievals from the OCO Missions: Biases, Information Content and Implications for Flux Inversions

The Orbiting Carbon Observatory-2 and Orbiting Carbon Observatory-3, launched in 2015 and 2019, respectively, are intended to collect and deliver high-resolution observations of CO2 with unprecedented space and time coverage. Observations of CO2 from these remote-sensing missions (also known as XCO2, or column-based average, dry air mole fraction of CO2) are then used by the global carbon cycle community to answer a wide range of science questions, from the distribution and quantification of global and regional CO2 source-sink patterns to quantification of anthropogenic sources at urban scales. Even though we have had the OCO-2 mission flying for a few years now, the retrieval algorithms are continuously evolving and improving to deliver XCO2 retrievals with very high precision and high accuracy (or low biases). In this presentation, we will discuss a simple yet effective quantitative framework that has been developed by the OCO-2 flux team to evaluate the information content of these XCO2 retrievals as soon as they are released, i.e., with lower latency than full-scale flux inversions. This framework serves as a precursor to advanced inverse modeling frameworks and is intended to provide an early but accurate assessment of the signal present in the satellite retrievals, the robustness of that signal, and the ability of these retrievals to resolve patterns in CO2 surface fluxes that cannot be resolved by our current network of surface sites. Specific results will tackle a tiered set of questions that are being addressed using this framework: (a) what are the distribution of retrievals in the different modes of operation and how do they vary in space and time? (b) what is the information that is being given to the inverse modeling frameworks from the space-based data, information above and beyond what is provided by the in-situ data? and (c) how do these factors influence our choices for doing flux inversions with the satellite retrievals? While the primary focus of the results will be on application of this technique to mature OCO-2 retrievals, we will show early results for a couple of months of OCO-3 retrievals. For the time-period that the retrievals from the two missions overlap, we will highlight how this framework allows us to effortlessly put the information from OCO-3 and OCO-2 on an equal footing, thus enabling easy comparison between the two pioneering missions.

Chatterjee, Abhishek↗

NASA NTRSDate not supplied

A Machine Learning Framework for Error Compensation in Radiative Transfer Calculations

Radiative heat transfer influences the amount of heat flux transferred to the surface of the hypersonic vehicle, which is essential to evaluate the performance of thermal protection systems. The radiative heat flux is found to be computationally prohibitive while accounting for the variation in spatial, angular, and spectral domains. A new methodology has been recently developed to alleviate the cost of computation in the spectral domain by constructing flow-agnostic reduced-order models (ROMs). The developed spectral ROM databases provide grouping strategies that account for non-equilibrium absorption and emission as well as interaction between disparate species due to spectral overlap in associated radiative processes. However, the developed ROMs need to be optimized for a specific combination of interacting gas species and would need to re-calibrated in case individual species are added/omitted. In this work, we use various machine learning (ML) techniques to approximate the radiative intensities determined by a ROM optimized for a specific gas mixture. The ML model relies on the ROM databases developed for a single species which ignores any spectral overlap. Thus, radiation evaluation starts with a simple summation of radiative intensities predicted using these non-calibrated ROMs for the contributing species. The ML framework then provides a correction to account for the interplay in the frequency, i.e., emission of photons by one species and absorption by another, and yields mixture-specific radiation fields. Once trained on the individual ROM databases, the ML framework offers instantaneous corrections that serves as a time/cost effective alternative to the optimization of ROMs for a specific gas mixture. The ML framework is trained on both the high fidelity and ROM evaluated line of sight (LOS) data from Orion, Stardust, and FIRE II cases to obtain a general purpose correction model for earth re-entry scenarios when radiation contributions from both atomic nitrogen and atomic oxygen are considered. A geometric length scale parameter is used in the training process to account for errors introduced in the ROM databases as a consequence of high optical thickness. The efficacy of the ML framework is underscored through extensive analysis of train and test errors with respect to all the re-entry scenarios. The applicability of such an ML framework was further corroborated by embedding it in a state-of-the-art US3D - NERO system for determining the radiative heat flux transferred to the hypersonic vehicle surface.

Radiation↗

NASA NTRS2009

(Quickly) Testing the Tester via Path Coverage

The configuration complexity and code size of an automated testing framework may grow to a point that the tester itself becomes a significant software artifact, prone to poor configuration and implementation errors. Unfortunately, testing the tester by using old versions of the software under test (SUT) may be impractical or impossible: test framework changes may have been motivated by interface changes in the tested system, or fault detection may become too expensive in terms of computing time to justify running until errors are detected on older versions of the software. We propose the use of path coverage measures as a "quick and dirty" method for detecting many faults in complex test frameworks. We also note the possibility of using techniques developed to diversify state-space searches in model checking to diversify test focus, and an associated classification of tester changes into focus-changing and non-focus-changing modifications.

evaluation of test systems↗

NASA NTRS2019

A Double Bootstrap Approach to Superposed Epoch Analysis to Evaluate Response Uncertainty

The association between climate variability and episodic events, such as the antecedent moisture conditions prior to wildfire or the cooling following volcanic eruptions, is commonly assessed using Superposed Epoch Analysis (SEA). In SEA the epochal response is typically calculated as the average climate conditions prior to and following all event years or their deviation from climatology. However, the magnitude and significance of the inferred climate association may be sensitive to the selection or omission of individual key years, potentially resulting in a biased assessment of the relationship between these events and climate. Here we describe and test a modified double-bootstrap SEA that generates multiple unique draws of the key years and evaluates the sign, magnitude, and significance of event-climate relationships within a probabilistic framework. This multiple resampling helps quantify multiple uncertainties inherent in conventional applications of SEA within dendrochronology and paleoclimatology. We demonstrate our modified SEA by evaluating the volcanic cooling signal in a Northern Hemisphere tree-ring temperature reconstruction and the link between drought and wildfire events in the western United States. Finally, we make our Matlab and R code available to be adapted for future SEA applications.

Dendroclimatology↗

NASA NTRS2003

Rapid Prototyping of an Aircraft Model in an Object-Oriented Simulation

A team was created to participate in the Mars Scout Opportunity. Trade studies determined that an aircraft provided the best opportunity to complete the science objectives of the team. A high fidelity six degree of freedom flight simulation was required to provide credible evidence that the aircraft design fulfilled mission objectives and to support the aircraft design process by providing performance evaluations. The team created the simulation using the Langley Standard Real-Time Simulation in C++ (LaSRS++) application framework. A rapid prototyping approach was necessary because the team had only three months to both develop the aircraft simulation model and evaluate aircraft performance as the design and mission parameters matured. The design of LaSRS++ enabled rapid-prototyping in several ways. First, the framework allowed component models to be designed, implemented, unit-tested, and integrated quickly. Next, the framework provides a highly reusable infrastructure that allowed developers to maximize code reuse while concentrating on aircraft and mission specific features. Finally, the framework reduces risk by providing reusable components that allow developers to build a quality product with a compressed testing cycle that relies heavily on unit testing of new components.

Kenney, P. Sean↗

NASA NTRS2024

Automation of the ICME Workflow Incorporating Material Digital Twins at Different Length Scales Within a Robust Information Management System

Recent successes in Integrated Computational Materials Engineering (ICME) have demonstrated the potential in designing ‘fit-for-purpose’ materials for a given application in a cost and time efficient manner. However, the material design process must contain a level of judicious automation in the material decision process; that is implementing some optimization algorithms to truly enable the benefits of ICME, particularly when considering materials at multiple length and time scales. Furthermore, the ability to effectively store developed material models, experimental data used for validation, and link models at multiple length and time scales must be implemented to ensure traceability across the material design process, such that the data gathered can be leveraged towards efficient material design. To enable such an optimization scheme a robust framework must exist: (1) that can capture changes made at a given length scale, (2) automatically propagate changes upstream to the highest scale, and (3) evaluate the material’s performance at the structural level. In this work, a developed framework for tracking material changes, automatically running the necessary simulations to determine the properties at the next highest scale, and saving each iteration of the design process to maintain the application’s digital thread is presented for polymer matrix composites (PMCs). The Automated Information Management Across Organizations and Scales (AIMAOS) program offers users an interactive graphical user interface (GUI) for defining constituent materials, building lamina and laminates, and applying effective laminate properties to finite element and composite optimization third party software. At each length scale, the necessary input files are automatically written, and subsequent analysis tools are called to solve for effective properties at the next scale, which are then read by the AIMAOS tool and displayed to the user. As changes are made to the material at lower length scales, information is automatically propagated upstream to higher length scales, and changes made are automatically tracked and versioned to maintain traceability during the design process. The AIMAOS tools serves as the first step in enabling optimized design of composites from the nano to the macroscale for a given application.

Brandon L Hearley↗

NASA NTRS1977

Methodology for back-contamination risk assessment for a Mars sample return mission

The risk of back-contamination from Mars Surface Sample Return (MSSR) missions is assessed. The methodology is designed to provide an assessment of the probability that a given mission design and strategy will result in accidental release of Martian organisms acquired as a result of MSSR. This is accomplished through the construction of risk models describing the mission risk elements and their impact on back-contamination probability. A conceptual framework is presented for using the risk model to evaluate mission design decisions that require a trade-off between science and planetary protection considerations.

Merkhofer, M. W.↗

NASA NTRS1996

Understanding and Improving High-Performance I/O Subsystems

This research program has been conducted in the framework of the NASA Earth and Space Science (ESS) evaluations led by Dr. Thomas Sterling. In addition to the many important research findings for NASA and the prestigious publications, the program has helped orienting the doctoral research program of two students towards parallel input/output in high-performance computing. Further, the experimental results in the case of the MasPar were very useful and helpful to MasPar with which the P.I. has had many interactions with the technical management. The contributions of this program are drawn from three experimental studies conducted on different high-performance computing testbeds/platforms, and therefore presented in 3 different segments as follows: 1. Evaluating the parallel input/output subsystem of a NASA high-performance computing testbeds, namely the MasPar MP- 1 and MP-2; 2. Characterizing the physical input/output request patterns for NASA ESS applications, which used the Beowulf platform; and 3. Dynamic scheduling techniques for hiding I/O latency in parallel applications such as sparse matrix computations. This study also has been conducted on the Intel Paragon and has also provided an experimental evaluation for the Parallel File System (PFS) and parallel input/output on the Paragon. This report is organized as follows. The summary of findings discusses the results of each of the aforementioned 3 studies. Three appendices, each containing a key scholarly research paper that details the work in one of the studies are included.

El-Ghazawi, Tarek A.↗

NASA NTRS2004

Oxygen Profile of a Thermo-Haliophilic Community in the Badwater Salt Flat

Badwater is the lowest, saltiest, and hottest, place in North America. This extreme environment is 86 meters (282 feet) below sea level surrounded by four mountain ranges. Due to the geographical location Badwater does not receive moisture from the prevailing winds, this intern results in a hot, arid, and salty environment. Despite these extreme living conditions, microbes manage to flourish within the salt flat. The salt acts as an insulator making life just beneath the surface more comfortable In this paper, we compare the microbial population versus oxygen concentration; and the importance of and the role of oxygen in metabolic functions by these thermo-haliophiles. Furthermore a model of the oxygen profile will also provide an insight to the oxygen cycle in salty environments. This research has implications for the limits of life on Earth and Mars. Recent results from the MER rovers show that water on Mars was very salty. Measuring the oxygen profile in these salty environments on Earth provides a framework within which potential life on Mars can be evaluated. The use of an oxygen profile could also be used as a search criteria for life.

Smith, H. D.↗

NASA NTRSDate not supplied

Understanding thermal stability in doped zirconia aerogels for high temperature applications

The high porosities and low densities of ceramic aerogels offer outstanding insulative performance in applications where weight is a critical factor. The high surface-to-volume ratios and specific surface areas provide extremely low thermal conductivity, but also contribute to rapid densification of the pore structure at elevated temperatures. This densification diminishes their favorable properties and inhibits use of aerogels in high temperature applications. The purpose of this work is the establishment of a design framework for thermally stable, highly porous materials. Here we will review our recent studies on the effects of synthetic parameters and dopant chemistry on the evolution of zirconia-based aerogels. The structural evolution was studied to 1200 °C using nitrogen physisorption, scanning and transmission electron microscopy, and x-ray diffraction. The role of dopant identity and concentration in thermal stability was elucidated. In context of the design framework, dopant chemistry is an aggregate for many closely related material properties, each of which may contribute to aerogel structural evolution. To develop a truly predictive design framework for ceramic-based aerogels, systematic and comprehensive evaluation of thermodynamic and kinetic properties must be performed in conjunction with studies on structural evolution.

aerogel↗

NASA NTRS2023

Thermal Stability of M x O y Doped Zirconia Aerogels (M = Y, Yb, Gd, Ce, Ca) Studied Through 1200°C

The high porosities and low densities of ceramic aerogels offer outstanding insulative performance in applications where weight is a critical factor. The high surface area-to-volume ratios and specific surface areas provide extremely low thermal conductivity, but also contribute to rapid densification of the pore structure at elevated temperatures. This densification diminishes their favorable properties and inhibits use of aerogels in high temperature applications. This work contributes to a design framework for thermally stable aerogels via the study of dopant chemistry (Y, Yb, Gd, Ca, Ce) in zirconia aerogels. The structural evolution was studied to 1200°C using nitrogen physisorption, scanning electron microscopy, and x-ray diffraction. The role of dopant identity and concentration in thermal stability was elucidated. In context of the design framework, dopant chemistry is an aggregate for many closely related material properties, each of which may contribute to aerogel structural evolution. To develop a truly predictive design framework for ceramic-based aerogels, systematic and comprehensive evaluation of thermodynamic and kinetic properties must be performed in conjunction with studies on structural evolution.

aerogel↗

NASA NTRS1995

Property-Based Software Engineering Measurement

Little theory exists in the field of software system measurement. Concepts such as complexity, coupling, cohesion or even size are very often subject to interpretation and appear to have inconsistent definitions in the literature. As a consequence, there is little guidance provided to the analyst attempting to define proper measures for specific problems. Many controversies in the literature are simply misunderstandings and stem from the fact that some people talk about different measurement concepts under the same label (complexity is the most common case). There is a need to define unambiguously the most important measurement concepts used in the measurement of software products. One way of doing so is to define precisely what mathematical properties characterize these concepts regardless of the specific software artifacts to which these concepts are applied. Such a mathematical framework could generate a consensus in the software engineering community and provide a means for better communication among researchers, better guidelines for analysis, and better evaluation methods for commercial static analyzers for practitioners. In this paper, we propose a mathematical framework which is generic, because it is not specific to any particular software artifact, and rigorous, because it is based on precise mathematical concepts. This framework defines several important measurement concepts (size, length, complexity, cohesion, coupling). It is not intended to be complete or fully objective; other frameworks could have been proposed and different choices could have been made. However, we believe that the formalism and properties we introduce are convenient and intuitive. In addition, we have reviewed the literature on this subject and compared it with our work. This framework contributes constructively to a firmer theoretical ground of software measurement.

Briand, Lionel↗

NASA NTRS1997

Property-Based Software Engineering Measurement

Little theory exists in the field of software system measurement. Concepts such as complexity, coupling, cohesion or even size are very often subject to interpretation and appear to have inconsistent definitions in the literature. As a consequence, there is little guidance provided to the analyst attempting to define proper measures for specific problems. Many controversies in the literature are simply misunderstandings and stem from the fact that some people talk about different measurement concepts under the same label (complexity is the most common case). There is a need to define unambiguously the most important measurement concepts used in the measurement of software products. One way of doing so is to define precisely what mathematical properties characterize these concepts, regardless of the specific software artifacts to which these concepts are applied. Such a mathematical framework could generate a consensus in the software engineering community and provide a means for better communication among researchers, better guidelines for analysts, and better evaluation methods for commercial static analyzers for practitioners. In this paper, we propose a mathematical framework which is generic, because it is not specific to any particular software artifact and rigorous, because it is based on precise mathematical concepts. We use this framework to propose definitions of several important measurement concepts (size, length, complexity, cohesion, coupling). It does not intend to be complete or fully objective; other frameworks could have been proposed and different choices could have been made. However, we believe that the formalisms and properties we introduce are convenient and intuitive. This framework contributes constructively to a firmer theoretical ground of software measurement.

Briand, Lionel C.↗

NASA NTRS2022

A Framework for Dynamic Architecture and Functional Allocations for Increasing Airspace Autonomy

To enable scalability of air travel for use cases such as cargo delivery, it is anticipated that future air traffic operations will involve unmanned aircraft operated by remote pilots. Of particular interest are schemes where a small number of pilots operate a large number of vehicles, mitigating high cost and pilot shortage issues. Such architectures require increased levels of automation and supervisory control modes. They also require ensuring safe operations when the command and control link to the vehicle is degraded or lost completely, rendering the vehicle autonomous. To evaluate these variable and dynamic architectures, this paper will present a framework for decomposing the functions necessary to ensure safe, orderly, and expeditious air travel, assessing the agents in the system, and classifying the levels of autonomy. Then, an example allocation to agents of roles for the function of separation assurance is presented, highlighting the dependency of the allocation on three main factors: time criticality of a potential separation violation, the ratio of pilots to vehicles, and the loss of the command and control link.

air traffic management, function allocation, auton↗

NASA NTRS2022

A Framework for Dynamic Architecture and Functional Allocations for Increasing Airspace Autonomy

To enable scalability of air travel for use cases such as cargo delivery, it is anticipated that future air traffic operations will involve unmanned aircraft operated by remote pilots. Of particular interest are schemes where a small number of pilots operate a large number of vehicles, mitigating high cost and pilot shortage issues. Such architectures require increased levels of automation and supervisory control modes. They also require ensuring safe operations when the command and control link to the vehicle is degraded or lost completely, rendering the vehicle autonomous. To evaluate these variable and dynamic architectures, this paper will present a framework for decomposing the functions necessary to ensure safe, orderly, and expeditious air travel, assessing the agents in the system, and classifying the levels of autonomy. Then, an example allocation to agents of roles for the function of separation assurance is presented, highlighting the dependency of the allocation on three main factors; time criticality of a potential separation violation, the ratio of pilots to vehicles, and the loss of the command and control link.

air traffic management↗