Results for “qualitative evaluation” — Page 17

NASA NTRS2008

A Three-Dimensional Analytic Model for the Scattering of a Spherical Bush

Advanced climate models require a more realistic description of canopy radiation with reasonable computational efficiency. This paper develops the mathematics of scattering from a spherical object conceptualized to be a spherical bush to provide a building block that helps to address this need of climate models. It is composed of a homogeneous distribution of individual smaller objects that scatter isotropically. In the limit of small optical depth, incident radiation will scatter isotropically as the sum of that scattered by all the individual scatterers, but at large optical depth the radiation leaving the spherical bush in a given direction is reduced by mutual shadowing of the smaller objects. In the single scattering limit, the scattering phase function and so the albedo are obtained by simple but accurate analytic expressions derived from analytic integration and numerical evaluation. Except in the limit of thin canopies, the scattering and hence albedos are qualitatively and quantitatively different than those derived from 1-D modeling.

Dickinson, Robert E.↗

NASA NTRS2011

The 2010 Desert Rats Science Operations Test: Outcomes and Lessons Learned

The Desert RATS 2010 Team tested a variety of science operations management techniques, applying experience gained during the manned Apollo missions and the robotic Mars missions. This test assessed integrated science operations management of human planetary exploration using real-time, tactical science operations to oversee daily crew science activities, and a night shift strategic science operations team to conduct strategic level assessment of science data and daily traverse results. In addition, an attempt was made to collect numerical metric data on the outcome of the science operations to assist test evaluation. The two most important outcomes were 1) the production of significant (almost overwhelming) volume of data produced during daily traverse operations with two rovers, advanced imaging systems and well trained, scientifically proficient crew-members, and 2) the degree to which the tactical team s interaction with the surface crew enhanced science return. This interaction depended on continuous real-time voice and data communications, and the quality of science return from any human planetary exploration mission will be based strongly on the aggregate interaction between a well trained surface crew and a dedicated science operations support team using voice and imaging data from a planet s surface. In addition, the scientific insight developed by both the science operations team and the crews could not be measurable by simple numerical quantities, and its value will be missed by a purely metric-based evaluation of test outcome. In particular, failure to recognize the critical importance of this qualitative type interaction may result in mission architecture choices that will reduce science return.

Eppler, D. B.↗

NASA NTRS2018

Toward Long-Term Aquatic Science Products from Heritage Landsat Missions

This paper aims at generating a long-term consistent record of Landsat-derived remote sensing reflectance (Rrs) products, which are central for producing downstream aquatic science products (e.g., concentrations of total suspended solids). The products are derived from Landsat-5 and Landsat-7 observations leading to Landsat-8 era to enable retrospective analyses of inland and nearshore coastal waters. In doing so, the data processing was built into the SeaWiFS Data Analysis System (SeaDAS) followed by vicariously calibrating Landsat-7 and -5 data using reference in situ measurements and near-concurrent ocean color products, respectively. The derived Rrs products are then validated using (a) matchups using the Aerosol Robotic Network (AERONET) data measured by in situ radiometers, i.e., AERONET-OC, and (b) ocean color products at select sites in North America. Following the vicarious calibration adjustments, it is found that the overall biases in Rrs products are significantly reduced. The root-mean-square errors (RMSE), however, indicate noticeable uncertainties due to random and systematic noise. Long-term (since 1984) seasonal Rrs composites over 12 coastal and inland systems are further evaluated to explore the utility of Landsat archive processed via SeaDAS. With all the qualitative and quantitative assessments, it is concluded that with careful algorithm developments, it is possible to discern natural variability in historic water quality conditions using heritage Landsat missions. This requires the changes in Rrs exceed maximum expected uncertainties, i.e., 0.0015 [1/sr], estimated from mean RMSEs associated with the matchups and intercomparison analyses. It is also anticipated that Landsat-5 products will be less susceptible to uncertainties in turbid waters with Rrs(660) > 0.004 [1/sr], which is equivalent of ~1.2% reflectance. Overall, end-users may utilize heritage Rrs products with "fitness-for-purpose" concept in mind, i.e., products could be valuable for one application but may not be viable for another. Further research should be dedicated to enhancing atmospheric correction to account for non-negligible near-infrared reflectance in CDOM-rich and extremely turbid waters.

Pahlevan, Nima↗

NASA NTRS1981

Remote sensing techniques for conservation and management of natural vegetation ecosystems

The importance of using remote sensing techniques, in the visible and near-infrared ranges, for mapping, inventory, conservation and management of natural ecosystems is discussed. Some examples realized in Brazil or other countries are given to evaluate the products from orbital platform (MSS and RBV imagery of LANDSAT) and aerial level (photography) for ecosystems study. The maximum quantitative and qualitative information which can be obtained from each sensor, at different level, are discussed. Based on the developed experiments it is concluded that the remote sensing technique is a useful tool in mapping vegetation units, estimating biomass, forecasting and evaluation of fire damage, disease detection, deforestation mapping and change detection in land-use. In addition, remote sensing techniques can be used in controling implantation and planning natural/artificial regeneration.

Parada, N. D. J.↗

NASA NTRS1967

Qualitative effects on safety and construction materials of 1300 deg F sodium-potassium spray in air and nitrogen

Shielding material evaluation for resistance to sodium potassium fires in air and nitrogen

POTASSIUM↗

DOE OSTI2025

A detailed study of pre-heating effects in electron beam melting powder bed fusion process

Metal-based additive manufacturing processes, such as powder bed fusion with electron beam (PBF-EB) process, also referred to as electron beam melting (EBM), can produce high-density parts with minimal residual stresses due to the uniform and coherent preheating of the powder bed. However, understanding and controlling the multiple stages of preheating is required to enable the production of high-quality, consistent parts of various materials. This work presents a large-scale, multi-layer, three-dimensional numerical analysis focused on studying the preheating stages for predicting thermal history during the PBF-EB process. The model follows a continuous multi-stage cyclic process, that incorporates all the main stages of the PBF-EB process for 316 L stainless steel. This includes the gradual deposition of a new powder layer, the first and second preheating levels of the powder bed, and the energy deposition during melting (excluding the actual melt-pool behavior simulation). The model employs an adaptive time-scaling approach that automatically adjusts the energy deposition for each solution time-increment. This allows for localized changes in time-resolution over an otherwise computationally expensive multi-layer procedure. The material property variations are also taken into account, with an emphasis on the subtle irreversible changes in powder effective thermal conductivity after the two requisite preheating stages of the powder bed. This effect is studied using simplified conductivity models from the literature for partially sintered powder, validated by a dedicated experiment and numerical simulation. The large-scale model is then used to estimate the actual temperatures during first and second preheating levels for 316 L steel, which is not yet fully supported commercially for PBF-EB. Model predictions are corroborated by experiments, using and analyzing IR images, taken at the completion of each layer by the machine’s built-in infrared camera. The current model also incorporates a qualitative assessment for the effects of conductivity change during pre-heating, as well as evaluates the applicability of the time-scaling approach.

36 MATERIALS SCIENCE↗

DOE OSTI2022

Approach and Model Used to Represent a Timeline Analysis for Security Design Enhancements

Next-generation reactors will be able to use risk to inform and performance base the licensing of many aspects of the reactor, facility, and site design, including attributes of physical security. There are several factors related to security, including site topography, reactor design, and physical protection system components, to consider when designing the physical protection system into the overall facility design and plan of operation. With the versatility of advanced reactors, especially micro reactors, methods are needed to simplify and quickly evaluate potential timelines for designing a site configuration. This report describes an approach to generate qualitative and quantitative insights using a risk-informed simulation. The modeling process is described in detail, focusing on three aspects: (1) the facility mission time (the time required to control the plant until safe), (2) the attacker timeline (the time to potential sabotage), and (3) the response timeline (the time to counter the facility attack). The modeling capabilities also are extended to include facility physical phenomena such as thermal-hydraulics and heat transfer to capture realistic representation of dynamic changes to a facility. While the plant models and examples are hypothetical and do not represent a real facility, these modeling approaches could be used for future security-by-design engineering in advanced reactors. The outputs and insights from the modeling approach may be used to modify and optimize the security posture of a facility by efficiently making modifications to the model and seeing the overall impact from the modification. Lastly, use of the approach described in this report can also provide the technical basis for a physical protection program, describing how the facility and security strategy will cope with off-normal events.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

NASA NTRS2007

Preliminary Evaluation of GEOS-5 Aerosol and CO Distributions Forecast During TC4

The NASA Tropical Composition, Cloud, and Climate Coupling (TC4) Mission was based out San Jose, Costa Rica during July and August 2007. During TC4 the NASA Global Modeling and Assimilation Office (GMAO) ran twice-daily 0.5 x 0.666 global 5-day forecasts of the Goddard Earth Observing System atmospheric general circulation model and data assimilation system (GEOS-5). This implementation of GEOS-5 contained an aerosol and carbon monoxide (CO) model to provide online forecast tropospheric distributions of dust, sea salt, sulfate, and carbonaceous aerosols and CO for both the planning of flights and for science. Here we provide a description of the aerosol and CO modeling system and give a preliminary evaluation of forecast tracer distributions. Our comparisons to satellite observations of aerosol and CO show qualitatively similar simulated distributions of tracers to those observed. During TC4 copious amounts of dust were observed in the Caribbean. The model generally reproduced the observations of the timing of dust events and the vertical structure in the lower atmosphere. However, the model simulations had too much aerosol at high altitudes relative to airborne Cloud Physics Lidar observations. The results were similar for biomass burning aerosol and CO tracers, where the model showed higher simulated concentrations of these tracers at aircraft flight altitude than observations.

Colarco, Peter R.↗

NASA NTRS2018

Evaluation of Multiple Flow Constrained Area Capacity Setting Methods for Collaborative Trajectory Options Program

The purpose of this study was to compare flow constrained area (FCA) capacity setting methods for Collaborative Trajectory Options Program (CTOP) as they pertain to the Integrated Demand Management (IDM) concept. IDM uses flow balancing to manage air traffic across multiple FCAs with a common downstream constraint, as well as constraints at the respective FCA locations. FCA capacity rates can be set manually, but generating capacities for multiple, interdependent FCAs could potentially over-burden a user. A new enhancement to CTOP called the FCA Balance Algorithm (FBA) was developed at NASA Ames Research Center to improve the process of allocating capacity across multiple flow constrained segments in the airspace. The FBA evaluates the predicted demand and capacity across multiple FCAs and dynamically generates capacity settings for the FCAs that best meet capacity limits for all identified constraints. In a human-in-the-loop simulation study, both manual and automated capacity setting methods were evaluated in terms of their overall feasibility using measures of system performance, human performance, and qualitative feedback. Subject matter experts were asked to use three different methods to allocate capacity to three FCAs, either (1) by manually setting capacity for every 60-minute time window, (2) by manually setting capacity for every 15-minute time window, or (3) by using the FBA capability to automatically generate capacity settings. Results showed no significant differences in terms of overall system performance, indicated by similar ground delay and airport throughput numbers between methods. However, differences in individual strategies afforded by the manual methods allowed some participants to achieve system-wide delay that was much lower than the average. The FBA was the fastest method of capacity setting, and it received the lowest subjective rating scores on physical task load, mental task load, task difficulty and task complexity out of the three methods. Finally, participants explained through qualitative feedback that there were many benefits to using the FBA, such as ease of use, accuracy, and low risk of human input error. Participants did not experience the same limitations with the FBA that they did with the manual methods, such as reduced accuracy in the 60-minute manual condition, or high complexity in the 15-minute/manual condition. These results suggest that the FBA automation enhancement to CTOP maintains system performance while improving human performance. Therefore, the FBA could be introduced as a way to mitigate operator workload while planning a CTOP.

NextGen↗

NASA NTRS2018

Evaluation of Multiple Flow Constrained Area Capacity Setting Methods for Collaborative Trajectory Options Program

The purpose of this study was to compare flow constrained area (FCA) capacity setting methods for Collaborative Trajectory Options Program (CTOP) as they pertain to the Integrated Demand Management (IDM) concept. IDM uses flow balancing to manage air traffic across multiple FCAs with a common downstream constraint, as well as constraints at the respective FCA locations. FCA capacity rates can be set manually, but generating capacities for multiple, interdependent FCAs could potentially over-burden a user. A new enhancement to CTOP called the FCA Balance Algorithm (FBA) was developed at NASA Ames Research Center to improve the process of allocating capacity across multiple flow constrained segments in the airspace. The FBA evaluates the predicted demand and capacity across multiple FCAs and dynamically generates capacity settings for the FCAs that best meet capacity limits for all identified constraints. In a human-in-the-loop simulation study, both manual and automated capacity setting methods were evaluated in terms of their overall feasibility using measures of system performance, human performance, and qualitative feedback. Subject matter experts were asked to use three different methods to allocate capacity to three FCAs, either (1) by manually setting capacity for every 60-minute time window, (2) by manually setting capacity for every 15-minute time window, or (3) by using the FBA capability to automatically generate capacity settings. Results showed no significant differences in terms of overall system performance, indicated by similar ground delay and airport throughput numbers between methods. However, differences in individual strategies afforded by the manual methods allowed some participants to achieve system-wide delay that was much lower than the average. The FBA was the fastest method of capacity setting, and it received the lowest subjective rating scores on physical task load, mental task load, task difficulty and task complexity out of the three methods. Finally, participants explained through qualitative feedback that there were many benefits to using the FBA, such as ease of use, accuracy, and low risk of human input error. Participants did not experience the same limitations with the FBA that they did with the manual methods, such as reduced accuracy in the 60-minute manual condition, or high complexity in the 15-minute/manual condition. These results suggest that the FBA automation enhancement to CTOP maintains system performance while improving human performance. Therefore, the FBA could be introduced as a way to mitigate operator workload while planning a CTOP.

NextGen↗

NASA NTRSDate not supplied

Qualitative Data Coding of User Experience with an Urban Air Mobility Fleet Manager Interface

The NASA Aeronautics Research Mission Directorate created the High Density Vertiplex (HDV) project to integrate and evaluate a prototype Urban Air Mobility (UAM) ecosystem. Part of the HDV testing environment included a prototype operator user interface called the Fleet Manager Interface (FMI). In 2023, HDV conducted flight testing with the FMI during which a user experience (UX) study was performed to assess the quality of UX and elicit design recommendations. As a result, a large database of open-ended, qualitative responses was generated and then coded using a new qualitative data coding technique called Directive String Coding, which used a blended coding approach to generate actionable heuristics that stakeholders (e.g., researchers, UI designers, software developers) could use to answer specific research questions and make future design decisions. The key themes that arose from the coded responses showed that the UX was pleasant, system notifications should be more salient, and information across multiple screens should be integrated into a central display. As the Fleet Manger operational role is still being defined, it is vital to increase our understanding of the tools and capabilities needed for such a role. The results from HDV work will eventually feed into standards for vertiport operations that will enable safe and scalable UAM operations in the United States.

high density vertiplex↗

NASA NTRSDate not supplied

Qualitative Data Coding of User Experience with an Urban Air Mobility Fleet Manager Interface

The NASA Aeronautics Research Mission Directorate created the High Density Vertiplex (HDV) project to integrate and evaluate a prototype Urban Air Mobility (UAM) ecosystem. Part of the HDV testing environment included a prototype operator user interface called the Fleet Manager Interface (FMI). In 2023, HDV conducted flight testing with the FMI during which a user experience (UX) study was performed to assess the quality of UX and elicit design recommendations. As a result, a large database of open-ended, qualitative responses was generated and then coded using a new qualitative data coding technique called Directive String Coding, which used a blended coding approach to generate actionable heuristics that stakeholders (e.g., researchers, UI designers, software developers) could use to answer specific research questions and make future design decisions. The key themes that arose from the coded responses showed that the UX was pleasant, system notifications should be more salient, and information across multiple screens should be integrated into a central display. As the Fleet Manger operational role is still being defined, it is vital to increase our understanding of the tools and capabilities needed for such a role. The results from HDV work will eventually feed into standards for vertiport operations that will enable safe and scalable UAM operations in the United States.

high density veriplex↗

DOE OSTI2020

Data Collection and Analysis Challenges and Mitigation Strategies for Quantitative Human Factors Research Studies in Nuclear Power Plant Modernization

The United States (U.S.) Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) program Plant Modernization Pathway is conducting targeted research and development (R&D) to address aging and reliability concerns with the legacy instrumentation and control and related information systems of the U.S. LWR fleet. In this effort, the application of human factors engineering (HFE) provides an important role in ensuring new digital plant technologies enable broad innovation and business improvement with continued operational safety. Evaluation is a key activity in HFE, which often occurs iteratively through the system design lifecycle. While qualitative methods are important in collecting information of how users perform tasks through observations, quantitative methods are equally important in assessing system design based on performance. In collecting and analyzing this quantitative data, there are notable challenges in the nuclear HFE domain that may threaten the validity and reliability of the inferences made in these studies. Notable challenges include small sample size and limited resources, large error variance and small effect size, an “adding test features to losing degrees of freedom” dilemma, non-normal distribution, and heterogeneity of variance. The results in control room usability studies are often statistically non-significant, which makes it hard to interpret. This work discusses these challenges across different scientific viewpoints and provides real-world examples of these challenges in practice. Collectively, the objective of this work is to position these challenges to the larger data science community as a means of identifying future opportunities to address these issues.

99 GENERAL AND MISCELLANEOUS↗

DOE OSTI2024

Godiva IV Simulated Radiation Field Characterization and Variance Reduction

Godiva IV is a system comprised of highly enriched uranium alloyed with molybdenum in the form of fuel plate rings. The reactor, along with its predecessors, was designed with the unique ability to satisfy interests in the super-prompt-critical reactor operation space. Originally, the reactor was part of the Los Alamos Critical Experiments Facility (LACEF) at Technical Area-18 (TA-18). The radiation field around Godiva at this facility was well characterized and understood. As a fast neutron system, the neutron spectrum in and around Godiva was close to a Watt Fission spectrum. The Kiva where Godiva IV was located at LACEF was made of thin, sheet metal walls which did not contribute significantly to the neutron spectrum. Following the transition of LACEF to the National Critical Experiments and Research Center (NCERC) in Nevada, Godiva-IV was moved from TA-18 to the Device Assembly Facility (DAF) at the Nevada National Security Site (NNSS). Part of this move brought renewed interest in radiation field characterization. The new facility introduced significant changes to the environment surrounding Godiva, and preliminary foil irradiation results suggested that the room contribution to the neutron spectrum was significant. Unlike at TA-18, a large thermal neutron signature was added to the fast spectrum from Godiva due to significant room return. A primary goal due to the additional complexity that the room return adds to the Godiva IV radiation emission spectrum was the development of an efficient Monte Carlo N-Particle (MCNP) calculation capable of characterizing the neutron spectrum anywhere in the room around Godiva. A campaign of activation foil irradiations and analysis were completed to support the validation of the MCNP model. The modeling of these foils in MCNP can be easily done with a standard volumetric neutron flux tally. However, given the multitude of locations and reaction rates to be modeled, further steps must be taken to increase the efficiency of these calculations in MCNP. During this study, a benchmark model currently under development for Godiva IV was used. A qualitative assessment of the thermal neutron contributors was performed using spatial neutron distribution plots. Additional detail was added to the model based on the qualitative results showing the thermal spectrum’s large sensitivity to hydrogenous material. Neutron energy spectra was evaluated at discrete locations in the room around Godiva to quantify the relative contribution of various components. It was discovered that the concrete walls are the largest contributor to the thermal signature, with minor contributions from plastic components surrounding Godiva. Following these results, two different variance reduction techniques were implemented to improve the problem efficiency in these calculations. In the first approach, an F5 point detector tally was implemented in the standard Godiva IV criticality problem. The second approach involved a weight-window generator implementation with an F5 point detector tally in a fixed source problem. The weight window implementation reduced the runtime from 42739.55 minutes to 1803.34 minutes (computer time), compared to the F5 KCODE implementation.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

NASA NTRS1992

Infrared thermographic analysis of polymer composites during ballistic impact

An infrared thermographic system was used to measure the surface temperature profile of composite materials under ballistic impact. The heat deposited was calculated based on the temperature profiles. The extent of damage induced in the materials was determined qualitatively from the measured data. The relative contribution of frictional heat to total energy absorption was evaluated, and the effect of the shape of the projectile on the extent of damage and temperature distribution in the composites was examined. The results show that graphite and polyethylene composite are more effective than Kevlar composite in dissipating heat during the ballistic impact penetration process due to their higher thermal conductivity.

Hsieh, C. Y.↗

NASA NTRS2018

Space Science and Technology Partnership Forum: Analysis for a Joint Demonstration of High Priority, In-Space Assembly Technology

The various U. S. government agencies that are pursuing in-space assembly technology have a common need to demonstrate technological capabilities on a space-based platform. Several of the agencies, and different mission developers within an agency, have independently begun planning such demonstrations. This paper reports on a study of how well the different planned platforms could support demonstrations of the agencies’ joint needs. The study first prioritized a comprehensive list of the needs for in-space assembly capabilities across the agencies against jointly agreed evaluation criteria. Each planned demonstration platform was characterized to a first order. The capability needs were qualitatively assessed against four figures of merit including their joint priority, and the platforms were assessed against five criteria to produce a quantitative weighting factor of reach capability need and each platform. A Quality Function Deployment (QFD) matrix was used to deploy the weighted capability needs against the weighted platforms capabilities. This first-order assessment showed that the platforms reflect a great deal of redundant capability without a strong reason to prefer one over the others. These results were largely insensitive to the details of the assumptions.

Hamill, Doris L↗

NASA NTRS1981

Evaluation of SLAR and thematic mapper MSS data for forest cover mapping using computer-aided analysis techniques

Training and test data sets for CAM1S from NS-001 MSS data for two dates (geometrically adjusted to 30 meter resolution) were used to evaluate wavelength band. Two sets of tapes containing digitized HH and HV polarization data were obtained. Because the SAR data on the 9 track tapes contained no meaningful data, the 7 track tapes were copied onto 9 track tapes at LARS. The LARSYS programs were modified and a program was written to reformat the digitized SAR data into a LARSYS format. The radar imagery is being qualitatively interpreted. Results are to be used to identify possible cover types, to produce a classification map to aid in the numerical evaluation classification of radar data, and to develop an interpretation key for radar imagery. The four spatial resolution data sets were analyzed. A program was developed to reduce the spatial distortions resulting from variable viewing distance, and geometrically adjusted data sets were generated. A flowchart of steps taken to geometrically adjust a data set from the NS-001 scanner is presented.

Hoffer, R. M.↗

DOE OSTI2023

Evaluating energy justice metrics in early-stage science and technology research using the JUST-R metrics framework

Embedding principles of energy justice throughout all aspects of clean energy technology research and development (R&D) can facilitate a more just energy transition; yet gaps remain in our understanding of how to best integrate energy justice from the earliest R&D stages. The Justice Underpinning Science and Technology Research (JUST-R) metrics framework has been developed to enable early-stage energy researchers to assess and address justice considerations associated with their research, but the impacts of the framework, and others like it, have yet to be evaluated. This study seeks to evaluate the JUST-R metrics framework in terms of its effectiveness and appeal to researchers engaged in early-stage technical R&D using qualitative analyses of documents and workshop transcripts. We find that the metrics framework helps researchers identify problems and potential solutions surrounding the energy justice implications of their work and spurs a change in perspective for researchers, but, simultaneously, there is no evidence of solution follow-through within the evaluation timeframe. Greater institutional support, specialization to research areas, knowledge of energy justice fundamentals, and earlier incorporation of energy justice considerations in research projects arise as factors needed to aid continued use of the framework and pursuit of identified solutions. This evaluation protocol and these findings can serve as a guide for improving other frameworks with similar goals of encouraging sociotechnical engagement in early-stage energy R&D.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗