Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “COMPARATIVE EVALUATIONS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Down-Selection of Four Common Habitat Variants

The Common Habitat is a large habitat that uses the Space Launch System core stage liquid oxygen tank as its primary structure. It has a gravity-independent internal architecture, such that identical units can be used on the lunar surface, Mars surface, and in microgravity. In developing the habitat, two key architectural questions emerged. Should the internal layout use a vertical or horizontal orientation of the tank? Should the crew size be four or eight? This led to the design of four variants: a four-crew horizontal, four-crew vertical, eight-crew horizontal, and eight-crew vertical. The primary consideration applied for down-selection was the crew experience living and working in the habitat, inclusive of crew productivity, well-being, and survivability. Based on this consideration, a series of seven assessments was performed to compare the four variants. A stowage assessment developed a standard logistics module and then considered the amounts of water to be stored in each variant. It then estimated how much stowage could be carried onboard each Common Habitat and how many logistics modules are required by each variant for a given mission duration. A functional analysis identified and compared the living and working functions across the four habitat, ranking them relative to each other. A crew time assessment first estimated the total crew time, building a weekly crew timeline for both four and eight-person crews. It then allocated time to activities linked to living and working functions, comparing how much time was available for each function in each variant. A science productivity assessment developed a relative metric using crew time, science stowage, and assumed rates of experiment consumables use to analytically compare the four variants. It also comparatively ranked the habitats with respect to a number of subjective parameters and a workstation acceptability rating. A maintenance capacity assessment identified and compared eleven generic maintenance capabilities across the four variants and also ranked the variants for their predicted ability to complete twelve fabrication, maintenance, and repair scenarios. A contingency responsiveness analysis examined twelve serious in-flight contingencies. For each scenario, the number of crew needed to respond was predicted and acceptability of various aspects of contingency response was evaluated, comparing the four variants against each other. Finally, in a habitability assessment, 120 habitability characteristics reflecting 13 major categories were evaluated for each habitat. These results were compared to identify the most acceptable habitat in each category. Ultimately, the data was shown to favor the horizontal orientation over the vertical and an eight-person crew over four. Implications of selecting this variant are discussed, including specific architectural challenges that result from the use of the full SLS liquid oxygen tank.

Habitability↗

Space storable propellant acquisition system

Surface tension propellant acquisition concepts for an advanced spacecraft propulsion system having a 10-year mission capability were investigated. Surface tension systems were specified because they were shown to be the best propellant acquisition technique for various interplanetery spacecraft in a prior study. A variety of surface tension concepts for accomplishing propellant acquisition were formulated for the baseline space storable propulsion module and Jupiter Orbiter mission. Analyses and evaluations were then conducted on each candidate concept to assess fabricability, performance capability, and spacecraft compatibility. A comparative evaluation of the results showed the Fruhof-class of low-g surface tension systems to be preferred for these interplanetary applications.

Tegart, J. R.↗

FATHOMS-RAG: A Framework for the Assessment of Thinking and Observation in Multimodal Systems that use Retrieval Augmented Generation

Retrieval-augmented generation (RAG) has emerged as a promising paradigm for improving factual accuracy in large language models (LLMs). We introduce a benchmark designed to evaluate RAG pipelines as a whole, evaluating a pipelines ability to ingest several modalities of information. We present (1) a curated dataset of 93 questions designed to evaluate a pipeline's ability to ingest textual data, tables, images, multimodal data, and cross-document multimodal data; (2) a phrase-level recall metric for correctness; (3) a nearest-neighbor embedding classifier in an attempt to classify pipeline hallucinations; (4) a comparative evaluation of 2 pipelines built with open-source retrieval mechanisms and 4 closed-source foundational models; and (5) a third-party human evaluation of the alignment of our correctness and hallucination metrics. We find that closed-source pipelines significantly outperform open-source pipelines in both the correctness and halucination metrics, with a wider performance gap in questions relying on multimodal and cross-document information. We also find after a human evaluation of our correctness and hallucination metric compared with our questions and pipeline responses, average agreement was 4.62 for correctness 4.53 for hallucination detection on a 1-5 Likert scale with 5 being strongly agree with our determination.

Hildebrand, Samuel [ORNL] (ORCID:0009000465963104)↗

Evaluation of tantalum 316 stainless steel transition joints

Tubular transition joints providing a metallurgically bonded connection between tantalum and 316 stainless steel pipe sections were comparatively evaluated for durability under thermal cycling conditions approximating the operation of a SNAP-8 mercury boiler. Both coextruded and vacuum brazed transition joints of 50mm (2 inch) diameter were tested by thermal cycling 100 times between 730 C and 120 C(1350 F and 250 F) in a high vacuum environment. The twelve evaluated transition joints survived the full test sequence without developing leaks, although liquid penetrant bond line indications eventually developed in all specimens. The brazed transition joints exhibited the best dimensional stability and bond line durability.

Stoner, D. R.↗

Total System Performance Ratio—A Systems Based Approach for Evaluating HVAC System Efficiency

The prescriptive path is the most widely used approach for commercial code compliance in the United States. Though easy to implement, prescriptive approaches do not typically discriminate between minimally compliant, high-performing and poorly performing HVAC system configurations. Hence, to meet aggressive energy and carbon reduction goals, it is clear that energy codes will need to transition from prescriptive to performance-based approaches, a transition that is riddled with several challenges. This paper discusses a new HVAC system-based performance approach (HVAC System Performance) which provides a simpler solution to HVAV system evaluation compared to whole building performance, while keeping tradeoffs limited to specific building systems. The Total System Performance Ratio (TSPR) is a metric for evaluation of overall system efficiency instead of individual component efficiency, a solution which could also eventually facilitate the transition to a 100% performance-based code structure. TSPR is a ratio that compares the annual heating and cooling load of a building to the annual energy consumed by the building’s HVAC system. A calculation software tool has been developed for determining a building’s TSPR. Already incorporated into the 2018 Washington State Energy Code, this approach is also being evaluated by ASHRAE Standard 90.l Project Committee and has the potential to provide a comprehensive performance-based approach for HVAC system evaluation and analysis.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

HVAC System Performance for Energy Codes (Technical Brief)

The prescriptive path is likely the most widely used approach for commercial code compliance in the United States. Though easy to implement, the prescriptive approach does not discriminate between high-performing and poorly performing heating, ventilation, air conditioning (HVAC) system configurations that are both minimally compliant. To meet aggressive energy and carbon reduction goals, energy codes will need to transition from prescriptive to performance-based approaches, a transition that is riddled with several challenges. HVAC System Performance is a discipline performance path and provides a simpler solution to HVAC system evaluation compared to whole building performance, while keeping tradeoffs limited to specific building systems. The Total System Performance Ratio (TSPR) is a metric for evaluation of overall system efficiency instead of individual component efficiency, a solution that could also eventually facilitate the transition to a 100% performance-based code structure. TSPR is a ratio that compares the annual heating and cooling load of a building to the annual energy consumed by the building’s HVAC system. A web-based calculation tool has been developed for determining a building’s TSPR. Already incorporated into the 2018 Washington State Energy Code, this approach has also been evaluated by the ASHRAE Standard 90.1 Project Committee and has the potential to provide a comprehensive performance-based approach for HVAC system evaluation and analysis

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Development of aircraft lavatory compartments with improved fire resistance characteristics. Phase 4: Sandwich panel decorative ink development

Five chemically different resin systems with improved fire resistance properties were studied for a possible screenprinting ink application. Fire resistance is hereby defined as the cured ink possessing improvements in flammability, smoke emission, and thermal stability. The developed ink is for application to polyvinyl fluoride film. Only clear inks without pigments were considered. Five formulations were evaluated compared with KC4900 clear acrylic ink, which was used as a baseline. The tests used in the screening evaluation included viscosity, smoke and toxic gas emission, limiting oxygen index (LOI), and polyvinyl fluoride film (PVF) printability. A chlorofluorocarbon resin (FPC461) was selected for optimization studies. The parameters for optimization included screenprinting process performance, quality of coating, and flammability of screenprinted 0.051-mm (0.002-in.) white Tedlar. The quality of the screenprinted coating on Tedlar is dependent on viscosity, curing time, adhesion to polyvinyl fluoride film, drying time (both inscreen and as an applied film), and silk screen mesh material and porosity.

Jayarajan, A.↗

Meta-analysis of North American Arctic and boreal aboveground biomass datasets: assessing accuracy, dynamics, and similarities

The North American arctic and boreal regions (ABRs) are rapidly warming and experiencing intensifying disturbances. Accurately quantifying aboveground biomass (AGB) is critical for understanding the impacts of these changes on the carbon cycle and for designing climate change mitigation strategies. Several AGB maps have been developed for the North American ABRs, including recent contributions from National Aeronautics and Space Administration’s Arctic-Boreal Vulnerability Experiment (ABoVE) campaign. However, these maps differ widely in training data, methodology, and resulting AGB density estimates. Presently, a comprehensive comparative evaluation is lacking, making it difficult for users to select datasets suited to their research or management needs. Here, in this study, we conducted a comparative analysis of nine AGB density datasets across North American ABRs, specifically for Alaska and Canada. We (1) summarized AGB by ecoregion and Canadian provinces, (2) evaluated their accuracy against field-based measurements, (3) analyzed spatial and temporal similarities among datasets, and (4) assessed their ability to capture disturbance (fire and harvest) impacts on AGB. We found substantial variation in regional and local AGB estimates across datasets, with overall accuracy ranging from R 2 = 0.25–0.62 and Bias% from −47.8% to 69.9% when validated against field plots. Despite these differences, most datasets have comparatively consistent spatial patterns in AGB (r > 0.8 for most cases). In contrast, agreement on the temporal patterns of AGB change is generally low. We found datasets with spatial resolutions ⩽300 m are capable of capturing disturbance impacts on AGB dynamics, though sensitivity varies across products. Our findings and dataset summary provide guidance for selecting appropriate AGB datasets for different applications within our study area. Our analysis also highlights the need to decrease map bias and increase capability to detect temporal change to decrease uncertainty of AGB datasets potentially by using training data which is representative of major plant functional types within the mapped area.

ABoVE↗

Evaluation of atmospheric density models and preliminary functional specifications for the Langley Atmospheric Information Retrieval System (LAIRS)

This document presents the results of an extensive survey and comparative evaluation of current atmosphere and wind models for inclusion in the Langley Atmospheric Information Retrieval System (LAIRS). It includes recommended models for use in LAIRS, estimated accuracies for the recommended models, and functional specifications for the development of LAIRS.

Lee, T.↗

Evaluation of a Dispatcher's Route Optimization Decision Aid to Avoid Aviation Weather Hazards

This document describes the results and analysis of the formal evaluation plan for the Honeywell software tool developed under the NASA AWIN (Aviation Weather Information) 'Weather Avoidance using Route Optimization as a Decision Aid' project. The software tool aims to provide airline dispatchers with a decision aid for selecting optimal routes that avoid weather and other hazards. This evaluation compares and contrasts route selection performance with the AWIN tool to that of subjects using a more traditional dispatcher environment. The evaluation assesses gains in safety, in fuel efficiency of planned routes, and in time efficiency in the pre-flight dispatch process through the use of the AWIN decision aid. In addition, we are interested in how this AWIN tool affects constructs that can be related to performance. The construct of Situation Awareness (SA), workload, trust in an information system, and operator acceptance are assessed using established scales, where these exist, as well as through the evaluation of questionnaire responses and subject comments. The intention of the experiment is to set up a simulated operations area for the dispatchers to work in. They will be given scenarios in which they are presented with stored company routes for a particular city-pair and aircraft type. A diverse set of external weather information sources is represented by a stand-alone display (MOCK), containing the actual historical weather data typically used by dispatchers. There is also the possibility of presenting selected weather data on the route visualization tool. The company routes have not been modified to avoid the weather except in the case of one additional route generated by the Honeywell prototype flight planning system. The dispatcher will be required to choose the most appropriate and efficient flight plan route in the displayed weather conditions. The route may be modified manually or may be chosen from those automatically displayed.

Dorneich, Michael C.↗

Evaluating long-term model-based scenarios of the energy system

Energy-economic models are used to provide science-based decision support in a variety of contexts. Analyses using these tools often involve defining a “reference” scenario, which serves as a counterfactual against which alternative scenarios are compared. Evaluating scenarios, including reference scenarios, is critical for establishing the credibility of the analyses these models support. We propose a framework for evaluating energy system scenarios which consists of three parts – a qualitative storyline, quantitative metrics, and evaluation criteria. We apply this framework to the reference scenario for GCAM-USA, a version of the global human-Earth system model GCAM (Global Change Assessment Model) with state-level detail in the United States, focusing on the evolution of the electric power sector. We develop new visual analytic tools to facilitate the evaluation of model outcomes in 51 sub-national regions, and demonstrate how scenario performance can be tracked and compared across four quantifications of the GCAM-USA reference scenario.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

A study of the comparative performance of six lightning warning systems

An analysis of mining explosive accident reports indicates a need for clear and ample warning of approaching electrical storms to reduce injuries/fatalities due to lightning induced premature detonation. Six lightning warning devices were comparatively evaluated at three test sites. The devices tested were: (1) flash counter, (2) corona point, (3) radioactive probe, (4) field mill, (5) azimuth/range locator, and (6) triangulation locator. The test sites were chosen to provide system evaluation under varied thunderstorm conditions. The devices were tested at San Antonio, Texas, Kennedy Space Center, Florida and Langmuir Laboratory, New Mexico. Comparative performance is reviewed in terms of alarm reliability, false alarm, and failure to alarm probabilities. Lightning warning system requirements and methods of relating warning criteria to explosive hazard are discussed.

Johnson, R. L.↗

Evaluating CO 2 mitigation strategies in SAF biorefineries: Techno-economic and life cycle analysis

The aviation sector requires scalable decarbonization strategies, and lignocellulosic sustainable aviation fuel (SAF) represents a promising pathway. This study comparatively evaluates the techno-economic analysis and life cycle assessment (LCA) of three CO 2 management strategies integrated within a U.S.-based gasification–Fischer–Tropsch SAF biorefinery: (i) catalytic hydrogenation of captured CO 2 to methanol, (ii) geological CO 2 sequestration, and (iii) mineralization to sodium bicarbonate (NaHCO 3 ). Techno-economic analysis indicates that methanol synthesis requires approximately 26% higher capital investment and 33% higher operating costs than mineralization. Although methanol co-production generates the highest gross revenue, NaHCO 3 production reduces the SAF minimum selling price by approximately 38% relative to both methanol synthesis and geological sequestration pathways, reflecting a more balanced cost allocation through mineral co-product valorization. Geological sequestration lowers operating costs by nearly 50% compared with methanol synthesis but remains highly dependent on carbon credit mechanisms. LCA reveals substantial divergence in climate performance. Relative to methanol synthesis, sequestration improves net greenhouse gas performance by approximately 163%, transitioning the system from net-positive to net-negative emissions. Mineralization further enhances carbon mitigation, achieving roughly 85% greater carbon reduction than sequestration and over sixfold improvement relative to methanol synthesis within the defined system boundary. Sensitivity analysis identified hydrogen price, co-product market value, and process emissions as dominant drivers. Under baseline assumptions, CO 2 mineralization is found to offer the most balanced pathway.

Carbon capture and storage↗

Sea based support aircraft alternatives

Multi-mission, CTOL, STOL and V/STOL aircraft concepts were sized for the Navy ASW/ASUW missions and comparatively evaluated. Initially, technology levels in the areas of lift, control systems, propulsion, advanced materials and avionics were established for the early 1990's. The aircraft were compared to determine the impact of multi-mission requirements, operational considerations, chiefly takeoff mode, as well as the technology advancements. Results presented include performance envelopes, weight statements, growth factors, impact of technology on weight reduction and the impact of design constraints.

Priestley, R. T.↗

Automated Plasma Spray (APS) process feasibility study

An automated plasma spray (APS) process was developed to apply two layer (NiCrAlY and ZrO2-12Y2O3) thermal barrier coatings to aircraft and stationary gas turbine engine blade airfoils. The APS process hardware consists of four subsystems: a mechanical positioning subsystem incorporating two interlaced six degree of freedom assemblies (one for coating deposition and one for coating thickness monitoring); a noncoherent optical metrology subsystem (for in process gaging of the coating thickness buildup at specified points on the specimen); a microprocessor based adaptive system controller (to achieve the desired overall thickness profile on the specimen); and commerical plasma spray equipment. Over fifty JT9D first stage aircraft turbine blade specimens, ten W501B utility turbine blade specimens and dozens of cylindrical specimens were coated with the APS process in preliminary checkout and evaluation studies. The best of the preliminary turbine blade specimens achieved an overall coating thickness uniformity of 53 micrometers (2.1 mils), much better than is achievable manually. Comparative evaluations of coating thickness uniformity for manually sprayed and APS coated specimens were performed. One of the preliminary turbine blade evaluation specimens was subjected to a torch test and metallographic evaluation. Some cylindrical specimens coated with the APS process survived up to 2000 cycles in subsequent burner rig testing.

Fetheroff, C. W.↗

Synergism of MODIS Aerosol Remote Sensing from Terra and Aqua

The MODerate-resolution Imaging Spectro-radiometer (MODIS) sensors, aboard the Earth Observing System (EOS) Terra and Aqua satellites, are showing excellent competence at measuring the global distribution and properties of aerosols. Terra and Aqua were launched on December 18, 1999 and May 4, 2002 respectively, with daytime equator crossing times of approximately 10:30 am and 1:30 pm respectively. Several aerosol parameters are retrieved at 10-km spatial resolution from MODIS daytime data over land and ocean surfaces. The parameters retrieved include: aerosol optical thickness (AOT) at 0.47, 0.55 and 0.66 micron wavelengths over land, and at 0.47, 0.55, 0.66, 0.87, 1.2, 1.6, and 2.1 microns over ocean; Angstrom exponent over land and ocean; and effective radii, and the proportion of AOT contributed by the small mode aerosols over ocean. Since the beginning of its operation, the quality of Terra-MODIS aerosol products (especially AOT) have been evaluated periodically by cross-correlation with equivalent data sets acquired by ground-based (and occasionally also airborne) sunphotometers, particularly those coordinated within the framework of the AErosol Robotic NETwork (AERONET). Terra-MODIS AOT data have been found to meet or exceed pre-launch accuracy expectations, and have been applied to various studies dealing with local, regional, and global aerosol monitoring. The results of these Terra-MODIS aerosol data validation efforts and studies have been reported in several scientific papers and conferences. Although Aqua-MODIS is still young, it is already yielding formidable aerosol data products, which are also subjected to careful periodic evaluation similar to that implemented for the Terra-MODIS products. This paper presents results of validation of Aqua-MODIS aerosol products with AERONET, as well as comparative evaluation against corresponding Terra-MODIS data. In addition, we show interesting independent and synergistic applications of MODIS aerosol data from both Terra and Aqua. In certain situations, this combined analysis of Terra- and Aqua-MODIS data offers an insight into the diurnal cycle of aerosol loading.

Ichoku, Charles↗

Comparative clumped isotope temperature relationships in freshwater carbonates

Lacustrine, riverine and spring carbonates represent archives of terrestrial climates and their geochemistry has been used to study palaeoenvironments. Clumped isotope thermometry is an emerging tool that has been applied to freshwater carbonates. Limited work has been done to evaluate comparative relationships between clumped isotopes and temperature in different types of modern freshwater carbonates. This study assembles an extensive calibration data set with 135 samples of modern freshwater carbonates from 96 sites and constrains the relationship between independent observations of water temperature and the clumped isotopic composition of carbonates (denoted by Δ 47 ), including new measurements, and recalculates published data in accordance with current community-defined standard values. For temperature reconstruction, the study reports a composite freshwater calibration and material-specific calibrations for biogenic carbonates (freshwater gastropods and bivalves), fine-grained carbonate (e.g. micrites), biologically mediated carbonates (microbialites and tufas) and travertines. Material-specific calibration trends show a convergence of slopes that are in agreement with recently published syntheses, but statistically significant differences in intercepts occur between some materials (e.g. some biogenics, fine-grained carbonates). These differences may arise due to unresolved seasonal biases, kinetic isotope effects and/or varying degrees of biological influence. The impact of different calibrations is shown through application to new data for glacial and deglacial age travertines from Austria and published data sets. While material-specific calibrations may yield more accurate results for biogenic and fine-grained carbonate samples, the use of material-specific and the composite freshwater calibrations generally produces values within 1.0–1.5°C of each other, and typically fall within calibration uncertainty given limitations of precision.

54 ENVIRONMENTAL SCIENCES↗

Monte Carlo Event Generation with Continuous Normalizing Flows

We apply continuous normalizing flows trained with the flow matching method to the problem of phase-space sampling in Monte Carlo event generation for high-energy collider physics. Focusing on lepton-pair and top-quark pair production with multiple jets, the two computationally most expensive processes at the Large Hadron Collider, we train helicity-conditioned continuous normalizing flows to remap the random numbers used in matrix element evaluation. Compared to standard methods, we achieve unweighting efficiency improvements by factors of up to 184 and 25 for the two processes at their respective highest jet number, at the cost of an increased evaluation time. When combining the advantages of continuous normalizing flows with the fast evaluation times of coupling-layer-based flows, using the RegFlow approach, we find parton-level unweighted event generation walltime gains of about a factor of 10 at the highest jet numbers. These substantial gains highlight the promise of samplers based on machine learning for next-generation collider experiments.

Bothmann, Enrico [CERN; Gottingen U.] (ORCID:00000↗