Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “COMPARATIVE EVALUATIONS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
NASA NTRS2009

Benchmarking Diagnostic Algorithms on an Electrical Power System Testbed

Diagnostic algorithms (DAs) are key to enabling automated health management. These algorithms are designed to detect and isolate anomalies of either a component or the whole system based on observations received from sensors. In recent years a wide range of algorithms, both model-based and data-driven, have been developed to increase autonomy and improve system reliability and affordability. However, the lack of support to perform systematic benchmarking of these algorithms continues to create barriers for effective development and deployment of diagnostic technologies. In this paper, we present our efforts to benchmark a set of DAs on a common platform using a framework that was developed to evaluate and compare various performance metrics for diagnostic technologies. The diagnosed system is an electrical power system, namely the Advanced Diagnostics and Prognostics Testbed (ADAPT) developed and located at the NASA Ames Research Center. The paper presents the fundamentals of the benchmarking framework, the ADAPT system, description of faults and data sets, the metrics used for evaluation, and an in-depth analysis of benchmarking results obtained from testing ten diagnostic algorithms on the ADAPT electrical power system testbed.

Kurtoglu, Tolga↗
NASA NTRS1978

Sensitivity of 30-cm mercury bombardment ion thruster characteristics to accelerator grid design

The design of ion optics for bombardment thrusters strongly influences overall performance and lifetime. The operation of a 30-cm thruster with accelerator grid open area fractions ranging from 43 to 24 percent, was evaluated and compared with previously published experimental and theoretical results. Ion optics properties measured included the beam current extraction capability, the minimum accelerator grid voltage to prevent backstreaming, ion beamlet diameter as a function of radial position on the grid and accelerator grid hole diameter, and the high energy, high angle ion beam edge location. Discharge chamber properties evaluated were propellant utilization efficiency, minimum discharge power per beam amp, and minimum discharge voltage.

Rawlin, V. K.↗
NASA NTRS2002

Wake Vortex Algorithm Scoring Results

This report compares the performance of two models of trailing vortex evolution for which interaction with the ground is not a significant factor. One model uses eddy dissipation rate (EDR) and the other uses the kinetic energy of turbulence fluctuations (TKE) to represent the effect of turbulence. In other respects, the models are nearly identical. The models are evaluated by comparing their predictions of circulation decay, vertical descent, and lateral transport to observations for over four hundred cases from Memphis and Dallas/Fort Worth International Airports. These observations were obtained during deployments in support of NASA's Aircraft Vortex Spacing System (AVOSS). The results of the comparisons show that the EDR model usually performs slightly better than the TKE model.

Robins, R. E.↗
NASA NTRS2023

Impact of Design Factors for ESA CCI Satellite Soil Moisture Data Assimilation over Europe

In this study, soil moisture retrievals of the combined active–passive ESA Climate Change Initiative (CCI) soil moisture product are assimilated into the Noah-MP land surface model over Europe using a one-dimensional ensemble Kalman filter and an 18-yr study period. The performance of the data assimilation (DA) system is evaluated by comparing it with a model-only experiment (at in situ sites) and by assessing statistics of innovations and increments as DA diagnostics (over the entire domain). For both assessments, we explore the impact of three design choices, resulting in the following insights. 1) The magnitude of the assumed observation errors strongly affects the skill improvements evaluated against in situ stations and internal diagnostics. 2) Choosing between climatological or monthly cumulative distribution function matching as the observation bias correction method only has a marginal effect on the in situ skill of the DA system. However, the internal diagnostics suggest a more robust system parameterization if the observations are rescaled monthly. 3) The choice of atmospheric reanalysis dataset to force the land surface model affects the model-only skill and the DA skill improvements. The model-only skill is higher with input from the MERRA-2 than with input from the ERA5 reanalysis, resulting in larger DA skill improvements for the latter. Additionally, we show that the added value of the DA strongly depends on the quality of the satellite retrievals and land cover, with the most substantial soil moisture skill improvements occurring over croplands and skill degradation occurring over densely forested areas.

Soil moisture↗
NASA NTRS1982

Characterization of advanced electric propulsion systems

Characteristics of several advanced electric propulsion systems are evaluated and compared. The propulsion systems studied are mass driver, rail gun, MPD thruster, hydrogen free radical thruster and mercury electron bombardment ion engine. These are characterized by specific impulse, overall efficiency, input power, average thrust, power to average thrust ratio and average thrust to dry weight ratio. Several important physical characteristics such as dry system mass, accelerator length, bore size and current pulse requirement are also evaluated in appropriate cases. Only the ion engine can operate at a specific impulse beyond 2000 sec. Rail gun, MPD thruster and free radical thruster are currently characterized by low efficiencies. Mass drivers have the best performance characteristics in terms of overall efficiency, power to average thrust ratio and average thrust to dry weight ratio. But, they can only operate at low specific impulses due to large power requirements and are extremely long due to limitations of driving current. Mercury ion engines have the next best performance characteristics while operating at higher specific impulses. It is concluded that, overall, ion engines have somewhat better characteristics as compared to the other electric propulsion systems.

Ray, P. K.↗
NASA NTRS1994

Accuracy metrics for judging time scale algorithms

Time scales have been constructed in different ways to meet the many demands placed upon them for time accuracy, frequency accuracy, long-term stability, and robustness. Usually, no single time scale is optimum for all purposes. In the context of the impending availability of high-accuracy intermittently-operated cesium fountains, we reconsider the question of evaluating the accuracy of time scales which use an algorithm to span interruptions of the primary standard. We consider a broad class of calibration algorithms that can be evaluated and compared quantitatively for their accuracy in the presence of frequency drift and a full noise model (a mixture of white PM, flicker PM, white FM, flicker FM, and random walk FM noise). We present the analytic techniques for computing the standard uncertainty for the full noise model and this class of calibration algorithms. The simplest algorithm is evaluated to find the average-frequency uncertainty arising from the noise of the cesium fountain's local oscillator and from the noise of a hydrogen maser transfer-standard. This algorithm and known noise sources are shown to permit interlaboratory frequency transfer with a standard uncertainty of less than 10(exp -15) for periods of 30-100 days.

Douglas, R. J.↗
NASA NTRS2011

Using Technical Performance Measures

All programs have requirements. For these requirements to be met, there must be a means of measurement. A Technical Performance Measure (TPM) is defined to produce a measured quantity that can be compared to the requirement. In practice, the TPM is often expressed as a maximum or minimum and a goal. Example TPMs for a rocket program are: vacuum or sea level specific impulse (lsp), weight, reliability (often expressed as a failure rate), schedule, operability (turn-around time), design and development cost, production cost, and operating cost. Program status is evaluated by comparing the TPMs against specified values of the requirements. During the program many design decisions are made and most of them affect some or all of the TPMs. Often, the same design decision changes some TPMs favorably while affecting other TPMs unfavorably. The problem then becomes how to compare the effects of a design decision on different TPMs. How much failure rate is one second of specific impulse worth? How many days of schedule is one pound of weight worth? In other words, how to compare dissimilar quantities in order to trade and manage the TPMs to meet all requirements. One method that has been used successfully and has a mathematical basis is Utility Analysis. Utility Analysis enables quantitative comparison among dissimilar attributes. It uses a mathematical model that maps decision maker preferences over the tradeable range of each attribute. It is capable of modeling both independent and dependent attributes. Utility Analysis is well supported in the literature on Decision Theory. It has been used at Pratt & Whitney Rocketdyne for internal programs and for contracted work such as the J-2X rocket engine program. This paper describes the construction of TPMs and describes Utility Analysis. It then discusses the use of TPMs in design trades and to manage margin during a program using Utility Analysis.

Garrett, Christopher J.↗
NASA NTRS1992

Dynamically-allocated multi-queue buffers for VLSI communication switches

Several buffer structures are discussed and compared in terms of implementation complexity, interswitch handshaking requirements, and their ability to deal with variations in traffic patterns and message lengths. A new design of buffers is presented that provide non-FIFO message handling and efficient storage allocation for variable size packets using linked lists managed by a simple on-chip controller. The new buffer design is evaluated by comparing it to several alternative designs in the context of a multistage interconnection network. The present modeling and simulations show that the new buffer outperforms alternative buffers and can thus be used to improve the performance of a wide variety of systems currently using less efficient buffers.

Tamir, Yuval↗
DOE OSTI2022

A Comparative Study of Machine Learning Algorithms for Industry-Specific Freight Generation Model

According to Bureau of Transportation Statistics, the U.S. transportation system handled 14,329 million ton-miles of freight per day in 2020. Understanding the generation of these freight shipments is crucial for transportation researchers, planners, and policymakers to design and plan for a more efficient and connected freight transportation system. Traditionally, the freight generation modeling has been based on Ordinary Least Square (OLS) regression, although more advanced Machine Learning (ML) algorithms have been evaluated and proven to have excellent performance in various transportation applications in recent years. Furthermore, one modeling approach applied for one industry might not always be applicable for another as their freight generation logics can be quite different. The objective of this study is to apply and evaluate alternative ML algorithms in the estimation of freight generation for each of 45 industry types. Seven alternative ML algorithms, along with the base OLS regression, were evaluated and compared. In addition, the study considered different combinations of variables in both the original and logarithmic form as well as hyperparameters of those ML algorithms in the model selection for each industry type. The results showed statistically significant improvements in the root mean square error reduction by the alternative ML algorithms over the OLS for over 80% of cases. The study suggests utilizing the alternative ML algorithms can reduce the root mean square error by about 30%, depending on industry types.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Learning curves for drug response prediction in cancer cell lines

Motivated by the size and availability of cell line drug sensitivity data, researchers have been developing machine learning (ML) models for predicting drug response to advance cancer treatment. As drug sensitivity studies continue generating drug response data, a common question is whether the generalization performance of existing prediction models can be further improved with more training data. We utilize empirical learning curves for evaluating and comparing the data scaling properties of two neural networks (NNs) and two gradient boosting decision tree (GBDT) models trained on four cell line drug screening datasets. The learning curves are accurately fitted to a power law model, providing a framework for assessing the data scaling behavior of these models. The curves demonstrate that no single model dominates in terms of prediction performance across all datasets and training sizes, thus suggesting that the actual shape of these curves depends on the unique pair of an ML model and a dataset. The multi-input NN (mNN), in which gene expressions of cancer cells and molecular drug descriptors are input into separate subnetworks, outperforms a single-input NN (sNN), where the cell and drug features are concatenated for the input layer. In contrast, a GBDT with hyperparameter tuning exhibits superior performance as compared with both NNs at the lower range of training set sizes for two of the tested datasets, whereas the mNN consistently performs better at the higher range of training sizes. Moreover, the trajectory of the curves suggests that increasing the sample size is expected to further improve prediction scores of both NNs. These observations demonstrate the benefit of using learning curves to evaluate prediction models, providing a broader perspective on the overall data scaling characteristics. A fitted power law learning curve provides a forward-looking metric for analyzing prediction performance and can serve as a co-design tool to guide experimental biologists and computational scientists in the design of future experiments in prospective research studies.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Environmental life cycle assessment methods applied to amine-ionic liquid hybrid CO 2 absorbents

A hybrid solvent mixture of triethyl(octyl)phosphonium cyanopyrrolide [P2228][2-CNPyr] and aqueous monoethanolamine (MEA) has the potential for absorbing CO 2 from post combustion flue gas. However, previous studies have found that the production of phosphonium based ionic liquids (IL) had significantly higher potential environmental impacts compared to MEA. Literature attributes these higher environmental impacts to the phosphine and phosgene-based intermediates required to produce the phosphonium ion of the ionic liquid. This study proposes a novel synthesis pathway that eliminates the need for phosphine and phosgene intermediates in the production of [P2228][2-CNPyr]. The environmental impacts of producing 1kg of the ionic liquid through this novel synthesis route was evaluated using the TRACI 2.1 methodology within the life cycle assessment (LCA) framework. Additionally, the environmental impacts for the production of 1kg of a hybrid solvent was also evaluated and compared against MEA. The life cycle inventory for the production of the IL and its hybrid solvent were calculated based on the stoichiometry and then scaled up. This study found that the IL and its hybrid solvents had higher environmental impacts among 9 of the 10 environmental impact categories calculated by the TRACI 2.1 methodology, except for the ecotoxicity potential. A sensitivity analysis indicated that these solvents were more sensitive to the assumptions of the material requirements of the phosphonium cation than the overall energy or transportation requirements. Despite this sensitivity, both the solvents demonstrated a lower Ecotoxicity Potential compared to MEA, the rest of the environmental impacts were still found to be higher than that of MEA, thereby underscoring the need to investigate novel synthesis routes for the production of phosphonium cation. The uncertainty analysis performed confirmed the findings that the IL has a higher environmental impact potentials across all categories except ecotoxicity potential. The uncertainty analysis also confirms that the phosphonium cation is a major hotspot in production route of these solvents and a source of uncertainty in the model compared to the anion. Altogether, this study underscores the need for investigating novel green chemistry pathway for the synthesis of phosphonium based ionic liquids, such as [P2228][2-CNPyr], to ensure that the these ILs can be a truly green alternative to MEA by not only offering superior CO₂ capture capacity compared to MEA but also being sustainably produced.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS2009

Science Operations for the 2008 NASA Lunar Analog Field Test at Black Point Lava Flow, Arizona

Surface science operations on the Moon will require merging lessons from Apollo with new operation concepts that exploit the Constellation Lunar Architecture. Prototypes of lunar vehicles and robots are already under development and will change the way we conduct science operations compared to Apollo. To prepare for future surface operations on the Moon, NASA, along with several supporting agencies and institutions, conducted a high-fidelity lunar mission simulation with prototypes of the small pressurized rover (SPR) and unpressurized rover (UPR) (Fig. 1) at Black Point lava flow (Fig. 2), 40 km north of Flagstaff, Arizona from Oct. 19-31, 2008. This field test was primarily intended to evaluate and compare the surface mobility afforded by unpressurized and pressurized rovers, the latter critically depending on the innovative suit-port concept for efficient egress and ingress. The UPR vehicle transports two astronauts who remain in their EVA suits at all times, whereas the SPR concept enables astronauts to remain in a pressurized shirt-sleeve environment during long translations and while making contextual observations and enables rapid (less than or equal to 10 minutes) transfer to and from the surface via suit-ports. A team of field geologists provided realistic science scenarios for the simulations and served as crew members, field observers, and operators of a science backroom. Here, we present a description of the science team s operations and lessons learned.

Garry W. D.↗
NASA NTRS1991

A Comparative Study of High and Low Fidelity Fan Models for Turbofan Engine System Simulation

In this paper, a heterogeneous propulsion system simulation method is presented. The method is based on the formulation of a cycle model of a gas turbine engine. The model includes the nonlinear characteristics of the engine components via use of empirical data. The potential to simulate the entire engine operation on a computer without the aid of data is demonstrated by numerically generating "performance maps" for a fan component using two flow models of varying fidelity. The suitability of the fan models were evaluated by comparing the computed performance with experimental data. A discussion of the potential benefits and/or difficulties in connecting simulations solutions of differing fidelity is given.

Reed, John A.↗
NASA NTRS1999

A Goal Seeking Strategy for Constructing Systems from Alternative Components

This paper describes a methodology to efficiently construct feasible systems then modify feasible systems to meet successive goals by selecting from alternative components, a problem recognized to be n-p complete. The methodology provides a means to catalog and model alternative components. A presented system modeling Structure is robust enough to model a wide variety of systems and provides a means to compare and evaluate alternative systems. These models act as input to a methodology for selecting alternative components to construct feasible systems and modify feasible systems to meet design goals and objectives. The presented algorithm's ability to find a restricted solution, as defined by a unique set of requirements, is demonstrated against an exhaustive search of a sample of proposed shuttle modifications. The utility of the algorithm is demonstrated by comparing results from the algorithm with results from three NASA shuttle evolution studies using their value systems and assumptions.

Valentine, Mark E.↗
NASA NTRS2011

Incorporating Ameriflux Data into LVT

This paper describes a new generic data reader that was developed in Fortran to handle the Ameriflux data for the LIS Verification Toolkit (LVT). Researchers at the Hydrological Sciences Branch of NASA Goddard Space Flight Center have created a high resolution land surface modeling and data assimilation system known as the Land Information System (LIS), which provides an infrastructure to integrate state-of-the-art land surface models, data assimilation algorithms, observations of land surface from satellite and remotely sensed platforms to provide estimates of land surface conditions such as soil moisture, evaporation, snowpack and runoff. These model predictions are typically evaluated by comparing them with data from observational networks. The observational data; however, are usually available in disparate data formats and require significant effort to process them into a structure amenable for use with the model data. The motivation to develop a uniform approach for land surface verification as a way to alleviate these processing efforts has led to the development of LVT which is designed to enable the rapid evaluation of land surface modeling and analysis products from LIS. LVT focuses on the use of observational datasets in their native format. As the formats of these datasets vary widely, a major part of LVT is creating programs to read and process the native datasets. The primary goal of this project is to enhance LVT capabilities by incorporating observational datasets from Ameriflux

Georgiev, Teodor↗
DOE OSTI2023

The June 2012 North American Derecho: A Testbed for Evaluating Regional and Global Climate Modeling Systems at Cloud‐Resolving Scales

Abstract In this paper, we introduce a testbed for evaluating and comparing climate modeling systems at cloud resolving scales using hindcasts of the June 2012 North American derecho. To demonstrate its utility for model intercomparison, the testbed is applied to two models: the regionally‐refined Simple Cloud‐Resolving E3SM Atmosphere Model (SCREAM) at 6.5, 3.25 and 1.625 km grid spacing and the Weather Research and Forecasting (WRF) model with 3.2 and 1.6 km grid spacing. We find the simulation results to be highly sensitive to the initial conditions (ICs), initialization time, and model configurations, with ICs from the Rapid Refresh producing the best simulation. Significant improvement is identified in both models as horizontal grid spacing is refined. While a propagation delay of approximately 2 hr is found in both models, SCREAM at 1.625 km simulates the observed bow echo structure of the derecho well and predicts strong surface gusts that exceed 30 m/s. In comparison, WRF has difficulty producing surface wind over 25 m/s, with wind gusts in WRF 42%–46% lower than in SCREAM. However, WRF has a lower bias in simulating cloud top temperature and extent, but overestimates precipitation intensity. Both models reproduce the observed outgoing longwave radiation spatial patterns well (Pearson correlation >0.88), but, compared with NEXRAD observations, simulate generally larger areas of composite radar reflectivity >40 dBZ and underestimate the precipitating area by ∼47%.

2012 North American derecho↗
NASA NTRS2001

Evaluation of Cirrus Cloud Simulations Using ARM Data - Development of a Case Study Data Set

Cloud-resolving models (CRMs) provide an effective linkage in terms of parameters and scales between observations and the parametric treatments of clouds in global climate models (GCMs). They also represent the best understanding of the physical processes acting to determine cloud system lifecycle. The goal of this project is to improve state-of-the-art CRMs used for studies of cirrus clouds and to establish a relative calibration with GCMs through comparisons among CRMs, single column model (SCM) versions of the GCMs, and observations. This project will compare and evaluate a variety of CRMs and SCMs, under the auspices of the GEWEX Cloud Systems Study (GCSS) Working Group on Cirrus Cloud Systems (WG2), using ARM data acquired at the Southern Great Plains (SGP) site. This poster will report on progress in developing a suitable WG2 case study data set based on the September 26, 1996 ARM IOP case - the Hurricane Nora outflow case. The environmental data (input) will be described as well as the wealth of validating cloud observations. We plan to also show results of preliminary simulations. The science questions to be addressed derive significantly from results of the GCSS WG2 cloud model comparison projects, which will be briefly summarized.

O'C.Starr, David↗
DOE OSTI2020

A Comprehensive Monte Carlo Framework for Jet-Quenching

This article presents the motivation for developing a comprehensive modeling framework in which different models and parameter inputs can be compared and evaluated for a large range of jet-quenching observables measured in relativistic heavy-ion collisions at RHIC and the LHC. Here, the concept of a framework is discussed within the context of recent efforts by the JET Collaboration, the authors of JEWEL, and the JETSCAPE collaborations. The framework ingredients for each of these approaches is presented with a sample of important results from each. The role of advanced statistical tools in comparing models to data is also discussed, along with the need for a more detailed accounting of correlated errors in experimental results.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗