Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluations”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Corroborative Evaluation of the Real-World Energy Saving Potentials of InfoRich Eco-Autonomous Driving (iREAD) System

There has been an increasing interest in exploring the potential to reduce energy consumption of future connected and automated vehicles. People have extensively studied various eco-driving implementations that leverage preview information provided by on-board sensors and connectivity, as well as the control authority enabled by automation. Quantitative real-world evaluation of eco-driving benefits is a challenging task. The standard regulatory driving cycles used for measuring exhaust emissions and fuel economy are not truly representative of real-world driving, nor for capturing how connectivity and automation might influence driving trajectories. To adequately consider real-world driving behavior and potential “off-cycle” impacts, this paper presents four collaborative evaluation methods: large-scale simulation, in-depth simulation, vehicle-in-the-loop testing, and vehicle road testing. These four approaches, spanning simulation and testing aspects, evaluate real-world fuel economy benefits with different ranges and resolutions. The large-scale simulations leverage an extensive real-world driving database to assess overall eco-driving benefits across a range of road network and driving scenarios. The real-world driving data are further leveraged to generate representative driving routes for deeper evaluation. Based on the representative routes, in-depth simulation relying on high-fidelity models investigates how different traffic scenarios can impact the eco-driving performance. The vehicle-in-the-loop setup reinforces the in-depth simulations by conducting tests with an actual vehicle operated on a chassis dynamometer; the measured energy savings were indeed found to agree with the in-depth simulation savings estimates. Finally, limited but representative road testing with the fully integrated vehicle will be conducted to demonstrate the eco-driving capability and conclude the overall evaluation regimen.

ARPA-E↗
DOE OSTI2021

Technical note: AQMEII4 Activity 1: evaluation of wet and dry deposition schemes as an integral part of regional-scale air quality models

We present in this technical note the research protocol for phase 4 of the Air Quality Model Evaluation International Initiative (AQMEII4). This research initiative is divided into two activities, collectively having three goals: (i) to define the current state of the science with respect to representations of wet and especially dry deposition in regional models, (ii) to quantify the extent to which different dry deposition parameterizations influence retrospective air pollutant concentration and flux predictions, and (iii) to identify, through the use of a common set of detailed diagnostics, sensitivity simulations, model evaluation, and reduction of input uncertainty, the specific causes for the current range of these predictions. Activity 1 is dedicated to the diagnostic evaluation of wet and dry deposition processes in regional air quality models (described in this paper), and Activity 2 to the evaluation of dry deposition point models against ozone flux measurements at multiple towers with multiyear observations (to be described in future submissions as part of the special issue on AQMEII4). The scope of this paper is to present the scientific protocols for Activity 1, as well as to summarize the technical information associated with the different dry deposition approaches used by the participating research groups of AQMEII4. In addition to describing all common aspects and data used for this multi-model evaluation activity, most importantly, we present the strategy devised to allow a common process-level comparison of dry deposition obtained from models using sometimes very different dry deposition schemes. The strategy is based on adding detailed diagnostics to the algorithms used in the dry deposition modules of existing regional air quality models, in particular archiving diagnostics specific to land use–land cover (LULC) and creating standardized LULC categories to facilitate cross-comparison of LULC-specific dry deposition parameters and processes, as well as archiving effective conductance and effective flux as means for comparing the relative influence of different pathways towards the net or total dry deposition. This new approach, along with an analysis of precipitation and wet deposition fields, will provide an unprecedented process-oriented comparison of deposition in regional air quality models. Examples of how specific dry deposition schemes used in participating models have been reduced to the common set of comparable diagnostics defined for AQMEII4 are also presented.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

The E3SM Diagnostics Package (E3SM Diags v2.7): a Python-based diagnostics package for Earth system model evaluation

Abstract. The E3SM Diagnostics Package (E3SM Diags) is a modern, Python-based Earth system model (ESM) evaluation tool (with Python module name e3sm_diags), developed to support the Department of Energy (DOE) Energy Exascale Earth System Model (E3SM). E3SM Diags provides a wide suite of tools for evaluating native E3SM output, as well as ESM data on regular latitude–longitude grids, including output from Coupled Model Intercomparison Project (CMIP) class models. E3SM Diags is modeled after the National Center for Atmospheric Research (NCAR) Atmosphere Model Working Group (AMWG, 2022) diagnostics package. In its version 1 release, E3SM Diags included a set of core essential diagnostics to evaluate the mean physical climate from model simulations. As of version 2.7, more process-oriented and phenomenon-based evaluation diagnostics have been implemented, such as analysis of the quasi-biennial oscillation (QBO), the El Niño–Southern Oscillation (ENSO), streamflow, the diurnal cycle of precipitation, tropical cyclones, ozone and aerosol properties. An in situ dataset from DOE's Atmospheric Radiation Measurement (ARM) program has been integrated into the package for evaluating the representation of simulated cloud and precipitation processes. This tool is designed with enough flexibility to allow for the addition of new observational datasets and new diagnostic algorithms. Additional features include customizable figures; streamlined installation, configuration and execution; and multiprocessing for fast computation. The package uses an up-to-date observational data repository maintained by its developers, where recent datasets are added to the repository as they become available. Finally, several applications for the E3SM Diags module were introduced to fit a diverse set of use cases from the scientific community.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Evaluating wind profiles in a numerical weather prediction model with Doppler lidar

We use Doppler lidar wind profiles from six locations around the globe to evaluate the wind profile forecasts in the boundary layer generated by the operational global Integrated Forecast System (IFS) from the European Centre for Medium-range Weather Forecasts (ECMWF). The six locations selected cover a variety of surfaces with different characteristics (rural, marine, mountainous urban, and coastal urban). We first validated the Doppler lidar observations at four locations by comparison with co-located radiosonde profiles to ensure that the Doppler lidar observations were of sufficient quality. The two observation types agree well, with the mean absolute error (MAE) in wind speed almost always less than 1 m s –1 . Large deviations in the wind direction were usually only seen for low wind speeds and are due to the wind direction uncertainty increasing rapidly as the wind speed tends to zero. Time–height composites of the wind evaluation with 1 h resolution were generated, and evaluation of the model winds showed that the IFS model performs best over marine and coastal locations, where the mean absolute wind vector error was usually less than 3 m s –1 at all heights within the boundary layer. Larger errors were seen in locations where the surface was more complex, especially in the wind direction. For example, in Granada, which is near a high mountain range, the IFS model failed to capture a commonly occurring mountain breeze, which is highly dependent on the sub-grid-size terrain features that are not resolved by the model. The uncertainty in the wind forecasts increased with forecast lead time, but no increase in the bias was seen. At one location, we conditionally performed the wind evaluation based on the presence or absence of a low-level jet diagnosed from the Doppler lidar observations. The model was able to reproduce the presence of the low-level jet, but the wind speed maximum was about 2 m s –1 lower than observed. This is attributed to the effective vertical resolution of the model being too coarse to create the strong gradients in wind speed observed. Our results show that Doppler lidar is a suitable instrument for evaluating the boundary layer wind profiles in atmospheric models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Rapid Evaluation Framework for the CMIP7 Assessment Fast Track

As Earth system models (ESMs) grow in complexity and in volume of output data, there is an increasing need for rapid, comprehensive evaluation of their scientific performance. The upcoming Assessment Fast Track for the Seventh Phase of the Coupled Model Intercomparison Project (CMIP7) will require expeditious response for model analyses designed to inform and drive integrated Earth system assessments. To meet this challenge, the Rapid Evaluation Framework (REF), a community-driven platform for benchmarking and performance assessment of ESMs, was designed and developed. The initial implementation of the REF, constructed to meet the near-term needs of the CMIP7 Assessment Fast Track, builds upon four disparate community evaluation and benchmarking tools that are coupled together using the Coordinated Model Evaluation Capabilities (CMEC) framework. The REF runs within a containerized workflow for portability and reproducibility and is aimed at generating and organizing diagnostics covering a variety of model variables. The REF leverages well documented observational datasets to provide assessments of model fidelity across a collection of diagnostics. All diagnostics were identified and selected with community involvement and consultation. Operational integration with the Earth System Grid Federation (ESGF) will permit automated execution of the REF for selected diagnostics as soon as model output data are published on ESGF by the originating modeling centers. The REF is designed to be portable across a range of current computational platforms to facilitate use by modeling centers for assessing the evolution of model versions or gauging the relative performance of CMIP simulations before being published on ESGF. When integrated into production simulation workflows, results from the REF provide immediate quantitative feedback that allows model developers and scientists to quickly identify model biases and performance issues. After the REF is released to the community, its subsequent development and support will be prioritized by an international consortium of scientists and engineers, enabling a broader impact across Earth science disciplines. For instance, the REF will facilitate improvements to models and will enhance confidence in model projections through process-based selection of models based on their performance with respect to observations. Production of reproducible diagnostics and community-based assessments are key features of the REF. Furthermore, providing interoperability with existing evaluation packages assures that contributions from previous community efforts will be available for use in future model intercomparison projects.

Hoffman, Forrest [ORNL] (ORCID:0000000158024134)↗
DOE OSTI2023

Test and Evaluation of Systems with Embedded Machine Learning Components

As Machine Learning (ML) continues to advance, it is being integrated into more systems. Often, the ML component represents a significant portion of the system that reduces the burden on the end user or significantly improves task performance. However, the ML component represents an unknown complex phenomenon that is learned from collected data without the need to be explicitly programmed. Despite the improvement in task performance, the models are often black boxes. Evaluating the credibility and the vulnerabilities of ML models poses a gap in current test and evaluation practice. For high consequence applications, the lack of testing and evaluation procedures represents a significant source of uncertainty and risk. To help reduce that risk, here we present considerations to evaluate systems embedded with an ML component within a red-teaming inspired methodology. We focus on (1) cyber vulnerabilities to an ML model, (2) evaluating performance gaps, and (3) adversarial ML vulnerabilities.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1962

A Survey and Evaluation of High Energy Liquid Chemical Propulsion Systems: Propellant Selection Criteria for Space Missions - Part 1

This report presents the results of a study to develop a procedure for evaluating liquid propellants in order (a) to select the most appropriate propellant (from among those under development) for each of several applications on each of the various missions in the NASA program, or (b) to select new propellants (from among those being proposed) for initiation or continuation of research and development. The analysis begins with a consideration of requirements--either for the specific application or for the various classes of applications. The known characteristics of the propellant or propellants to be evaluated are then put into a convenient form for evaluation. The next step is to determine whether or not there are requirements that simply cannot be met by the propellant. If the propellant passes this test, an optimum vehicle configuration using the propellant (and meeting all requirements) is estimated. (The configuration should be optimized with respect to the total resource consumption for all aspects of the mission, including R&D, production, logistics, and operation.) The total resource consumption for this configuration is then compared with that for similar configurations using other propellants (and meeting all requirements equally well). If all factors have been properly taken into account, this comparison of resource consumption will complete the evaluation. Such an evaluation may be performed several times, in increasing detail and with correspondingly increasing accuracy, as an R&D program proceeds, and the accuracy of the data as well as the cost of the next step in the program increase. The procedure is superior to those in common use in that it minimizes both the amount of analytical work and the number of points at which subjective value judgments are made.

LIQUID PROPELLANT↗
NASA NTRS1972

Development of techniques and instrumentation for the nondestructive evaluation of multi-layer insulation

The development of techniques and instrumentation suitable for the nondestructive evaluation of purged multilayer insulation prior to and after a shuttle orbiter flight is discussed. These techniques are desirable because of the possibility that corrosion, compression, or tearing of the insulation will degrade the performance of the insulation during subsequent flights. A second objective was the application of the developed technology to the evaluation of other insulation types. Three nondestructive evaluation techniques, electromagnetic, thermal, and acoustic were selected for evaluation. The results of evaluation of each technique are described.

Source record↗
NASA NTRS1975

Skylab program earth resouces experiment package. Volume 4: Sensor performance evaluation (S193 R/S)

The results of the sensor performance evaluation of the 13.9 GHz radiometer/scatterometer, which was part of the earth resources experiment package on Skylab. Findings are presented in the areas of housekeeping parameters, antenna gain and scanning performance, dynamic range, linearity, precision, resolution, stability, integration time, and transmitter output. Supplementary analyses covering performance anomalies, data stream peculiarities, aircraft sensor data comparisons, scatterometer saturation characteristics, and RF heating effects are reported. Results of the evaluation show that instrument performance was generally as expected, but capability degradations were observed to result from three major anomalies. Conclusions are drawn from the evaluation results, and recommendations for improving the effectiveness of a future program are offered. An addendum describes the special evaluation techniques developed and applied in the sensor performance evaluation tasks.

Kenney, G. P.↗
NASA NTRS1975

Apollo flight crew cardiovascular evaluations

The results of the lower body negative pressure (LBNP) and passive stand tests are presented, and the efficacy of the experimental antihypotensive garments is evaluated. Results of the tests are presented in tabular form. Postflight orthostatic evaluations during the Apollo program indicate that reduction in orthostatic tolerance is a consequence of space flight exposure. Heart rate, the most reliable index, was increased, while systolic and pulse pressures were decreased during immediate postflight evaluations using lower body negative pressure and passive standing as the orthostatic stress. Elevation in resting heart rate was a less frequent finding. There was considerable variability in the magnitude of these changes between individual crewmembers and in the persistence of the changes over subsequent postflight evaluations. Postflight changes in leg volume during LBNP were equal to or less than those seen during preflight baseline evaluations. Body weight, resting calf girth, supine leg volume, and cardiothoracic ratios were all diminished immediately postflight, and return to preflight values was not complete within the postflight testing time frame.

G.W. Hoffler↗
NASA NTRS1977

Evaluation of Viking Lander barometric pressure sensor

Variable reluctance type pressure sensors were evaluated to determine their performance characteristics related to Viking Mission environment levels. Static calibrations were performed throughout the evaluation over the full range of the sensors using two point contact manometer standards. From the beginning of the evaluation to the end of the evaluation, the zero shift in the two sensors was within 0.5 percent, and the sensitivity shift was 0.05 percent. The maximum thermal zero coefficient exhibited by the sensors was 0.032 percent over the temperature range of -28.89 C to 71.11 C. The evaluation results indicated that the sensors are capable of making high accuracy pressure measurements while being exposed to the conditions mentioned.

Mitchell, M.↗
NASA NTRS1980

FBI fingerprint identification automation study. AIDS 3 evaluation report. Volume 4: Economic feasibility

The results of the economic analysis of the AIDS 3 system design are presented. AIDS 3 evaluated a set of economic feasibility measures including life cycle cost, implementation cost, annual operating expenditures and annual capital expenditures. The economic feasibility of AIDS 3 was determined by comparing the evaluated measures with the same measures, where applicable, evaluated for the current system. A set of future work load scenarios was constructed using JPL's environmental evaluation study of the fingerprint identification system. AIDS 3 and the current system were evaluated for each of the economic feasibility measures for each of the work load scenarios. They were compared for a set of performance measures, including response time and accuracy, and for a set of cost/benefit ratios, including cost per transaction and cost per technical search. Benefit measures related to the economic feasibility of the system are also presented, including the required number of employees and the required employee skill mix.

Mulhall, B. D. L.↗
NASA NTRS1982

Description and planned use of a data distribution evaluation system for fiber optic data buses

A general description of a laboratory data distribution evaluation system (DDES) is given and some plans to use the system. The DDES is a microprocessor-based evaluation system consisting of three identical terminals. The DDES provides the capability for evaluating different system protocols and data word structures by making appropriate software changes, and different transmission medias and modulation schemes by making front-end hardware changes. A data multiplexing standard for fiber optic data transmission which parallels MIL-STD-1553B has proposed four different data modulation schemes. These modulation schemes will be described along with the plans to evaluate each of them. NASA-Langley has also been developing a data distribution approach which utilizes optical wavelength division multiplexing (WDM). A four-port, four-wavelength WDM system will be described along with the planned use of the DDES to evaluate its performance.

Spencer, J. L.↗
NASA NTRS1984

Thematic Mapper geometric correction performance evaluation

The present paper provides a performance evaluation of geometric correction in the Thematic Mapper Image Processing System (TIPS). TIPS forms a part of NASA's Thematic Mapper (TM) data processing facility. During a TM Research and Development period which continues through 1984, the geometric accuracy of TIPS is currently being evaluated. A description is given of TIPS geometric correction system, taking into account a TIPS overview and the Thematic Mapper Control Point Library Build (TCL). TCL generates control point chips which are stored in a library for subsequent use in the Thematic Mapper Archive Generation (TAG). Methods for measuring accuracy are discussed, giving attention to error estimation and direct measurement. A temporal registration evaluation from error estimation is considered along with a temporal evaluation from direct measurement, and a geodetic correction evaluation.

Brooks, J.↗
NASA NTRS1987

An evaluation of candidate oxidation resistant materials

Ground based testing of materials considered for Kapton solar array blanket protection, graphite epoxy structural member protection, and high temperature radiators was performed in an RF plasma asher. Ashing rates for Kapton were correlated with rates measured on STS-8 to determine the exposure time equivalent to one year in low Earth orbit (LEO) at a constant density space station orbital flux. Protective coatings on Kapton from Tekmat, Andus Corporation, and LeRC were evaluated in the plasma asher and mass loss rates per unit area were measured for each sample. All samples evaluated provided some protection to the underlying surface but ion beam sputter deposited samples of SiO2 and SiO2 with 8% polytetrafluoroethylene (PTFE) showed no evidence of degradation after 47 hours of exposure. Mica paint was evaluated as a protective coating for graphite epoxy structural members. Mica appears to be resistant to attack by atomic oxygen but only offers some limited protection as a paint because the paint vehicles evaluated to date were not resistant to atomic oxygen. Four materials were selected for evaluation as candidate radiator materials: stainless steel, copper, niobium-1% zirconium, and titanium-6% aluminum-4% vanadium. These materials were surface textured by various means to improve their emittance. Emittances as high as 0.93 at 2.5 microns for stainless steel and 0.89 at 2.5 microns for Nb-1 Zr were obtained from surface texturing. There were no significant changes in emittance after asher exposure.

Rutledge, Sharon↗
NASA NTRS1987

A methodology for evaluation of an interactive multispectral image processing system

Because of the considerable cost of an interactive multispectral image processing system, an evaluation of a prospective system should be performed to ascertain if it will be acceptable to the anticipated users. Evaluation of a developmental system indicated that the important system elements include documentation, user friendliness, image processing capabilities, and system services. The criteria and evaluation procedures for these elements are described herein. The following factors contributed to the success of the evaluation of the developmental system: (1) careful review of documentation prior to program development, (2) construction and testing of macromodules representing typical processing scenarios, (3) availability of other image processing systems for referral and verification, and (4) use of testing personnel with an applications perspective and experience with other systems. This evaluation was done in addition to and independently of program testing by the software developers of the system.

Kovalick, William M.↗
NASA NTRS1987

An evaluation plan of bus architectures and protocols using the NASA Ames intelligent redundant actuation system

Means for evaluating data bus architectures and protocols for highly integrated flight control system applications are needed. Described are the criteria and plans to do this by using the NASA/Ames Intelligent Redundant Actuation System (IRAS) experimental set-up. Candidate bus architectures differ from one another in terms of: topology, access control, message transfer schemes, message characteristics, initialization. data flow control, transmission rates, fault tolerance, and time synchronization. The evaluation criteria are developed relative to these features. A preliminary, analytical evaluation of four candidate busses (MIL-STD-1553B, DATAC, Ethernet, and HSIS) is described. A bus must be exercised in a real-time environment to evaluate its dynamic characteristics. A plan for real-time evaluation of these four busses using a combination of hardware and simulation techniques is presented.

Defeo, P.↗
NASA NTRS1987

Evaluation of high temperature structural adhesives for extended service, phase 5

The evaluation of 3 experimental polymers from NASA-Langley and a commercially produced polymer from Mitsui Toatsu Chemicals as high temperature structural adhesives is presented. A polyphenylquinoxaline (PPQ), polyimide (STPI/LaRC-2), and a polyarylene ether (PAE-SO2) were evaluated as metal-to-metal adhesives. Lap shear, crack extension, and climbing drum peel specimens were fabricated from all three polymers and tested after thermal, combined thermal/humidity, and stressed hydraulic fluid (Skydrol) exposure. The fourth polymer, LARC-TPI was evaluated as an adhesive for titanium honeycomb sandwich structure. All three experimental polymers performed well as metal-to-metal adhesives from 219 K (-65 F) to 505 K (450 F), including humidity exposure. Structural adhesive strength was also maintained at 505 K for a minimum of 3000 hours. LaRC-TPI was evaluated as a high temperature (505 K) adhesive for titanium honeycomb sandwich structure. The LaRC-TPI bonding process development concentrated on improving the honeycomb core-to-skin bond. The most promising approach of those evaluated combined a LaRC-TPI polymer solution with a semi-crystalline LaRC-TPI powder for adhesive film fabrication and fillet formation.

Hendricks, C. L.↗