Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “independent evaluation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

Initial Ada components evaluation

The SAIC has the responsibility for independent test and validation of the SSE. They have been using a mathematical functions library package implemented in Ada to test the SSE IV and V process. The library package consists of elementary mathematical functions and is both machine and accuracy independent. The SSE Ada components evaluation includes code complexity metrics based on Halstead's software science metrics and McCabe's measure of cyclomatic complexity. Halstead's metrics are based on the number of operators and operands on a logical unit of code and are compiled from the number of distinct operators, distinct operands, and total number of occurrences of operators and operands. These metrics give an indication of the physical size of a program in terms of operators and operands and are used diagnostically to point to potential problems. McCabe's Cyclomatic Complexity Metrics (CCM) are compiled from flow charts transformed to equivalent directed graphs. The CCM is a measure of the total number of linearly independent paths through the code's control structure. These metrics were computed for the Ada mathematical functions library using Software Automated Verification and Validation (SAVVAS), the SSE IV and V tool. A table with selected results was shown, indicating that most of these routines are of good quality. Thresholds for the Halstead measures indicate poor quality if the length metric exceeds 260 or difficulty is greater than 190. The McCabe CCM indicated a high quality of software products.

Moebes, Travis↗

Bringing Automated Fault Detection and Diagnostics Tools for HVAC&R Into the Mainstream

Heating, ventilation, and air-conditioning (HVAC) systems consume over 5 quads of energy annually, representing 30% of energy consumption in the U.S. commercial buildings. Additionally, commercial refrigeration (R) systems add about 1 quad to commercial buildings energy consumption. Most HVAC systems operate with one or more faults that result in increased energy consumption. Fault detection and diagnostics (FDD) tools have been developed to address this national issue, and many tools are commercially available. FDD tools have the potential to save considerable energy for an existing commercial rooftop unit (RTU) and refrigeration systems. These devices can be used for both retro commissioning and, when faults are addressed, continuous commissioning. However, there appears to be multiple market barriers for this technology. Although there are efforts to develop FDD tool standards, currently there are no standards and methods to define functions, capabilities, accuracy, and reliability of FDD tools in the field. Moreover, most of the commercial FDD tools have not been verified in the field independently. This paper presents a comprehensive approach for bringing HVAC FDD tools into the mainstream. The approach involves demonstrating ten commercially available FDD tools installed at ten different sites, independent testing and evaluation of the FDD tools, communication with various stakeholders, identifying and assessing market barriers, creating a process evaluation methodology, and assisting utility companies in developing incentive programs. Furthermore, the preliminary baseline results from the case study demonstrate how the use of an independent monitoring system (IMS) can be used for ground-truth in evaluating FDD tools in the field.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Radiometric Characterization Results for the IKONOS, Quickbird, and OrbView-3 Sensor

Radiometric calibration of commercial imaging satellite products is required to ensure that science and application communities better understand commercial imaging satellite properties. Inaccurate radiometric calibrations can lead to erroneous decisions and invalid conclusions and can limit intercomparisons with other systems. To address this calibration need, the NASA Applied Sciences Directorate (ASD) at Stennis Space Center established a commercial satellite imaging radiometric calibration team consisting of three independent groups: NASA ASD, the University of Arizona Remote Sensing Group, and South Dakota State University. Each group independently determined the absolute radiometric calibration coefficients of available high-spatial-resolution commercial 4-band multispectral products, in the visible though near-infrared spectrum, from GeoEye(tradeMark) (formerly SpaceImaging(Registered TradeMark)) IKONOS, DigitalGlobe(Regitered TradeMark) QuickBird, and GeoEye (formerly ORBIMAGE(Registered TradeMark) OrbView. Each team member employed some variant of reflectance-based vicarious calibration approach, requiring ground-based measurements coincident with image acquisitions and radiative transfer calculations. Several study sites throughout the United States that covered a significant portion of the sensor's dynamic range were employed. Satellite at-sensor radiance values were compared to those estimated by each independent team member to evaluate the sensor's radiometric accuracy. The combined results of this evaluation provide the user community with an independent assessment of these sensors' absolute calibration values.

Holekamp, Kara↗

Using Atmospheric Trace Gas Vertical Profiles to Evaluate Model Fluxes: A Case Study of Arctic-Cap Observations and Geos Simulations for the ABoVE Domain

Accurate estimates of carbon–climate feedbacks require an independent means for evaluating surface flux models at regional scales. The altitude-integrated enhancement (AIE) derived from the Arctic Carbon Atmospheric Profiles (Arctic-CAP) project demonstrates the utility of this bulk quantity for surface flux model evaluation. This bulk quantity leverages background mole fraction values from the middle free troposphere, is agnostic to uncertainties in boundary layer height, and can be derived from model estimates of mole fractions and vertical gradients. To demonstrate the utility of the bulk quantity, six airborne profiling surveys of atmospheric carbon dioxide (CO2), methane (CH4), and carbon monoxide (CO) throughout Alaska and northwestern Canada between April and November 2017 were completed as part of NASA’s Arctic–Boreal Vulnerability Experiment (ABoVE). The Arctic-CAP sampling strategy involved acquiring vertical profiles of CO2, CH4, and CO from the surface to 5 km altitude at 25 sites around the ABoVE domain on a 4- to 6-week time interval. All Arctic-CAP measurements were compared to a global simulation using the Goddard Earth Observing System (GEOS) modeling system. Comparisons of the AIE bulk quantity from aircraft observations and GEOS simulations of atmospheric CO2, CH4, and CO highlight the fidelity of the modeled surface fluxes. The model–data comparison over the ABoVE domain reveals that while current state-of-the-art models and flux estimates are able to capture broad-scale spatial and temporal patterns in near-surface CO2 and CH4 concentrations, more work is needed to resolve fine-scale flux features that are captured in CO observations

CO2↗

Final Report on Aerosol Pretreatment Technology Performance and Benchmarking

Solvent-based post-combustion CO 2 capture (PCC) technology remains one of the leading methods to combat global CO 2 emissions produced from large-scale coal-fired power production. Advanced solventbased PCC technology has made significant improvements in design and performance that reduce capital and operating costs to enable its commercial use. Key to low cost, manageable logistics, and environmentally safe operation of solvent-based PCC technology are minimal solvent losses from the process through the treated gas stream exiting PCC plant absorbers. High flue gas aerosol particle concentrations (>10 5 particles/cm 3 ) for particles in the range of 70-200 nm have been shown to cause significant amine solvent losses for solvent-based PCC processes through several mechanisms including absorption of solvent and water into growing aerosol particles. Flue gas aerosol pretreatment technology is the only realistic and economically attractive method to reduce very high aerosol particle concentrations (>10 7 particles/cm 3 ) to enable solvent-based PCC for existing power plants lacking sufficient particle removal systems, such as baghouses. The overall goal of this project was to design, construct, independently test, and evaluate three flue gas aerosol pretreatment technologies identified to significantly reduce high aerosol particle concentrations (>10 7 particles/cm 3 ) in the 70-200 nm particle size range: (1) a novel, high-velocity water injection spray concept developed by RWE and tested by Linde, (2) an innovative electrostatic precipitator (ESP) device with optimized operating conditions developed by Washington University in St. Louis (WUSTL), and (3) a non-regenerative sorbent-based filter technology developed by InnoSepra for SO x and NO x removal from coal-fired power plant flue gas. Each technology has been validated with tests on 500-1000 scfm of actual coal-fired flue gas and evaluated in terms of particle removal efficiency (%), cost competitiveness, and environmental impact. Aerosol measurements were performed upstream and downstream of each aerosol reduction unit during independent testing of each technology using advanced instrumentation and analytical methods provided by WUSTL. To perform aerosol measurements, isokinetic probes were inserted into flanged pipes attached to the flue gas piping, and a small suction pump was used to sample gas containing aerosol particles. Aerosol particle number concentrations (# particles/cm 3 ) and size distributions were then measured using a scanning mobility particle sizer (SMPS, TSI Inc.) for very fine particles (<1,000 nm) and a particle counter manufactured by GRIMM for particles larger than 1,000 nm. This report summarizes the aerosol removal performance results from pilot scale testing of each technology. Performance results have been benchmarked against pre-defined targets and other flue gas aerosol pretreatment technologies with documented performance. Linde Gas North America LLC has been the prime contractor to DOE responsible for overall project management and provided the design for the high-velocity water spray-based aerosol removal technology based on a design concept developed by the German utility company RWE.

20 FOSSIL-FUELED POWER PLANTS↗

Propulsion Diagnostic Method Evaluation Strategy (ProDiMES) User's Guide

This report is a User's Guide for the Propulsion Diagnostic Method Evaluation Strategy (ProDiMES). ProDiMES is a standard benchmarking problem and a set of evaluation metrics to enable the comparison of candidate aircraft engine gas path diagnostic methods. This Matlab (The Mathworks, Inc.) based software tool enables users to independently develop and evaluate diagnostic methods. Additionally, a set of blind test case data is also distributed as part of the software. This will enable the side-by-side comparison of diagnostic approaches developed by multiple users. The Users Guide describes the various components of ProDiMES, and provides instructions for the installation and operation of the tool.

Simon, Donald L.↗

Risk Model for EM Decision Support Toolsets

The Government Office of Accountability (GAO) has published several reports identifying the need for the Department of Energy (DOE) Office of Environmental Management (EM) to address mounting costs for DoE's cleanup program. DOEEM could greatly benefit from independent decision tool-sets/models that would allow them to evaluate options and inform business decision at the enterprise level considering site-specific life cycle costs and system plans. Program goals: Develop a tool-set that enables EM to independently evaluate alternatives, assess outcomes from different contracting strategies, and inform critical decisions for the enterprise. Project goals: Adaption of a Risk Model for integration with complimentary tool-sets for project level decision making. Operational Events: Discrete event model that evaluates operational variables (e.g. capacity, throughput, maintenance constraints) and identify bottlenecks. Identifying and Bounding Project Risk: Identify risks that impact confidence in meeting goals (e.g. production). Life cycle Cost: Evaluate impacts of staffing levels, inventory, and capital investments on life cycle costs. Methods and approach: Monte Carlo Analysis is being executed to generate results: Input derives from Risk Register Data; Tied to Projected and Target Schedules; Incorporates float duration within the model. Assumes associated risk mitigation actions being completed within a timeline of five fiscal years. Metrics include: Confidence in Meeting Production Goal; Confidence in Safety Standards; Confidence in Continuous Operation; Other Metrics can be added as appropriate regarding specific site needs. The adapted risk model can be used as a stand alone decision tool or can be integrated complementary tool-sets (i.e. process and cost models) for project-specific decisions. These support tool-sets can then be integrated with others for site- and complex- level evaluations. Future work includes designing an adaptable and modular framework that would allow integration of multiple tool-sets for holistic and/or targeted evaluation of alternative strategies for decision making that could lead to risk and cost reduction across the enterprise.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Conduct and results of the Interagency Nuclear Safety Review Panel's evaluation of the Ulysses space mission

After over a year of detailed independent review, a Safety Evaluation Report (SER) was prepared for the nuclear-powered Ulysses spacecraft mission. An overview is presently given of the Ulysses mission as well as the conduct and results of the evaluation process; in the final analysis, the SER was supportive of a positive launch decision. Eleven key accident scenarios were carried through to complete analysis, out of a total of 19 possible accidents involving a potential for fuel release to the spacecraft environment.

Sholtis, Joseph A., Jr.↗

Identification of terms to define unconstrained air transportation demands

The factors involved in the evaluation of unconstrained air transportation systems were carefully analyzed. By definition an unconstrained system is taken to be one in which the design can employ innovative and advanced concepts no longer limited by present environmental, social, political or regulatory settings. Four principal evaluation criteria are involved: (1) service utilization, based on the operating performance characteristics as viewed by potential patrons; (2) community impacts, reflecting decisions based on the perceived impacts of the system; (3) technological feasibility, estimating what is required to reduce the system to practice; and (4) financial feasibility, predicting the ability of the concepts to attract financial support. For each of these criteria, a set of terms or descriptors was identified, which should be used in the evaluation to render it complete. It is also demonstrated that these descriptors have the following properties: (a) their interpretation may be made by different groups of evaluators; (b) their interpretations and the way they are used may depend on the stage of development of the system in which they are used; (c) in formulating the problem, all descriptors should be addressed independent of the evaluation technique selected.

Jacobson, I. D.↗

Synthetic Vision Enhances Situation Awareness and RNP Capabilities for Terrain-Challenged Approaches

The Synthetic Vision Systems (SVS) Project of Aviation Safety Program is striving to eliminate poor visibility as a causal factor in aircraft accidents as well as enhance operational capabilities of all aircraft through the display of computer generated imagery derived from an onboard database of terrain, obstacle, and airport information. To achieve these objectives, NASA 757 flight test research was conducted at the Eagle-Vail, Colorado airport to evaluate three SVS display types (Head-Up Display, Head-Down Size A, Head-Down Size X) and two terrain texture methods (photo-realistic, generic) in comparison to the simulated Baseline Boeing-757 Electronic Attitude Direction Indicator and Navigation / Terrain Awareness and Warning System displays. These independent variables were evaluated for situation awareness, path error, and workload while making approaches to Runway 25 and 07 and during simulated engine-out Cottonwood 2 and KREMM departures. The results of the experiment showed significantly improved situation awareness, performance, and workload for SVS concepts compared to the Baseline displays and confirmed the retrofit capability of the Head-Up Display and Size A SVS concepts. The research also demonstrated that the pathway and pursuit guidance used within the SVS concepts achieved required navigation performance (RNP) criteria.

Kramer, Lynda J.↗

Groundwater Recharge Estimated by Land Surface Models: An Evaluation in the Conterminous U.S.

Estimating diffuse recharge of precipitation is fundamental to assessing groundwater sustainability. Diffuse recharge is also the process through which climate and climate change directly affect groundwater. In this study, we evaluated diffuse recharge over the conterminous U.S. simulated by a suite of land surface models (LSMs) that were forced using a common set of meteorological input data. Simulated annual recharge exhibited spatial patterns that were similar among the LSMs, with the highest values in the eastern U.S. and Pacific Northwest. However, the magnitudes of annual recharge varied significantly among the models and were associated with differences in simulated ET, runoff and snow. Evaluation against two independent datasets did not answer the question of whether the ensemble mean performs the best, due to inconsistency between those datasets. The amplitude and timing of seasonal maximum recharge differed among the models, influenced strongly by model physics governing deep soil moisture drainage rates and, in cold regions, snowmelt. Evaluation using in situ soil moisture observations suggested that true recharge peaks 1-3 months later than simulated recharge, indicating systematic biases in simulating deep soil moisture. However, recharge from lateral flows and through preferential flows cannot be inferred from soil moisture data, and the seasonal cycle of simulated groundwater storage actually compared well with in situ groundwater observations. Long-term trends in recharge were not consistently correlated with either precipitation trends or temperature trends. This study highlights the need to employ dynamic flow models in LSMs, among other improvements, to enable more accurate simulation of recharge.

snowmelt↗

A method for determining the weak statistical stationarity of a random process

A method for determining the weak statistical stationarity of a random process is presented. The core of this testing procedure consists of generating an equivalent ensemble which approximates a true ensemble. Formation of an equivalent ensemble is accomplished through segmenting a sufficiently long time history of a random process into equal, finite, and statistically independent sample records. The weak statistical stationarity is ascertained based on the time invariance of the equivalent-ensemble averages. Comparison of these averages with their corresponding time averages over a single sample record leads to a heuristic estimate of the ergodicity of a random process. Specific variance tests are introduced for evaluating the statistical independence of the sample records, the time invariance of the equivalent-ensemble autocorrelations, and the ergodicity. Examination and substantiation of these procedures were conducted utilizing turbulent velocity signals.

Sadeh, W. Z.↗

Effectiveness of Shield Termination Techniques Tested with TEM Cell and Bulk Current Injection

This paper presents experimental results of the effectiveness of various shield termination techniques. Each termination technique is evaluated by two independent noise injection methods; transverse electromagnetic (TEM) cell operated from 3 MHz 400 MHz, and bulk current injection (BCI) operated from 50 kHz 400 MHz. Both single carrier and broadband injection tests were investigated. Recommendations as to how to achieve the best shield transfer impedance (i.e. reduced coupled noise) are made based on the empirical data. Finally, the noise injection techniques themselves are indirectly evaluated by comparing the results obtained from the TEM Cell to those from BCI.

Bradley, Arthur T.↗

Comparison of Image Restoration Methods for Lunar Epithermal Neutron Emission Mapping

Orbital measurements of neutrons by the Lunar Exploring Neutron Detector (LEND) onboard the Lunar Reconnaissance Orbiter are being used to quantify the spatial distribution of near surface hydrogen (H). Inferred H concentration maps have low signal-to-noise (SN) and image restoration (IR) techniques are being studied to enhance results. A single-blind. two-phase study is described in which four teams of researchers independently developed image restoration techniques optimized for LEND data. Synthetic lunar epithermal neutron emission maps were derived from LEND simulations. These data were used as ground truth to determine the relative quantitative performance of the IR methods vs. a default denoising (smoothing) technique. We review and used factors influencing orbital remote sensing of neutrons emitted from the lunar surface to develop a database of synthetic "true" maps for performance evaluation. A prior independent training phase was implemented for each technique to assure methods were optimized before the blind trial. Method performance was determined using several regional root-mean-square error metrics specific to epithermal signals of interest. Results indicate unbiased IR methods realize only small signal gains in most of the tested metrics. This suggests other physically based modeling assumptions are required to produce appreciable signal gains in similar low SN IR applications.

McClanahan, T. P.↗

Evaluating Behaviorally Oriented Aviation Maintenance Resource Management (MRM) Training and Programs: Methods, Results, and Conclusions

Assessment of the impact of Aviation Resource Management Programs on aviation culture and performance has compelled a considerable body of research (Taylor & Robertson, 1995; Taylor, 1998; Taylor & Patankar, 2001). In recent years new methods have been applied to the problem of maintenance error precipitated by factors such as the need for self-assessment of communication and trust. The present study - 2002 -- is an extension of that past work. This research project was designed as the conclusion of a larger effort to help understand, evaluate and validate the impact of Maintenance Resource Management (MRM) training programs, and other MRM interventions on participant attitudes, opinions, behaviors, and ultimately on enhanced safety performance. It includes research and development of evaluation methodology as well as examination of psychological constructs and correlates of maintainer performance. In particular, during 2002, three issues were addressed. First, the evaluation of two (independent & different) MRM programs for changing behaviors was undertaken. In one case we were able to further apply the approach to measuring written communication developed during 2001 (Taylor, 2002; Taylor & Thomas, 2003). Second, the MRM/TOQ surveys were made available for completion on the internet. The responses from these on-line surveys were automatically linked to a results calculator (like the one developed and described in Taylor, 2002) to aid industry users in analyzing and evaluating their local survey data on the internet. Third, the main trends and themes from our research about MRM programs over the past dozen years were reviewed.

Taylor, James C.↗

GeoRePORT Case Study Examples: Reporting Using the Geothermal Resource Portfolio Optimization and Reporting Technique (GeoRePORT)

The Geothermal Research Portfolio Optimization & Reporting Technique (GeoRePORT) was developed with funding from the United States Department of Energy's Geothermal Technologies Office (GTO) to assist in identifying and pursuing long-term investment strategies through the development of a resource reporting protocol. GeoRePORT provides scientists and non-scientists a comprehensive and quantitative means of reporting: (1) features intrinsic to geothermal sites (project grade) and, (2) maturity of the development (project readiness). Because geothermal feasibility is not determined by any single factor (e.g. temperature, permeability, permitting), a site's project grade and readiness are evaluated on twelve independent attributes pertaining to geological, technical, or socio-economic feasibility. In this paper, we present case studies illustrating how GeoRePORT can be used to compare geological, technical, and socio-economic attributes between geothermal systems. The consistent and objective assessment protocols used in GeoRePORT allow for comparison of project attributes across unique locations and geological settings. GeoRePORT case studies outline the geological, socio-economic and technical features of four individual geothermal sites: Coso, Chena, Dixie Valley, and White Sands Missile Range. The case studies presented herein illustrate the usefulness of GeoRePORT in evaluating project risk/return, identifying gaps in reported data, evaluating R&D impact, and gathering insights on successes/failures as applicable to future projects.

40 EE - Geothermal Technologies Office (EE-4G)↗

Field Evaluations of Sampling, Interviewing, and Flight Tracking of NASA's Low Boom Flight Demonstrator Aircraft

The first year’s effort identified sampling and interviewing as the principal risks to assessment of prompt reactions to overflights producing low-amplitude sonic booms. It also 1) established the utility of geo-information system-based route planning for LBFD flight missions, 2) developed and demonstrated a prototype of a geographically-distributed, Internet-enabled instrumentation system capable of wide-area tracking of LBFD aircraft in near-real time. The latter system permits synchronizing the conduct of interviews in multiple overflown communities with arrival times of shock waves at interviewing sites; and of measuring, archiving, and processing their acoustic signatures. Means were also recommended for constructing representative, telephone-based samples of eligible respondents living in households within carpet boom corridors adjacent to LBFD flight tracks, and for conducting interviews with cross-sectional (independent) samples of such respondents about their prompt reactions to exposure to low-amplitude sonic booms. A detailed study design was prepared and accepted by NASA for a set of single-contact attempt telephone interviews with a nationally representative sample of households. The study design focused on testing automated and live agent interview completion rates obtainable without callbacks. A minimal (two monitoring station) version of the aircraft tracking system was built and installed near a civil airport in a successful demonstration of the system’s ability to detect and track aircraft movements. The field exercise also demonstrated the ability of the system to capture the acoustic emissions of departing aircraft, and to serve aircraft position and sound level information to remote, geographically-distributed analysts in near-real time. Upon approval of OMB and IRB of the detailed study plan, a stratified, nationally representative sample of landline and wireless telephone-subscribing households was constructed. A total of 12,734 telephone interview contact attempts of the sort required by a straightforward cross-sectional study design were then made. These contact attempts demonstrated the impracticality of conducting a time-critical, cross-sectional study of prompt community response to low-amplitude sonic booms by means of “independent” (single contact attempt per respondent for each LBFD flight mission) telephone samples of respondents. The observed interview completion rates for these single telephone contact attempts were so low (~ 1% to 3% for automated and live agent interviews, respectively) that: 1) the representativeness of collected opinions would be susceptible to intuitive challenge as inadequate, even absent conclusive evidence of non-representativeness. Refuting challenges to representativeness would have to demonstrate that the composition of the actual sample did not differ from that of the target population, a task that is tantamount to proving a negative; 2) the information required to refute allegations of non-representativeness would require a questionnaire considerably lengthier than that required simply to determine the prevalence of boom-induced startle and annoyance. Such a questionnaire would have to inquire about potentially sensitive and intrusive matters, including respondents’ age, gender, education, employment, home ownership, income, ethnicity, family size, and other demographic factors; and 3) unreasonable numbers of attempts would be required to re-contact households with unsuccessful initial contact attempts, given the limited time available for doing so. For example, if about 500 completed interviews were desired in a supersonically overflown community, approximately 50,000 automated interview attempts would have to be made within ten to fifteen minutes of each LBFD overflight. Such large numbers of contact attempts could well exceed the numbers of households available for interview in areas of similar boom exposure levels in some communities near LBFD flight tracks. Such large numbers of interviews could be cost-effectively undertaken only by means of automated (i.e., outgoing interactive voice response) interviewing, a data collection method ill-suited for complex and sensitive questionnaire items. The infeasibility of independent sampling for evaluating prompt responses to LBFD overflights in a cross-sectional study is due in large part to simple non-response: that is, potential respondents – particularly those contacted on wireless telephones – refusing to answer calls with unfamiliar caller IDs. It is also due in part, however, to 1) the lack of time to attempt to contact the same respondent more than once within a few minutes after the arrival of a shock wave at the respondent’s location; and 2) the need to place calls during weekday/daytime hours, when response rates are notably lower than during evenings and weekends. Despite the poor interview completion rates achieved under the above constraints, cross sectional assessments of delayed reactions to LBFD overflights could still be feasible, if multiple attempts could be made to contact respondents during evening and weekend time periods, over extended time periods. Detailed plans for a longitudinal (panel) sample were developed as an alternative to a cross sectional sample design.

Fidell, Sanford↗