Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “reliability metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
NASA NTRS1992

Comparison of three controllers applied to helicopter vibration

A comparison was made of the applicability and suitability of the deterministic controller, the cautious controller, and the dual controller for the reduction of helicopter vibration by using higher harmonic blade pitch control. A randomly generated linear plant model was assumed and the performance index was defined to be a quadratic output metric of this linear plant. A computer code, designed to check out and evaluate these controllers, was implemented and used to accomplish this comparison. The effects of random measurement noise, the initial estimate of the plant matrix, and the plant matrix propagation rate were determined for each of the controllers. With few exceptions, the deterministic controller yielded the greatest vibration reduction (as characterized by the quadratic output metric) and operated with the greatest reliability. Theoretical limitations of these controllers were defined and appropriate candidate alternative methods, including one method particularly suitable to the cockpit, were identified.

Leyland, Jane A.↗
NASA NTRS2008

The Super Tuesday Outbreak: Forecast Sensitivities to Single-Moment Microphysics Schemes

Forecast precipitation and radar characteristics are used by operational centers to guide the issuance of advisory products. As operational numerical weather prediction is performed at increasingly finer spatial resolution, convective precipitation traditionally represented by sub-grid scale parameterization schemes is now being determined explicitly through single- or multi-moment bulk water microphysics routines. Gains in forecasting skill are expected through improved simulation of clouds and their microphysical processes. High resolution model grids and advanced parameterizations are now available through steady increases in computer resources. As with any parameterization, their reliability must be measured through performance metrics, with errors noted and targeted for improvement. Furthermore, the use of these schemes within an operational framework requires an understanding of limitations and an estimate of biases so that forecasters and model development teams can be aware of potential errors. The National Severe Storms Laboratory (NSSL) Spring Experiments have produced daily, high resolution forecasts used to evaluate forecast skill among an ensemble with varied physical parameterizations and data assimilation techniques. In this research, high resolution forecasts of the 5-6 February 2008 Super Tuesday Outbreak are replicated using the NSSL configuration in order to evaluate two components of simulated convection on a large domain: sensitivities of quantitative precipitation forecasts to assumptions within a single-moment bulk water microphysics scheme, and to determine if these schemes accurately depict the reflectivity characteristics of well-simulated, organized, cold frontal convection. As radar returns are sensitive to the amount of hydrometeor mass and the distribution of mass among variably sized targets, radar comparisons may guide potential improvements to a single-moment scheme. In addition, object-based verification metrics are evaluated for their utility in gauging model performance and QPF variability.

Molthan, Andrew L.↗
NASA NTRS2004

Software Quality Assurance Metrics

Software Quality Assurance (SQA) is a planned and systematic set of activities that ensures conformance of software life cycle processes and products conform to requirements, standards and procedures. In software development, software quality means meeting requirements and a degree of excellence and refinement of a project or product. Software Quality is a set of attributes of a software product by which its quality is described and evaluated. The set of attributes includes functionality, reliability, usability, efficiency, maintainability, and portability. Software Metrics help us understand the technical process that is used to develop a product. The process is measured to improve it and the product is measured to increase quality throughout the life cycle of software. Software Metrics are measurements of the quality of software. Software is measured to indicate the quality of the product, to assess the productivity of the people who produce the product, to assess the benefits derived from new software engineering methods and tools, to form a baseline for estimation, and to help justify requests for new tools or additional training. Any part of the software development can be measured. If Software Metrics are implemented in software development, it can save time, money, and allow the organization to identify the caused of defects which have the greatest effect on software development. The summer of 2004, I worked with Cynthia Calhoun and Frank Robinson in the Software Assurance/Risk Management department. My task was to research and collect, compile, and analyze SQA Metrics that have been used in other projects that are not currently being used by the SA team and report them to the Software Assurance team to see if any metrics can be implemented in their software assurance life cycle process.

McRae, Kalindra A.↗
NASA NTRSDate not supplied

Anisotropic Goal-Based Mesh Adaptation Metric Clarification and Development

Adaptive unstructured mesh techniques have a limited, but growing impact on production analysis workflows to control discretization error for reliable simulation results. Multiple independent implementations of flow solvers, anisotropic metric construction methods, and anisotropic mesh adaptation mechanics have matured. Goal-based metrics target estimated error in output functions, such as lift and drag, through the guidance of an adjoint solution. A unification of goal-based anisotropic metrics is presented for steady viscous flows, which is an active area of research. These goal-based metrics drive robust and efficient anisotropic mesh adaptation for the calculation of output functions. The super-convergent functional output error behavior of stabilized finite-element methods is exploited without a formal proof, and evidence of super-convergence is shown in numerical experiments. Mesh adapted drag and lift outputs for two simple bodies in compressible viscous flow show convergence of error to less than a single drag count. Asymptotic behavior established for relatively coarse meshes shows the efficiency of this goal-based metric when compared to solution interpolation error control and expert-guided meshing. Anisotropic mesh adaptation techniques are applied to a transport aircraft in a high-lift configuration where variation between approaches decreases with mesh refinement, but asymptotic behavior is not observed with available resources.

goal-based↗
NASA NTRS2002

Integrating Formal Methods and Testing 2002

Traditionally, qualitative program verification methodologies and program testing are studied in separate research communities. None of them alone is powerful and practical enough to provide sufficient confidence in ultra-high reliability assessment when used exclusively. Significant advances can be made by accounting not only tho formal verification and program testing. but also the impact of many other standard V&V techniques, in a unified software reliability assessment framework. The first year of this research resulted in the statistical framework that, given the assumptions on the success of the qualitative V&V and QA procedures, significantly reduces the amount of testing needed to confidently assess reliability at so-called high and ultra-high levels (10-4 or higher). The coming years shall address the methodologies to realistically estimate the impacts of various V&V techniques to system reliability and include the impact of operational risk to reliability assessment. Combine formal correctness verification, process and product metrics, and other standard qualitative software assurance methods with statistical testing with the aim of gaining higher confidence in software reliability assessment for high-assurance applications. B) Quantify the impact of these methods on software reliability. C) Demonstrate that accounting for the effectiveness of these methods reduces the number of tests needed to attain certain confidence level. D) Quantify and justify the reliability estimate for systems developed using various methods.

Cukic, Bojan↗
DOE OSTI2025

Functional protein mining with conformal guarantees

Molecular structure prediction and homology detection offer promising paths to discovering protein function and evolutionary relationships. However, current approaches lack statistical reliability assurances, limiting their practical utility for selecting proteins for further experimental and in-silico characterization. To address this challenge, we introduce a statistically principled approach to protein search leveraging principles from conformal prediction, offering a framework that ensures statistical guarantees with user-specified risk and provides calibrated probabilities (rather than raw ML scores) for any protein search model. Our method (1) lets users select many biologically-relevant loss metrics (i.e. false discovery rate) and assigns reliable functional probabilities for annotating genes of unknown function; (2) achieves state-of-the-art performance in enzyme classification without training new models; and (3) robustly and rapidly pre-filters proteins for computationally intensive structural alignment algorithms. Our framework enhances the reliability of protein homology detection and enables the discovery of uncharacterized proteins with likely desirable functional properties.

59 BASIC BIOLOGICAL SCIENCES↗
NASA NTRSDate not supplied

Improving Quantification of Medical Evacuation Risk for Human Spaceflight in the IMPACT Model

Traditionally, mission planners have used a heuristic and qualitative approach to design medical systems based on prior experience however this approach may result in high variability and implicit bias to design and could put missions and crewmembers at risk. The Informing Mission Planning through Analysis of Complex Tradespaces (IMPACT) tool is a probabilistic risk assessment tool designed by NASA to model medical risk in long duration missions outside of low earth orbit (LEO). This tool can quantify risk metrics such as risk of crewmember death, loss of crew task time, and the need for medical evacuation to better inform and augment the more traditional approach to medical system design. The risk metric of Return to Definitive Care (RTDC) represents the likelihood of requiring medical evacuation and has been difficult to quantify in a reliable and standardized manner. Initial development of the RTDC metric was constrained due to the complexity of a medical evacuation decision and the lack of prior spaceflight data and this presentation will discuss the review and improvement process used to increase model accuracy and fidelity.

Prashant Parmar↗
NASA NTRSDate not supplied

Artemis Missions Probabilistic Risk Assessment (PRA) & Reliability Assessment Overview

To show how NASA’s Moon-to-Mars (M2M) program can use hardware reliability assessments and comprehensive Probabilistic Risk Assessments (PRAs) to estimate risk of future missions and manage risk via safety and mission success requirements from the top down, so that this risk can be balanced with other program metrics like cost and schedule. These PRAs and reliability assessments can be used during the concept, design, and operational phases of development. This presentation also provides some thoughts on how the NASA process can be utilized for a lunar surface power grid development.

PRA↗
NASA NTRS1994

A Planar Approximation for the Least Reliable Bit Log-likelihood Ratio of 8-PSK Modulation

The optimum decoding of component codes in block coded modulation (BCM) schemes requires the use of the log-likelihood ratio (LLR) as the signal metric. An approximation to the LLR for the least reliable bit (LRB) in an 8-PSK modulation based on planar equations with fixed point arithmetic is developed that is both accurate and easily realizable for practical BCM schemes. Through an error power analysis and an example simulation it is shown that the approximation results in 0.06 dB in degradation over the exact expression at an E(sub s)/N(sub o) of 10 dB. It is also shown that the approximation can be realized in combinatorial logic using roughly 7300 transistors. This compares favorably to a look up table approach in typical systems.

Thesling, William H.↗
DOE OSTI2021

PV Evolution in the Light of Circular Economy

As photovoltaic (PV) deployments increase, the increasing material volume presents challenges throughout the supply chain and eventual end-of-life waste management. Quantifying the value of lifetime, reliability, repair, reuse, recycling, and other options to increase circularity can help identify and prioritize research and technical solutions required to integrate PV into the circular economy. In this work, we capture the evolution of PV module technology, such as improved reliability and increasing market share of glass/glass modules. We discuss material circularity, lifetime and reliability, and energy return on investment as quantitative metrics to define and evaluate circularity and sustainability.

bifacial↗
DOE OSTI2024

Establishing performance metrics for quantitative non-targeted analysis: a demonstration using per- and polyfluoroalkyl substances

Abstract Non-targeted analysis (NTA) is an increasingly popular technique for characterizing undefined chemical analytes. Generating quantitative NTA (qNTA) concentration estimates requires the use of training data from calibration “surrogates,” which can yield diminished predictive performance relative to targeted analysis. To evaluate performance differences between targeted and qNTA approaches, we defined new metrics that convey predictive accuracy, uncertainty (using 95% inverse confidence intervals), and reliability (the extent to which confidence intervals contain true values). We calculated and examined these newly defined metrics across five quantitative approaches applied to a mixture of 29 per- and polyfluoroalkyl substances (PFAS). The quantitative approaches spanned a traditional targeted design using chemical-specific calibration curves to a generalizable qNTA design using bootstrap-sampled calibration values from “global” chemical surrogates. As expected, the targeted approaches performed best, with major benefits realized from matched calibration curves and internal standard correction. In comparison to the benchmark targeted approach, the most generalizable qNTA approach (using “global” surrogates) showed a decrease in accuracy by a factor of ~4, an increase in uncertainty by a factor of ~1000, and a decrease in reliability by ~5%, on average. Using “expert-selected” surrogates ( n = 3) instead of “global” surrogates ( n = 25) for qNTA yielded improvements in predictive accuracy (by ~1.5×) and uncertainty (by ~70×) but at the cost of further-reduced reliability (by ~5%). Overall, our results illustrate the utility of qNTA approaches for a subclass of emerging contaminants and present a framework on which to develop new approaches for more complex use cases. Graphical Abstract

Pu, Shirley (ORCID:0000000201223797)↗
DOE OSTI2022

Temporal complementarity of marine renewables with wind and solar generation: Implications for GB system benefits

Wave and tidal energy have the potential to provide benefits to power systems with high proportions of stochastic renewable generation. This is particularly applicable with wind and solar photovoltaics, as the offsetting of these renewable resources results in more reliable renewable generation. This study utilises ten metrics to quantify the temporal complementarity and supply-demand balancing requirements of the energy mix in Great Britain, to investigate the potential magnitude of these system benefits. Wave and tidal generation profiles are created using historical resource data and hydrodynamic models. The results show that the inclusion of wave and tidal generation creates a renewable energy mix which is more available under multiple conditions: throughout a year of operation; at times of peak demand; for multiple consecutive hourly time periods; and at times when wind and solar generation are not available. Three regional case studies also show that the inclusion of marine energy allows for improved regional supply-demand matching, reducing instances of energy shortage and excess and potentially relieving transmission congestion at particularly constrained locations within GB. Finally, the implications of these findings are discussed in terms of GB wholesale market operation, system balancing and system security.

16 TIDAL AND WAVE POWER↗
DOE OSTI2023

Building water resilience in the face of cascading wildfire risks

Severe wildfire is altering the natural and the built environment and posing risks to environmental and societal health and well-being, including cascading impacts to water systems and built water infrastructure. Research on wildfire-resilient water systems is growing but not keeping pace with the scale and severity of wildfire impacts, despite their intensifying threat. In this study, we evaluate the state of knowledge regarding wildfire-related hazards to water systems. We propose a holistic framework to assess interactions and feedback loops between water quality, quantity, and infrastructure hazards as determinants of post-fire water availability and access. Efforts to address the evolving threat of wildfires to water systems will require more interdisciplinary research on the complex relationships shaping wildfire’s threat to water availability and access. To support this, we need reliable long-term data availability, consistent metrics, greater research in shared contexts, more extensive research beyond the burn area, and multistakeholder collaboration on wildfire risks to water systems.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2010

Rapid Response Risk Assessment in New Project Development

A capability for rapidly performing quantitative risk assessments has been developed by JSC Safety and Mission Assurance for use on project design trade studies early in the project life cycle, i.e., concept development through preliminary design phases. A risk assessment tool set has been developed consisting of interactive and integrated software modules that allow a user/project designer to assess the impact of alternative design or programmatic options on the probability of mission success or other risk metrics. The risk and design trade space includes interactive options for selecting parameters and/or metrics for numerous design characteristics including component reliability characteristics, functional redundancy levels, item or system technology readiness levels, and mission event characteristics. This capability is intended for use on any project or system development with a defined mission, and an example project will used for demonstration and descriptive purposes, e.g., landing a robot on the moon. The effects of various alternative design considerations and their impact of these decisions on mission success (or failure) can be measured in real time on a personal computer. This capability provides a high degree of efficiency for quickly providing information in NASA s evolving risk-based decision environment

Graber, Robert R.↗
NASA NTRS2018

An Assessment of the Differences Between Spatial Resolution and Grid Size for the SMAP Enhanced Soil Moisture Product over Homogeneous Sites

Satellite-based passive microwave remote sensing typically involves a scanning antenna that makes measurements at irregularly spaced locations. These locations can change on a day to day basis. Soil moisture products derived from satellite-based passive microwave remote sensing are usually resampled to a fixed Earth grid that facilitates their use in applications. In many cases the grid size is finer than the actual spatial resolution of the observation, and often this difference is not well understood by the user. Here, this issue was examined for the Soil Moisture Active Passive (SMAP) enhanced version of the passive-based soil moisture product, which has a grid size of 9-km and a nominal spatial resolution of 33-km. In situ observations from core validation sites were used to compute comparison metrics. For sites that satisfied the established reliability and scaling criteria, the impact of validating the 9-km grid product with in situ data collected over a 9-km versus a 33-km domain was very small for the sites studied (0.039 cu. m/cu. m unbiased root mean square difference for the 9-km case versus 0.037 cu. m/cu. m for the 33-km case). This result does not mean that the resolution of the product is 9-km but that for the conditions studied here the soil moisture estimated from in situ observations over 9-km is a close approximation of the soil moisture estimated from in situ observations over the 33-km resolution. The implication is that using the enhanced SMAP product at its grid resolution of 9-km should not introduce large errors in most applications.

Soil moisture↗
DOE OSTI2023

A Hot‐Swappable, Fault‐Tolerant, Modular Power Converter System for Solar Photovoltaic Plants

The performance metrics of the state-of-the-art commercial solar inverters, such as system cost, operation and maintenance (O&M) cost, service life, reliability, maintainability, and power density are much lower than the target metrics needed to achieve SunShot’s 2030 levelized cost of energy (LCOE) goals. To overcome the shortcomings of the existing solar inverters, this project proposed a novel Hot-Swappable, Fault-Tolerant, Modular Power Converter (HSFT-MPC) concept for solar photovoltaic (PV) plants and proved the concept through the design, fabrication, and laboratory test validation of a single-phase HSFT-MPC prototype. The HSFT-MPC has the following distinct advantages over the state-of-the-art: 1) elimination of harmonic/ electromagnetic interference (EMI) filter in the inverter stage due to the novel topology, 2) lower system cost and higher power density due to the modular design, elimination of harmonic/EMI filter, and lower cooling requirement, 3) higher efficiency due to lower switching frequencies, 4) higher reliability and longer (50 years) service life due to simpler cooling and fault tolerance capability, 5) easier installation, lower O&M cost, and improved maintainability due to the modular design and hot-swappable power electronic building blocks (PEBBs), and 6) improved manufacturability due to the modular design. This project developed a single-phase HSFT-MPC prototype with 25kW nominal output power, 2.4kV, 60Hz nominal AC output, lower than 5% AC output voltage total harmonic distortion, over 5 kW/L inverter power density, and 99.4% inverter peak efficiency, being tolerant to failure of single and multiple PEBBs, and capable of hot swapping of the failed PEBB(s). The HSFT-MPC enables uninterruptable operation of the solar PV plant when failure of single or multiple PEBBs or PV modules occurs. Compared with the existing solar inverters in the market, the HSFT-MPC is expected to reduce the inverter failure-caused downtime and energy losses of solar PV plants by more than 60% and 50%, respectively. Project findings have been presented at major conferences in the field and published in peer-reviewed papers, which added new knowledge to the field of power electronics for solar PV systems. A minicourse on Solar PV Systems was developed for outreach activities. The minicourse will help attract young individuals to the renewable energy profession which has a significant talent shortage. The HSFT-MPC is expected to overcome all of the shortcomings of the state-of-the-art solar inverters in terms of cost, efficiency, service life, reliability, maintainability, and manufacturability targets needed to achieve SunShot’s 2030 LCOE goals. Therefore, the proposed HSFT-MPC concept has the great potential to disrupt the current solar inverter market. This project created a pathway towards industry adoption of the HSFT-MPC to help achieve 50-year service life solar PV systems. Since the solar PV plants using the HSFT-MPC will feature with higher reliability, longer service life, and easier maintenance, they are particularly useful for the rural areas with underserved populations that demand reliable and affordable clean electricity. The outcomes of the project have the strong potential to address national needs in the field of renewable energy to reduce CO 2 emissions from the electricity sector, reduce imports of energy from foreign sources, and improve energy security, efficiency, and sustainability. Since electricity is used in almost all of society’s sectors, the outcomes of the project will benefit various sectors of society and economy.

14 SOLAR ENERGY↗
DOE OSTI2020

Scalable and Actionable Performance Measures for Traffic Signal Systems using Probe Vehicle Trajectory Data

Scalable and actionable performance measures for traffic signal systems provide opportunities for practitioners to measure and improve the transportation network. Historically, traffic signal improvements have relied on scheduled signal retiming based on limited data collection, or on the public to call and alert engineers of an issue. This inefficient method of improving signal timing led to the creation of automated traffic signal performance measures (ATSPMs). These metrics rely on expensive infrastructure, including detection and communications, which has produced barriers for numerous agencies to fully adopt. Recently, third-party data providers have begun to release vehicle trajectory data, which allows for enhanced signal metrics with no investment in physical equipment. The purpose of this study is to demonstrate the use of these data and summarize the scalability of the created metrics. This work builds on previous efforts to quantify signal performance on nine intersections in Michigan, U.S. Ten signalized corridors in Columbus, Ohio, were chosen to scale a performance assessment using crowdsourced trajectory data. A total of 136 intersections were assessed in 2-h intervals using data from all weekdays in 2017. High-level corridor summary metrics including average percent of vehicles stopping (18%–32%), average delay (9.4–20.5 s), and level of travel time reliability (1.23–2.73) were calculated for each corridor direction. Intersection-level metrics were also introduced, which can be used by practitioners to identify problems, improve signal timings, and prioritize future infrastructure investments.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2021

How different power plant types contribute to electric grid reliability, resilience, and vulnerability: a comparative analytical framework

Abstract This work explores the dependability tradeoffs provided by the most common types of central power plants in the United States. Historically, the electricity sector has lacked consensus on how reliability , resilience , and vulnerability differ and how those metrics change depending on the power plant fleet composition. We propose distinct definitions for these metrics and an analytical framework to evaluate power plant fleet dependability. Using data analysis and literature review, we identify fifteen dependability attributes across which we rank eleven power plant types relative to natural gas combined-cycle (NGCC) plants. We use NGCC as the benchmark because it is common to many locations and is of relatively recent vintage. The framework shows that each power plant type has unique dependability benefits and drawbacks. We provide examples of how researchers may use the framework to evaluate grid dependability qualitatively under different scenarios. We find that assuming all attributes that contribute to grid dependability are equally important and additive, electric grid dependability is best supported when power plant fleets include a mixture of power generation technologies. Then, we discuss scenario characteristics that could alter the prioritization and relationships of attributes. We also find that if current capacity installation trends continue to favor low- and zero-carbon power plants, US power grids may benefit from increased resilience and reduced vulnerability at the cost of decreased reliability. We conclude by recommending methods for adapting the framework and quantifying relationships between attributes in individual scenarios.

Ramirez-Meyers, K. (ORCID:0000000291216952)↗