SEARCH · Engineering Papers

Results for “reliability metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Advanced Performance Metrics and their Application to the Sensitivity Analysis for Model Validation and Calibration

High-quality generator dynamic models are critical to reliable and accurate power systems studies and planning. With the availability of PMUs, measurement-based approach for model validation has gained significant prominence. In this approach, the quality of a model is analyzed by visually comparing measured generator response with the model-based simulated response for large system disturbances. This paper proposes a new set of performance metrics to assess the model validation results to facilitate automation of the model validation process. In the proposed methodology, first, the slow governor response and comparatively faster oscillatory response are separated, and then a separate set of performance metrics is calculated for each of these two components. These proposed metrics quantify the mismatch between the actual and model-based response in a comprehensive manner without missing any information enabling automation of the process. Furthermore, in this paper, we are also proposing that the sensitivity analysis for model calibration be performed with respect to the proposed metrics for the systematic identification of key parameters. In this work, results obtained using both simulated and real-world case-studies validate the effectiveness of the proposed performance metrics for model validation and their application to the sensitivity analysis for model calibration.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS2000

Probabilistic Multi-Scale, Multi-Level, Multi-Disciplinary Analysis and Optimization of Engine Structures

Aircraft engines are assemblies of dynamically interacting components. Engine updates to keep present aircraft flying safely and engines for new aircraft are progressively required to operate in more demanding technological and environmental requirements. Designs to effectively meet those requirements are necessarily collections of multi-scale, multi-level, multi-disciplinary analysis and optimization methods and probabilistic methods are necessary to quantify respective uncertainties. These types of methods are the only ones that can formally evaluate advanced composite designs which satisfy those progressively demanding requirements while assuring minimum cost, maximum reliability and maximum durability. Recent research activities at NASA Glenn Research Center have focused on developing multi-scale, multi-level, multidisciplinary analysis and optimization methods. Multi-scale refers to formal methods which describe complex material behavior metal or composite; multi-level refers to integration of participating disciplines to describe a structural response at the scale of interest; multidisciplinary refers to open-ended for various existing and yet to be developed discipline constructs required to formally predict/describe a structural response in engine operating environments. For example, these include but are not limited to: multi-factor models for material behavior, multi-scale composite mechanics, general purpose structural analysis, progressive structural fracture for evaluating durability and integrity, noise and acoustic fatigue, emission requirements, hot fluid mechanics, heat-transfer and probabilistic simulations. Many of these, as well as others, are encompassed in an integrated computer code identified as Engine Structures Technology Benefits Estimator (EST/BEST) or Multi-faceted/Engine Structures Optimization (MP/ESTOP). The discipline modules integrated in MP/ESTOP include: engine cycle (thermodynamics), engine weights, internal fluid mechanics, cost, mission and coupled structural/thermal, various composite property simulators and probabilistic methods to evaluate uncertainty effects (scatter ranges) in all the design parameters. The objective of the proposed paper is to briefly describe a multi-faceted design analysis and optimization capability for coupled multi-discipline engine structures optimization. Results are presented for engine and aircraft type metrics to illustrate the versatility of that capability. Results are also presented for reliability, noise and fatigue to illustrate its inclusiveness. For example, replacing metal rotors with composites reduces the engine weight by 20 percent, 15 percent noise reduction, and an order of magnitude improvement in reliability. Composite designs exist to increase fatigue life by at least two orders of magnitude compared to state-of-the-art metals.

Chamis, Christos C.↗

NASA NTRS2018

Serious Gaming for Building a Basis of Certification via Trust and Trustworthiness of Autonomous Systems

Autonomous systems governed by a variety of adaptive and nondeterministic algorithms are being planned for inclusion into safety-critical environments, such as unmanned aircraft and space systems in both civilian and military applications. However, until autonomous systems are proven and perceived to be capable and resilient in the face of unanticipated conditions, humans will be reluctant or unable to delegate authority, remaining in control aided by machine-based information and decision support. Proving capability, or trustworthiness, is a necessary component of certification. Perceived capability is a component of trust. Trustworthiness is an attribute of a cyber-physical system that requires context-driven metrics to prove and certify. Trust is an attribute of the agents participating in the system and is gained over time and multiple interactions through trustworthy behavior and transparency. Historically, artificial intelligence and machine learning systems provide answers without explanation - without a rationale or insight into the machine “thinking”. In order to function as trusted teammates, machines must be able to explain their decisions and actions. This transparency is a product of both content and communication. NASA’s Autonomy Teaming & TRAjectories for Complex Trusted Operational Reliability (ATTRACTOR) project seeks to build a basis for certification of autonomous systems via establishing metrics for trustworthiness and trust in multi-agent team interactions, using AI (Artificial Intelligence) explainability and persistent modeling and simulation, in the context of mission planning and execution, with analyzable trajectories. Inspired by Massively Multiplayer Online Role Playing Games (MMORPG) and Serious Gaming, the proposed ATTRACTOR modeling and simulation environment is similar to online gaming environments in which player (aka agent) participants interact with each other, affect their environment, and expect the simulation to persist and change regardless of any individual agent’s active participation. This persistent simulation environment will accommodate individual agents, groups of self-organizing agents, and large-scale infrastructure behavior. The effects of the emerging adaptation and coevolution can be observed and measured to building a basis of measurable trustworthiness and trust, toward certification of safety-critical autonomous systems.

Allen, B. Danette↗

DOE OSTI2021

Translating dosimetry of Dibenzo[ def,p ]chrysene (DBC) and metabolites across dose and species using physiologically based pharmacokinetic (PBPK) modeling

We report that Dibenzo[def,p]chrysene (DBC) is an environmental polycyclic aromatic hydrocarbon (PAH) that causes tumors in mice and has been classified as a probable human carcinogen by the International Agency for Research on Cancer. Animal toxicity studies often utilize higher doses than are found in relevant human exposures. Additionally, like many PAHs, DBC requires metabolic bioactivation to form the ultimate toxicant, and species differences in DBC and DBC metabolite metabolism have been observed. To understand the implications of dose and species differences, a physiologically based pharmacokinetic model (PBPK) for DBC and major metabolites was developed in mice and humans. Metabolism parameters used in the model were obtained from experimental in vitro metabolism assays using mice and human hepatic microsomes. PBPK model simulations were evaluated against mice dosed with 15 mg/kg DBC by oral gavage and human volunteers orally microdosed with 29 ng of DBC. DBC and its primary metabolite DBC-11,12-diol were measured in blood of mice and humans, while in urine, the majority of DBC metabolites were obeserved as conjugated DBC-11,12-diol, conjugated DBC tetrols, and unconjugated DBC tetrols. The PBPK model was able to predict the time course concentrations of DBC, DBC-11,12-diol, and other DBC metabolites in blood and urine of human volunteers and mice with reasonable accuracy. Agreement between model simulations and measured pharmacokinetic data in mice and human studies demonstrate the success and versatility of our model for interspecies extrapolation and applicability for different doses. Furthermore, our simulations show that internal dose metrics used for risk assessment do not necessarily scale allometrically, and that PBPK modeling provides a reliable approach to appropriately account for interspecies differences in metabolism and physiology.

59 BASIC BIOLOGICAL SCIENCES↗

NASA NTRS2014

Don't Trust a Management Metric, Especially in Life Support

Goodhart's law states that metrics do not work. Metrics become distorted when used and they deflect effort away from more important goals. These well-known and unavoidable problems occurred when the closure and system mass metrics were used to manage life support research. The intent of life support research should be to develop flyable, operable, reliable systems, not merely to increase life support system closure or to reduce its total mass. It would be better to design life support systems to meet the anticipated mission requirements and user needs. Substituting the metrics of closure and total mass for these goals seems to have led life support research to solve the wrong problems.

life support↗

DOE OSTI2023

Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons

Neural networks (NNs) are currently changing the computational paradigm on how to combine data with mathematical laws in physics and engineering in a profound way, tackling challenging inverse and ill-posed problems not solvable with traditional methods. However, quantifying errors and uncertainties in NN-based inference is more complicated than in traditional methods. This is because in addition to aleatoric uncertainty associated with noisy data, there is also uncertainty due to limited data, but also due to NN hyperparameters, overparametrization, optimization and sampling errors as well as model misspecification. Although there are some recent works on uncertainty quantification (UQ) in NNs, there is no systematic investigation of suitable methods towards quantifying the total uncertainty effectively and efficiently even for function approximation, and there is even less work on solving partial differential equations and learning operator mappings between infinite-dimensional function spaces using NNs. In this work, we present a comprehensive framework that includes uncertainty modeling, new and existing solution methods, as well as evaluation metrics and post-hoc improvement approaches. Further, to demonstrate the applicability and reliability of our framework, we present an extensive comparative study in which various methods are tested on prototype problems, including problems with mixed input-output data, and stochastic problems in high dimensions. In the Appendix, we include a comprehensive description of all the UQ methods employed. Further, to help facilitate the deployment of UQ in Scientific Machine Learning research and practice, we present and develop in [1] an open-source Python library (github.com/Crunch-UQ4MI/neuraluq), termed NeuralUQ, that is accompanied by an educational tutorial and additional computational experiments.

11 physics-informed neural networks↗

DOE OSTI2022

Optimal Portfolio Design of Distributed Energy Resources on Puerto Rico Distribution Feeders with Long Outages after Hurricane Maria

This work details a project to design reliable, resilient, and cost-effective networked microgrids considering grid constraints and resilience metrics focused on Puerto Rico distribution feeder locations with long outages after Hurricane Maria. The project consisted primarily of modeling and simulation tasks that accomplished the following objectives: 1. Selected 10 distribution feeder models in vulnerable areas. The sample feeders are geographically distributed across Puerto Rico and vary in length to capture the wide variety of feeders on the island. 2. Determined the optimal location and sizing of distributed energy resources (DERs) on the identified distribution feeders. The systems considered as part of the microgrid solutions were solar photovoltaic (PV), battery energy storage systems (BESS) and distributed fossil fuel generation (DFFG). 3. Estimated the cost-benefit of the proposed DER portfolios. 4. Provided a set of final recommendations that inform decision making on how to do targeted planning analysis for microgrids that can supply energy to critical infrastructures.

24 POWER TRANSMISSION AND DISTRIBUTION↗

NASA NTRS1985

Numerical Estimation of the Curvature of Biological Surfaces

Many biological systems may profitably be studied as surface phenomena. A model consisting of isotropic growth of a curved surface from a flat sheet is assumed. With such a model, the Gaussian curvature of the final surface determines whether growth rate of the surface is subharmonic or superharmonic. These properties correspond to notions of convexity and concavity, and thus to local excess growth and local deficiency of growth. In biological models where the major factors controlling surface growth are intrinsic to the surface, researchers thus gained from geometrical study information on the differential growth undergone by the surface. These ideas were applied to an analysis of the folding of the cerebral cortex, a geometrically rather complex surface growth. A numerical surface curvature technique based on an approximation to the Dupin indicatrix of the surface was developed. A metric for comparing curvature estimates is introduced, and considerable numerical testing indicated the reliability of this technique.

Todd, P. H.↗

NASA NTRS2004

Adaptive Bio-Inspired Wireless Network Routing for Planetary Surface Exploration

Wireless mobile networks suffer connectivity loss when used in a terrain that has hills, and valleys when line of sight is interrupted or range is exceeded. To resolve this problem and achieve acceptable network performance, we have designed an adaptive, configurable, hybrid system to automatically route network packets along the best path between multiple geographically dispersed modules. This is very useful in planetary surface exploration, especially for ad-hoc mobile networks, where computational devices take an active part in creating a network infrastructure, and can actually be used to route data dynamically and even store data for later transmission between networks. Using inspiration from biological systems, this research proposes to use ant trail algorithms with multi-layered information maps (topographic maps, RF coverage maps) to determine the best route through ad-hoc network at real time. The determination of best route is a complex one, and requires research into the appropriate metrics, best method to identify the best path, optimizing traffic capacity, network performance, reliability, processing capabilities and cost. Real ants are capable of finding the shortest path from their nest to a food source without visual sensing through the use of pheromones. They are also able to adapt to changes in the environment using subtle clues. To use ant trail algorithms, we need to define the probability function. The artificial ant is, in this case, a software agent that moves from node to node on a network graph. The function to calculate the fitness (evaluate the better path) includes: length of the network edge, the coverage index, topology graph index, and pheromone trail left behind by other ant agents. Each agent modifies the environment in two different ways: 1) Local trail updating: As the ant moves between nodes it updates the amount of pheromone on the edge; and 2) Global trail updating: When all ants have completed a tour the ant that found the shortest route updates the edges in its path.

Alena, Richard I.↗

NASA NTRS2009

The Case for Distributed Engine Control in Turbo-Shaft Engine Systems

The turbo-shaft engine is an important propulsion system used to power vehicles on land, sea, and in the air. As the power plant for many high performance helicopters, the characteristics of the engine and control are critical to proper vehicle operation as well as being the main determinant to overall vehicle performance. When applied to vertical flight, important distinctions exist in the turbo-shaft engine control system due to the high degree of dynamic coupling between the engine and airframe and the affect on vehicle handling characteristics. In this study, the impact of engine control system architecture is explored relative to engine performance, weight, reliability, safety, and overall cost. Comparison of the impact of architecture on these metrics is investigated as the control system is modified from a legacy centralized structure to a more distributed configuration. A composite strawman system which is typical of turbo-shaft engines in the 1000 to 2000 hp class is described and used for comparison. The overall benefits of these changes to control system architecture are assessed. The availability of supporting technologies to achieve this evolution is also discussed.

Culley, Dennis E.↗

DOE OSTI2020

PV_ICE: PV in the Circular Economy, Dynamic Energy and Materials Tool (PV ICE)

This open-source tool implements Circular Economy metrics for photovoltaic (PV) materials. It can be used to quantify and assign a value framework to efforts on re-design, reduction, replacement, reuse, recycling, and lifetime and reliability increases in the PV value chain. The PV_ICE is leveraging published data from different sources on PV manufacturing and predicted technological changes. Input data is being compiled here This tool will help implement circularity metrics, quantify and assign a value framework to efforts on re-design, reduction, replacement, reusage, recycling, and lifetime and reliability increases on PV.

Ovaitt, nee Ayala Pelaez, Silvana↗

DOE OSTI2022

Assessment methods for determining small changes in hearing performance over time

Although the behavioral pure-tone threshold audiogram is considered the gold standard for quantifying hearing loss, assessment of speech understanding, especially in noise, is more relevant to quality of life but is only partly related to the audiogram. Metrics of speech understanding in noise are therefore an attractive target for assessing hearing over time. However, speech-in-noise assessments have more potential sources of variability than pure-tone threshold measures, making it a challenge to obtain results reliable enough to detect small changes in performance. Here, this review examines the benefits and limitations of speech-understanding metrics and their application to longitudinal hearing assessment, and identifies potential sources of variability, including learning effects, differences in item difficulty, and between- and within-individual variations in effort and motivation. We conclude by recommending the integration of non-speech auditory tests, which provide information about aspects of auditory health that have reduced variability and fewer central influences than speech tests, in parallel with the traditional audiogram and speech-based assessments.

60 APPLIED LIFE SCIENCES↗

DOE OSTI2022

Power System Resilience Evaluation Framework and Metric Review: Preprint

Power system resilience is an emerging hot topic in recent years to study the increasing threats of extreme events such as natural disasters, severe weather, and cyberattacks. Although many research works have been done to define, model, and quantify resilience from different aspects, the lack of universally accepted resilience metrics and evaluation methods makes it difficult to assess and compare resilience across different power systems like what is typically done in power system reliability studies. In this paper, we first review the definitions of resilience and summarized two core concepts shared by most literature. On top of that, we conduct a thorough review of resilience metrics and develop a new framework to assess power system resilience from two perspectives, i.e., pre-event estimation and post-event evaluation, to capture system resilience performance in both general and specific fashions. Existing resilience metrics are summarized and categorized using the proposed framework, where recommendations are also proposed to capture core concepts of resilience.

power system resilience↗

DOE OSTI2024

Prioritizing circular economy strategies for sustainable PV deployment at the TW scale

Global decarbonization requires an unprecedented scale-up of photovoltaic (PV) manufacturing and deployment. The material demand and eventual end of life management associated with multi-TW scale deployment poses many challenges. Circular Economy (CE) and it's associated R-Actions (Reduce, Reuse, Recycle) have been proposed to mitigate end of life management and material sourcing concerns. However, CE metrics typically focus on a single product and only consider mass, excluding energy flows. This work leverages the PV in Circular Economy (PV ICE) tool to quantify the deployment, mass, and energy impacts of R-Actions and proposed sustainable PV designs in the context of achieving energy transition deployment goals (75 TW in 2050). 13 module scenarios are established and evaluated across 6 capacity, mass and energy metrics to identify tradeoffs and priorities. We find that increasing module efficiency can reduce near-term material demands up to 30% and improve energy metrics by up to 9%. Material circularity (recycling) can minimize lifecycle wastes and reduce material demands at the cost of higher energy demands. Increasing module lifetime, including reliability improvements and reuse strategies, is effective at reducing both material (>10%) and energy demands (24%). Uniquely, lifetime improvements maximize benefits and minimize the harms across all six metrics while achieving multi-TW scale deployment.

Photovoltaics↗

NASA NTRS2012

Availability Estimation for Facilities in Extreme Geographical Locations

A value added analysis for the Reliability. Availability and Maintainability of McMurdo Ground Station was developed, which will be a useful tool for system managers in sparing, maintenance planning and determining vital performance metrics needed for readiness assessment of the upgrades to the McMurdo System. Output of this study can also be used as inputs and recommendations for the application of Reliability Centered Maintenance (RCM) for the system. ReliaSoft's BlockSim. a commercial Reliability Analysis software package, has been used to model the availability of the system upgrade to the National Aeronautics and Space Administration (NASA) Near Earth Network (NEN) Ground Station at McMurdo Station in the Antarctica. The logistics challenges due to the closure of access to McMurdo Station during the Antarctic winter was modeled using a weighted composite of four Weibull distributions. one of the possible choices for statistical distributions throughout the software program and usually used to account for failure rates of components supplied by different manufacturers. The inaccessibility of the antenna site on a hill outside McMurdo Station throughout one year due to severe weather was modeled with a Weibull distribution for the repair crew availability. The Weibull distribution is based on an analysis of the available weather data for the antenna site for 2007 in combination with the rules for travel restrictions due to severe weather imposed by the administrating agency, the National Science Foundation (NSF). The simulations resulted in an upper bound for the system availability and allowed for identification of components that would improve availability based on a higher on-site spare count than initially planned.

Fischer, Gerd M.↗

DOE OSTI2020

Using the red chromatic coordinate to characterize the phenology of forest canopy photosynthesis

Vegetation phenology has received increasing attention in climate change research. Near-surface sensing using digital repeat photography has proven to be useful for ecosystem-scale monitoring of vegetation phenology. However, our understanding of the link between phenological metrics derived from digital repeat photography and the phenology of forest canopy photosynthesis is still incomplete, especially for evergreen plant species. Using 49 site-years of digital images from the PhenoCam Network from eight evergreen needleleaf forest (ENF) and six deciduous broadleaf forest (DBF) sites in North America, we explored the potential of the green chromatic (GCC) and red chromatic coordinates (RCC) in tracking forest canopy photosynthesis by comparing camera-derived start- and end-of-growing season (SOS and EOS, respectively) with corresponding estimates derived from eddy covariance-derived daily gross primary productivity (GPP). We found that for DBF sites, both GCC and RCC performed comparable in capturing SOS and EOS. However, similar to earlier studies, GCC had limited potential in capturing GPP-based SOS or EOS for ENF sites. In contrast, we found RCC was a better predictor of both GPP-based SOS and EOS for ENF sites. Environmental and ecological explanations were both provided that phenological transitions derived from RCC were highly correlated with spring and autumn meteorological conditions, as well as having overall higher correlations with phenology based on LAI, a critical variable for describing canopy development. Our results demonstrate that RCC is an underappreciated metric for tracking vegetation phenology, especially for ENF sites where GCC failed to provide reliable estimates for GPP-based SOS or EOS. Our results improve confidence in using digital repeat photography to characterize the phenology of canopy photosynthesis across forest types.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Pervaporative Dehydration of 2,3-Butanediol by Dense Poly(vinylidene fluoride) Hollow Fiber Membranes: Parameter Estimation, Process Design, and Technoeconomic Evaluation under Uncertainty

Pervaporation, combined with other separation processes, can effectively remove water from fermentation product streams, making it highly suitable for purifying alcohols like 2,3-butanediol (BDO). In this study, a dense poly(vinylidene fluoride) (PVDF) hollow fiber membrane module prototype was fabricated for BDO dehydration, achieving >0.2 LMH total flux and >95% BDO rejection. With a Markov chain Monte Carlo (MCMC) approach, Bayesian inference was used to quantify the uncertainty of the permeance parameters. A membrane cascade model was developed to scale up a process that purifies a preconcentrated BDO feed (70 wt %) to high purity (90 wt %). Through propagation of the uncertainty of the parameters and sensitivity analyses of the process variables, a cascade design was recommended. Despite data and model limitations, the framework enabled a reliable system analysis and economic evaluation, validated through tight confidence intervals in key process metrics, establishing the foundation for future applications of Bayesian methods in membrane-based processes.

Animal feed↗

NASA NTRS2005

Safety and Mission Assurance Performance Metric

The safety and mission assurance (S&MA) performance metric is a method that provides a process through which the managers of a large, complex program can readily understand and assess the accepted risk, the problems, and the associated reliability of the program. Conceived for original use in helping to assure the safety and success of the International Space Station (ISS) program, the S&MA performance metric also can be applied to other large and complex programs and projects. The S&MA-performance-metric data products comprise one or more tables (possibly also one or more graphs) that succinctly display all of the information relevant (and no information that is irrelevant) to management decisions that must be made to assure the safety and success of a program or project, thereby facilitating such decisions.

Holsomback, Jerry↗