Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “reliability metrics”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

A Predictive Approach to Eliminating Errors in Software Code

NASA s Metrics Data Program Data Repository is a database that stores problem, product, and metrics data. The primary goal of this data repository is to provide project data to the software community. In doing so, the Metrics Data Program collects artifacts from a large NASA dataset, generates metrics on the artifacts, and then generates reports that are made available to the public at no cost. The data that are made available to general users have been sanitized and authorized for publication through the Metrics Data Program Web site by officials representing the projects from which the data originated. The data repository is operated by NASA s Independent Verification and Validation (IV&V) Facility, which is located in Fairmont, West Virginia, a high-tech hub for emerging innovation in the Mountain State. The IV&V Facility was founded in 1993, under the NASA Office of Safety and Mission Assurance, as a direct result of recommendations made by the National Research Council and the Report of the Presidential Commission on the Space Shuttle Challenger Accident. Today, under the direction of Goddard Space Flight Center, the IV&V Facility continues its mission to provide the highest achievable levels of safety and cost-effectiveness for mission-critical software. By extending its data to public users, the facility has helped improve the safety, reliability, and quality of complex software systems throughout private industry and other government agencies. Integrated Software Metrics, Inc., is one of the organizations that has benefited from studying the metrics data. As a result, the company has evolved into a leading developer of innovative software-error prediction tools that help organizations deliver better software, on time and on budget.

Source record↗

Creating A Consistent Historical NASA POWER Solar Radiation Dataset to Support Renewable Energy, Building Energy Efficiency and Agro-Climatology Decisions

Prediction of Worldwide Energy Resources (POWER) project provides irradiance dataset to support renewable energy, building energy efficiency and agricultural needs. These datasets are derived from Global Energy and Water Cycle Experiment Surface Radiation Budget (GEWEX SRB) and Clouds and the Earth’s Radiant Energy System (CERES SYN1Deg). A systematic bias has been reported between these two datasets for the years with overlapping observations. For obtaining a consistent climate data record spanning the entire time record of observations, it is crucial to understand and remove the bias in the irradiance dataset. Inconsistency in solar radiation data can lead to inaccurate conclusions about solar energy potential and obscure real trends in solar radiation patterns that would impact energy availability assessments. In this study, we adapt quantile mapping approach to remove the systematic bias and to improve reliability of shortwave and longwave irradiance data. We present a validation of the bias corrected data against ground truth. For each 1° latitude and 1° longitude grid box across the globe, we match the CDFs of the reference dataset (CERES SYN1Deg) to that of the SRB dataset, thereby, adjusting the irradiance values to match the empirical distribution of two different measurements. The performance of quantile mapping is evaluated by using the metrics such as Mean Absolute Deviation (MAD). The results indicate that the quantile mapping significantly improves the accuracy and reliability of solar irradiance dataset especially for the weather conditions associated with high cloud cover and extreme irradiance values. The initial range of MAD for the studied sites for daily data was 4 to 19 Wm-2. After correction these reduced to 3 to 7 Wm-2. The findings from this study have important implications for solar energy system design, agricultural planning, and climate modeling community. Reducing the inconsistency and biases in solar irradiance dataset can enable better planning and operation of solar energy systems, leading to increased efficiency and cost-effectiveness. Additionally, this work also contributes to the statistical post-processing techniques in the renewable energy domain and highlights the potential of historical and near-real-time NASA POWER dataset as a valuable resource for solar energy research and applications.

POWER↗

Exploratory study on performance measures as indicators of IS effectiveness

The Information Systems Directorate at JSC/NASA has undertaken the reevaluation of its performance measures process and measures. Under the direction of a quality approach it is essential to identify an external perspective of how well an organization is performing. This study was conducted with two major objectives: (1) survey and summarize the academic literature on performance measures as indicators of information systems (IS) effectiveness; and (2) survey organizations for their experience in measuring for IS effectiveness. Four approaches to measuring the effectiveness of IS performance were identified: (1) listen to the customer for the things they need; (2) align with corporate goals; (3) benchmark against well-respected organizations; and (4) ask yourself what critical factors lead to success. The list of known methods for soliciting customer feedback are as follows: (1) executive visit; (2) survey, interview, and focus group; (3) complaints and compliments; and (4) service level agreements. A common set of characteristics that satisfy customers was identified from the literature. The list includes elements such as the following: accuracy, timeliness, relevance, understandability, reliability, and completeness. Future research in this topic area should prove beneficial to determine the metrics for external validity.

Bishop, Peter C.↗

Normative Baseline Oculomotor Performance

Future missions to the moon or Mars will require the crew to monitor and assess their health and performance more autonomously, necessitating approaches that are easily useable and interpretable by non-clinicians. The eye-movement-based performance metrics (oculometrics) obtained from NASA-patented technology developed at the Visuomotor Control Laboratory can reliably detect and discriminate the source of mild neural impairment relative to an individual’s baseline visual and sensorimotor performance and can predict performance in manual control tasks. This study aims to develop a database of normative performance that can be used to enable the detection of impairment without a within-subject baseline and to facilitate power analyses for the design of future studies.

eye movements↗

Review of Controllers for Low-Power Free-Piston Stirling Convertors

This work provides a complete review of controller strategies for free-piston Stirling convertors (FPSCs) and design recommendations for future controller development efforts. First, basic operating principles for the control of FPSCs and design variables for the FPSC controllers are described. Next, past controller approaches are reviewed and categorized based on the design variables. State-of-the-art controllers are also summarized by their key features, which are then hypothetically extended to flight designs to compare their critical metrics such as the estimated size, weight, and power with cost (SWaP-C) and reliability. Finally, design recommendations are suggested to improve the state-of-the-art controllers for future flight development projects.

Free-Piston Stirling Convertor Controller↗

Lithium-Ion Battery Diagnostics Using Electrochemical Impedance via Machine-Learning

Diagnosing battery states such as health, state-of-charge, or temperature is crucial for ensuring the safety and reliability of electrochemical energy storage systems. While some states, such as temperature, may be measured using cheap sensors, accurate diagnosis of battery health metrics usually requires time-consuming performance measurements, making them infeasible for use in real-world operation. These health metrics can be measured during lab-testing and then estimated on-line using predictive life models or via state observer algorithms such as Kalman filters, but these predictive methods should be supplemented by actual measurement of battery health whenever possible to ensure reliability. Rapid measurement of battery health may be done by various types of fast diagnostic techniques such as electrochemical impedance spectroscopy (EIS), which can be performed in only a few minutes and require only a fraction of the energy and power needed for a full charge and discharge measurement. But there is a substantial challenge for estimating battery health using EIS data, as EIS is sensitive to cell temperature, state-of-charge, current, and resting time in addition to health. Thus, utilizing EIS data to predict battery capacity requires correcting for all these additional variables, a task that is extremely difficult to handle analytically. This talk utilizes machine-learning methods to estimate the effectiveness of battery capacity prediction from EIS data, leveraging a data set of hundreds of EIS measurements recorded at varying temperature and state-of-charge throughout a 500-day aging study of 32 commercial, large-format NMC-Graphite lithium-ion batteries. Using EIS as input to machine-learning models is complicated by the nonlinear response of impedance to battery health, temperature, and state-of-charge, as well as the collinearity between the impedance response at neighboring frequencies, which can easily lead to overfit models. To train robust models, features from EIS data need to be extracted from the data or some subset of critical frequencies selected. Many approaches for extracting and selecting features from EIS data from electrochemical analysis and machine-learning fields were identified for analysis: using the entire raw spectra; selection of one, two, or many frequencies from the entire spectra; selecting interesting points from the EIS measurement using domain knowledge; fitting EIS with an equivalent-circuit model; calculating statistics on the raw impedance values; and reducing the dimensionality of the data using unsupervised linear (principal component analysis) and non-linear (uniform manifold approximation and projection) methods. These approaches were rigorously compared using a machine-learning pipeline approach, training linear, Gaussian process, and random forest regression models and quantifying performance using cross-validation as well as a held-out test set. An artificial neural network model trained on the raw spectra was also tested. Promising pipelines were fine-tuned via Bayesian hyperparameter optimization using cross-validation loss and training with class-specific weights to counter data set imbalance. The most reliable method for utilizing impedance in this work was the selection of two optimal frequencies through an exhaustive search, resulting in about 2% mean absolute error on test data for both Gaussian process and random forest model architectures. Interrogation of a variety of models reveals critical frequencies of 100 Hz and 103 Hz for this data set, though the optimal set of frequencies is not necessarily intuitive, i.e., the best performing models are not simply those that use impedance at frequencies that have the highest correlation to the relative discharge capacity. The best performing model is an ensemble model, which is able to predict battery capacity with 1.9% mean absolute error for unseen cells using impedance recorded at a variety of temperatures and states-of-charge.

battery↗

A Briefing on Metrics and Risks for Autonomous Decision-Making in Aerospace Applications

Significant technology advances will enable future aerospace systems to safely and reliably make decisions autonomously, or without human interaction. The decision-making may result in actions that enable an aircraft or spacecraft in an off-nominal state or with slightly degraded components to achieve mission performance and safety goals while reducing or avoiding damage to the aircraft or spacecraft. Some key technology enablers for autonomous decision-making include: a continuous state awareness through the maturation of the prognostics health management field, novel sensor development, and the considerable gains made in computation power and data processing bandwidth versus system size. Sophisticated algorithms and physics based models coupled with these technological advances allow reliable assessment of a system, subsystem, or components. Decisions that balance mission objectives and constraints with remaining useful life predictions can be made autonomously to maintain safety requirements, optimal performance, and ensure mission objectives. This autonomous approach to decision-making will come with new risks and benefits, some of which will be examined in this paper. To start, an account of previous work to categorize or quantify autonomy in aerospace systems will be presented. In addition, a survey of perceived risks in autonomous decision-making in the context of piloted aircraft and remotely piloted or completely autonomous unmanned autonomous systems (UAS) will be presented based on interviews that were conducted with individuals from industry, academia, and government.

Frost, Susan↗

A Data Quality-Aware Framework to Reliably Forecast Photovoltaic Generation and Consumer Load for an Improved Resilience of Microgrids

Photovoltaic (PV) power and consumer load forecasting plays a critical role to ensure operational resilience of the electric grid. Most data-driven forecasting algorithms rely heavily on the continuous availability of good quality data for periodic training and validation. When deployed at the grid’s edge, prolonged disruptions to communications during extreme events degrade data quality. Factors such as missing observations, epistemic uncertainties, data drift, and concept drift are manifestations of data quality that impact the generalization of such field-deployed forecasting models. Currently, there exists no mechanism in the literature to dynamically switch between models under varying degrees of data quality as quantified by certain metrics for each factor highlighted above. This paper addresses this shortcoming by conceptually introducing a data qualityaware framework for reliable PV generation and consumer load forecasting. The framework’s design incorporates components of missing values, divergence tests, and continuous monitoring of generalization performance to detect changes in data quality caused by communications disruptions and trigger specific classes of forecasting models grouped under three use cases (UC1- UC3). As a first step towards validating this framework, real data collected from an actual field microgrid system is used to demonstrate the viability of the three use cases. Results show that the performance is the best in UC1 with an unadjusted R-square value of 0.954, followed by 0.939 for UC2 and 0.757 for UC3.

Sundararajan, Aditya↗

Optimized Biasing of Pump Laser Diodes in a Highly Reliable Metrology Source for Long-Duration Space Missions

Optical metrology system reliability during a prolonged space mission is often limited by the reliability of pump laser diodes. We developed a metrology laser pump module architecture that meets NASA SIM Lite instrument optical power and reliability requirements by combining the outputs of multiple single-mode pump diodes in a low-loss, high port count fiber coupler. We describe Monte-Carlo simulations used to calculate the reliability of the laser pump module and introduce a combined laser farm aging parameter that serves as a load-sharing optimization metric. Employing these tools, we select pump module architecture, operating conditions, biasing approach and perform parameter sensitivity studies to investigate the robustness of the obtained solution.

808 mm diode pumps↗

Connected Thermostat Alternatives for Room Air Conditioners and Minisplit Heat Pumps

The availability of smart, connected thermostats has improved climate control, energy efficiency, and grid demand-response programs for central HVAC systems. However, a significant gap exists in addressing integrated control systems for point-source heating and cooling systems such as window air-conditioners (window ACs) and mini-split heat pumps (MSHPs). This report examines the emerging market of third-party connected thermostats tailored for these systems, focusing on their effectiveness, reliability, and potential barriers to adoption.This study evaluates several commercially available products designed for room ACs and MSHPs through a series of laboratory tests. While these infrared-based (IR-based) thermostats offer remote temperature control and scheduling via mobile apps, our findings reveal that none are seamless, with reliability of basic functions being a critical factor. Promising features include integration of indoor air quality metrics and time-of-use pricing, but the latter are not yet available in the U.S. Barriers to broad user acceptance include non-seamless setup processes, challenges in thermostat placement, and unclear product differentiation. There is a pressing need for research and development in enabling MSHPs and central thermostats to coordinate, enhancing energy savings and comfort in retrofit applications. This study underscores the importance of further innovation in connected thermostat technology to address the diverse needs of single-zone HVAC systems and promote efficient energy management in households.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

A Resilient Integrated Resource Planning Framework for Transmission Systems: Analysis and Optimization

This article presents a resilient Integrated Resource Planning (IRP) framework designed for transmission systems, with a specific focus on analyzing and optimizing responses to High-Impact Low-Probability (HILP) events. The framework aims to improve the resilience of transmission networks in the face of extreme events by prioritizing the assessment of events with significant consequences. Unlike traditional reliability-based planning methods that average the impact of various outage durations, this work adopts a metric based on the proximity of outage lines to generators to select HILP events. The system’s baseline resilience is evaluated by calculating load curtailment in different parts of the network resulting from HILP outage events. The transmission network is represented as an undirected graph. Graph-theoretic techniques are used to identify islands with or without generators, potentially forming segmented grids or microgrids. This article introduces Expected Load Curtailment (ELC) as a metric to quantify the system’s resilience. The framework allows for the re-evaluation of system resilience by integrating additional generating resources to achieve desired resilience levels. Optimization is performed in the re-evaluation stage to determine the optimal placement of distributed energy resources (DERs) for enhancing resilience, i.e., minimizing ELC. Case studies on the IEEE 24-bus system illustrate the effectiveness of the proposed framework. In the broader context, this resilient IRP framework aligns with energy sustainability goals by promoting robust and resilient transmission networks, as the optimal placement of DERs for resilience enhancement not only strengthens the system’s ability to withstand and recover from disruptions but also contributes to efficient resource utilization, advancing the overarching goal of energy sustainability.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Verification of Viscous Goal-Based Anisotropic Mesh Adaptation

Adaptive unstructured mesh techniques have a limited, but growing impact on production analysis workflows where the control of discretization error is critical to obtaining reliable simulation results. Recent progress has matured a number of independent implementations of flow solvers, anisotropic metric construction methods, and anisotropic mesh adaptation mechanics. A key ingredient for the broader acceptance of unstructured mesh adaptation is the verification of these implementations. Anisotropic metric construction methods are evaluated with analytically defined primal fields and the corresponding entropy variables as adjoint fields. This allows the comparison of different metric formulations and different implementations of the same formulation without the complications of a flow and adjoint solution method. The convergence of the output associated with the entropy variable adjoint is studied for mesh adaptation to these fields and a manufactured solution. Mesh adapted drag output is studied for two simple wings in compressible laminar flow to show fine-mesh convergence of multiple metric construction methods to less than a single drag count. The documentation of these verification exercises helps to prepare these goal-based methods for routine use in more complex simulations for production workflows.

mesh adaptation↗

Increasing Discovery and Usability of Earth Science Satellite Data with My NASA Data

For 20 years, the My NASA Data project at NASA Langley Research Center has developed innovative approaches to increase the use of NASA’s satellite data by learners. My NASA Data offers a variety of authentic Earth Science datasets and a data visualization tool, eliminating the need for educators and/or learners to obtain specialized knowledge of GIS data formats and software to access and use authentic Earth Science data. While there is no shortage of available data, as federal government agencies such as NASA house petabytes of freely accessible Earth Science datasets, much of the data are only available for download and visualization in specialized formats and software, limiting their accessibility to educators and learners, especially those in primary and secondary school. Using the Google Earth Engine platform, the My NASA Data team has recently reinvented their data visualization tool, called the Earth System Data Explorer (ESDE). The ESDE gives users the capability to explore over 60 Earth Science satellite datasets in a multitude of formats such as maps, graphs, and data table Its new and improved user interface design was developed based on the preferences of educators, whom the My NASA Data project has over 20 years’ experience working with. Earth Science and GIS Subject Matter Experts (SMEs) structured the data in a professional and scientific manner. During Fiscal Year 2023, the My NASA Data website received over 1 million digital engagements, with over one-third being visitors to the data visualization tool. These metrics highlight the interest in a visualization tool that is simple and free to use with reliable and trusted datasets. The ESDE empowers users to readily relate and analyze NASA Earth Science data within their area of interest. The team used a user-centered design (UCD) framework to receive and incorporate feedback into the application’s design. Core requested features include the ability to create time series graphs, comparative analysis of maps, and download the data as CSV file. Responses indicate that advances in data visualization tools such as the ESDE make authentic Earth Science data more accessible. This presentation will cover how the My NASA Data project develops tools to enhance data discovery and accessibility, as well as how SME and user suggestions are incorporated.

Desiray Wilson↗

Preliminary evaluation of a micro-based repeated measures testing system

A need exists for an automated performance test system to study the effects of various treatments which are of interest to the aerospace medical community, i.e., the effects of drugs and environmental stress. The ethics and pragmatics of such assessment demand that repeated measures in small groups of subjects be the customary research paradigm. Test stability, reliability-efficiency and factor structure take on extreme significance; in a program of study by the U.S. Navy, 80 percent of 150 tests failed to meet minimum metric requirements. The best is being programmed on a portable microprocessor and administered along with tests in their original formats in order to examine their metric properties in the computerized mode. Twenty subjects have been tested over four replications on a 6.0 minute computerized battery (six tests) and which compared with five paper and pencil marker tests. All tests achieved stability within the four test sessions, reliability-efficiencies were high (r greater than .707 for three minutes testing), and the computerized tests were largely comparable to the paper and pencil version from which they were derived. This computerized performance test system is portable, inexpensive and rugged.

Kennedy, Robert S.↗

Power System Resilience Evaluation Framework and Metric Review

Power system resilience has been an emerging hot topic in recent years to investigate the increasing threats of extreme events, such as natural disasters, severe weather, and cyberattacks. Although much research has been done to define, model, and quantify resilience from different aspects, the lack of universally accepted evaluation methods and resilience metrics makes it difficult to assess and compare resilience across different power systems, such as what is typically done in power system reliability studies. In this paper, first, we review the definitions of resilience, and we summarize two core concepts shared by most of the literature. Then, we develop a new framework to assess power system resilience from two perspectives - i.e., pre-event estimation and post-event evaluation - to capture system resilience performance in both general and specific fashions. We conduct a thorough review of existing resilience metrics and categorize them using the proposed framework, where recommendations are also proposed to capture core concepts of resilience.

power system resilience↗

ARCSTONE: Calibration of Lunar Spectral Reflectance from Space

Detecting and improving the scientific understanding of global trends in complex Earth systems, such as climate, increasingly depends on assimilating datasets from multiple instruments and platforms over decadal timescales. Calibration accuracy, stability, and inter-consistency among different instruments are key to developing reliable composite data records from sensors in low Earth and geostationary orbits, but achieving sufficiently low uncertainties for these performance metrics poses a significant challenge. Space-borne instruments commonly carry on-board references for calibration at various wavelengths, but these increase mass and mission complexity, and are subject to degradation in the space environment. The Moon can be considered a natural solar diffuser which can be observed as a calibration target by most spaceborne Earth-observing instruments. Since the lunar surface reflectance is effectively time-invariant, establishing the Moon as a high-accuracy calibration reference enables broad inter-calibration opportunities even between temporally non-overlapping instruments and provides an exo-atmospheric absolute radiometric standard. The ARCSTONE mission goal is to establish the Moon as a reliable reference for high-accuracy on-orbit calibration in the visible and near-infrared spectral region. The ARCSTONE instrument is a compact spectrometer, which will be packaged on a CubeSat intended for low Earth orbit. It will measure the lunar spectral reflectance with accuracy < 0.5% (k=1), sufficient to establish an SI-traceable absolute lunar calibration standard when referenced to the spectral solar irradiance across the 350 to 2300 nm spectral range. This lunar reference will help to enable high-accuracy absolute calibration and inter-calibration of past, current, and future Earth-observing sensors, meteorological imagers, and long-term climate monitoring satellite systems. The ARCSTONE team will present the development status of a full-spectral-range (FSR) instrument, the intended approach to calibration and characterization, and the planned path toward mission implementation.

calibration↗

ARCSTONE: Calibration of Lunar Spectral Reflectance from Space

Detecting and improving the scientific understanding of global trends in complex Earth systems, such as climate, increasingly depends on assimilating datasets from multiple instruments and platforms over decadal timescales. Calibration accuracy, stability, and inter-consistency among different instruments are key to developing reliable composite data records from sensors in low Earth and geostationary orbits, but achieving sufficiently low uncertainties for these performance metrics poses a significant challenge. Space-borne instruments commonly carry on-board references for calibration at various wavelengths, but these increase mass and mission complexity, and are subject to degradation in the space environment. The Moon can be considered a natural solar diffuser which can be observed as a calibration target by most spaceborne Earth-observing instruments. Since the lunar surface reflectance is effectively time-invariant, establishing the Moon as a high-accuracy calibration reference enables broad inter-calibration opportunities even between temporally non-overlapping instruments and provides an exo-atmospheric absolute radiometric standard. The ARCSTONE mission goal is to establish the Moon as a reliable reference for high-accuracy on-orbit calibration in the visible and near-infrared spectral region. The ARCSTONE instrument is a compact spectrometer, which will be packaged on a CubeSat intended for low Earth orbit. It will measure the lunar spectral reflectance with accuracy < 0.5% (k=1), sufficient to establish an SI-traceable absolute lunar calibration standard when referenced to the spectral solar irradiance across the 350 to 2300 nm spectral range. This lunar reference will help to enable high-accuracy absolute calibration and inter-calibration of past, current, and future Earth-observing sensors, meteorological imagers, and long-term climate monitoring satellite systems. The ARCSTONE team will present the development status of a full-spectral-range (FSR) instrument, the intended approach to calibration and characterization, and the planned path toward mission implementation.

Moon↗