SEARCH · Engineering Papers

Results for “METRICS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,333 records · Page 74

All sources NASA NTRS DOE OSTI

DOE OSTI2023

Anomalous behavior detection by an artificial intelligence-enabled system with multiple correlated sensors

Multi-metric artificial intelligence (AI)/machine learning (ML) models for detection of anomalous behavior of a machine/system are disclosed. The multi-metric AI/ML models are configured to detect anomalous behavior of systems having multiple sensors that measure correlated sensor metrics such as coolant distribution units (CDUs). The multi-metric AI/ML models perform the anomalous system behavior detection in a manner that enables both a reduction in the amount of sensor instrumentation needed to monitor the system's operational behavior as well as a corresponding reduction in the complexity of the firmware that controls the sensor instrumentation. As such, AI-enabled systems and corresponding methods for anomalous behavior detection disclosed herein offer a technical solution to the technical problem of increased failure rates of existing multi-sensor systems, which is caused by the presence of redundant sensor instrumentation that necessitates complex firmware for controlling the sensor instrumentation.

Serebryakov, Sergey↗

DOE OSTI2023

Acoustic signal based analysis of batteries

Systems and methods for acoustic signal based analysis, include obtaining acoustic response signal data of at least a portion of a battery cell, the acoustic response signal data comprising waveforms generated by transmitting one or more acoustic excitation signals into at least the portion of the battery cell and recording response vibration signals to the one or more acoustic excitation signals. One or more metrics are determined from at least the acoustic response signal data, the one or more metrics being determined based on correlation of the one or more metrics to one or more characteristics of battery cells and a reference model is generated from the one or more metrics. A test battery can be evaluated using the reference model. Actionable insights or recommendations can be generated based on the evaluation. The reference model can also be updated based on the evaluation.

Van Tassell, Barry J.↗

DOE OSTI2024

Measuring Sustainability of Solar Modules for Energy Transition: Mass, Energy, and Circularity

Transition to a carbon-free energy system is crucial for global decarbonization and underpins Circular Economy (CE) goals. Photovoltaic (PV) technology is required for Energy Transition, but manufacturing and circular pathways can be material, energy, and carbon-intensive. Therefore, we need a prioritization of sustainability strategies for PV evolution and lifecycle management in the context of Energy Transition. This study employs a suite of quantitative metrics to compare different proposed sustainability strategies for PV modules on their ability to achieve Energy Transition. Proposals for sustainable PV range from high-yield, high-efficiency paradigms, to short-lived and fully recyclable, to long-lasting, indestructible modules. We leverage a global decarbonization deployment schedule through 2100 with the open-source PV in Circular Economy (PV ICE) tool to quantify the impacts of different evolving module design scenarios covering the range of proposed sustainability strategies. First, modules are compared on effective capacity and required replacements to meet and maintain decarbonization capacity targets through 2100. We demonstrate the effects of lifetime, degradation, and reliability on effective capacity. Next, we quantify and compare virgin material demands and lifecycle wastes, examining the impacts of lifetime and recycling rates. Finally, and critically for renewable energy technologies, we quantify the energy demands required to achieve the decarbonization capacity targets and calculate energy balance metrics (net energy, energy return on investment). These results are then summarized into a metric matrix, demonstrating tradeoffs and the importance of longevity. Our suite of mass and energy metrics provides stakeholders and decision-makers with quantitative data on circular economy choices for PV in the energy transition, enabling informed evaluation of tradeoffs of different PV module designs and CE pathways.

circular economy↗

DOE OSTI2024

Life-Cycle Assessment Integration into Scalable Open-Source Numerical Models (LiAISON) for Analyzing Emerging Low-Carbon Technologies

Decarbonizing the industrial sector is a significant challenge in achieving a net-zero greenhouse gas (GHG) emissions economy by 2050 and the Paris Agreement, i.e., a global climate change mitigation target of achieving a maximum average temperature change potential of 1.5 degrees Celsius or less by 2100 with respect to pre-industrial levels. In the United States (US), the industrial sector accounts for 23% of total GHG emissions and is home to a number of hard-to-electrify activities. The chemicals subsector has the single largest subsector emissions profile after direct emissions from fossil fuel combustion and leakage from fossil fuel distribution systems. Within the chemicals subsector, many processes depend on hydrogen or ammonia precursors. Decarbonizing these two commodities would contribute significantly to decarbonizing the industrial sector as hydrogen could also be used for low carbon steel production (e.g., hydrogen-based direct reduction of iron) and other industrial applications. Emerging technologies require the application of prospective life cycle assessment (LCA), which can account for technology (foreground) scaling and process improvements via learning-by-doing, among others. In many cases, the future system context (background) in which the technologies are assumed to operate in is equally relevant. Background scenarios generated by integrated assessment models (IAM) can coherently incorporate potential future dynamics of the energy-climate-human-land system. Further, IAM scenarios are harmonized across socioeconomic and climate change mitigation pathways, which facilitates the comparability of prospective LCAs using different IAMs. We introduce an open source prospective LCA framework, the Life-cycle Assessment Integration into Scalable Open-source Numerical models (LiAISON), to analyze the non-linear relationships between technology foreground and the future energy system background across a series of midpoint and resource use metrics. The integration of LCA and IAM data is achieved using prospective environmental Impact assessment (PREMISE). We showcase it by assessing two Power-to-Hydrogen (PtH2) processes, namely Solid Oxide Electrolysis (SOE) and Polymer Electrolyte Membrane Electrolysis (PEME). We compare the technologies to a baseline of hydrogen production via natural gas-based Steam Methane Reforming (SMR) in a US context of multiple energy system and climate change mitigation futures. Besides providing an analysis that specifies the LCA results ranges with temporal and geospatial explicitness across the two technologies, metrics, and impact assessment methods, this research also aims to establish a base framework that can be expanded to use other IAM generated scenarios and US open-source life cycle inventory (LCI) databases. We find that the temporal environmental performance of either technology or their difference to SMR is directly influenced by the underlying background dynamics. Under baseline projections (i.e., no decarbonization goals), neither process reaches parity with the incumbent technology across several environmental metrics. Under the decarbonization scenarios, the underlying sectoral shifts result in declining impacts over time, compared to 2020 levels, except for metal depletion levels, which increase. The background shifts postulate a heavily decarbonized economy and energy system, which help technologies reach parity with SMR between 2040-2050 (RCP2.6) and 2030-2040 (RCP1.9) for global warming. Despite declines across several other metrics over time, neither PtH2 technology break even with SMR by 2100 besides for global warming.

decarbonizing↗

DOE OSTI2024

Life-Cycle Assessment Integration into Scalable Open-Source Numerical Models (LiAISON) for Analyzing Emerging Low-Carbon Technologies

Decarbonizing the industrial sector is a significant challenge in achieving a net-zero greenhouse gas (GHG) emissions economy by 2050 and the Paris Agreement, i.e., a global climate change mitigation target of achieving a maximum average temperature change potential of 1.5 degrees C or less by 2100 with respect to pre-industrial levels. In the United States (US), the industrial sector accounts for 23% of total GHG emissions and is home to a number of hard-to-electrify activities. The chemicals subsector has the single largest subsector emissions profile after direct emissions from fossil fuel combustion and leakage from fossil fuel distribution systems. Within the chemicals subsector, many processes depend on hydrogen or ammonia precursors. Decarbonizing these two commodities would contribute significantly to decarbonizing the industrial sector as hydrogen could also be used for low carbon steel production (e.g., hydrogen-based direct reduction of iron) and other industrial applications. Emerging technologies require the application of prospective life cycle assessment (LCA), which can account for technology (foreground) scaling and process improvements via learning-by-doing, among others. In many cases, the future system context (background) in which the technologies are assumed to operate in is equally relevant. Background scenarios generated by integrated assessment models (IAM) can coherently incorporate potential future dynamics of the energy-climate-human-land system. Further, IAM scenarios are harmonized across socioeconomic and climate change mitigation pathways, which facilitates the comparability of prospective LCAs using different IAMs. We introduce an open source prospective LCA framework, the Life-cycle Assessment Integration into Scalable Open-source Numerical models (LiAISON), to analyze the non-linear relationships between technology foreground and the future energy system background across a series of midpoint and resource use metrics The integration of LCA and IAM data is achieved using prospective environmental Impact assessment (PREMISE). We showcase it by assessing two Power-to-Hydrogen (PtH2) processes, namely Solid Oxide Electrolysis (SOE) and Polymer Electrolyte Membrane Electrolysis (PEME). We compare the technologies to a baseline of hydrogen production via natural gas-based Steam Methane Reforming (SMR) in a US context of multiple energy system and climate change mitigation futures. Besides providing an analysis that specifies the LCA results ranges with temporal and geospatial explicitness across the two technologies, metrics, and impact assessment methods, this research also aims to establish a base framework that can be expanded to use other IAM generated scenarios and US open-source life cycle inventory (LCI) databases. We find that the temporal environmental performance of either technology or their difference to SMR is directly influenced by the underlying background dynamics. Under baseline projections (i.e., no decarbonization goals), neither process reaches parity with the incumbent technology across several environmental metrics. Under the decarbonization scenarios, the underlying sectoral shifts result in declining impacts over time, compared to 2020 levels, except for metal depletion levels, which increase. The background shifts postulate a heavily decarbonized economy and energy system, which help technologies reach parity with SMR between 2040-2050 (RCP2.6) and 2030-2040 (RCP1.9) for global warming. Despite declines across several other metrics over time, neither PtH2 technology break even with SMR by 2100 besides for global warming.

decarbonizing↗

DOE OSTI2024

Acoustics-based noninvasive wafer defect detection

Techniques are provided for detecting wafer defects. Example techniques include exciting a wafer using an acoustic signal to cause the wafer to exhibit vibrations, measuring one or more of linear frequency response metrics or nonlinear frequency responses metrics associated with the vibrations, and identifying any defects in the wafer based at least in part on one or more of the linear frequency response metrics or nonlinear frequency responses metrics. In embodiments, the wafer includes bismuth telluride (Bi 2 Te 3 ).

Pantea, Cristian↗

DOE OSTI2026

FATHOMS-RAG: A Framework for the Assessment of Thinking and Observation in Multimodal Systems that use Retrieval Augmented Generation

Retrieval-augmented generation (RAG) has emerged as a promising paradigm for improving factual accuracy in large language models (LLMs). We introduce a benchmark designed to evaluate RAG pipelines as a whole, evaluating a pipelines ability to ingest several modalities of information. We present (1) a curated dataset of 93 questions designed to evaluate a pipeline's ability to ingest textual data, tables, images, multimodal data, and cross-document multimodal data; (2) a phrase-level recall metric for correctness; (3) a nearest-neighbor embedding classifier in an attempt to classify pipeline hallucinations; (4) a comparative evaluation of 2 pipelines built with open-source retrieval mechanisms and 4 closed-source foundational models; and (5) a third-party human evaluation of the alignment of our correctness and hallucination metrics. We find that closed-source pipelines significantly outperform open-source pipelines in both the correctness and halucination metrics, with a wider performance gap in questions relying on multimodal and cross-document information. We also find after a human evaluation of our correctness and hallucination metric compared with our questions and pipeline responses, average agreement was 4.62 for correctness 4.53 for hallucination detection on a 1-5 Likert scale with 5 being strongly agree with our determination.

Hildebrand, Samuel [ORNL] (ORCID:0009000465963104)↗

DOE OSTI2022

Utilization of the Critic Subnetwork of a Generative Adversarial Network as Detector of Morphological Material Change in Image Data

The resolution of computed tomography (CT) has become high enough to monitor morphological changes due to aging in materials in long-term applications. For this work, we explored the utility of the critic of a generative adversarial network (GAN) to automatically detect such changes. The GAN was trained with images of pristine Pharmatose, which is used as a surrogate energetic material. It is important to note that images of the material with altered morphology were only used during the test phase. The GAN-generated images reproduced the microstructure of Pharmatose well, although some unrealistic particle fusion was seen. Calculated morphological metrics (volume fraction, interfacial line length, and local thickness) for the synthetic images also showed good agreement with the training data, albeit with signs of mode collapse in the interfacial line length. While the critic exposed changes in particle size, it showed limited ability to distinguish images by particle shape. The detection of shape differences was also a more challenging task for the selected morphological metrics that related to energetic material performance. We further tested the critic with images of aged Pharmatose. Subtle changes due to aging are difficult for the human analyst to detect; but both critic and morphological metrics analysis showed image differentiation.

36 MATERIALS SCIENCE↗

DOE OSTI2022

A quantitative and holistic circular economy assessment framework at the micro level

Circular Economy (CE) aims to solve resource, waste, and emission challenges by creating a production-to-consumption supply chain that is restorative and environmentally benign. A variety of metrics has been developed with the focus mainly on the macro and meso levels. This work introduces a micro level CE assessment framework that provides i) a set of indicators and metrics with sector-specific dimensions, ii) quantitative and holistic CE overall and category-based metrics, iii) media for data visualization and analysis of CE indicators, and iv) an analytical tool to assess multi-national businesses and the multi-scale and interconnected CE supply chains. Using this quantitative tool, companies are able to track their transition towards CE, conduct temporal analysis, and benchmark their performance against their peers and industry’s standards. The applicability and the capabilities of the developed CE assessment framework is demonstrated through three case studies, with the results demonstrating a clear trend towards circularity.

42 ENGINEERING↗

DOE OSTI2020

Simulation-aided occupant-centric building design: A critical review of tools, methods, and applications

Occupants are active participants in their built environment, affecting its performance while simultaneously being affected by its design and indoor environmental conditions. With recent advances in computer modeling, simulation tools, and analysis techniques, topics such as human-building interactions and occupant behavior have gained significant interest in the literature given their premise of improving building design processes and operating strategies. In practice, the focus of occupant-centric literature has been mostly geared towards the latter (i.e., operation), leaving the implications on building design practices underexplored. This paper fills the gap by providing a critical review of existing studies applying computer-based modeling and simulation to guide occupant-centric building design. The reviewed papers are organized along four main themes, namely occupant-centric: (i) metrics of building performance, (ii) modeling and simulation approaches, (iii) design methods and applications, and (iv) supporting practices and mechanisms. Important barriers are identified for a more effective application of occupant-centric building design practices, including the limited consideration of metrics beyond energy efficiency (e.g., occupant well-being and space planning), the limited implementation and validation of the proposed methods, and the lack of integration of occupant behavior modeling in existing building performance simulation tools. Finally, future research directions are discussed, covering large-scale international data collection efforts to move from generic assumptions about occupant behavior to specific/localized knowledge, improved metrics of measuring building performance, and improved industry practices, such as building codes, to promote an occupant-in-the-loop approach to the building design process.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2023

Evaluating Retention Index Score Assumptions to Refine GC–MS Metabolite Identification

As metabolomics grows into a high-throughput and high demand research field, current metrics for the identification of small molecules in gas chromatography–mass spectrometry (GC–MS) still require manual verification. Though steps have been taken to improve scoring metrics by combining spectral similarity (SS) and retention index (RI), the problem persists. A large body of literature has analyzed and refined SS scores, but few studies have explicitly studied improvements to RI scores. Here, in this study, we examined whether uninvestigated assumptions of the RI score are valid and propose ways to improve them. Query RIs were matched to library RI with a generous window of ±35 to avoid unintentional removal of valid compound identifications. Each match was manually verified as a true positive (TP), true negative, or unknown. Metabolites with at least 30 TP identifications were included in downstream analyses, resulting in a total of 87 metabolites from samples of varying complexity and type (e.g., amino acid mixtures, human urine, fungal species, and so on.). Our results showed that the RI score assumptions of normality, consistent variance across metabolites, and a mean error centered at 0 are often violated. We demonstrated through a cross-validation analysis that modifying these underlying assumptions according to empirical metabolite-specific distributions improved the TP and negative rankings. Further, we statistically determined the minimum number of samples required to estimate distributional parameters for scoring metrics. Overall, this work proposes a robust statistical pipeline to reduce the time bottleneck of metabolite identification by improving RI scores and thus minimize the effort to complete manual verification.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2025

Quantifying Distribution System Resilience From Utility Data: Large Event Risk and Benefits of Investments

We focus on blackouts in electric distribution systems that have a large cost to customers. To quantify resilience to these events, we show how to calculate risk metrics from the historical outage data routinely collected by utilities' outage management systems. Risk is defined using a customer cost exceedance curve. The exceedance curve has a heavy tail that implies large fluctuations in large blackout costs, and this makes estimating the mean large cost in the usual way impractical. To avoid this problem, we use new resilience metrics describing the large event risk; these metrics are the probability of a large cost event, the annual log cost resilience index, and the average of the logarithm of the cost of large-cost events or the slope magnitude of the tail on a log–log exceedance curve. Resilience can be improved by planned investments to upgrade system components or speed up restoration. The benefits that these investments would have had if they had been made in the past can be quantified by “rerunning history” with the effects of the investment included, and then recalculating the large event risk to find the improvement in resilience. An example using utility data shows a 2% reduction in the probability of a large cost event due to 10% wind hardening and 6%–7% reduction due to 10% faster restoration in two different areas of a distribution utility. This new data-driven approach to quantify resilience and resilience investments is realistic and much easier to apply than complicated approaches based on modeling all the phases of resilience. Moreover, an appeal to improvements to past lived experience may well be persuasive to customers and regulators in making the case for resilience investments.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2024

Underwater Target Detection Software Demonstration on the RivGen Turbine

This repository contains data and processing scripts necessary to train the object detection models utilized in the underwater target detection software demonstration on the RivGen turbine project and to produce performance metrics (precision, recall, mAP50, mAP50-95). - Contents - Data consist of "images" and "labels". Each image has an associated label, both share the same time string in its file name (e.g., 2024_05_25_09_01_57.98.jpg and 2024_05_25_09_01_57.98.txt). Time strings have the format %yyyy_%mm_%dd_%HH_%MM_%SS.%3f. Images and labels were curated from 2021 and 2024 smolt outmigration periods at the project site in Igiugig, AK. Images are monochrome 8-bit images of objects (smolt, debris, and other) passing through the field of view of the deployed cameras during various operational stages of the RivGen turbine. Labels are text files indicating the class and bounding polygon of each object in an image. The provided labels use the "YOLO" label format. - Requirements - Python3.8+ is required to install and run the train and validation script. The README.md provides instruction for installing the requirements from the requirements.py file. - Instructions - The "example_train.py" file ingests the provided data, trains a model, and produces model performance metrics at completion. NOTE: model performance metrics will vary from run to run as a consequence of the random selection of training and validation data.

16 TIDAL AND WAVE POWER↗

DOE OSTI2025

JUSTIFI: Software for Improving Performance Objectives via Energy Efficiency

With growing energy supply concerns and rising costs, energy efficiency is a critical component of industrial energy resilience and competitiveness by directly reducing energy operating costs. Energy efficiency projects in manufacturing also yield valuable benefits to other key metrics, such as improved quality, reduced maintenance costs, improved safety, decreased pollution, and enhanced productivity. However, it is difficult to receive approval for energy efficiency projects, so implementation rates are low, even when meeting capital project payback period criteria. The inclusion and quantification of non-energy benefits (NEBs) in the decision-making process for energy efficiency projects can improve the overall financial payback period while demonstrating a positive impact on the firm's key performance metrics and business strategy. Despite their significant financial and strategic value, NEBs are rarely factored into decision-making due to lack of tools to effectively identify and quantify them. Therefore, a comprehensive and integrative approach is needed for the rapidly evolving energy landscape. To address these challenges, through funding from U.S. Department of Energy, our new assessment methodology integrates common continuous improvement six sigma concepts, such as the DMAIC process, and a protocol of guiding questions, into energy efficiency assessments to identify NEBs. We have also developed open-source software, JUSTIFI, to guide users through this process, data collection, and quantification. It is designed to be used concurrently with DOE energy system analysis software suite, MEASUR. Our methodology and tools inform energy assessors, firm engineering, decision makers, and workforce seeking to increase energy resilience and to maximize benefits aligned with performance metrics.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2026

JUSTIFI: Software for Improving Performance Objectives via Energy Efficiency

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

DOE OSTI2021

Do We Need a New Sustainability Assessment Method for the Circular Economy? A Critical Literature Review

The goal of the circular economy (CE) is to transition from today's take-make-waste linear pattern of production and consumption to a circular system in which the societal value of products, materials, and resources is maximized over time. Yet circularity in and of itself does not ensure social, economic, and environmental performance (i.e., sustainability). Sustainability of CE strategies needs to be measured against their linear counterparts to identify and avoid strategies that increase circularity yet lead to unintended externalities. The state of the practice in quantitatively comparing sustainability impacts of circular to linear systems is one of experimentation with various extant methods developed in other fields and now applied here. While the proliferation of circularity metrics has received considerable attention, to-date, there is no critical review of the methods and combinations of methods that underlie those metrics and that specifically quantify sustainability impacts of circular strategies. Our critical review herein analyzes identified methods according to six criteria: temporal resolution, scope, data requirements, data granularity, capacity for measuring material efficiency potentials, and sustainability completeness. Results suggest that the industrial ecology and complex systems science fields could prove complementary when assessing the sustainability of the transition to a CE. Both fields include quantitative methods differing primarily with regard to their inclusion of temporal aspects and material efficiency potentials. Moreover, operations research methods such as multiple-criteria decision-making (MCDM) may alleviate the common contradictions which often exist between circularity metrics. This review concludes by suggesting guidelines for selecting quantitative methods most appropriate to a particular research question and making the argument that while there are a variety of existing methods, additional research is needed to combine existing methods and develop a more holistic approach for assessing sustainability impacts of CE strategies.

36 MATERIALS SCIENCE↗

DOE OSTI2021

NMR Spectroscopy for Protein Higher Order Structure Similarity Assessment in Formulated Drug Products

Peptide and protein drug molecules fold into higher order structures (HOS) in formulation and these folded structures are often critical for drug efficacy and safety. Generic or biosimilar drug products (DPs) need to show similar HOS to the reference product. The solution NMR spectroscopy is a non-invasive, chemically and structurally specific analytical method that is ideal for characterizing protein therapeutics in formulation. However, only limited NMR studies have been performed directly on marketed DPs and questions remain on how to quantitively define similarity. Here, NMR spectra were collected on marketed peptide and protein DPs, including calcitonin-salmon, liraglutide, teriparatide, exenatide, insulin glargine and rituximab. The 1D 1 H spectral pattern readily revealed protein HOS heterogeneity, exchange and oligomerization in the different formulations. Principal component analysis (PCA) applied to two rituximab DPs showed consistent results with the previously demonstrated similarity metrics of Mahalanobis distance (D M ) of 3.3. The 2D 1 H- 13 C HSQC spectral comparison of insulin glargine DPs provided similarity metrics for chemical shift difference (Δδ) and methyl peak profile, i.e., 4 ppb for 1 H, 15 ppb for 13 C and 98% peaks with equivalent peak height. Finally, 2D 1 H- 15 N sofast HMQC was demonstrated as a sensitive method for comparison of small protein HOS. The application of NMR procedures and chemometric analysis on therapeutic proteins offer quantitative similarity assessments of DPs with practically achievable similarity metrics.

59 BASIC BIOLOGICAL SCIENCES↗

NASA NTRS2009

Real-Time Assessment of Robot Performance During Remote Exploration Operations

To ensure that robots are used effectively for exploration missions, it is important to assess their performance during operations. We are investigating the definition and computation of performance metrics for assessing remote robotic operations in real-time. Our approach is to monitor data streams from robots, compute performance metrics, and provide Web-based displays of these metrics for assessing robot performance during operations. We evaluated our approach for measuring robot performance with the K10 rovers from NASA Ames Research Center during a field test at Moses Lake Sand Dunes (WA) in June 2008. In this paper we present the results of evaluating our software for robot performance and discuss our conclusions from this evaluation for future robot operations.

SBIR TOPIC X7.02 PHASE 1↗