Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “METRICS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 649 records · Page 36
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

A quantitative and holistic circular economy assessment framework at the micro level

Circular Economy (CE) aims to solve resource, waste, and emission challenges by creating a production-to-consumption supply chain that is restorative and environmentally benign. A variety of metrics has been developed with the focus mainly on the macro and meso levels. This work introduces a micro level CE assessment framework that provides i) a set of indicators and metrics with sector-specific dimensions, ii) quantitative and holistic CE overall and category-based metrics, iii) media for data visualization and analysis of CE indicators, and iv) an analytical tool to assess multi-national businesses and the multi-scale and interconnected CE supply chains. Using this quantitative tool, companies are able to track their transition towards CE, conduct temporal analysis, and benchmark their performance against their peers and industry’s standards. The applicability and the capabilities of the developed CE assessment framework is demonstrated through three case studies, with the results demonstrating a clear trend towards circularity.

42 ENGINEERING↗
DOE OSTI2020

Simulation-aided occupant-centric building design: A critical review of tools, methods, and applications

Occupants are active participants in their built environment, affecting its performance while simultaneously being affected by its design and indoor environmental conditions. With recent advances in computer modeling, simulation tools, and analysis techniques, topics such as human-building interactions and occupant behavior have gained significant interest in the literature given their premise of improving building design processes and operating strategies. In practice, the focus of occupant-centric literature has been mostly geared towards the latter (i.e., operation), leaving the implications on building design practices underexplored. This paper fills the gap by providing a critical review of existing studies applying computer-based modeling and simulation to guide occupant-centric building design. The reviewed papers are organized along four main themes, namely occupant-centric: (i) metrics of building performance, (ii) modeling and simulation approaches, (iii) design methods and applications, and (iv) supporting practices and mechanisms. Important barriers are identified for a more effective application of occupant-centric building design practices, including the limited consideration of metrics beyond energy efficiency (e.g., occupant well-being and space planning), the limited implementation and validation of the proposed methods, and the lack of integration of occupant behavior modeling in existing building performance simulation tools. Finally, future research directions are discussed, covering large-scale international data collection efforts to move from generic assumptions about occupant behavior to specific/localized knowledge, improved metrics of measuring building performance, and improved industry practices, such as building codes, to promote an occupant-in-the-loop approach to the building design process.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

Evaluating Retention Index Score Assumptions to Refine GC–MS Metabolite Identification

As metabolomics grows into a high-throughput and high demand research field, current metrics for the identification of small molecules in gas chromatography–mass spectrometry (GC–MS) still require manual verification. Though steps have been taken to improve scoring metrics by combining spectral similarity (SS) and retention index (RI), the problem persists. A large body of literature has analyzed and refined SS scores, but few studies have explicitly studied improvements to RI scores. Here, in this study, we examined whether uninvestigated assumptions of the RI score are valid and propose ways to improve them. Query RIs were matched to library RI with a generous window of ±35 to avoid unintentional removal of valid compound identifications. Each match was manually verified as a true positive (TP), true negative, or unknown. Metabolites with at least 30 TP identifications were included in downstream analyses, resulting in a total of 87 metabolites from samples of varying complexity and type (e.g., amino acid mixtures, human urine, fungal species, and so on.). Our results showed that the RI score assumptions of normality, consistent variance across metabolites, and a mean error centered at 0 are often violated. We demonstrated through a cross-validation analysis that modifying these underlying assumptions according to empirical metabolite-specific distributions improved the TP and negative rankings. Further, we statistically determined the minimum number of samples required to estimate distributional parameters for scoring metrics. Overall, this work proposes a robust statistical pipeline to reduce the time bottleneck of metabolite identification by improving RI scores and thus minimize the effort to complete manual verification.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Quantifying Distribution System Resilience From Utility Data: Large Event Risk and Benefits of Investments

We focus on blackouts in electric distribution systems that have a large cost to customers. To quantify resilience to these events, we show how to calculate risk metrics from the historical outage data routinely collected by utilities' outage management systems. Risk is defined using a customer cost exceedance curve. The exceedance curve has a heavy tail that implies large fluctuations in large blackout costs, and this makes estimating the mean large cost in the usual way impractical. To avoid this problem, we use new resilience metrics describing the large event risk; these metrics are the probability of a large cost event, the annual log cost resilience index, and the average of the logarithm of the cost of large-cost events or the slope magnitude of the tail on a log–log exceedance curve. Resilience can be improved by planned investments to upgrade system components or speed up restoration. The benefits that these investments would have had if they had been made in the past can be quantified by “rerunning history” with the effects of the investment included, and then recalculating the large event risk to find the improvement in resilience. An example using utility data shows a 2% reduction in the probability of a large cost event due to 10% wind hardening and 6%–7% reduction due to 10% faster restoration in two different areas of a distribution utility. This new data-driven approach to quantify resilience and resilience investments is realistic and much easier to apply than complicated approaches based on modeling all the phases of resilience. Moreover, an appeal to improvements to past lived experience may well be persuasive to customers and regulators in making the case for resilience investments.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Underwater Target Detection Software Demonstration on the RivGen Turbine

This repository contains data and processing scripts necessary to train the object detection models utilized in the underwater target detection software demonstration on the RivGen turbine project and to produce performance metrics (precision, recall, mAP50, mAP50-95). - Contents - Data consist of "images" and "labels". Each image has an associated label, both share the same time string in its file name (e.g., 2024_05_25_09_01_57.98.jpg and 2024_05_25_09_01_57.98.txt). Time strings have the format %yyyy_%mm_%dd_%HH_%MM_%SS.%3f. Images and labels were curated from 2021 and 2024 smolt outmigration periods at the project site in Igiugig, AK. Images are monochrome 8-bit images of objects (smolt, debris, and other) passing through the field of view of the deployed cameras during various operational stages of the RivGen turbine. Labels are text files indicating the class and bounding polygon of each object in an image. The provided labels use the "YOLO" label format. - Requirements - Python3.8+ is required to install and run the train and validation script. The README.md provides instruction for installing the requirements from the requirements.py file. - Instructions - The "example_train.py" file ingests the provided data, trains a model, and produces model performance metrics at completion. NOTE: model performance metrics will vary from run to run as a consequence of the random selection of training and validation data.

16 TIDAL AND WAVE POWER↗
DOE OSTI2025

JUSTIFI: Software for Improving Performance Objectives via Energy Efficiency

With growing energy supply concerns and rising costs, energy efficiency is a critical component of industrial energy resilience and competitiveness by directly reducing energy operating costs. Energy efficiency projects in manufacturing also yield valuable benefits to other key metrics, such as improved quality, reduced maintenance costs, improved safety, decreased pollution, and enhanced productivity. However, it is difficult to receive approval for energy efficiency projects, so implementation rates are low, even when meeting capital project payback period criteria. The inclusion and quantification of non-energy benefits (NEBs) in the decision-making process for energy efficiency projects can improve the overall financial payback period while demonstrating a positive impact on the firm's key performance metrics and business strategy. Despite their significant financial and strategic value, NEBs are rarely factored into decision-making due to lack of tools to effectively identify and quantify them. Therefore, a comprehensive and integrative approach is needed for the rapidly evolving energy landscape. To address these challenges, through funding from U.S. Department of Energy, our new assessment methodology integrates common continuous improvement six sigma concepts, such as the DMAIC process, and a protocol of guiding questions, into energy efficiency assessments to identify NEBs. We have also developed open-source software, JUSTIFI, to guide users through this process, data collection, and quantification. It is designed to be used concurrently with DOE energy system analysis software suite, MEASUR. Our methodology and tools inform energy assessors, firm engineering, decision makers, and workforce seeking to increase energy resilience and to maximize benefits aligned with performance metrics.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2026

JUSTIFI: Software for Improving Performance Objectives via Energy Efficiency

With growing energy supply concerns and rising costs, energy efficiency is a critical component of industrial energy resilience and competitiveness by directly reducing energy operating costs. Energy efficiency projects in manufacturing also yield valuable benefits to other key metrics, such as improved quality, reduced maintenance costs, improved safety, decreased pollution, and enhanced productivity. However, it is difficult to receive approval for energy efficiency projects, so implementation rates are low, even when meeting capital project payback period criteria. The inclusion and quantification of non-energy benefits (NEBs) in the decision-making process for energy efficiency projects can improve the overall financial payback period while demonstrating a positive impact on the firm's key performance metrics and business strategy. Despite their significant financial and strategic value, NEBs are rarely factored into decision-making due to lack of tools to effectively identify and quantify them. Therefore, a comprehensive and integrative approach is needed for the rapidly evolving energy landscape. To address these challenges, through funding from U.S. Department of Energy, our new assessment methodology integrates common continuous improvement six sigma concepts, such as the DMAIC process, and a protocol of guiding questions, into energy efficiency assessments to identify NEBs. We have also developed open-source software, JUSTIFI, to guide users through this process, data collection, and quantification. It is designed to be used concurrently with DOE energy system analysis software suite, MEASUR. Our methodology and tools inform energy assessors, firm engineering, decision makers, and workforce seeking to increase energy resilience and to maximize benefits aligned with performance metrics.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Do We Need a New Sustainability Assessment Method for the Circular Economy? A Critical Literature Review

The goal of the circular economy (CE) is to transition from today's take-make-waste linear pattern of production and consumption to a circular system in which the societal value of products, materials, and resources is maximized over time. Yet circularity in and of itself does not ensure social, economic, and environmental performance (i.e., sustainability). Sustainability of CE strategies needs to be measured against their linear counterparts to identify and avoid strategies that increase circularity yet lead to unintended externalities. The state of the practice in quantitatively comparing sustainability impacts of circular to linear systems is one of experimentation with various extant methods developed in other fields and now applied here. While the proliferation of circularity metrics has received considerable attention, to-date, there is no critical review of the methods and combinations of methods that underlie those metrics and that specifically quantify sustainability impacts of circular strategies. Our critical review herein analyzes identified methods according to six criteria: temporal resolution, scope, data requirements, data granularity, capacity for measuring material efficiency potentials, and sustainability completeness. Results suggest that the industrial ecology and complex systems science fields could prove complementary when assessing the sustainability of the transition to a CE. Both fields include quantitative methods differing primarily with regard to their inclusion of temporal aspects and material efficiency potentials. Moreover, operations research methods such as multiple-criteria decision-making (MCDM) may alleviate the common contradictions which often exist between circularity metrics. This review concludes by suggesting guidelines for selecting quantitative methods most appropriate to a particular research question and making the argument that while there are a variety of existing methods, additional research is needed to combine existing methods and develop a more holistic approach for assessing sustainability impacts of CE strategies.

36 MATERIALS SCIENCE↗
DOE OSTI2021

NMR Spectroscopy for Protein Higher Order Structure Similarity Assessment in Formulated Drug Products

Peptide and protein drug molecules fold into higher order structures (HOS) in formulation and these folded structures are often critical for drug efficacy and safety. Generic or biosimilar drug products (DPs) need to show similar HOS to the reference product. The solution NMR spectroscopy is a non-invasive, chemically and structurally specific analytical method that is ideal for characterizing protein therapeutics in formulation. However, only limited NMR studies have been performed directly on marketed DPs and questions remain on how to quantitively define similarity. Here, NMR spectra were collected on marketed peptide and protein DPs, including calcitonin-salmon, liraglutide, teriparatide, exenatide, insulin glargine and rituximab. The 1D 1 H spectral pattern readily revealed protein HOS heterogeneity, exchange and oligomerization in the different formulations. Principal component analysis (PCA) applied to two rituximab DPs showed consistent results with the previously demonstrated similarity metrics of Mahalanobis distance (D M ) of 3.3. The 2D 1 H- 13 C HSQC spectral comparison of insulin glargine DPs provided similarity metrics for chemical shift difference (Δδ) and methyl peak profile, i.e., 4 ppb for 1 H, 15 ppb for 13 C and 98% peaks with equivalent peak height. Finally, 2D 1 H- 15 N sofast HMQC was demonstrated as a sensitive method for comparison of small protein HOS. The application of NMR procedures and chemometric analysis on therapeutic proteins offer quantitative similarity assessments of DPs with practically achievable similarity metrics.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Adapting Existing Energy Models for Resilience Analysis

Resilience is a topic receiving much attention in relation to energy systems, with particular attention being paid to the supply of electricity. As a result of the growing interest in energy sector resilience, research communities have proposed a plethora of candidate resilience indicators and metrics, most of which remain immature at different scales and segments within the energy system. A necessary focus of the research community lies in implementing, testing, and validating resilience metrics and analysis approaches in energy sector models, which will be invaluable for informing resilience planning and investment decisions. Recognizing these challenges that need to be addressed, we explore how to effectively integrate resilience considerations into energy sector models and tools. The overarching goal of the effort was to evaluate the data needs, methodologies, and outcomes—including consequences and/or changes in investment or operational decisions due to avoided consequences—based on resilience analysis in a range of existing tools. In particular, we selected five models originally built at NREL to explore non-resilience energy research questions to implement and exercise resilience metrics and analysis approaches. To demonstrate the importance of perspective, we selected models that represent different segments of the energy sector, geographic scales, and modeling approaches. A second important aspect of our effort was the development of generalized power interruption scenarios. These scenarios were intended to help establish a framework for simulating the effects of real-world threats in terms of their impacts on system components and, in turn, power interruption.

ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2024

Environmental and economic analyses of chemical recycling via dissolution of waste polyethylene terephthalate

Globally, more than 1000 organizations and 175 nations are facing the plastic waste problem and have realized the need to transition from “linear-to-circular” economy of plastics. While the current mechanical recycling technologies for plastics are struggling to increase the U.S. plastic recycling rates beyond 9%, chemical recycling technologies become important complementary technologies to the predominant mechanical recycling that are needed to realize the circular economy in plastics supply chains. Dissolution is one such chemical recycling technology that can recycle waste plastic back into high-quality virgin grade plastic. However, the environmental and economic impacts of chemical recycling of waste polyethylene terephthalate (PET) via dissolution technology using a green solvent are unknown. Our study evaluated environmental metrics such as greenhouse gas (GHG) emissions and cumulative energy demand, and economic metrics such as net present value (NPV), minimum selling price, payback period, return on investment, and discounted internal rate of return for three dissolution processes with polymer recovery via anti-solvent, evaporation, and cooling precipitation techniques. The dissolution process with evaporation technique was the most economically favorable, whereas that with cooling technique was the most environmentally favorable. The anti-solvent approach had low economic performance and the highest environmental impacts. The NPV for all of these technologies ranged from $2.67 MM to $10.93 MM for a capacity of 8,400 MT/year and was found to be the highest for dissolution with evaporation approach and the least for anti-solvent approach. The cradle-to-gate GHG emissions and energy demand for PET dissolution processes ranged from 1.33-3.77 kg CO2-eq/kg of chemically recycled (CR) PET and18.9-56.1 MJ/kg of CR-PET, respectively. These economic and environmental metrics will be helpful in evaluating the sustainability of circular PET supply chains in the U.S.

09 BIOMASS FUELS↗
DOE OSTI2023

New Chemical Dopant and Counterion Mechanism for Organic Electrochemical Transistors and Organic Mixed Ionic–Electronic Conductors

Abstract Organic mixed ionic–electronic conductors (OMIECs) have varied performance requirements across a diverse application space. Chemically doping the OMIEC can be a simple, low‐cost approach for adapting performance metrics. However, complex challenges, such as identifying new dopant materials and elucidating design rules, inhibit its realization. Here, these challenges are approached by introducing a new n‐dopant, tetrabutylammonium hydroxide (TBA‐OH), and identifying a new design consideration underpinning its success. TBA‐OH behaves as both a chemical n‐dopant and morphology additive in donor acceptor co‐polymer naphthodithiophene diimide‐based polymer, which serves as an electron transporting material in organic electrochemical transistors (OECTs). The combined effects enhance OECT transconductance, charge carrier mobility, and volumetric capacitance, representative of the key metrics underpinning all OMIEC applications. Additionally, when the TBA + counterion adopts an “edge‐on” location relative to the polymer backbone, Coulombic interaction between the counterion and polaron is reduced, and polaron delocalization increases. This is the first time such mechanisms are identified in doped‐OECTs and doped‐OMIECs. The work herein therefore takes the first steps toward developing the design guidelines needed to realize chemical doping as a generic strategy for tailoring performance metrics in OECTs and OMIECs.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Supply chain system for a centralized biorefinery system based on switchgrass grown on marginal land in Michigan

The feedstock supply chain of a biorefinery is critical in determining the economic and environmental performance of biofuels. This study investigated various feedstock supply chains for a centralized biorefinery based on different objective functions and how the biorefinery responds to supply chain volatility. For the analysis, a hypothetical centralized biorefinery was supplied by switchgrass grown on marginal lands in Michigan. From an economic standpoint, the minimum biofuel selling price is an important metric of the objective function, and the greenhouse gas mitigation potential is a similarly important metric for global warming impact. A trade-off therefore exists between the economic and environmental performance of biofuels under this system configuration. However, when a carbon tax credit is applied to soil organic carbon sequestration in switchgrass production, the metric of the objective function may not be the key factor in establishing a supply chain as long as it is associated with biorefinery capacity as well as economic or environmental values. Year-to-year fluctuations in cellulosic biomass yield (supply chain volatility) might also prevent the biorefinery from operating at full capacity. Cellulosic biomass pellets can serve as an auxiliary feedstock for the centralized cellulosic biorefinery system to minimize the impacts of supply chain volatility.

09 BIOMASS FUELS↗
DOE OSTI2020

An international laboratory comparison of dissolved organic matter composition by high resolution mass spectrometry: Are we getting the same answer?

Since the first detailed compositional analysis of dissolved organic matter (DOM) in 1995, high resolution mass spectrometry (HRMS) has become a vital tool for DOM characterization. While the upward trend in HRMS for molecular level analysis of DOM continues, so do the challenges of data comparison and interpretation between laboratories operating instruments of differing performance and user operating conditions. It is therefore essential that the community establishes whether data and trends can be compared robustly between research groups. To this end, 4 identically prepared DOM samples were each studied by 16 laboratories, representing 17 commercially-purchased instruments located across 8 countries, using positive- and negative-mode electrospray ionization (ESI) HRMS analysis. Despite the widely reported sensitivity of ESI experiments to sample matrix, preparation, ion source operation and instrument tuning parameters, the instruments used in this study showed relatively low variability between results for the same samples, particularly H/C and AI metrics. The variation in O/C and m/z metrics was relatively higher, although z-score graphs and %BCD demonstrated that the relative difference in general chemical composition between samples was the same for each instrument included in the study. As such, these metrics can be used for fingerprinting of DOM samples and as a benchmark for quality control in participating laboratories, with future strides in environmental science building upon these efforts.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Shrubs Strongly Influence Snow Properties in Two Subarctic Watersheds

Understanding changes in snow distribution in permafrost ecosystems is fundamental to predicting their response to future climate change. The expansion of tall shrubs into tundra ecosystems can trap snow and insulate permafrost ecosystems during the winter, but the overall insulation effect is dependent upon many ecosystem properties. To study shrub–snow–ground interactions, small temperature sensors were deployed at two research sites on the Seward Peninsula of Alaska, USA, during the 2019–2020 winter. Snow temperatures were used to extrapolate multiple metrics, including freezing n-factors, the snow insulation effect, snow cover duration, and the length of the snowmelt period. Statistical and spatial analysis showed that shrub patches were a dominant control on all snow metrics. Within shrub patches, average ground temperatures were 2.1°C warmer, snow persisted 50 days longer, snow insulation was double, and a longer, later spring snowmelt period occurred compared to nonshrubby areas. Site-level differences contributed relatively little to variation in snow metrics, indicating that shrub presence is an overarching driver of snow–ground interactions at the locations examined. Shrub expansion, which is anticipated under climate change, will strongly impact future permafrost distribution and Arctic energy, water, and carbon cycles through snow–shrub–ground feedbacks.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Persistent Classification: Understanding Adversarial Attacks by Studying Decision Boundary Dynamics

ABSTRACT There are a number of hypotheses underlying the existence of adversarial examples for classification problems. These include the high‐dimensionality of the data, the high codimension in the ambient space of the data manifolds of interest, and that the structure of machine learning models may encourage classifiers to develop decision boundaries close to data points. This article proposes a new framework for studying adversarial examples that does not depend directly on the distance to the decision boundary. Similarly to the smoothed classifier literature, we define a (natural or adversarial) data point to be ( γ , σ)‐stable if the probability of the same classification is at least for points sampled in a Gaussian neighborhood of the point with a given standard deviation . We focus on studying the differences between persistence metrics along interpolants of natural and adversarial points. We show that adversarial examples have significantly lower persistence than natural examples for large neural networks in the context of the MNIST and ImageNet datasets. We connect this lack of persistence with decision boundary geometry by measuring angles of interpolants with respect to decision boundaries. Finally, we connect this approach with robustness by developing a manifold alignment gradient metric and demonstrating the increase in robustness that can be achieved when training with the addition of this metric.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Dimensionality reduction using elastic measures

With the recent surge in big data analytics for hyperdimensional data, there is a renewed interest in dimensionality reduction techniques. In order for these methods to improve performance gains and understanding of the underlying data, a proper metric needs to be identified. This step is often overlooked, and metrics are typically chosen without consideration of the underlying geometry of the data. Here, in this paper, we present a method for incorporating elastic metrics into the t-distributed stochastic neighbour embedding (t-SNE) and Uniform Manifold Approximation and Projection (UMAP). We apply our method to functional data, which is uniquely characterized by rotations, parameterization and scale. If these properties are ignored, they can lead to incorrect analysis and poor classification performance. Through our method, we demonstrate improved performance on shape identification tasks for three benchmark data sets (MPEG-7, Car data set and Plane data set of Thankoor), where we achieve 0.77, 0.95 and 1.00 F1 score, respectively.

97 MATHEMATICS AND COMPUTING↗