Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “COMPARATIVE EVALUATIONS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Novel reusable animal model for comparative evaluation of in vivo growth and protein-expression of Escherichia coli O157 strains in the bovine rumen

Previously, we had demonstrated that Escherichia coli O157:H7 (O157) strain 86–24 expresses proteins involved in survival rather than virulence in vitro in rumen fluid from dairy cattle limit fed a maintenance diet. Here, we verified if this observation would be true for different O157 strains grown in vitro in rumen fluid from, and in vivo in the rumen of, animals on contrasting maintenance (high fiber) and lactation (high energy-protein) diets usually limit fed to dairy cattle. For the in vivo studies, an economical, novel, reusable and non-terminal rumen-fistulated animal model permitting simultaneous evaluation of multiple bacterial strains in the bovine rumen was developed. All experiments were conducted in duplicate using different animals to account for host-related variations. The O157 strains included, 86–24, EDL933 and the super shed SS-17. E. coli Nal R (#5735), derived from a bovine intestinal commensal E. coli, was included as a control. As expected, diet influenced ruminal pH and volatile fatty acid (VFA) composition. The pH ranged from 6.2–7.0 and total VFA concentrations from 109–141 μM/ml, in animals fed the maintenance diet. In comparison, animals fed the lactation diet had a ruminal pH ranging between 5.18–6.0, and total VFA of 125–219 μM/ml. Strain dependent differences in O157 recovery from the rumen fluid of cattle fed either diet was observed, both in vitro and in vivo, with O157 strains 86–24 and EDL933 demonstrating similar survival patterns. Analysis of the O157 proteomes expressed in the rumen fluid/rumen verified previous observations of adaptive responses. Any difference in the adaptive response was mainly influenced by the animal’s diet and growth conditions (in vitro and in vivo) and not the O157 strain. These new insights into the O157 responses could help formulate modalities to control O157 across strains in cattle at all stages of husbandry.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Performance Evaluation of Comparative Vacuum Monitoring and Piezoelectric Sensors for Structural Health Monitoring of Rotorcraft Components

The costs associated with the increasing maintenance and surveillance needs of aging structures are rising at an unexpected rate. Multi-site fatigue damage, hidden cracks in hard-to-reach locations, disbonded joints, erosion, impact, and corrosion are among the major flaws encountered in today’s extensive fleet of aging aircraft and space vehicles. Aircraft maintenance and repairs represent about a quarter of a commercial fleet’s operating costs. The application of Structural Health Monitoring (SHM) systems using distributed sensor networks can reduce these costs by facilitating rapid and global assessments of structural integrity. The use of in-situ sensors for real-time health monitoring can overcome inspection impediments stemming from accessibility limitations, complex geometries, and the location and depth of hidden damage. Reliable, structural health monitoring systems can automatically process data, assess structural condition, and signal the need for human intervention. The ease of monitoring an entire on-board network of distributed sensors means that structural health assessments can occur more often, allowing operators to be even more vigilant with respect to flaw onset. SHM systems also allow for condition-based maintenance practices to be substituted for the current time-based or cycle-based maintenance approach thus optimizing maintenance labor. The Federal Aviation Administration has conducted a series of SHM validation and certification programs intended to comprehensively support the evolution and adoption of SHM practices into routine aircraft maintenance practices. This report presents one of those programs involving a Sandia Labs-aviation industry effort to move SHM into routine use for aircraft maintenance. The Airworthiness Assurance NDI Validation Center (AANC) at Sandia Labs, in conjunction with Sikorsky, Structural Monitoring Systems Ltd., Anodyne Electronics Manufacturing Corp., Acellent Technologies Inc., and the Federal Aviation Administration (FAA) carried out a trial validation and certification program to evaluate Comparative Vacuum Monitoring (CVM) and Piezoelectric Transducers (PZT) as a structural health monitoring solution to specific rotorcraft applications. Validation tasks were designed to address the SHM equipment, the health monitoring task, the resolution required, the sensor interrogation procedures, the conditions under which the monitoring will occur, the potential inspector population, adoption of CVM and PZT systems into rotorcraft maintenance programs and the document revisions necessary to allow for their routine use as an alternate means of performing periodic structural inspections. This program addressed formal SHM technology validation and certification issues so that the full spectrum of concerns, including design, deployment, performance and certification were appropriately considered. Sandia Labs designed, implemented, and analyzed the results from a focused and statistically relevant experimental effort to quantify the reliability of a CVM system applied to Sikorsky S-92 fuselage frame application and a PZT system applied to an S-92 main gearbox mount beam application. The applications included both local and global damage detection assessments. All factors that affect SHM sensitivity were included in this program: flaw size, shape, orientation and location relative to the sensors, as well as operational and environmental variables. Statistical methods were applied to performance data to derive Probability of Detection (POD) values for SHM sensors in a manner that agrees with current nondestructive inspection (NDI) validation requirements and is acceptable to both the aviation industry and regulatory bodies. The validation work completed in this program demonstrated the ability of both CVM and PZT SHM systems to detect cracks in rotorcraft components. It proved the ability to use final system response parameters to provide a Green Light/Red Light (“GO” – “NO GO”) decision on the presence of damage. In additional to quantifying the performance of each SHM system for the trial applications on the S-92 platform, this study also identified specific methods that can be used to optimize damage detection, guidance on deployment scenarios that can affect performance and considerations that must be made to properly apply CVM and PZT sensors. These results support the main goal of safely integrating SHM sensors into rotorcraft maintenance programs. Additional benefits from deploying rotorcraft Health and Usage Monitoring Systems (HUMS) may be realized when structural assessment data, collected by an SHM system, is also used to detect structural damage to compliment the operational environment monitoring. The use of in-situ sensors for health monitoring of rotorcraft structures can be a viable option for both flaw detection and maintenance planning activities. This formal SHM validation will allow aircraft manufacturers and airlines to confidently make informed decisions about the proper utilization of CVM and PZT technology. It will also streamline future regulatory actions and formal certification measures needed to assure the safe application of SHM solutions.

42 ENGINEERING↗
DOE OSTI2025

Comparative Performance Evaluation of Large Language Models for Extracting Molecular Interactions and Pathway Knowledge

Understanding the interactions and regulatory relationships among biomolecules is essential for deciphering complex biological systems and elucidating the mechanisms behind diverse biological functions. Traditionally, the collection of such molecular interaction data has relied on expert curation, a process that is both time-consuming and labor-intensive. To address these limitations, this study explores the use of large language models (LLMs) to automate the genome-scale extraction of molecular interaction knowledge. Here, we evaluate the performance of various LLMs on key biological tasks, including the identification of protein-protein interactions, detection of genes associated with pathways influenced by low-dose radiation, and inference of gene regulatory relationships. Our findings demonstrate that larger LLMs tend to perform better, particularly in extracting intricate gene and protein interactions. Despite their strengths, these models face challenges in recognizing functionally diverse gene groups and highly correlated regulatory relationships. Through a comprehensive analysis using established molecular interaction and pathway databases, we show that LLMs possess the potential to identify relevant biomolecules and predict their interactions, offering valuable insights and marking a significant step toward AI-driven biological knowledge discovery.

63 RADIATION, THERMAL, AND OTHER ENVIRON. POLLUTAN↗
DOE OSTI2022

Bayesian model updating with finite element vs surrogate models: Application to a miter gate structural system

Bayesian finite element (FE) model updating using direct model evaluations of large-scale high-fidelity FE models is extremely computationally expensive. Surrogate models can be used as fast emulators of FE models to accelerate the model calibration process. The physics/mechanics-based FE models are still the underpinning behind the surrogate models. Here, this paper evaluates the loss in accuracy and the gain in computational time while performing Bayesian model updating by using surrogate model evaluations compared to using direct FE model evaluations. This evaluation is crucial before entirely relying on surrogate models in model updating for structural health monitoring (SHM) and damage prognosis (DP) purposes. This paper also demonstrates Bayesian updating and surrogate model construction of large-scale high-fidelity FE models of infrastructure systems. In this regard, the miter gate structural system is considered as the testbed structure. Three predominant damage modes (loss of contact between gate and wall, loss of thickness due to corrosion, and loss of tension in the diagonal rods) are considered for model updating purposes. Bayesian model updating is performed using direct FE evaluations by leveraging parallel computing. Two types of surrogates, namely polynomial chaos expansion (PCE) and Gaussian process regression (GPR), are developed for the miter gate. Model updating is performed again using the trained surrogate models, and the updating results are compared with their counterparts obtained using the direct FE evaluation results. The posterior distribution of the FE model parameters obtained using the trained surrogates are sufficiently accurate with respect to the posterior obtained utilizing the direct FE evaluations. In addition, an approximate 4-fold decrease in the computational time was observed when using surrogate model evaluations instead of direct FE evaluations for model updating.

42 ENGINEERING↗
DOE OSTI2020

Data-driven Detection and Early Prediction of Thermoacoustic Instability in a Multi-nozzle Combustor

Thermoacoustic instability (TAI) is a critical issue in modern lean-burn gas-turbine combustors, which is induced by a strong coupling between the resonant combustor acoustics and fluctuations in the heat release rate. This instability may lead to high-amplitude pressure waves that generate undesirable noise levels as well as fatigue stresses in mechanical structures of the combustor. The intense pressure fluctuations due to TAI may also cause large flow perturbations and possibly flow reversal that may lead to flame oscillations, flame liftoff, and even flame blow-out. Hence, there is a strong need for exercising control actions in a timely fashion to mitigate the TAI phenomena. Anomaly detection is an essential prerequisite to the design of a good controller and such a detector must be able to reliably predict a forthcoming TAI. To detect and predict the onset of a TAI from an ensemble of pressure time series, this paper investigates three data-driven methods: Fast Fourier transform (FFT), symbolic time series analysis (STSA), and hidden Markov modeling (HMM). The main focus of the paper is to make a comparative evaluation of these three anomaly detection methods for classification of the current regime of operation into stable and unstable categories as well as for real-time identification of precursors to impending instabilities with short-length time series of measured variables (e.g., pressure oscillations). Finally, the results, generated on experimental data from a multi-nozzle combustor apparatus, have been compared to evaluate the performance of FFT, STSA, and HMM methods for TAI analysis.

42 ENGINEERING↗
DOE OSTI2022

Quantifying Energy Flows in PV Circularity Processes

As sustainable deployment and end-of-life management become a hot topic to timely address in the PV community, a dynamic comparative evaluation of the benefits of circular pathways such as reuse, and remanufacturing, recycling has not been performed holistically beyond material flows or LCA analysis. Energy flows are critical for evaluating energy generation technologies. Previously they have been used to compare renewables to fossil generation and then between PV technologies. This paper quantifies energy flows to evaluate circular pathways for PV. The energy flows tracking manufacturing, generation, and losses complementary to the mass flows of silicon are quantified leveraging the PV ICE framework.

circular economy↗
DOE OSTI2021

Reinforcement learning in discrete action space applied to inverse defect design

Abstract Reinforcement learning (RL) algorithms that include Monte Carlo Tree Search (MCTS) have found tremendous success in computer games such as Go, Shiga and Chess. Such learning algorithms have demonstrated super-human capabilities in navigating through an exhaustive discrete action search space. Motivated by their success in computer games, we demonstrate that RL can be applied to inverse materials design problems. We deploy RL for a representative case of the optimal atomic scale inverse design of extended defects via rearrangement of chalcogen (e.g. S) vacancies in 2D transition metal dichalcogenides (e.g. MoS 2 ). These defect rearrangements and their dynamics are important from the perspective of tunable phase transition in 2D materials i.e. 2H (semi-conducting) to 1T (metallic) in MoS 2 . We demonstrate the ability of MCTS interfaced with a reactive molecular dynamics simulator to efficiently sample the defect phase space and perform inverse design—starting from randomly distributed S vacancies, the optimal defect rearrangement of defects corresponds a line defect of S vacancies. We compare MCTS performance with evolutionary optimization i.e. genetic algorithms and show that MCTS converges to a better optimal solution (lower objective) and in fewer evaluations compared to GA. We also comprehensively evaluate and discuss the effect of MCTS hyperparameters on the convergence to solution. Overall, our study demonstrates the effectives of using RL approaches that operate in discrete action space for inverse defect design problems.

42 ENGINEERING↗
DOE OSTI2026

FATHOMS-RAG: A Framework for the Assessment of Thinking and Observation in Multimodal Systems that use Retrieval Augmented Generation

Retrieval-augmented generation (RAG) has emerged as a promising paradigm for improving factual accuracy in large language models (LLMs). We introduce a benchmark designed to evaluate RAG pipelines as a whole, evaluating a pipelines ability to ingest several modalities of information. We present (1) a curated dataset of 93 questions designed to evaluate a pipeline's ability to ingest textual data, tables, images, multimodal data, and cross-document multimodal data; (2) a phrase-level recall metric for correctness; (3) a nearest-neighbor embedding classifier in an attempt to classify pipeline hallucinations; (4) a comparative evaluation of 2 pipelines built with open-source retrieval mechanisms and 4 closed-source foundational models; and (5) a third-party human evaluation of the alignment of our correctness and hallucination metrics. We find that closed-source pipelines significantly outperform open-source pipelines in both the correctness and halucination metrics, with a wider performance gap in questions relying on multimodal and cross-document information. We also find after a human evaluation of our correctness and hallucination metric compared with our questions and pipeline responses, average agreement was 4.62 for correctness 4.53 for hallucination detection on a 1-5 Likert scale with 5 being strongly agree with our determination.

Hildebrand, Samuel [ORNL] (ORCID:0009000465963104)↗
DOE OSTI2021

Total System Performance Ratio—A Systems Based Approach for Evaluating HVAC System Efficiency

The prescriptive path is the most widely used approach for commercial code compliance in the United States. Though easy to implement, prescriptive approaches do not typically discriminate between minimally compliant, high-performing and poorly performing HVAC system configurations. Hence, to meet aggressive energy and carbon reduction goals, it is clear that energy codes will need to transition from prescriptive to performance-based approaches, a transition that is riddled with several challenges. This paper discusses a new HVAC system-based performance approach (HVAC System Performance) which provides a simpler solution to HVAV system evaluation compared to whole building performance, while keeping tradeoffs limited to specific building systems. The Total System Performance Ratio (TSPR) is a metric for evaluation of overall system efficiency instead of individual component efficiency, a solution which could also eventually facilitate the transition to a 100% performance-based code structure. TSPR is a ratio that compares the annual heating and cooling load of a building to the annual energy consumed by the building’s HVAC system. A calculation software tool has been developed for determining a building’s TSPR. Already incorporated into the 2018 Washington State Energy Code, this approach is also being evaluated by ASHRAE Standard 90.l Project Committee and has the potential to provide a comprehensive performance-based approach for HVAC system evaluation and analysis.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2021

HVAC System Performance for Energy Codes (Technical Brief)

The prescriptive path is likely the most widely used approach for commercial code compliance in the United States. Though easy to implement, the prescriptive approach does not discriminate between high-performing and poorly performing heating, ventilation, air conditioning (HVAC) system configurations that are both minimally compliant. To meet aggressive energy and carbon reduction goals, energy codes will need to transition from prescriptive to performance-based approaches, a transition that is riddled with several challenges. HVAC System Performance is a discipline performance path and provides a simpler solution to HVAC system evaluation compared to whole building performance, while keeping tradeoffs limited to specific building systems. The Total System Performance Ratio (TSPR) is a metric for evaluation of overall system efficiency instead of individual component efficiency, a solution that could also eventually facilitate the transition to a 100% performance-based code structure. TSPR is a ratio that compares the annual heating and cooling load of a building to the annual energy consumed by the building’s HVAC system. A web-based calculation tool has been developed for determining a building’s TSPR. Already incorporated into the 2018 Washington State Energy Code, this approach has also been evaluated by the ASHRAE Standard 90.1 Project Committee and has the potential to provide a comprehensive performance-based approach for HVAC system evaluation and analysis

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2026

Meta-analysis of North American Arctic and boreal aboveground biomass datasets: assessing accuracy, dynamics, and similarities

The North American arctic and boreal regions (ABRs) are rapidly warming and experiencing intensifying disturbances. Accurately quantifying aboveground biomass (AGB) is critical for understanding the impacts of these changes on the carbon cycle and for designing climate change mitigation strategies. Several AGB maps have been developed for the North American ABRs, including recent contributions from National Aeronautics and Space Administration’s Arctic-Boreal Vulnerability Experiment (ABoVE) campaign. However, these maps differ widely in training data, methodology, and resulting AGB density estimates. Presently, a comprehensive comparative evaluation is lacking, making it difficult for users to select datasets suited to their research or management needs. Here, in this study, we conducted a comparative analysis of nine AGB density datasets across North American ABRs, specifically for Alaska and Canada. We (1) summarized AGB by ecoregion and Canadian provinces, (2) evaluated their accuracy against field-based measurements, (3) analyzed spatial and temporal similarities among datasets, and (4) assessed their ability to capture disturbance (fire and harvest) impacts on AGB. We found substantial variation in regional and local AGB estimates across datasets, with overall accuracy ranging from R 2 = 0.25–0.62 and Bias% from −47.8% to 69.9% when validated against field plots. Despite these differences, most datasets have comparatively consistent spatial patterns in AGB (r > 0.8 for most cases). In contrast, agreement on the temporal patterns of AGB change is generally low. We found datasets with spatial resolutions ⩽300 m are capable of capturing disturbance impacts on AGB dynamics, though sensitivity varies across products. Our findings and dataset summary provide guidance for selecting appropriate AGB datasets for different applications within our study area. Our analysis also highlights the need to decrease map bias and increase capability to detect temporal change to decrease uncertainty of AGB datasets potentially by using training data which is representative of major plant functional types within the mapped area.

ABoVE↗
DOE OSTI2020

Evaluating long-term model-based scenarios of the energy system

Energy-economic models are used to provide science-based decision support in a variety of contexts. Analyses using these tools often involve defining a “reference” scenario, which serves as a counterfactual against which alternative scenarios are compared. Evaluating scenarios, including reference scenarios, is critical for establishing the credibility of the analyses these models support. We propose a framework for evaluating energy system scenarios which consists of three parts – a qualitative storyline, quantitative metrics, and evaluation criteria. We apply this framework to the reference scenario for GCAM-USA, a version of the global human-Earth system model GCAM (Global Change Assessment Model) with state-level detail in the United States, focusing on the evolution of the electric power sector. We develop new visual analytic tools to facilitate the evaluation of model outcomes in 51 sub-national regions, and demonstrate how scenario performance can be tracked and compared across four quantifications of the GCAM-USA reference scenario.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2026

Evaluating CO 2 mitigation strategies in SAF biorefineries: Techno-economic and life cycle analysis

The aviation sector requires scalable decarbonization strategies, and lignocellulosic sustainable aviation fuel (SAF) represents a promising pathway. This study comparatively evaluates the techno-economic analysis and life cycle assessment (LCA) of three CO 2 management strategies integrated within a U.S.-based gasification–Fischer–Tropsch SAF biorefinery: (i) catalytic hydrogenation of captured CO 2 to methanol, (ii) geological CO 2 sequestration, and (iii) mineralization to sodium bicarbonate (NaHCO 3 ). Techno-economic analysis indicates that methanol synthesis requires approximately 26% higher capital investment and 33% higher operating costs than mineralization. Although methanol co-production generates the highest gross revenue, NaHCO 3 production reduces the SAF minimum selling price by approximately 38% relative to both methanol synthesis and geological sequestration pathways, reflecting a more balanced cost allocation through mineral co-product valorization. Geological sequestration lowers operating costs by nearly 50% compared with methanol synthesis but remains highly dependent on carbon credit mechanisms. LCA reveals substantial divergence in climate performance. Relative to methanol synthesis, sequestration improves net greenhouse gas performance by approximately 163%, transitioning the system from net-positive to net-negative emissions. Mineralization further enhances carbon mitigation, achieving roughly 85% greater carbon reduction than sequestration and over sixfold improvement relative to methanol synthesis within the defined system boundary. Sensitivity analysis identified hydrogen price, co-product market value, and process emissions as dominant drivers. Under baseline assumptions, CO 2 mineralization is found to offer the most balanced pathway.

Carbon capture and storage↗
DOE OSTI2024

Comparative clumped isotope temperature relationships in freshwater carbonates

Lacustrine, riverine and spring carbonates represent archives of terrestrial climates and their geochemistry has been used to study palaeoenvironments. Clumped isotope thermometry is an emerging tool that has been applied to freshwater carbonates. Limited work has been done to evaluate comparative relationships between clumped isotopes and temperature in different types of modern freshwater carbonates. This study assembles an extensive calibration data set with 135 samples of modern freshwater carbonates from 96 sites and constrains the relationship between independent observations of water temperature and the clumped isotopic composition of carbonates (denoted by Δ 47 ), including new measurements, and recalculates published data in accordance with current community-defined standard values. For temperature reconstruction, the study reports a composite freshwater calibration and material-specific calibrations for biogenic carbonates (freshwater gastropods and bivalves), fine-grained carbonate (e.g. micrites), biologically mediated carbonates (microbialites and tufas) and travertines. Material-specific calibration trends show a convergence of slopes that are in agreement with recently published syntheses, but statistically significant differences in intercepts occur between some materials (e.g. some biogenics, fine-grained carbonates). These differences may arise due to unresolved seasonal biases, kinetic isotope effects and/or varying degrees of biological influence. The impact of different calibrations is shown through application to new data for glacial and deglacial age travertines from Austria and published data sets. While material-specific calibrations may yield more accurate results for biogenic and fine-grained carbonate samples, the use of material-specific and the composite freshwater calibrations generally produces values within 1.0–1.5°C of each other, and typically fall within calibration uncertainty given limitations of precision.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Monte Carlo Event Generation with Continuous Normalizing Flows

We apply continuous normalizing flows trained with the flow matching method to the problem of phase-space sampling in Monte Carlo event generation for high-energy collider physics. Focusing on lepton-pair and top-quark pair production with multiple jets, the two computationally most expensive processes at the Large Hadron Collider, we train helicity-conditioned continuous normalizing flows to remap the random numbers used in matrix element evaluation. Compared to standard methods, we achieve unweighting efficiency improvements by factors of up to 184 and 25 for the two processes at their respective highest jet number, at the cost of an increased evaluation time. When combining the advantages of continuous normalizing flows with the fast evaluation times of coupling-layer-based flows, using the RegFlow approach, we find parton-level unweighted event generation walltime gains of about a factor of 10 at the highest jet numbers. These substantial gains highlight the promise of samplers based on machine learning for next-generation collider experiments.

Bothmann, Enrico [CERN; Gottingen U.] (ORCID:00000↗
DOE OSTI2020

Vertex Reordering for Real-world Graphs and Applications: An Empirical Evaluation

Vertex reordering is a way to improve locality in graph computations. Given an input (or ``natural'') order, reordering aims to compute an alternate permutation of the vertices that is aimed at maximizing a locality-based objective. Given decades of research on this topic, there are tens of graph reordering schemes, and there are also several linear arrangement ``gap'' measures for treatment as objectives. However, a comprehensive empirical analysis of the efficacy of the ordering schemes against the different gap measures, and against real-world applications is currently lacking. In this study, we present an extensive empirical evaluation of up to 11 ordering schemes, taken from different classes of approaches, on a set of 34 real-world graphs emerging from different application domains. Our study is presented in two parts: a) a thorough comparative evaluation of the different ordering schemes on their effectiveness to optimize different linear arrangement gap measures, relevant to preserving locality; and b) extensive evaluation of the impact of the ordering schemes on two real-world, parallel graph applications, namely, community detection and influence maximization. Our studies show a significant divergence among the ordering schemes (up to $40\times$ between the best and the poor) in their effectiveness to reduce the gap measures; and a wide ranging impact of the ordering schemes on various aspects including application runtime (up to $4\times$), memory and cache use, load balancing, and parallel work and efficiency. The comparative study also help reveal the nuances of a parallel environment (compared to serial) on the ordering schemes and their role in optimizing applications.

Barik, Reet↗