Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “METRICS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,477 records · Page 82

Reliable Measures of Spread in High Dimensional Latent Spaces

Understanding geometric properties of the latent spaces of natural language processing models allows the manipulation of these properties for improved performance on downstream tasks. One such property is the amount of data spread in a model’s latent space, or how fully the available latent space is being used. We demonstrate that the commonly used measures of data spread, average cosine similarity and a partition function min/max ratio I (V), do not provide reliable metrics to compare the use of latent space across data distributions. We propose and examine six alternative measures of data spread, all of which improve over these current metrics when applied to seven synthetic data distributions. Of our proposed measures, we recommend one principal component-based measure and one entropy-based measure that provide reliable, relative measures of spread and can be used to compare models of different sizes and dimensionalities.

97 MATHEMATICS AND COMPUTING↗

Neural Network Analysis of Nuclear Magnetic Resonance and Infrared Spectra

Nuclear magnetic resonance (NMR) spectroscopy and infrared (IR) spectroscopy are powerful chemical characterization techniques with broad general usage. However, the manual evaluation of the resulting spectra is time-consuming and requires significant expertise, preventing insights from being used in real-time applications. With recent advances in computation and artificial intelligence (AI), new tools are available for automating spectral interpretation. In this work, machine learning (ML) algorithms using 1-dimensional convolutional neural networks (CNNs) were applied to identify common functional groups from spectral information. Raw spectra were collected virtually from the Human Metabolome Database (HMDB) and National Institute of Standards and Technology (NIST) Chemistry WebBook and processed into a suitable standard. Algorithm design was tailored to best fit the nature of the problem, with built-in flexibility to accommodate relevant parameters beyond the raw spectral input, specifically solvent identity and magnetic frequency for NMR. The predictive capability of the algorithm in identifying functional groups is displayed in several examples. This methodology has been compiled into a code repository and could easily be modified to adapt alternative data sources, including other spectrum types. To mitigate overfitting, a common problem in mathematical modeling where overfamiliarity with training data produces trends that are not representative of the general data, a novel metric was developed, referred to as Accufit. Accufit includes a parameter that penalizes substantial differences in the training accuracy and the accuracy of an independent validation set. Examples are presented showing the effectiveness of Accufit in maintaining the model’s predictive capability while controlling the overfitting when used as a custom metric for hyperparameter tuning.

Sturgill, James↗

Surrogate Neural Architecture Codesign Package (SNAC-Pack)

Neural Architecture Search is a powerful approach for automating model design, but existing methods struggle to accurately optimize for real hardware performance, often relying on proxy metrics such as bit operations. We present Surrogate Neural Architecture Codesign Package (SNAC-Pack), an integrated framework that automates the discovery and optimization of neural networks focusing on FPGA deployment. SNAC-Pack combines Neural Architecture Codesign's multi-stage search capabilities with the Resource Utilization and Latency Estimator, enabling multi-objective optimization across accuracy, FPGA resource utilization, and latency without requiring time-intensive synthesis for each candidate model. We demonstrate SNAC-Pack on a high energy physics jet classification task, achieving 63.84% accuracy with resource estimation. When synthesized on a Xilinx Virtex UltraScale+ VU13P FPGA, the SNAC-Pack model matches baseline accuracy while maintaining comparable resource utilization to models optimized using traditional BOPs metrics. This work demonstrates the potential of hardware-aware neural architecture search for resource-constrained deployments and provides an open-source framework for automating the design of efficient FPGA-accelerated models.

Weitz, Jason [UC, San Diego] (ORCID:00090004631535↗

Electrical and Thermal Load Impacts of Three District Heating and Cooling Designs for an Existing Community in Washington, DC

District energy systems that provide building heating and cooling are a promising option to provide low-cost, energy efficient heating and cooling solutions for communities. As some buildings move to electrified heating designs, there may be significant increases in electric grid demand, particularly in the winter. District energy systems may be able to help reduce these high electrical demands on the electric grid, while also providing decreased overall energy consumption with an increased flexibility in electrical energy usage. To evaluate and compare district options for district-based heating and cooling, an existing neighborhood of Washington, DC consisting of 35 existing buildings was selected as a case study. This study models and compares a fourth generation (4G) district heating and cooling systems that provide hot and chilled water from a central plant directly to each prosumer and a fifth generation (5G) district heating and cooling systems that provide near-ambient water via geothermal boreholes to interface with an energy transfer station with a heat pump at each prosumer. These district systems are compared against a baseline of the non-connected buildings with their self-contained and current HVAC systems. The neighborhood was selected as a good candidate for a district system for its diversity of loads (mix of different commercial and property types) and the availability of building characteristics. URBANopt District Energy Systems (DES) was used to model the buildings and create initial Modelica models for the district systems. Finally, the DES models were tuned and simulated in Dymola, and the outputs were post-processed for verification and to calculate the electrical and thermal grid impact metrics. This is the first documentation of this workflow and its complete analysis. The three designed systems are compared via their grid metrics including: total energy consumption, electrical demand peak loads, daily peak-to-valley ratios, and system ramping. A detailed discussion is provided about how each system impacts and interacts with the electric grid.

15 GEOTHERMAL ENERGY↗

Leveraging Large Language Models for Real-World Data Evidence: A Framework for Automated Treatment Extraction and Data Harmonization

Background: The ability to comprehensively collect treatment information from cancer patient medical records would enable studies to evaluate real-world benefits and risks tied to specific treatments. Currently, it is difficult to system- atically collect high-quality treatment information because it is often stored in unstructured text. Manually extracting and standardizing drug and regimen data is time-intensive. Recent advances in large language models (LLMs) offer a potential solution for automated extraction of structured treatment information from clinical text. Objective: This study systematically evaluates the utility of four LLMs from the Llama family for automated extraction of oncology treatment information from clinical text. This information can guide researchers using cancer registry data to provide insights into cancer care and outcomes beyond clinical trials. Methods: Four instruction-tuned Llama models with varying parameter counts (1B, 3B, 8B, and 70B) were evaluated for their ability to extract treatment information from clinical documents. A unified oncology knowledge base integrating seven major public data sources was developed to standardize and normalize extracted entities—a critical step for harmonizing data from diverse sources. Extracted treatment data were compared against expert-annotated ground truth. Model performance was assessed using accuracy metrics (Precision, Recall, F1-Score) and opera- tional feasibility metrics, including processing speed and structural compliance of the output. Results: A strong positive correlation was observed between model size and extraction accuracy. F1-score improved from 0.609 for the 1B model to 0.710 (3B), 0.807 (8B), and 0.828 (70B). While larger models demonstrated superior accuracy and compliance, they incurred higher computational costs. The modest performance difference between 8B and 70B suggests diminishing returns with increasing model size. Conclusions: LLMs represent a viable technology for automating oncology treatment extraction. The 8B-parameter model emerged as a highly effective option, balancing high accuracy and computational efficiency. Selecting an appropriate LLM for deployment in cancer registries involves a trade-off between desired accuracy and available operational resources. Harmonizing extracted entities with the oncology knowledge base facilitates standardized integration into common data models, enhancing data quality for real-world evidence analyses.

artificial intelligence↗

Job Scheduler-Driven Power Gateway for High Performance Computing

Power gateways in the form of a microgrid can incorporate multiple distributed energy resources (DER) in either grid forming or grid following mode and support high performance computing (HPC) power profiles including the large load-follow requirements observed in multi-user HPC systems. The microgrid’s flexibility to operate in either grid forming or grid following mode and to actively switch between these modes enables baseline power from multiple non-baseline DER while maintaining high power quality metrics for the HPC system. But this enormous flexibility in demand response and time of use shifting is generally programmed independently of any integration with an HPC job scheduler which can better inform the load shaping by the microgrid. While there are many existing approaches where the HPC job scheduler takes in information from the grid to make queue scheduling decisions, this work takes the opposite view and explores a scheduler where the jobs in the queue can directly impact the settings of the grid. Several HPC scheduler strategies are tested where the jobs in the queue directly impact the settings of a microgrid designed for HPC operation which is driving a datacenter with three classes of HPC architectures. The scheduler operation is shown using a microgrid with 64 kW of solar capacity and 320 kWh of battery over a period of 21 days operating with significant low-follow swings, a throttled grid, cloudy conditions, switching between grid following and grid forming modes, and a wide range of battery states-of-charge all while maintaining high quality power metrics. The scheduler provides a mechanism for the job queue to directly impact a power gateway like a microgrid and to improve HPC power outcomes such as maximizing renewable energy usage

microgrid↗

Natural Language Processing to Inform Agent-Based Modeling: With Application to Modeling Adoption of Medium-Duty Electric Vehicles

Agent-based socio-technical modeling of medium- and heavy-duty (MDHD) electric vehicle (EV) adoption has the potential to provide analysis, prediction, and gui. This paper describes new applications of text analysis developed through machine learning (ML) to build and understand relevant topics and their saliency in the published discourse on adoption of MDHD EVs. This work contributes to the state of the art in topic mining models by defining a new metric of topic ranking (START) that quantifies the importance of predefined topics within the corpus using weighted results for predefined topics from two topic modeling approaches: Latent Dirichlet Allocation (LDA) and BERTopic. The START metric is then demonstrated in practice to model how academia and industry view the EV adoption process based on the respective texts published by these groups. Results show that academic literature places more emphasis on categories of interests such as norms/attitudes and adopter knowledge, while trade journals tend to emphasize long-term cost more than academia. The two bodies of literature agree on the importance of policy and incentives in MDHD EV adoption. Together these results illustrate the potential to use ML-based text analysis to populate the characteristics of agent-based socio-technical models.

Electric vehicle adoption, fleet electrification, ↗

Spent nuclear fuel receipt rate analysis within an integrated waste management system (IWMS) architecture that includes consolidated storage

A key parameter in analyzing the performance of an integrated waste management system (IWMS) architecture for the disposition of spent nuclear fuel (SNF) is the SNF receipt rate from reactor and other custodian sites. Receipt rate in this paper means how much SNF is accepted per year for transport in the IWMS from such sites. Introducing one or more federal consolidated interim storage facilities (CISFs) into the IWMS architecture can potentially accelerate the receipt rate profile over time relative to system architectures without a CISF. This raises the question of what an optimal SNF receipt rate profile for an IWMS architecture might be in view of practical constraints and desired system performance attributes and associated metrics. This paper describes a sensitivity study on SNF receipt rates and the associated results for a selected set of IWMS scenarios aimed at informing near-term planning for interim storage capabilities and transportation assets. Two different strategies for CISF operation while awaiting availability of a disposal system to receive SNF are compared: one that relatively quickly fills an initial CISF and then idles the transportation system; and another that aims for more continuous use of transportation assets and receipt capabilities at the CISF. This study examines cost considerations and other factors, such as the timing of clearing reactor sites of SNF, efficient use of capital assets, and some other metrics that might be important to a CISF host community. Based on the analysis, an initial approach is presented that targets a continuous receipt strategy while maintaining the flexibility to step up receipt capabilities to a reasonable degree when needed and beneficial, within overall system constraints.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Benefits and challenges of vibroacoustic process monitoring to support operators in nuclear research facilities

According to the International Atomic Energy Agency in 2024, global nuclear energy capacity is projected to increase by 2.5 times by 2050 in their high-case scenario. Research on the laboratory scale of innovative processes within nuclear facilities is ongoing to improve nuclear fuel cycle capabilities. Vibroacoustic process monitoring of these techniques has potential to inform operators regarding process specific metrics, predictive maintenance, and anomalous event detection. Additionally, this type of monitoring has the potential to aid in nuclear safeguards. Challenges regarding vibroacoustic monitoring in these environments include high temperature, high radiation, limited access, and shielded equipment limiting sensor type and placement. Microphones, accelerometers, and temperature sensors were deployed both inside and near these environments during operation to determine environment driven limitations, sound attenuation of containment enclosures, process specific metrics, and future potential of vibroacoustic monitoring in these environments. Overall, this work highlights the benefits and limitations of vibroacoustic process monitoring in nuclear research facilities.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Spent Nuclear Fuel Receipt Rate Analysis within an Integrated Waste Manage-ment System Architecture that Includes Consolidated Interim Storage

A key parameter in analyzing the performance of an integrated waste management system (IWMS) architecture for the disposition of spent nuclear fuel (SNF) is the SNF receipt rate from reactor and other custodian sites. The introduction of one or more federal consolidated interim storage facilities (CISFs) into the IWMS architecture can enable the receipt rate profile as a function of time to be accelerated relative to system architectures without a CISF. The question then arises as to what an optimal SNF receipt rate profile for an IWMS architecture might be in view of practical constraints and desired system performance attributes and associated metrics. This paper describes a sensitivity study on SNF receipt rates and the associated results for a selected set of IWMS scenarios aimed an in-forming near-term planning for interim storage capabilities and transportation assets. Two different strategies are compared, one that fills an initial CISF quickly and then idles the transportation system while a disposal system is prepared, and a second strategy that aims to provide a more continuous use of transportation assets and receipt capabilities at the IWMS while the disposal system is readied for SNF receipt. Cost considerations and other factors such as impact on timing of clearing reactor sites of SNF, efficient use of capital assets, and other metrics, including those which may be important to a CISF host community, are examined. Based on the analysis, an initial approach is presented targeting a continuous receipt strategy while having the flexibility to step up receipt capabilities to a reasonable degree when needed and beneficial within overall system constraints.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Surrogate Neural Architecture Codesign Package (SNAC-Pack)

Neural architecture search (NAS) is a powerful approach for automating model design, but existing methods often optimize for accuracy alone or rely on proxy metrics such as bit operations (BOPs) that correlate poorly with hardware cost. This gap is particularly large for FPGA deployment, where cost is dominated by a multi-dimensional budget of lookup tables, DSPs, flip-flops, BRAM, and latency. We present the Surrogate Neural Architecture Codesign Package (SNAC-Pack), an open-source AutoML framework for hardware-aware neural architecture codesign and end-to-end FPGA deployment. SNAC-Pack runs a multi-objective global search with Optuna and NSGA-II, loading trials to a shared SQLite store that enables parallel workers across compute nodes. A hardware surrogate model outputs per-trial resource and latency estimates, avoiding the synthesis cost that would otherwise dominate the search loop. A local search stage then applies quantization-aware training (QAT) together with iterative magnitude pruning in a combined compression loop, after which the final model is synthesized to FPGA firmware via the hls4ml Python library. A YAML configuration and an optional agentic frontend let users run the pipeline on new datasets without modifying the framework. We demonstrate SNAC-Pack on jet classification at the Large Hadron Collider and superconducting qubit readout, discovering compact architectures that match or exceed strong baselines on the task metric while reducing FPGA resource utilization and, in the qubit readout case, reducing the design space exploration process from months of manual fine-tuning to hours of automated search.

Weitz, Jason [UC, San Diego]↗

Infrared-enhanced Photometric Redshifts for the Dark Energy Survey Y6 Gold catalogue

The Dark Energy Survey (DES) provides optical data across 5000 square degrees of the southern sky, enabling a broad range of extragalactic and cosmological studies. Combining DES data with infrared surveys offers the opportunity to improve its photometric redshift (photo-z) estimates. We aim to investigate improvements in photometric redshift estimation achieved by combining DES optical data with infrared measurements from the VISTA Hemisphere Survey (VHS) and the Wide-field Infrared Survey Explorer (WISE), and release an updated version of the catalogue. We performed a positional sky cross-match between the DES Y6 Gold catalogue matched to a spectroscopic dataset, the 2013 AllWISE Data Release, and VHS Data Release 5, in order to test these improvements using the Directional Neighbourhood Fitting (DNF) algorithm (Y6 Gold catalogue reference estimator). We additionally matched it to the unWISE catalogue to verify the performance against this deeper dataset. Adding infrared data reduces all the metrics (scatter, bias and outlier fraction) in photo-z estimates, particularly at higher redshifts in comparison with only using optical data from DES. The obtained results are globally better for the DES+WISE sample, with improvements that are statistically significant. On the other hand, the addition of the VHS bands to available depth is only marginal. The combined use of DES and WISE W1 and W2 data improves the photometric redshift metrics analysed here. The addition of VHS data at the DES and VHS depths explored here does not provide any further improvement at z less than 1.5, indicating that, under these constraints, WISE data may already capture the key infrared features and depth needed for accurate photo-z estimation. In addition, low signal-to-noise (less than 10) infrared data does not contribute to any improvement beyond the DES optical dataset.

Puebla, M. M. [Madrid U.; Madrid, CIEMAT]↗

Enhancing Operational Safety via Agentic Dialogue Hazard Identification Analysis

Operational safety in high-stakes domains such as industrial process control, autonomous, and safety-critical systems demand reliable hazard identification. While large language models (LLMs) have shown promise in automating safety analysis tasks, single-turn, monolithic inference is brittle: it lacks the self-correction, deliberation, and contextual refinement that safety engineers apply iteratively. In this paper, we introduce HAZDIAL, a framework that investigates whether structured agentic dialogue (multi-agent, multi-turn interactions) improves the quality of NLP-based hazard identification over single-pass baselines. We systematically compare two dialogue modalities: adversarial debate and constructive discussion, and propose an genetic algorithm-based agentic interaction optimization. We evaluate all configurations against a curated golden dataset using standard classification metrics (accuracy, precision, recall, F1) and a novel dialogue metrics. This work advances the intersection of dialogue systems, multi-agent reasoning, and AI safety, providing empirical evidence for dialogue-driven hazard analysis.

Das, Sanjay [ORNL] (ORCID:0009000542591915)↗

Oversizing and Part-Load Problems

Oversizing, the common engineering practice of specifying devices with capacity exceeding the actual load requirement, is a widespread practice across virtually all building technologies end-use categories, including HVAC, electrical systems, lighting, appliances, and plug loads. This practice, driven by factors like design uncertainty, institutional pressures, and risk aversion, results in wasted capital investment, control difficulties, and excessive energy consumption due to inefficient part-load operation. Part-load operation, where devices run below maximum capacity, is the dominant operating mode in most energy systems and presents a complex design challenge. Solutions to match output to load fall into three broad categories: constraining the output, adjusting the device’s internal behavior, and linking output to energy storage or other waste-heat reuse applications. The energy implications of part-load are critical, as efficiency often drops sharply as load decreases across a wide diversity of devices. To quantify the extent of this problem, we derive a dimensionless Part-Load Metric (PLM) based on device efficiency and its frequency distribution of operating hours at various output levels. The PLM quantifies the deviation of a device's actual efficiency from its maximum design efficiency. This metric also serves as a measure of "capital inefficiency," enabling engineers to compare the impact of different part-load solutions and providing a unified framework for evaluating performance across various devices and systems.

Meier, Alan↗

Database Performance Monitoring for DUNE

This report presents the research, design, and implementation of improved PostgreSQL monitoring for DUNE Rucio database services using Checkmk. The project began with a request to improve dashboard visibility for database performance metrics, including connection usage, configured connection limits, lock activity, wait behavior, storage trends, query performance, and saturation alerts. The initial implementation focused on the dune_rucio_prod database on the rucio_prod PostgreSQL instance because connection saturation and lock contention are direct reliability risks for database-backed services. Existing Checkmk PostgreSQL monitoring was investigated, and several gaps were identified. Built-in connection monitoring did not clearly separate active, idle, idle-in-transaction, total, and usage-percent metrics, while the built-in lock monitoring simplified PostgreSQL lock modes into shared and exclusive categories. To address these gaps, two DSG-specific Checkmk local checks were created: one for connection-state monitoring and one for lock-state monitoring. These checks supplement the built-in PostgreSQL checks and provide additional performance data for dashboard graphs, service states, and alerts.

Bowers, Elliot [Cabrillo Coll.]↗

Evaluating the impacts of Variable Message Signs on Airport Curbside Performance Using Microsimulation

Curbs play a vital role in facilitating vehicle access and egress for individuals at airports. Inefficiently allocating this resource hinders airport accessibility and productivity, resulting in congestion, longer travel times, and increased pollution. As airport demand fluctuates throughout the day and grows over time, airports face intensified curbside pressure. Yet, curb management research is significantly less robust at airports than in urban areas. Given the unbalanced nature of airport demand—riders tend to arrive simultaneously at specific entrances at certain hours—Variable Message Sign (VMS) arises as a cost-effective technology to divert vehicles from congested to underutilized curbs. Still, VMS implementation faces a significant challenge. Historically, airports have managed VMS heuristically and by intuition rather than an evidence-based approach. This research investigates the impacts of implementing VMS on curb performance at airports. By considering different driver compliance rates (DCR), we aim to determine when the sign should be turned on and off to diverge traffic to avoid undesired externalities while enhancing curb performance. Using a validated agent-based microsimulation model, VISSIM, we analyzed the Seattle-Tacoma (SeaTac) Airport as a case study. We modeled sixteen VMS management scenarios and a baseline where the message sign is not displayed, diverging vehicles between the departures and arrivals access levels at four different moments (early morning, morning, afternoon, and late night). We quantified the effects of VMS using seven metrics, including curb productivity index (CPI), curb accessibility (CA), queue length, queue duration, delay, vehicle counts, and emissions. The results of each scenario were compared against the baseline using absolute and relative changes and Repeated Measures ANOVA. Overall, VMS improved curb performance and traffic conditions at the airport, reducing emissions by 14.8% to 8.9%. Moreover, significant reductions in queue length (1,150 ft to 100 ft) and duration (15 to 144 minutes) were observed in the sending link under all VMS policies. However, impacts on the receiving link varied based on congestion, with significant increases in queue duration (9.8 to 24 min) when congested but no substantial changes in free flow. Notably, diverging vehicles to congested links resulted in non-significant results, and activating late and deactivating late VMS affected curb productivity (-5.8% to -61.4%), curb accessibility (-16.5% to -25.8%), cumulative counts (-33.4% to -59.4%), and vehicle delay (95.98% to 594.3%). Activating VMS before congestion begins in the sending link and deactivating before a queue forms in the receiving link yield the most significant improvements: 8.1% to 10.1% in CPI, 9.4% to9.6% in CA, -29.3% to -77.9% in total delay, -11.6% to -13.9% in total emissions, and 101% to 103% in cumulative counts. As the analysis was made with a wide range of time periods, access levels, driver compliance rates, and scenarios, we believe our findings can provide valuable insights into how airports should manage VMS. Our work introduces a novel approach to the scientific airport literature, as some of our metrics were previously unexplored. Additionally, we propose a methodology that other airports can adopt to maximize their curb performance.

Gutierrez, Jorge D.↗

Fundamentals of Resource Adequacy for Modern Power Systems

This webinar covers and introduction to power system resource adequacy, examining definitions, metrics, an overview of probabilistic methods, incentives and capacity credits, storage modeling, and evolving practices.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Discovery of an Intrinsic Antiferromagnetic Semiconductor EuSc 2 Te 4 With Magnetism‐Driven Nonlinear Transport

Magnetic topological materials have recently emerged as a promising platform for studying quantum geometry by the nonlinear transport in thin film devices. In this work, an antiferromagnetic (AFM) semiconductor EuSc₂Te₄ as the first bulk crystal that exhibits quantum geometry-driven nonlinear transport is reported. This material crystallizes into an orthorhombic lattice with AFM order below 5.2 K and a bandgap of less than 50 meV. The calculated band structure aligns with the angle-resolved photoemission spectroscopy spectrum. The AFM order preserves combined space-time inversion symmetry but breaks both spatial inversion and time-reversal symmetry, leading to the nonlinear Hall effect (NLHE). Nonlinear Hall voltage measured in bulk crystals appears at zero field, peaks near the spin-flop transition as the field increases, and then diminishes as the spin moments align into a ferromagnetic order. This field dependence, along with the scaling analysis of the nonlinear Hall conductivity, suggests that the NLHE of EuSc₂Te₄ involves contributions from quantum metric, in addition to extrinsic contributions, such as spin scattering and junction effects. Furthermore, this NLHE is found to have the functionality of broadband frequency mixing, indicating its potential applications in electronics. This work reveals a new avenue for studying magnetism-induced nonlinear transport in magnetic materials.

36 MATERIALS SCIENCE↗