Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Evaluation Framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

Resilience Metrics for Solar Photovoltaics

This workshop presentation proposes the development of solar photovoltaic (PV) system resilience metrics and a methodology and framework for evaluation of PV resilience metrics. PV resilience metrics are needed to establish a consistent basis for reporting, evaluation, and data collection by industry, evaluate performance of PV systems that have been subject to natural hazards, correlating resilience to system attributes, and predicting resilience for any PV system. PV resilience metrics can guide improved system design, standards, and insurance coverage. Establishing consistent metrics can foster data collection on impacts of natural hazards on PV systems.

14 SOLAR ENERGY↗

The ABCs of On-Demand Transit (ODT)

On-demand mobility - also referred to as on-demand transit (ODT) - is a form of public mobility that is flexible with respect to where and when service is provided, and ODT deployments have increased significantly in recent years. Transit agencies are becoming increasingly interested in ODT, and due to differing definitions and various service design and business model options, it can be difficult to learn about the emerging industry. This work provides an overview of the definitions of ODT, recent trends internationally and in the U.S., ODT's benefits and challenges (particularly compared to fixed-route transit), three primary service design options, system costs and funding considerations, and a metrics framework for evaluating ODT systems to ensure continued successful performance. Identified benefits include increased service areas, short ride and wait times, increased user flexibility, potential to reduce energy consumption and emissions through shared trips and smaller, right-sized vehicles, increased safety and comfort through door-to-door service, and rich data streams including granular spatio-temporal data that can be analyzed to continuously improve the service. Challenges include scaling ODT service up as small increases in ridership require additional supply to keep service quality high, serving peak times including keeping low wait times, the lack of fixed schedule being challenging for commuters, integrating ODT services with nearby transit systems, and equity for riders without smartphones who cannot track the vehicle in a mobile app. Finally, an overview of seven ODT case studies (in Texas, Missouri, New York, and Ontario, Canada) performed by NREL and related analysis of travel time, energy and emissions, costs, and equity are presented. Initial key findings include: ODT can be cost- and energy-effective compared to fixed-route transit, ODT serves more people than other transit options, and ODT system deployments can be followed by rapid growth.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Atmospheric Modeling and Denoising for Millimeter-Wave Line Intensity Mapping

Line-intensity mapping (LIM) offers a promising approach to mapping large-scale cosmic structure, and the greatest obstacle for ground-based observations at millimeter wavelengths is foreground contamination from atmospheric emission. In this work, we present a simulation and denoising framework designed to isolate and subtract atmospheric fluctuations from LIM data, modeled after the instrument parameters of the South Pole Telescope Summertime Line Intensity Mapper (SPT-SLIM). We generate mock observations spanning 125-175 GHz containing cosmic signals, precipitable water vapor screens, ice crystal fluctuations, and photon noise. We then implement a spatial-spectral atmospheric removal pipeline combining per-pixel linear template regression with a two-dimensional Fourier-domain filter. The framework is evaluated under simulated conditions in the South Pole and the Atacama Desert across three key metrics: cosmic signal preservation, foreground subtraction efficiency, and instrument noise injection. Our pipeline achieves atmospheric suppression at large spatial scales, and these results establish a physically grounded foundation for atmosphere removal in ground-based LIM data collection.

Saye, Laney [UC, Berkeley (main)] (ORCID:000900078↗

Utilizing Reinforcement Learning to Continuously Improve a Primitive-Based Motion Planner

We report in this paper describes how the performance of motion primitive-based planning algorithms can be improved using reinforcement learning. Specifically, we describe and evaluate a framework that autonomously improves the performance of a primitive-based motion planner. The improvement process consists of three phases: exploration, extraction, and reward updates. This process can be iterated continuously to provide successive improvement. The exploration step generates new trajectories, and the extraction step identifies new primitives from these trajectories. These primitives are then used to update rewards for continued exploration. This framework required novel shaping rewards, development of a primitive extraction algorithm, and modification of the Hybrid A* algorithm. The framework is tested on a navigation task using a nonlinear F-16 model. The framework autonomously added 91 motion primitives to the primitive library and reduced average path cost by 21.6 seconds, or 35.75% of the original cost. The learned primitives are applied to an obstacle field navigation task, which was not used in training, and reduced path cost by 16.3 seconds, or 24.1%. Additionally, two heuristics for the modified Hybrid A* algorithm are designed to improve effective branching factor.

42 ENGINEERING↗

Design of solvent-assisted plastics recycling: Integrated economics and environmental impacts analysis

In 2018, the United States generated over 35. 7 million tons of plastic waste, with only 8.4% being recycled and the other 91.6% incinerated or disposed of in a landfill. The continued growth of the polymer market has raised concerns over the end of life of plastics. Currently, the waste management system is faced with issues of inefficient sorting methods and low-efficiency recycling methods when it comes to plastics recycling. Mechanical recycling is the commonest recycling method but presents a lower-valued recycled material due to the material incompatibilities introduced via the inefficient sorting methods. Chemical recycling offers a promising alternative as it potentially allows for plastics to maintain their original properties. To that end, there is the need to investigate feasible chemical recycling methods to help mitigate the challenging problem posed by plastics at the end-of-life stage. This work proposes a conceptual solvent-assisted plastics recycling framework based on a superstructure optimization approach. This framework is evaluated using a representative case study to recover Polyethylene Terephthalate (PET). In this case study, it is found that polymer recovery is both economically and environmentally favorable when compared to traditional methods of disposal such as incineration.

Lehr, Austin L.↗

Farm Practice Typologies as a Strategy for Management-Relevant Land Use and Land Cover Mapping in the Great Lakes Region (Version 1) [Dataset]

Dataset overview and development This dataset provides spatially explicit agricultural land-use and land-management typologies developed for the Great Lakes Region (GLR) at the farm-parcel level. The typologies were designed to characterize not only the land-use and land-cover (LULC) associated with individual agricultural farm parcels, but also the land-management practices (LMPs), including irrigation, tile drainage, and conservation easements, occurring within those parcels and how these characteristics change through time. The dataset contains four related typology products: Annual integrated typology – describes the combined LULC and land-management characteristics for each farm parcel for individual years. LULC transition typology – describes the temporal pattern of LULC change for each farm parcel across the study period (2008-2023). LMP trend typology – describes the temporal pattern in the occurrence of LMPs for each farm parcel across the study period. Multi-year integrated typology – combines the LULC transition typology and LMP trend typology to provide an integrated characterization of long-term land-use and management patterns. Purpose of the dataset The purpose of these products is to provide a management-relevant integrated and consistent framework for evaluating the spatial and temporal organization of agricultural landscapes across the GLR. The resulting typologies can: support landscape-scale environmental and land-use analysis; provide spatial information relevant to land-management strategies, conservation planning, policy development, and program evaluation; characterize spatial patterns of agricultural land use and management; examine changes in agricultural landscapes through time; and identify persistent, transitional, and changing agricultural systems. Please refer to the README file provided in Files for more details.

Agriculture↗

OpenCGRA: An Open-Source Unified Framework for Modeling,Testing, and Evaluating CGRAs

Coarse-grained reconfigurable arrays (CGRAs),loosely defined as arrays of functional units (e.g, adder, sub-tractor, multiplier, divider, or larger multi-operation units, butsmaller than a general-purpose core) interconnected through aNetwork-on-Chip, provide higher flexibility than domain-specificASIC accelerators while offering increased hardware efficiencywith respect to fine-grained reconfigurable devices, such as FieldProgrammable Gate Arrays (FPGAs). The fast evolving fieldsof machine learning and edge computing, which are seeing acontinuous flow of novel algorithms and larger models, makeCGRAs ideal target architectures to allow domain specializationwithout loosing too much generality. They also generally offerquicker and more effective reconfigurability than FPGAs, po-tentially allowing adaptation during actual algorithm execution,and implement a dataflow programming paradigm that adaptswell to these emerging workloads. Designing and generating aCGRA, however, still requires to define the type and number ofthe specific functional units, implement their interconnect andthe network topology, and perform its simulation and validation,given a variety of workloads of interest.In this paper, we propose OpenCGRA, a Python-based unifiedframework that integrates generation, modeling, testing and eval-uation for CGRAs. OpenCGRA is the first open-source integratedframework able to support the full top-to-bottom design flow forspecializing and implementing CGRAs: modeling at different ab-straction levels (functional level, cycle level, register-transfer level),generation, simulation, testing at different granularities (unit test-ing, integration testing, property-based testing), and characteriza-tion (area, power, and timing). OpenCGRAs will be made availableon GitHub.

CGRA, synthesis↗

Developing Metrics to Assess Justice and Equity Implications of Early-Stage Research

Physics is a central component of early-stage research and development of virtually all technologies. But as much as scientific breakthroughs often hold the key to high efficiencies, long lifetimes, and high stability in devices for countless applications, early-stage choices research choices, such as those around materials or processing, can also serve to lock in long-term social and equity impacts of deployed technologies. This talk will explore how scientists can start to consider these impacts, with a focus on the energy sector and achieving a just and sustainable energy transition. We discuss development of the Justice Underpinning Science and Technology Research (JUST-R) metrics framework - a suite of metrics targeted at early-stage researchers to assess justice considerations in their work. The framework is evaluated for its appeal to researchers and effectiveness at promoting integration of justice considerations into research through case studies, which reveal its ability to broaden researcher perspectives and key avenues for future improvement.

energy equity↗

A CHIL Validation of Machine Learning-Assisted Methods for Real-Time Controls of Solar PV for Grid Services: Preprint

Recent research has highlighted the potential for solar to act as a zero-marginal-cost and zero-emission flexibility resource on the bulk power system when operated with advanced control systems. To increase the performance of these systems, leading technologies, including machine learning (ML) and hierarchical inverter set point allocation, have been proposed; however, these technologies lack comprehensive validation under real-world application scenarios. This paper addresses this gap by designing and developing a controller-hardware-in-the-loop framework to evaluate the performance of different flexible solar technologies in responding to automatic generation control signals in a closed-loop fashion. Simulation results indicate the superior performance of an ML-based approach compared to the conventional reference-control grouping-based approach, showcasing its potential to support grid stability and operational efficiency.

closed-loop validation↗

NE-COST plug-in: Expanding ACCERT's Capabilities for Life-Cycle Cost Modeling

The Algorithm for the Capital Cost Estimation of Reactor Technologies (ACCERT) is a structured methodology and software tool designed to simplify and standardize cost estimation for nuclear reactor technologies [1]. By utilizing a relational database structure and modular cost estimation algorithms, ACCERT delivers a robust, flexible, and scalable framework for evaluating costs across various reactor types and configurations [2]. The recent integration of the NE-COST plugin further expands ACCERT’s scope by introducing detailed life-cycle cost modeling and probabilistic analysis of uncertainties. This addition enables users to evaluate costs across front-end processes such as uranium enrichment and fabrication, as well as back-end activities including waste disposal and geologic storage. Through Monte Carlo statistical cost simulations, the plugin provides probabilistic insights into cost ranges, offering critical decision-making support for stakeholders including reactor developers, policymakers, and researchers.

Zhou, Jia↗

Oversizing and Part-Load Problems

Oversizing, the common engineering practice of specifying devices with capacity exceeding the actual load requirement, is a widespread practice across virtually all building technologies end-use categories, including HVAC, electrical systems, lighting, appliances, and plug loads. This practice, driven by factors like design uncertainty, institutional pressures, and risk aversion, results in wasted capital investment, control difficulties, and excessive energy consumption due to inefficient part-load operation. Part-load operation, where devices run below maximum capacity, is the dominant operating mode in most energy systems and presents a complex design challenge. Solutions to match output to load fall into three broad categories: constraining the output, adjusting the device’s internal behavior, and linking output to energy storage or other waste-heat reuse applications. The energy implications of part-load are critical, as efficiency often drops sharply as load decreases across a wide diversity of devices. To quantify the extent of this problem, we derive a dimensionless Part-Load Metric (PLM) based on device efficiency and its frequency distribution of operating hours at various output levels. The PLM quantifies the deviation of a device's actual efficiency from its maximum design efficiency. This metric also serves as a measure of "capital inefficiency," enabling engineers to compare the impact of different part-load solutions and providing a unified framework for evaluating performance across various devices and systems.

Meier, Alan↗

Atmospheric Modeling and Denoising for Millimeter-Wave Line Intensity Mapping

Line-intensity mapping (LIM) offers a promising approach to mapping large-scale cosmic structure, and the greatest obstacle for ground-based observations at millimeter wavelengths is foreground contamination from atmospheric emission. In this work, we present a simulation and denoising framework designed to isolate and subtract atmospheric fluctuations from LIM data, modeled after the instrument parameters of the South Pole Telescope Summertime Line Intensity Mapper (SPT-SLIM). We generate mock observations spanning 125-175 GHz containing cosmic signals, precipitable water vapor screens, ice crystal fluctuations, and photon noise. We then implement a spatial-spectral atmospheric removal pipeline combining per-pixel linear template regression with a two-dimensional Fourier-domain filter. The framework is evaluated under simulated conditions in the South Pole and the Atacama Desert across three key metrics: cosmic signal preservation, foreground subtraction efficiency, and instrument noise injection. Our pipeline achieves atmospheric suppression at large spatial scales, and these results establish a physically grounded foundation for atmosphere removal in ground-based LIM data collection.

Saye, L. K. [UC, Berkeley (main)] (ORCID:000900078↗

TRIM: AI Guided Random Number Generation for Resource-Constrained IoT Systems

Random numbers often serve as the backbone for many security solutions in diverse domains such as cryptography, side channel leakage prevention, and moving target defense. However, generating true random numbers requires a physical source of entropy (e.g. hardware, quantum, environmental phenomenon) making it difficult to realize at a large scale and at a low cost. On the flip side, pseudorandom number generators (easy to implement) following a specific distribution (e.g. Gaussian) can be easily compromised given a sufficient amount of traces. In this work, we have developed a machine learning-guided generative approach that can be used to create portable, resource-efficient, and cost-effective random number generators with high throughput and true randomness characteristics. We implement the proposed approach as a highly parameterized framework and perform extensive evaluation for different settings. The framework was able to learn from true random sources such as irrational numbers and environmental audio noise and imitate those sources towards generating new good quality random numbers on demand. We have generated more than 1 billion bits and observed robust performance in terms of true randomness metrics obtained from NIST SP 800-22 and FIPS 140-1 randomness test suites achieving a throughput of up to 142.85 Mbps. Compared to the state-of-the-art (SOTA) technique, the iso-cost setup of our framework can achieve more than 500 Mbps in a distributed setting. We have evaluated the efficacy of running the true randomness imitation AI models on target edge devices such as Raspberry Pi 4 (Model B), Nvidia Jetson Nano, Nvidia Jetson Orin Nano and Nvidia Jetson Xavier. We have also looked at the security of the TRIM framework itself against different adversarial threat models.

Cybersecurity↗

Updating the BNB Flux Prediction at SBND

Precise, accurate neutrino flux predictions for neutrino beam experiments are crucial for physics results. Flux uncertainties contribute significantly to the total systematic uncertainties seen in modern accelerator neutrino measurements such as cross sections, oscillations, and BSM studies.. For over a decade, experiments utilizing Fermilab’s Booster Neutrino Beam (BNB) have relied on the 2009 MiniBooNE flux prediction. However, the high-statistics era of the Short-Baseline Neutrino (SBN) program, with both SBND and ICARUS now operating, demands a modernized flux model and framework. While SBND is using the MiniBooNE flux model for its Generation 1 analyses, including many upcoming cross section measurements, future work will be based on a new flux model. In this talk, the ongoing work towards this new model will be outlined, including a new simulation framework (G4BNB), a new evaluation of model parameters from hadron scattering data, and a new framework for evaluating systematic uncertainties (BNBFP). Additionally, expansions to the flux model to include BSM contributions from neutral mesons will be discussed, as well as the PRISM capabilities of SBND to observe a wide range of off axis angles of the neutrino beam.

Paton, Josephine [Fermilab]↗

Updating the BNB Flux Prediction at SBND

Precise, accurate neutrino flux predictions for neutrino beam experiments are crucial for physics results. Flux uncertainties contribute significantly to the total systematic uncertainties seen in modern accelerator neutrino measurements such as cross sections, oscillations, and BSM studies.. For over a decade, experiments utilizing Fermilab’s Booster Neutrino Beam (BNB) have relied on the 2009 MiniBooNE flux prediction. However, the high-statistics era of the Short-Baseline Neutrino (SBN) program, with both SBND and ICARUS now operating, demands a modernized flux model and framework. While SBND is using the MiniBooNE flux model for its Generation 1 analyses, including many upcoming cross section measurements, future work will be based on a new flux model. In this talk, the ongoing work towards this new model will be outlined, including a new simulation framework (G4BNB), a new evaluation of model parameters from hadron scattering data, and a new framework for evaluating systematic uncertainties (BNBFP). Additionally, expansions to the flux model to include BSM contributions from neutral mesons will be discussed, as well as the PRISM capabilities of SBND to observe a wide range of off axis angles of the neutrino beam.

Paton, Josephine [Fermilab]↗

AutoBEM: A scalable framework for nationwide building energy simulation and retrofit evaluation in the United States

This paper presents AutoBEM, an integrated, automated framework for nationwide building energy modeling and retrofit evaluation in the United States. Unlike prior UBEM platforms that either rely primarily on representative stock sampling or operate at city scale, AutoBEM automates the generation of building-resolved, physics-based EnergyPlus/OpenStudio simulation models at national scale using GIS-derived geometry, prototype-based assumptions, and standardized scalable workflows. Leveraging the Model America dataset and high-performance computing, AutoBEM generates and simulates energy models for 122.9 million buildings, representing 97.8% of the U.S. building stock. These models are being made publicly and freely available as the Model America v1.0 (MAv1) dataset. AutoBEM supports detailed, building-level assessments of energy consumption, CO2 emissions, and post-processed anthropogenic heat emissions (AHE), and evaluates 151 energy conservation measures (ECMs) using localized utility pricing and building characteristics. In addition, AutoBEM incorporates both typical and future climate conditions through integration with Typical Meteorological Year (TMY) and Future TMY (fTMY) weather data derived from IPCC scenarios. In a case study of Phoenix, Arizona, AutoBEM identified several high-efficiency HVAC upgrades and selected envelope measures with short modeled payback periods (1.5 years) for certain building types and standards. Simulations under future climate scenarios (SSP5–RCP8.5) project an 11.3% increase in electricity use and a 32% reduction in natural gas demand by 2100, underscoring the need for climate-adaptive retrofit planning. By enabling reproducible, bottom-up, and location-specific analysis at scale, AutoBEM provides a step toward a national digital twin of the built environment and supports data-driven screening and planning for decarbonization, resilience, and energy equity.

Li, Hang [ORNL] (ORCID:0000000306001920)↗

Evaluation and Demonstration of Blockchain Applicability Framework

Blockchain technology has been gaining great interest from a variety of industry sectors, including financial, food processing, and power and energy markets. Realizing the strength of blockchain technology beyond the successful application in the cryptocurrency arena, researchers have been evaluating and using blockchain for applications such as supply chain management, transactive industry (both financial and energy), system integrity, device cybersecurity, identity management, and much more. One of the unique elements of the blockchain technology that made it such a captivating technology to researchers is its plethora of features. Some of the features include smart contracts, cryptocurrency and tokenizing, immutable distributed ledger, cryptographic hashing, and digital signature. In addition, there are multiple types of blockchains, such as permissioned/private and permissionless/public, and various consensus models, such as proof-of-work, proof-of-authority, proof-of-burn, and proof-of-stake. Therefore, it is often non-trivial to determine if an application requires a blockchain. If so, what kind of blockchain and consensus is most appropriate? This paper discusses the blockchain applicability framework (BAF), which was specifically designed with the purpose to answer those questions. BAF is divided into five domains, 18 subdomains, and about 100 controls. It is designed to ingest detailed user requirements to perform a weighted evaluation that is built on mathematical constructs to determine the ideal combination of blockchain that is appropriate for an application. Along with the core logical formulation of BAF, this paper depicts the efficacy of BAF through two use cases

Gourisetti, Sri Nikhil G.↗

Evaluation of the Self Retrieval Augmented Generation Technique on Common Security Advisory Framework Data

This small experimental report evaluates a variation of Retrieval Augmented Generation (RAG), called Self-RAG. This method uses a generative language model that incorporates retrieved facts into its generation and is explicitly trained to be able to determine whether retrieved information is enough to answer the input query, with a user-defined threshold for confidence. We performed an experiment using data from the publicly available CISA Common Security Advisory Framework (CSAF) repository (https://github.com/cisagov/CSAF) as the database of facts to be used in retrieval. Qualitative results from the experiment demonstrate that the Self-RAG method has some ability to provide reasonable answers to queries that are in the dataset and will often ignore irrelevant information when asked outside of domain questions (e.g., general facts). In settings with deliberately confusing questions (the question is within domain, but asks about a fabricated advisory), it was able to refuse 40% of the time without further adjustments to the original framework. While this performance is not sufficient for current practical use, further improvements to data formatting, disambiguating results, and leveraging threshold values could improve performance significantly. However, evaluating this will require more extensive evaluations on larger datasets and potentially better models.

97 MATHEMATICS AND COMPUTING↗