Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 667 records · Page 37

Evaluating Basin-Scale Forest Adaptation Scenarios: Wildfire, Streamflow, Biomass, and Economic Recovery Synergies and Trade-Offs

Active forest management is applied in many parts of the western United States to reduce wildfire severity, mitigate vulnerability to drought and bark beetle mortality, and more recently, to increase snow retention and late-season streamflow. A rapidly warming climate accelerates the need for these restorative treatments, but the treatment priority among forest patches varies considerably. We simulated four treatment scenarios across the 3,450 km 2 Wenatchee River basin in eastern Washington, United States. We used a decision support tool (DST) to assess trade-offs and synergies within and among treatments on wildfire risk and smoke emissions, water yield and snow retention, biomass production, and economic return. Treatment scenarios emphasized prescribed burning ( BurnOnly ), biomass production ( MaxBiomass ), gap-based thinning to optimize water yield ( IdealWater ), and a principle-based restoration scenario ( RA1 ). Fire hazard, smoke emissions, and biomass production metrics were evaluated across scenarios using the Forest Vegetation Simulator, and water yields were modeled using the Distributed Hydrology Soil Vegetation Model. Simulations were summarized to both patch- (10 1 –10 2 ha) and subwatershed- (10 3 –10 4 ha) scales, and treatment effects were evaluated against an untreated baseline landscape. We used logic models to rank effect sizes by scenario across metrics along a continuum between −1 (no or weak effect) to +1 (large effect). All treatments produced benefits across one or more ecosystem services and led to synergistic benefits to water yield and wildfire hazard reduction. Tradeoffs among resource benefits were clear in wilderness where reliance on prescribed burning without mechanical treatment increased costs and eliminated the potential for biomass recovery. The BurnOnly scenario improved fire risk metrics and streamflow, but effect sizes were lower compared to other treatments. IdealWater showed the strongest benefits overall, demonstrating the ability to capture multiple resource benefits through spatially explicit thinning. Our study provides a framework for integrating strategic and tactical models that evaluate tradeoffs and synergies gained through varied management approaches. We demonstrate the utility of decision support modeling to enhance management synergies across large landscapes.

54 ENVIRONMENTAL SCIENCES↗

On the feasibility of using physics-informed machine learning for underground reservoir pressure management

In this work, we evaluate the feasibility of using physics-informed machine learning (PIML) for underground energy-related pressure management. To this end, we develop a PIML framework to manage underground reservoir pressures by training neural networks to determine fluid extraction rates for dedicated extraction wells during fluid injection operations given a range of reservoir conditions (e.g., transmissivity and storativity). We implement an automatically-differentiable analytical physics model of fluid flow in porous media within the PIML framework as a proxy for more complicated models. This allows us to execute a sufficient number of training scenarios to fully evaluate the feasibility of using PIML to support pressure management activities. We quantify the number of physics-model parameters required for automatic differentiation to become more efficient than finite-difference gradient calculations. We use a simple scenario with a single injector, extractor, and critical location for our feasibility analysis. We evaluate the effect of the size of the training dataset (i.e., the number of reservoir condition samples) on the accuracy and efficiency of the PIML framework. For an equivalent number of model evaluations, the larger training dataset took less time to train and produced a neural network that was able to more accurately manage reservoir pressures. We also evaluate the effect of the training dataset batch size (i.e., number of reservoir condition samples used to update the neural network coefficients during training; i.e., how the training dataset is partitioned). While training ran faster with larger batch sizes, they produced neural networks that managed pressures less accurately. We demonstrate the approach on a more complex scenario involving 10 injectors, 10 extractors, and 4 critical locations (a relatively high well density of 20 wells/km2). We provide the number of forward and adjoint model evaluations required in each case as an indication of the feasibility of using PIML for pressure management when more complicated physics models with longer execution times are used.

54 ENVIRONMENTAL SCIENCES↗

Unraveling the 2021 Central Tennessee flood event using a hierarchical multi-model inundation modeling framework

Flood prediction systems need hierarchical atmospheric, hydrologic, and hydraulic models to predict rainfall, runoff, streamflow, and floodplain inundation. The accuracy of such systems depends on the error propagation through the modeling chain, sensitivity to input data, and choice of models. In this study, we used multiple precipitation forcings (hindcast and forecast) to drive hydrologic and hydrodynamic models to analyze the impacts of various drivers on the estimates of flood inundation depth and extent. We implement this framework to unravel the August 2021 extreme flooding event that occurred in Central Tennessee, USA. We used two radar-based quantitative precipitation estimates (STAGE4 and MRMS) as well as quantitative precipitation forecasts (QPF) from the National Weather Service Weather Prediction Center (WPC) to drive a series of models in the hierarchical framework, including the Variable Infiltration Capacity (VIC) land surface model, the Routing Application for Parallel Computation of Discharge (RAPID) river routing model, and the AutoRoute and TRITON inundation models. An evaluation with observed high-water marks demonstrates that the framework can reasonably simulate flood inundation. Despite the complex error propagation mechanism of the modeling chain, we show that inundation estimates are most sensitive to rainfall estimates. Most notably, QPF significantly underestimates flood magnitudes and inundations leading to unanticipated severe flooding for all stakeholders involved in the event. Finally, we discuss the implications of the hydrodynamic modeling framework for real-time flood forecasting.

54 ENVIRONMENTAL SCIENCES↗

Avoiding pitfalls in molecular simulation of vapor sorption: Example of propane and isobutane in metal–organic frameworks for adsorption cooling applications

This study introduces recommendations for conducting molecular simulations of vapor adsorption, with an emphasis on enhancing the accuracy, reproducibility, and comparability of results. The first aspect we address is consistency in the implementation of some details of typical molecular models, including tail corrections and cutoff distances, due to their significant influence on generated data. We highlight the importance of explicitly calculating the saturation pressures at relevant temperatures using methods such as Gibbs ensemble Monte Carlo simulations and illustrate some pitfalls in extrapolating saturation pressures using this method. For grand canonical Monte Carlo (GCMC) simulations, the input fugacity is usually calculated using an equation of state, which often requires the critical parameters of the fluid. We show the importance of using critical parameters derived from the simulation with the same model to ensure internal consistency between the simulated explicit adsorbate phase and the implicit bulk phase in GCMC. We show the advantages of presenting isotherms on a relative pressure scale to facilitate easier comparison among models and with experiment. Extending these guidelines to a practical case study, we evaluate the performance of various isoreticular metal–organic frameworks (MOFs) in adsorption cooling applications. This includes examining the advantages of using propane and isobutane as working fluids and identifying MOFs with a superior performance.

36 MATERIALS SCIENCE↗

Machine learning guided selection of broad-spectrum epitope-specific functional antibodies for "Disease X"

Our project established and demonstrated a transfer learning framework that enables prediction of antibody–antigen interactions across related viruses. The approach focused on three major activities: 1. Conserved region and epitope identification – We compared viral protein structures and sequences to identify shared receptor-binding domains and neutralizing epitope regions across variants and related viruses. These conserved features formed the foundation for discovering broadly functional antibodies. 2. Machine learning model development – We built neural network–based models that integrate epitope features with antibody sequence information. Instead of relying solely on structural or physical properties, the models learned transferable patterns that describe antibody binding potential across different viral families. 3. Transfer learning and validation – Using SARS-CoV-2 and Ebola as source systems, we successfully transferred learned epitope features to predict antibody interactions for SARS CoV-1 and Marburg virus. Iterative cycles of dataset generation, retraining, and evaluation improved generalization and predictive power, ensuring the framework can adapt to new threats.

59 BASIC BIOLOGICAL SCIENCES↗

Arm and shoulder muscle segmentation in axial MRI with UNet deep learning model

Quantifying individual upper-limb muscle volumes from MRI provides key insight into muscle-specific strength, deficits, and adaptations. Manual delineation is the gold standard but time‑intensive, and the performance of current deep learning approaches, particularly for small or anatomically complex muscles, remains incompletely characterized. We evaluated a state‑of‑the‑art deep learning framework across the entire upper limb and analyzed factors governing segmentation performance, with attention to the forearm. Three previously published MRI datasets (1.5 T, 3D GRE T1‑weighted; total n = 39) spanning young, middle‑aged, and older adults were curated and quality‑checked, including expert manual segmentations for 31 muscles. Following multiclass mask reconstruction, we trained three 3D nnU‑Net multiclass models matched to the muscle subsets present across datasets, using five‑fold cross‑validation and a composite Dice Similarity Coefficient (DSC) + cross entropy loss. Segmentation accuracy was assessed with DSC. Performance varied across muscles (mean DSC = 0.806 ± 0.098), ranging from 0.920 (Deltoid) to 0.461 (Extensor pollicis brevis). In uncertainty‑weighted regressions, muscle volume was positively associated with DSC (R2 = 0.36, p < 0.001), whereas training segmentation count and muscle orientation showed negligible associations (R2 ≤ 0.06). A weighted mixed‑effects model identified volume as the strongest evaluated predictor, explaining 23.9% of variance in DSC; orientation and training count each contributed <1%, leaving 61.5% unexplained. These results indicate that deep learning–based segmentation can accurately quantify muscle volume for many upper‑limb muscles but remains constrained for small, low‑contrast forearm muscles.

Gillespie, Samuel↗

Sim-Situ: A Framework for the Faithful Simulation of in situ Processing

The amount of data generated by numerical simulations in various scientific domains led to a fundamental redesign of how the analysis and visualization of simulation outputs are performed. The throughput and capacity of storage subsystems have not evolved as fast as the computing power in extreme-scale supercomputers, making the classical post-hoc approach highly inefficient. In situ processing has then emerged as a solution in which simulation and data analysis/visualization are intertwined for better performance and greater interactivity.Determining the best allocation, i.e., how many resources to allocate to simulation and analysis respectively, mapping, i.e., where and at which frequency to run the analysis/visualization, and data transfer mode is a complex task whose performance assessment is crucial to the efficient execution of in situ processing. However, such a performance evaluation of different strategies usually relies either on directly running them on the targeted execution environments, which can rapidly become extremely time- and resource-consuming, or on resorting to simplified models of the components of an in situ application, which can lack of realism. In both cases, the validity of the performance evaluation is limited.In this paper, we present Sim-Situ, a simulation-based framework for the faithful performance evaluation of in situ processing strategies. We designed Sim-Situ to reflect the typical features of in situ processing systems. Thanks to its modular design, Sim-situ has the necessary flexibility to easily and faithfully evaluate the behavior and performance of various allocation, mapping, and data transfer strategies. We illustrate the simulation capabilities of Sim-Situ on a Molecular Dynamics use case. We study the impact of different strategies on performance and show how users can leverage Sim-Situ to determine interesting tradeoffs when adding analysis/visualization components to their application.

Honoré, Valentin↗

Proactive Frequency Stability Scheme: A Distributed Framework Based on Particle Filters and Synchrophasors

The reactive nature of traditional under-frequency load shedding schemes can lead to delayed response and unnecessary loss of load. This work presents a proactive framework for power system frequency stability. Bayesian filters and synchrophasors are leveraged to produce predictions after disturbances are detected. By being able to estimate the future state of frequency corrective actions can be taken before the system reaches a critical condition. This proactive approach makes it possible to optimize the response to a disturbance, which results in a decrease in the amount of compensation utilized. The framework is tested via Matlab simulations based on Kundur’s Two-Area System, and the IEEE 14-Bus System. Performance metrics are provided and evaluated against other contemporary solutions found in literature. During testing this framework outperformed other solutions by drastically reducing the amount of load dropped during compensation.

42 ENGINEERING↗

Graph-based Simulation Framework for Power Resilience Estimation and Enhancement

The increasing frequency of extreme weather events poses significant risks to power distribution systems, leading to widespread outages and severe economic and social consequences. This paper presents a novel simulation framework for assessing and enhancing the resilience of power distribution networks under such conditions. Resilience is estimated through Monte Carlo simulations, which simulate extreme weather scenarios and evaluate the impact on infrastructure fragility. Due to the proprietary nature of power network topology, a distribution network is synthesized using publicly available data. To generate the weather scenarios, an extreme weather generation method is developed. To enhance resilience, renewable resources such as solar panels and energy storage systems (batteries in this study) are incorporated. A customized Genetic Algorithm is proposed to determine the optimal locations and capacities for solar panels and battery installations, maximizing resilience while balancing cost constraints. Experiment results demonstrate that on a large-scale synthetic distribution network with more than 300,000 nodes and 300,000 edges, the proposed framework can efficiently evaluate the resilience, and enhance the resilience through the installations of distributed energy resources (DERs), providing utilities with valuable insights for community-level power system resilience estimation and enhancement.

Wang, Xuesong [Wayne State Univ., Detroit, MI (Uni↗

NuQKD: A Modular Quantum Key Distribution Simulation Framework for Engineering Applications

Abstract An experimental Quantum Key Distribution (QKD) implementation requires advanced costly hardware, unavailable in most research environments, making protocol testing and performance evaluation complicated. This has been a major motivation for the development of QKD simulation frameworks, to allow researchers to obtain insight before proceeding into practical implementations. Several simulators have been introduced over the recent years. However, only four are publicly available, only one of which models equipment imperfections. Currently, no open‐source simulator includes all following capabilities: channel attenuation modelling, equipment imperfections and effects on key rates, estimation of elapsed time during classical and quantum‐ channel processes, use of truly random binary sequences for qubits and measurement bases, shared ‐bit fraction customization. In this paper, we present NuQKD, an open‐ source modular, intuitive simulator, featuring all the above capabilities. NuQKD establishes communication between two computer terminals, accepts custom user inputs (iterations, raw key size, attacker interception rate etc.) and evaluates the sifted key, Quantum Bit Error Rate (QBER), elapsed communication time, and more. NuQKD capabilities include optical fiber and free ‐space simulation, modeling of equipment/channel imperfections, bitstrings from True Random Number Generators, decoy‐state protocol support, and automated evaluation of performance metrics. We expect NuQKD to enable convenient and accurate representation of actual experimental conditions.

Gkouliaras, Konstantinos↗

Radiation-resistant metal-organic framework enables efficient separation of krypton fission gas from spent nuclear fuel

Capture and storage of volatile radionuclides that result from processing of used nuclear fuel is a major challenge. Solid adsorbents, in particular ultra-microporous metal-organic frameworks, could be effective in capturing these volatile radionuclides, including 85 Kr. However, metal-organic frameworks are found to have higher affinity for xenon than for krypton, and have comparable affinity for Kr and N 2 . Also, the adsorbent needs to have high radiation stability. To address these challenges, here we evaluate a series of ultra-microporous metal-organic frameworks, SIFSIX-3-M (M = Zn, Cu, Ni, Co, or Fe) for their capability in 85 Kr separation and storage using a two-bed breakthrough method. These materials were found to have higher Kr/N 2 selectivity than current benchmark materials, which leads to a notable decrease in the nuclear waste volume. The materials were systematically studied for gamma and beta irradiation stability, and SIFSIX-3-Cu is found to be the most radiation resistant.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Initiating a Roadmap for Solar Fuels R&D: Imagining Beyond Thermochemical Cycles

Sandia National Laboratories in collaboration with the National Renewable Energy Laboratory outline a framework for developing a solar fuels roadmap based on novel concepts for hybridizing gas-splitting thermochemical cycle s with high-temperature electro chemical steps. We call this concept SoHyTEC, a Solar Hybrid Thermochemical-Electrochemical Cycle. The strategy focuses on transforming purely thermochemical cycles that split water (H 2 O) and carbon dioxide (CO 2 ) to produce hydrogen (H 2 ) and carbon monoxide (CO) , respectively, the fundamental chemical building blocks for diverse fuels and chemicals , by substituting thermochemical reactions with high-temperature electrochemical steps. By invoking high-temperature electrochemistry, the energy required to complete the gas-splitting cycle is divided into a thermal component (process temperature) and an electrical component (applied voltage). These components, sourced from solar energy, are independently variable knobs to maximize overall process efficiency. Furthermore, a small applied voltage can reduce cycle process temperature by hundreds of degrees , opening the door to cost-effective solar concentrators and practical receiver/reactor de signs. Using the SoHyTEC concept as a backdrop, we outline a framework that advocates developing methods for automating information gathering, critically evaluating thermochemical cycles for adapting into SoHyTEC, establishing requirements based on thermodynamic analysis, and developing a model-based approach to benchmarking a SoHyTEC system against a baseline concentrating solar thermal integrated electrolysis plant. We feel these framework elements are a necessary precursor to creating a robust and adaptive technology development roadmap for producing solar fuels using SoHyTEC. In one example, we introduce high-temperature electrochemistry as a method to manipulate a fully stoichiometric two-step metal oxide cycle that circumvents costly separation processes and ultra-high cycle temperatures. We also identify and group water-splitting chemistries that are conceptually amenable to hybridization.

14 SOLAR ENERGY↗

Community Solar Policy Screening Workbook for Multifamily Affordable Housing Building Portfolios

This workbook provides a framework to help Multifamily Affordable Housing (MFAH) housing providers evaluate state-level policy landscapes as they relate to MFAH community solar development. By exploring which programs and policies aid in the technical and economic feasibility of MFAH solar projects, building owners can gain an understanding of where in their portfolios to begin developing solar. This workbook also walks through the steps necessary for MFAH building owners to assess their portfolios using this framework. It was developed by the National Renewable Energy Laboratory for the National Community Solar Partnership (NCSP) Multifamily Affordable Housing Collaborative. Learn more about the NCSP and the Collaborative at https://www.energy.gov/communitysolar/community-solar.

14 SOLAR ENERGY↗

Library for Evolutionary Algorithms in Python (LEAP)

There are generally three types of scientific software users: users that solve problems using existing science software tools, researchers that explore new approaches by extending existing code, and educators that teach students scientific concepts. Python is a general-purpose programming language that is accessible to beginners, such as students, but also as a language that has a rich scientific programming ecosystem that facilitates writing research software. Additionally, as high-performance computing (HPC) resources become more readily available, software support for parallel processing becomes more relevant to scientific software.There currently are no Python-based evolutionary computation frameworks that support all three types of scientific software users. Moreover, some support synchronous concurrent fitness evaluation that do not efficiently use HPC resources. We pose here a new Python-based EC framework that uses an established generalized unified approach to EA concepts to provide an easy to use toolkit for users wishing to use an EA to solve a problem, for researchers to implement novel approaches, and for providing a low-bar to entry to EA concepts for students. Additionally, this toolkit provides a scalable asynchronous fitness evaluation implementation friendly to HPC that has been vetted on hardware ranging from laptops to the world’s fastest supercomputer, Summit.

Coletti, Mark↗

Evaluating Implementations of the Immersed Boundary Method in the Weather Research and Forecasting Model

The terrain-following coordinate system used by many atmospheric models can cause numerical instabilities due to discretization errors as resolved terrain slopes increase and the grid becomes highly skewed. The immersed boundary (IB) method, which does not require the grid to conform to the terrain, has been shown to alleviate these errors, and has been used successfully for high-resolution atmospheric simulations over steep terrain, including vertical building surfaces. Since many previous applications of IB methods to atmospheric models have used very fine grid resolution (5 m or less), the present study seeks to evaluate IB method performance over a range of grid resolutions and aspect ratios. Two classes of IB algorithms, velocity reconstruction and shear stress reconstruction, are tested within the common framework of the Weather Research and Forecasting (WRF) Model. Performance is evaluated in two test cases, one with flat terrain and the other with the topography of Askervein Hill, both under neutrally stratified conditions. WRF-IB results are compared to similarity theory, observations, and native WRF results. Despite sensitivity to the location at which the IB intersects the model grid, the velocity reconstruction IB method shows consistent performance when used with a hybrid RANS/LES surface scheme. The shear stress reconstruction IB method is not sensitive to the grid intersection, but is less consistent and near-surface velocity errors can occur at coarse resolutions. This study represents an initial investigation of IB method variability across grid resolutions in WRF. Future work will focus on improving IB method performance at intermediate to coarse resolutions.

54 ENVIRONMENTAL SCIENCES↗

CROWBAR: Natively Fuzzing Trusted Applications Using ARM CoreSight

Abstract Trusted execution environments (TEE) are deployed on many platforms to provide both confidentiality and integrity, and their extensive use offers a secure environment for privacy-sensitive operations. Despite TEE prevalence in the smartphone and tablet market, vulnerability research into TEE security is relatively rare. This is, in part, due to the strong isolation guarantees provided by its implementation. In this paper, we propose a hardware assisted fuzzing framework, CROWBAR, that bypasses TEE isolation to natively evaluate trusted applications (TAs) on mobile devices by leveraging ARM CoreSight components. CROWBAR performs feedback-driven fuzzing on commercial, closed source TAs while running in a TEE protected environment. We implement CROWBAR on 2 prototype commercial-off-the-shelf (COTS) smartphones and one development board, finding 3 unique crashes in 5 closed source TAs that are previously unreported in the TrustZone fuzzing literature.

Shan, Haoqi↗

Hybrid data-driven cement-stabilized soil design: An integration of machine learning, multi-objective optimization, and life cycle assessment

Soil stabilization is crucial in geotechnical engineering, yet conventional methods are often time-consuming, resource-intensive, and environmentally unsustainable. Despite growing interest in Machine Learning (ML) and optimization tools for mix design, few studies integrate these methods with decision-making techniques and environmental assessment to support practical implementation. This study proposes a hybrid data-driven framework for predicting strength, optimizing mix compositions, and evaluating environmental impacts via life cycle assessment of cement-stabilized soft soils. Six ML models were evaluated, and the top-performing eXtreme Gradient Boosting (XGB) model was further improved using the Grey Wolf Optimizer (GWO). The optimized XGB-GWO model, integrated with a polynomial cost function, served as the objective function in a multi-objective optimization problem solved via the Non-Dominated Sorting Genetic Algorithm II (NSGA-II), with final mix selection guided by the entropy-weighted TOPSIS method. Validation through a case study produced mix designs offering superior strength-cost trade-offs, with the optimal mix achieving 2243.2 kPa unconfined compressive strength and a 16.07 % reduction in carbon emissions compared to the highest-cost design. In conclusion, this study offers a sustainable, scalable approach to soil stabilization and supports informed decision-making in construction.

Life cycle assessment↗

A method and tool to determine the colorimetric and photobiological properties of light transmitted through glass and other optical materials

Here we describe a framework and provides a computational tool to characterize the color quality and biological potential of light that is transmitted through glass and other optical materials. The IES TM-30 framework and Excel computation tool were adapted to evaluate color quality, with measures from CIE S 026, UL 24480, and the WELL Building Standard v2 added to evaluate biological potential. The user selects a pre-transmittance spectral power distribution (SPD), such as for a CIE D-Series Illuminant, Planckian radiation, electric lamp, or measurement of daylight at a building site. The tool allows a user to populate a database with spectral data for glazing or other optical materials, comprising spectral transmission and the spectral reflectance of both sides. The user creates a unit with one, two, or three panes of glazing or other optical materials, and the tool calculates the composite spectral transmittance accounting for reflections between materials. The tool computes the transmitted SPD, then determines colorimetric and photobiological outputs using the transmitted SPD as the test source. Glass, window, and skylight manufacturers can employ the tool to optimize glazing spectral transmission to achieve intentional colorimetric and photobiological performance with transmitted light. Electric lighting manufacturers, designers, and researchers can use the tool to evaluate the impact of glazing units and other optical materials on the color quality and biological potential of transmitted light.

36 MATERIALS SCIENCE↗