Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 793 records · Page 44
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

FlexDRL v1.0

FlexDRL is a framework that can be used as a benchmarking tool to evaluate different Deep Reinforcement Learning algorithms in a building energy management setting. OpenAI Gym was used as wrapper to the co-simulation environment (EnergyPlus for envelope and HVAC and Modelica for battery and PV) to interface with DRL algorithms, which are developed in PyTorch. In order to facilitate the usage of FlexDRL, all the required dependencies have been packaged into a Docker container.

Touzani, Samir↗
DOE OSTI2025

Advanced Materials and Manufacturing Technologies (AMMT) 2025 Roadmap

The mission of the Advanced Materials and Manufacturing Technologies (AMMT) program is to accelerate the development, qualification, demonstration, and deployment of advanced materials and manufacturing technologies in support of the United States (US) leadership in a broad range of nuclear energy applications. The vision of the AMMT program is the expansion of reliable and economical nuclear energy enabled by advanced materials and manufacturing technologies. Four major goals were set to realize the mission and vision of the AMMT program, including: (1) develop advanced materials and manufacturing technologies that have cross-reactor applications, (2) establish and demonstrate a rapid qualification framework that supports diverse materials and manufacturing technology needs, (3) evaluate materials performance in a range of nuclear environments, and (4) accelerate commercialization of new technologies through technology maturation. The program is designed to deliver solutions that support a wide range of reactor technologies.

36 MATERIALS SCIENCE↗
DOE OSTI2024

StructuredFuzzer: Fuzzing Structured Text-Based Control Logic Applications

Rigorous testing methods are essential for ensuring the security and reliability of industrial controller software. Fuzzing, a technique that automatically discovers software bugs, has also proven effective in finding software vulnerabilities. Unsurprisingly, fuzzing has been applied to a wide range of platforms, including programmable logic controllers (PLCs). However, current approaches, such as coverage-guided evolutionary fuzzing implemented in the popular fuzzer American Fuzzy Lop Plus Plus (AFL++), are often inadequate for finding logical errors and bugs in PLC control logic applications. They primarily target generic programming languages like C/C++, Java, and Python, and do not consider the unique characteristics and behaviors of PLCs, which are often programmed using specialized programming languages like Structured Text (ST). Furthermore, these fuzzers are ill suited to deal with complex input structures encapsulated in ST, as they are not specifically designed to generate appropriate input sequences. This renders the application of traditional fuzzing techniques less efficient on these platforms. To address this issue, this paper presents a fuzzing framework designed explicitly for PLC software to discover logic bugs in applications written in ST specified by the IEC 61131-3 standard. The proposed framework incorporates a custom-tailored PLC runtime and a fuzzer designed for the purpose. We demonstrate its effectiveness by fuzzing a collection of ST programs that were crafted for evaluation purposes. We compare the performance against a popular fuzzer, namely, AFL++. The proposed fuzzing framework demonstrated its capabilities in our experiments, successfully detecting logic bugs in the tested PLC control logic applications written in ST. On average, it was at least 83 times faster than AFL++, and in certain cases, for example, it was more than 23,000 times faster.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

High-Resolution ESM Projections for Energy Applications Over the CONUS

Assessing energy resources under future scenarios requires high-resolution meteorological information that is physically consistent and suitable for regional-scale analysis. While Earth system model (ESM) projections provide valuable large-scale information, their coarse resolution and systematic biases limit direct applicability for energy system modeling and planning. In this study, we develop a high-resolution dynamical downscaling framework based on the Weather Research and Forecasting (WRF) model to translate global-scale ESM data into energy-relevant regional projections over the contiguous United States (CONUS). The framework identifies an optimized WRF configuration through numerical experiments and evaluates raw and bias-corrected ESM initial and boundary conditions, with soil moisture (SM) and soil temperature (ST) bias correction implemented as an integral part of the bias-corrected ESM forcing to improve land-atmosphere coupling prior to WRF dynamical downscaling. Using an optimized WRF configuration at 4-km resolution, we show that raw ESM forcing introduces systematic dry and cold soil biases that propagate into pronounced warm biases in near-surface air temperature and positive biases in solar irradiance, particularly during summer. Applying bias-corrected atmospheric forcing together with bias-corrected SM and ST substantially reduces these downstream biases and improves the surface energy balance and near-surface atmospheric fields. These results demonstrate that bias-aware treatment of initial conditions is critical for producing high-resolution downscaled projections suitable for energy system modeling and planning applications.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Computational Analysis of Different Sparging Systems and their Influence in the Fluid-Dynamic Behavior of Bubble Column Reactors

Bubble column bioreactors are being actively considered for gas fermentation applications, specifically for CO2 utilization, and sugars to fuels conversion. Their main advantages include good mass transfer without any moving parts and low-cost of operation and maintenance. However, the design and scale-up of such reactors is challenging specifically for carbon capture applications where a mixture of gases (e.g. CO2/CO/H2) with variable solubilities is used. The overall performance of scaled-up bioreactors (e.g., mass transfer rate) is largely affected by gas holdup, bubble size distribution (BSD), and multiphase hydrodynamics. We investigate the effect of gas sparger designs on the performance of these large-scale bioreactors using computational fluid dynamics simulations in this work, so as to improve CO2 conversion at scale. The gas distribution systems in bubble column reactors not only determines operational regime, but also affects the evolution of the BSD, which in turn influences interfacial mass transfer and ultimately the efficiency of the gas-liquid exchange process. In addition to the BSD, uniformity in gas sparging affects gas holdup and bubble residence time which constitute important metrics of performance in gas-liquid systems. In this work, we use computational models to simulate high fidelity representations of different sparger designs and their effect on the operation of a bubble column reactor. Four different types of spargers have been selected for the computational study (Fig. 1): ladder, multi-ring, single-ring and toroidal. Their effect on superficial velocity, gas holdup mixing efficiency, and BSD will be evaluated in this work. The model uses a multiphase Eulerian framework similar to [1] and include a composition of mixtures of H2/CO/CO2 gases, common in fermentation applications.

BIOMASS FUELS,MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Gains in operational flexibility, safety margins, and cost efficiencies via integrated Plant Reload Optimization platform

The U.S. Department of Energy Light Water Reactor Sustainability Program Risk-Informed Systems Analysis Pathway Plant Reload Optimization Project aims to develop an integrated, comprehensive framework offering an all-in-one solution for reload evaluations with a special focus on optimizing core design. Optimizing the fuel loading pattern is one of the most important considerations in reducing the amount of new fuel used in the core. Due to thousands of possible core configuration options, finding optimal solutions is an unachievable task for a human. The Plant ReLoad Optimization platform, which supports artificial-intelligence-based reactor core designing, is now fully capable of handling realistic problems. The Plant ReLoad Optimization platform development project aims to build a reactor core design tool that includes reactor safety and fuel performance analyses and uses artificial intelligence to support the optimization of core design solutions. The NSGA-II (Non-dominated Sorting Genetic Algorithm II) optimizer was developed and tested within RAVEN (Risk Analysis and Virtual ENvironment) to handle many constraints by using an augmented objectives methodology. The demonstration was performed with constrained multiobjective optimization of a 17 × 17 pressurized-water reactor core loading patterns to minimize fuel cost and maximize fuel cycle length.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2024

Pressurized-Water Reactor Core Design using Multi-Objective Plant Fuel Reload Optimization Platform

The United States (U.S.) Department of Energy (DOE) Light Water Reactor Sustainability (LWRS) Program Risk-Informed Systems Analysis (RISA) Pathway Plant Reload Optimization Project aims to develop an integrated, comprehensive framework offering an all-in-one solution for reload evaluations with a special focus on optimization of core design. The optimization of the fuel loading pattern is one of the most important considerations in reducing the amount of new fuel used in the core. Due to thousands of possible options of core configuration, finding optimal solutions is an unachievable task for a human. The Plant ReLoad Optimization (PRLO) platform which supports artificial-intelligence-based reactor core designing is now fully capable of handling realistic problems. The PRLO Platform development project aims to build a reactor core design tool that includes reactor safety and fuel performance analyses and uses artificial intelligence to support the optimization of core design solutions. The NSGA-II (Non-dominated Sorting Genetic Algorithm-II) optimizer was developed and tested within RAVEN (Risk Analysis and Virtual Environment) to handle many constraints by using an augmented objectives methodology. The demonstration was performed with constrained multi-objective optimization of a 17 × 17 pressurized-water reactor core loading patterns to minimize fuel cost and maximize fuel cycle length.

42 ENGINEERING↗
DOE OSTI2024

Developing IEEE Std 2800-Compliant Algorithms for Transmission-Connected Inverter-Based Resources

This study addresses the compliance of Inverter-based Resources (IBRs) with IEEE Standard 2800, a leading standard that defines interconnection and interoperability requirements for IBRs integrated into transmission systems. Focusing on abnormal grid scenarios, the research evaluates the specific demands on IBRs, proposing a controller development framework for abnormal grid conditions. This framework caters to maintaining ride-through operation in line with IEEE Std 2800, alongside managing currents during voltage ride-through scenarios. The effectiveness of this proposed controller framework is rigorously validated through case studies, employing a MATLAB/Simulink model of an IBR to test its performance under diverse grid fault conditions, ensuring the IBRs' alignment with standard requirements and their robust performance in enhancing grid reliability.

grid↗
DOE OSTI2020

BioSTEAMDevelopmentGroup/thermosteam

BioSTEAM is a fast and flexible package for the design, simulation, and techno-economic analysis of biorefineries under uncertainty. BioSTEAM’s framework is built to streamline and automate early-stage technology evaluations and to enable rigorous sensitivity and uncertainty analyses. Complete biorefinery configurations are available at the Bioindustrial-Park GitHub repository, BioSTEAM’s premier repository for biorefinery models and results. The long-term growth and maintenance of BioSTEAM is supported through both community-led development and the research institutions invested in BioSTEAM. Through the open-source and community-lead platform, BioSTEAM aims to foster communication and transparency within the biorefinery research community for an integrated effort to expedite the evaluation of candidate biofuels and bioproducts. Additionally, an agile life cycle assessment (LCA) platform has been designed to interface with BioSTEAM, BioSTEAM-LCA. This open-source, installable package allows users to perform streamlined LCAs of biorefineries. The focus of BioSTEAM-LCA is to streamline and automate early-stage environmental impact analyses of processes and technologies, and to enable rigorous sensitivity and uncertainty analyses linking process design, performance, economics, and environmental impacts. ThermoSTEAM is a standalone thermodynamic engine capable of estimating mixture properties, solving thermodynamic phase equilibria, and modeling stoichiometric reactions. ThermoSTEAM builds upon chemicals, the chemical properties component of the Chemical Engineering Design Library, with a robust and flexible framework that facilitates the creation of property packages. The Biorefinery Simulation and Techno-Economic Analysis Modules (BioSTEAM) is dependent on ThermoSTEAM for the simulation of unit operations.

Cortes-Peña, Yoel↗
DOE OSTI2024

Risk-based System Upgrade Planning for AOT-IC [Capstone Project]

The Accelerator Operations and Technology Instrumentation and Controls Group (AOT IC) at the Los Alamos Neutron Science Center (LANSCE) has established a comprehensive risk-based system upgrade planning strategy which has been utilized as a tool to prioritize system upgrade projects for the past several years. A challenge with the current system is that the group often lacks the data to quantify the probability and impact of system failures, so the group often relies instead on qualitative risk assessments as well as evaluations of a potential upgrade project’s alignment with group strategy and vision to prioritize projects. A proposed framework to enable quantitative risk probability and impact assessments has been developed and tested on three pilot systems chosen to broadly represent the types and conditions of equipment used by the group. The proposed framework incorporates availability data gathered from the LANSCE control room logbooks and the LANSCE work control system, as well as a system health evaluation which is conducted from a variety of resources to assess the probability of equipment failure. The impact of system failure is viewed from the perspective of impact on mission and schedule, where spares status, system documentation status, and the functional distribution of deployed systems are used to quantify these impacts.

43 PARTICLE ACCELERATORS↗
DOE OSTI2021

Evaluating Spatial Accelerator Architectures with Tiled Matrix-Matrix Multiplication

There is a growing interest in custom spatial accelerators for machine learning applications. These accelerators employ a spatial array of processing elements (PEs) interacting via custom buffer hierarchies and networks-on-chip. The efficiency of these accelerators comes from employing optimized dataflow (i.e., spatial/temporal partitioning of data across the PEs and fine-grained scheduling) strategies to optimize data reuse. The focus of this work is to evaluate these accelerator architectures using a tiled general matrix-matrix multiplication (GEMM) kernel. To do so, we develop a framework that finds optimized mappings (dataflow and tile sizes) for a tiled GEMM for a given spatial accelerator and workload combination, leveraging an analytical cost model for runtime and energy. Our evaluations over five spatial accelerators demonstrate that the tiled GEMM mappings systematically generated by our framework achieve high performance on various GEMM workloads and accelerators.

43 PARTICLE ACCELERATORS↗
DOE OSTI2026

A Benchmark Suite for Evaluating Scientific AI Workloads on GPUs

AI applications have been steadily increasing in the allocation portfolio among leadership computing facilities. These applications depend on deep learning frameworks with hardware acceleration and underlying software systems. With the rapid development of applications, software stacks, and hardware devices, it is essential to evaluate the performance of core operations in AI workloads for direction of optimizations and procurement of next-generation high-performance computing (HPC) infrastructures. Currently, most benchmarks lack scientific AI workloads. So, we present DeepKernelBench and the experimental results of evaluating the benchmark suite for early observations and performance comparisons on datacenter GPUs using representative workloads for scientific AI, including Attentions, General matrix multiplications, Geometrics and Fourier neural operations.

Jin, Zheming [Advanced Micro Devices (AMD)]↗
DOE OSTI2026

Evaluating lightweight unsupervised online IDS for masquerade attacks in CAN

Vehicular controller area networks (CANs) are susceptible to masquerade attacks by malicious adversaries. In masquerade attacks, adversaries silence a targeted ID and then send malicious frames with forged content at the expected timing of benign frames. As masquerade attacks could seriously harm vehicle functionality and are the stealthiest attacks to detect in CAN, recent work has devoted attention to compare frameworks for detecting masquerade attacks in CAN. However, most existing works report offline evaluations using CAN logs already collected using simulations that do not comply with the domain’s real-time constraints. Here we contribute to advance the state of the art by presenting a comparative evaluation of four different non-deep learning (DL)-based unsupervised online intrusion detection systems (IDS) for masquerade attacks in CAN. Our approach differs from existing comparative evaluations in that we analyze the effect of controlling streaming data conditions in a sliding window setting. In doing so, we use realistic masquerade attacks being replayed from the ROAD dataset. We show that although evaluated IDS are not effective at detecting every attack type, the method that relies on detecting changes in the hierarchical structure of clusters of time series produces the best results at the expense of higher computational overhead. We discuss limitations, open challenges, and how the evaluated methods can be used for practical unsupervised online CAN IDS for masquerade attacks.

Anomaly detection↗
DOE OSTI2021

Evaluating Spatial Accelerator Architectures with Tiled Matrix-Matrix Multiplication.

There is a growing interest in custom spatial accelerators for machine learning applications. These accelerators employ a spatial array of processing elements (PEs) interacting via custom buffer hierarchies and networks-on-chip. The efficiency of these accelerators comes from employing optimized dataflow (i.e., spatial/temporal partitioning of data across the PEs and fine-grained scheduling) strategies to optimize data reuse. The focus of this work is to evaluate these accelerator architectures using a tiled general matrix-matrix multiplication (GEMM) kernel. To do so, we develop a framework that finds optimized mappings (dataflow and tile sizes) for a tiled GEMM for a given spatial accelerator and workload combination, leveraging an analytical cost model for runtime and energy. Finally, our evaluations over five spatial accelerators demonstrate that the tiled GEMM mappings systematically generated by our framework achieve high performance on various GEMM workloads and accelerators.

42 ENGINEERING↗
DOE OSTI2023

An Architectural Survey of the Area 25 Nuclear Rocket Development Station, Nevada National Security Site, Nye County, Nevada, Volume 3 Of 3

The purpose of the current architectural survey is to record the Nuclear Rocket Development Station (NRDS) district, which is recommended eligible for listing to the NRHP under all four criteria as the testing center for the Rover/NERVA (Nuclear Engine for Rocket Vehicle Application) program from the initial boundary survey in 1956 to the end of nuclear rocket development on January 5, 1973. Resources in the NRDS district were evaluated to determine if they are eligible for listing to the National Register as district contributors. The current survey identified the Reactor Control Point subdistrict to provide a temporal, spatial, and functional framework for organizing the resources involved in the NRDS operations. However, the subdistrict was not evaluated as a potential historic district against the Secretary of Interior’s Significance Criteria.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

SeqMask: Behavior Extraction Over Cyber Threat Intelligence Via Multi-Instance Learning

Abstract Identification and extraction of Tactics, Techniques and Procedures (TTPs) for Cyber Threat Intelligence (CTI) restore the full picture of cyber attacks and guide the analysts to assess the system risk. Existing frameworks can hardly provide uniform and complete processing mechanisms for TTPs information extraction without adequate knowledge background. A multi-instance learning approach named SeqMask is proposed in this paper as a solution. SeqMask extracts behavior keywords from CTI evaluated by the semantic impact, and predicts TTPs labels by conditional probabilities. Still, the framework has two mechanisms to determine the validity of keywords. One using expert experience verification. The other verifies the distortion of the classification effect by blocking existing keywords. In the experiments, SeqMask reached 86.07% and 73.99% in F1 scores for TTPs classifications. For the top 20% of keywords, the expert approval rating is 92.20%, where the average repetition of keywords whose scores between 100% and 90% is 60.02%. Particularly, when the top 65% of the keywords were blocked, the F1 decreased to about 50%; when removing the top 50%, the F1 was under 31%. Further, we also validate the possibility of extracting TTPs from full-size CTI and malware whose F1 are improved by 2.16% and 0.81%.

Ge, Wenhan↗
DOE OSTI2021

Considerations regarding the Use of Computer Vision Machine Learning in Safety-Related or Risk-Significant Applications in Nuclear Power Plants

With the advancements made to date in the field of artificial intelligence (AI), significant potential exists to utilize AI capabilities for nuclear power plant (NPP) applications. AI can replicate human decision making and it is usually faster and more accurate than humans. For implementations that impact critical NPP applications (e.g., safety-related or non-safety systems that potentially affect overall plant risk), a deeper safety analysis of the AI methods is necessary. AI applied to NPP operations could resemble the use of digital I&C (DI&C) because such applications involve digital computer hardware and custom-designed software that input plant data, execute complex software algorithms, and output the results to a system or licensed human operator to potentially provoke an action. For AI methods to be compliant with current safety requirements for DI&C, AI compatibility must be evaluated, and AI-related gaps may exist that prevent the prompt deployment of AI in NPPs. This effort aims to evaluate how example AI technologies align with the DI&C safety framework, and discusses how they could be analyzed, modeled, tested, and validated in a manner similar to typical DI&C technologies. Because AI is a broad field that encompasses areas such as machine learning (ML), natural language processing, and computer vision, this research focused on a subset of methods categorized as the computer vision ML (CVML) methods. This report explores two CVML use cases, gauge reading and fire watch, considered relevant to the DI&C standards, as they could play a safety-critical role. For the gauge reading use case, a CVML-enabled technology that can read gauges at oblique angles is utilized. For the fire watch use case, a CVML-enabled technology is utilized that migrates fire watch from a manual (human) approach to automated fire detection. These use cases are mainly intended to give context to the CVML system discussion. This effort assumes the worst-case scenario, with the CVML system being used to replace a safety-related or risk-significant system, thus requiring evaluation. Evaluating CVML against most of the relevant safety requirements for DI&C yielded several CVML-specific considerations due to the uniqueness of its characteristics in comparison with typical DI&C systems. For example, CVML models often employ commonly used (open-source) datasets, and it is not always possible to determine the level of overlap among open-source datasets. Therefore, the independence of the developed CVML models when demonstrating diversity is questionable, therefore creating vulnerability to common cause failure (CCF). The design verification process is also impacted since the data overlap could result in overestimation of the software validation and verification (V&V) performance results. Section 2 of this report evaluates a list of the identified CVML-specific characteristics and discusses the resulting considerations and potential solutions in the context of each referenced requirement. A summation is provided in Section 3. This report is not to be used as a guideline. It was developed to identify and consider issues in the implementation of ML technologies used to augment activities that may have a bearing on plant operation. The report draws parallels to the use of DI&C technologies, for which many standards are available to guide their use in nuclear plant operation. It considers the technologies and some of the potential implications of their use in safety-related applications but is not intended to address regulatory or licensing related issues.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2025

Disentangling atmospheric, hydrological, and coupling uncertainties in compound flood modeling within a coupled Earth system model

Compound riverine and coastal flooding is usually driven by complex interactions among meteorological, hydrological, and ocean extremes. However, existing efforts to model this phenomenon often do not integrate hydrological processes across atmosphere–land–river–ocean systems, leading to substantial uncertainties that have not been fully examined. To bridge this gap, we leverage the new capabilities of the Energy Exascale Earth System Model (E3SM) that enable a multi-component framework that integrates coastally refined atmospheric, terrestrial, and oceanic components. We evaluate compound uncertainties arising from two-way land–river–ocean coupling in E3SM and track the cascading meteorological and hydrological uncertainties through ensemble simulations over the Delaware River basin and estuary during Hurricane Irene (2011). Our findings highlight the importance of two-way river–ocean coupling to compound flood modeling and demonstrate E3SM's capability in capturing compound flood extent near the coast, with a hit rate over 0.75. Our study shows the growing uncertainties that transition from atmospheric forcings to flood distribution and severity. Furthermore, an analysis based on artificial neural networks is used to assess the roles of hydrological drivers, such as infiltration and soil moisture, in the generation of compound flooding. The response of compound floods to tropical cyclones (TCs) is found to be susceptible to these often overlooked drivers. For instance, the flooded area could increase more than 2-fold (∼2.4) if Hurricane Irene were preceded by an extreme antecedent soil moisture condition (AMC). The results not only support the use of a multi-component framework for interactive flooding processes, but also underscore the necessity of broader definitions of compound flooding that encompass the simultaneous occurrence of intense precipitation, storm surge, and high AMC during TCs.

Feng, Dongyu [Pacific Northwest National Laborator↗