Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “evaluation framework”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,207 records · Page 67

Risk-based System Upgrade Planning for AOT-IC [Capstone Project]

The Accelerator Operations and Technology Instrumentation and Controls Group (AOT IC) at the Los Alamos Neutron Science Center (LANSCE) has established a comprehensive risk-based system upgrade planning strategy which has been utilized as a tool to prioritize system upgrade projects for the past several years. A challenge with the current system is that the group often lacks the data to quantify the probability and impact of system failures, so the group often relies instead on qualitative risk assessments as well as evaluations of a potential upgrade project’s alignment with group strategy and vision to prioritize projects. A proposed framework to enable quantitative risk probability and impact assessments has been developed and tested on three pilot systems chosen to broadly represent the types and conditions of equipment used by the group. The proposed framework incorporates availability data gathered from the LANSCE control room logbooks and the LANSCE work control system, as well as a system health evaluation which is conducted from a variety of resources to assess the probability of equipment failure. The impact of system failure is viewed from the perspective of impact on mission and schedule, where spares status, system documentation status, and the functional distribution of deployed systems are used to quantify these impacts.

43 PARTICLE ACCELERATORS↗

Framework for Structural Online Health Monitoring of Aging and Degradation of Secondary Systems due to some Aspects of Erosion

This report describes the current state of research related to critical aspects of erosion and selected aspects of degradation of secondary components in nuclear power plants. The report also proposes a framework for online health monitoring of aging and degradation of secondary components. The framework consists of an integrated multi-sensor modality system which can be used to monitor different piping configurations under different degradation conditions. The report analyses the currently known degradation mechanisms and available predictive models. Based on this analysis, the structural health monitoring framework is proposed. The Light Water Reactor Sustainability Program began to evaluate technologies that could be used to perform online monitoring of piping and other secondary system structural components in commercial NPPs. These online monitoring systems have the potential to identify when a more detailed inspection is needed using real-time measurements, rather than at a pre-determined inspection interval. This transition to condition-based, risk informed automated maintenance will contribute to a significant reduction of operations and maintenance costs that account for the majority of nuclear power generation costs. There is unanimous agreement between industry experts and academic researchers that identifying and prioritizing inspection locations in secondary piping systems (for example, in raw water piping or diesel piping) would eliminate many excessive in-service inspections. The proposed structural health monitoring framework takes aim at answering this challenge by combining long-range guided wave technologies with other monitoring techniques, which can significantly increase the inspection length and pinpoint the locations that degraded the most. More widely, the report suggests research efforts aimed at developing, validating, and deploying online corrosion monitoring techniques for complex geometries, which are pervasive in NPPs.

47 OTHER INSTRUMENTATION↗

Development and Verification of a Numerical Library for Solving Global Terrestrial Multiphysics Problems

Abstract Current generation Land Surface Models (LSMs) routinely simulate many nonlinear multiphysics processes. The complexity of future generation LSMs is expected to increase as critical new biophysical and biogeochemical processes are incorporated. Current generation LSMs have several shortcomings including the lack of robust numerical methods to solve discretized equations, monolithic software design that hinders testing of a process representation in isolation from other components, and absence of a flexible coupling framework to solve tightly coupled multiphysics problems. While the LSMs community vigorously evaluates the accuracy of the conceptual model to represent reality (validation), the accuracy of the numerical implementation of the conceptual model (verification) is seldom examined. Method of Manufactured Solutions is a technique for verifying complex codes when analytical solutions are unavailable. In this work, we present a stand‐alone, open source numerical library for solving global terrestrial multiphysics processes that includes a flexible coupling framework. Robust numerical solution for linear and nonlinear equations is provided via the Portable, Extensible Toolkit for Scientific Computation library. We verify the numerical library using Method of Manufactured Solutions for a range of problems comprising single and multiphysics steady state problems that are applied in one or multiple physical domains. This work provides an example of incorporating code verification as an integral part of Land Surface Model development activities.

58 GEOSCIENCES↗

Evaluating Spatial Accelerator Architectures with Tiled Matrix-Matrix Multiplication

There is a growing interest in custom spatial accelerators for machine learning applications. These accelerators employ a spatial array of processing elements (PEs) interacting via custom buffer hierarchies and networks-on-chip. The efficiency of these accelerators comes from employing optimized dataflow (i.e., spatial/temporal partitioning of data across the PEs and fine-grained scheduling) strategies to optimize data reuse. The focus of this work is to evaluate these accelerator architectures using a tiled general matrix-matrix multiplication (GEMM) kernel. To do so, we develop a framework that finds optimized mappings (dataflow and tile sizes) for a tiled GEMM for a given spatial accelerator and workload combination, leveraging an analytical cost model for runtime and energy. Our evaluations over five spatial accelerators demonstrate that the tiled GEMM mappings systematically generated by our framework achieve high performance on various GEMM workloads and accelerators.

43 PARTICLE ACCELERATORS↗

A Benchmark Suite for Evaluating Scientific AI Workloads on GPUs

AI applications have been steadily increasing in the allocation portfolio among leadership computing facilities. These applications depend on deep learning frameworks with hardware acceleration and underlying software systems. With the rapid development of applications, software stacks, and hardware devices, it is essential to evaluate the performance of core operations in AI workloads for direction of optimizations and procurement of next-generation high-performance computing (HPC) infrastructures. Currently, most benchmarks lack scientific AI workloads. So, we present DeepKernelBench and the experimental results of evaluating the benchmark suite for early observations and performance comparisons on datacenter GPUs using representative workloads for scientific AI, including Attentions, General matrix multiplications, Geometrics and Fourier neural operations.

Jin, Zheming [Advanced Micro Devices (AMD)]↗

Evaluating lightweight unsupervised online IDS for masquerade attacks in CAN

Vehicular controller area networks (CANs) are susceptible to masquerade attacks by malicious adversaries. In masquerade attacks, adversaries silence a targeted ID and then send malicious frames with forged content at the expected timing of benign frames. As masquerade attacks could seriously harm vehicle functionality and are the stealthiest attacks to detect in CAN, recent work has devoted attention to compare frameworks for detecting masquerade attacks in CAN. However, most existing works report offline evaluations using CAN logs already collected using simulations that do not comply with the domain’s real-time constraints. Here we contribute to advance the state of the art by presenting a comparative evaluation of four different non-deep learning (DL)-based unsupervised online intrusion detection systems (IDS) for masquerade attacks in CAN. Our approach differs from existing comparative evaluations in that we analyze the effect of controlling streaming data conditions in a sliding window setting. In doing so, we use realistic masquerade attacks being replayed from the ROAD dataset. We show that although evaluated IDS are not effective at detecting every attack type, the method that relies on detecting changes in the hierarchical structure of clusters of time series produces the best results at the expense of higher computational overhead. We discuss limitations, open challenges, and how the evaluated methods can be used for practical unsupervised online CAN IDS for masquerade attacks.

Anomaly detection↗

Evaluating Spatial Accelerator Architectures with Tiled Matrix-Matrix Multiplication.

There is a growing interest in custom spatial accelerators for machine learning applications. These accelerators employ a spatial array of processing elements (PEs) interacting via custom buffer hierarchies and networks-on-chip. The efficiency of these accelerators comes from employing optimized dataflow (i.e., spatial/temporal partitioning of data across the PEs and fine-grained scheduling) strategies to optimize data reuse. The focus of this work is to evaluate these accelerator architectures using a tiled general matrix-matrix multiplication (GEMM) kernel. To do so, we develop a framework that finds optimized mappings (dataflow and tile sizes) for a tiled GEMM for a given spatial accelerator and workload combination, leveraging an analytical cost model for runtime and energy. Finally, our evaluations over five spatial accelerators demonstrate that the tiled GEMM mappings systematically generated by our framework achieve high performance on various GEMM workloads and accelerators.

42 ENGINEERING↗

An Architectural Survey of the Area 25 Nuclear Rocket Development Station, Nevada National Security Site, Nye County, Nevada, Volume 3 Of 3

The purpose of the current architectural survey is to record the Nuclear Rocket Development Station (NRDS) district, which is recommended eligible for listing to the NRHP under all four criteria as the testing center for the Rover/NERVA (Nuclear Engine for Rocket Vehicle Application) program from the initial boundary survey in 1956 to the end of nuclear rocket development on January 5, 1973. Resources in the NRDS district were evaluated to determine if they are eligible for listing to the National Register as district contributors. The current survey identified the Reactor Control Point subdistrict to provide a temporal, spatial, and functional framework for organizing the resources involved in the NRDS operations. However, the subdistrict was not evaluated as a potential historic district against the Secretary of Interior’s Significance Criteria.

54 ENVIRONMENTAL SCIENCES↗

Quantifying and reducing model-form uncertainties in Reynolds-averaged Navier–Stokes simulations: A data-driven, physics-informed Bayesian approach

Despite their well-known limitations, Reynolds-Averaged Navier–Stokes (RANS) models are still the workhorse tools for turbulent flow simulations in today's engineering analysis, design and optimization. While the predictive capability of RANS models depends on many factors, for many practical flows the turbulence models are by far the largest source of uncertainty. As RANS models are used in the design and safety evaluation of many mission-critical systems such as airplanes and nuclear power plants, quantifying their model-form uncertainties has significant implications in enabling risk-informed decision-making. In this work we develop a data-driven, physics-informed Bayesian framework for quantifying model-form uncertainties in RANS simulations. Uncertainties are introduced directly to the Reynolds stresses and are represented with compact parameterization accounting for empirical prior knowledge and physical constraints (e.g., realizability, smoothness, and symmetry). An iterative ensemble Kalman method is used to assimilate the prior knowledge and observation data in a Bayesian framework, and to propagate them to posterior distributions of velocities and other Quantities of Interest (QoIs). We use two representative cases, the flow over periodic hills and the flow in a square duct, to evaluate the performance of the proposed framework. Both cases are challenging for standard RANS turbulence models. Simulation results suggest that, even with very sparse observations, the obtained posterior mean velocities and other QoIs have significantly better agreement with the benchmark data compared to the baseline results. At most locations the posterior distribution adequately captures the true model error within the developed model form uncertainty bounds. The framework is a major improvement over existing black-box, physics-neutral methods for model-form uncertainty quantification, where prior knowledge and details of the models are not exploited. This approach has potential implications in many fields in which the governing equations are well understood but the model uncertainty comes from unresolved physical processes. - Highlights: • Proposed a physics–informed framework to quantify uncertainty in RANS simulations. • Framework incorporates physical prior knowledge and observation data. • Based on a rigorous Bayesian framework yet fully utilizes physical model. • Applicable for many complex physical systems beyond turbulent flows.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

ORIGEN-based Nuclear Fuel Inventory Module for Fuel Cycle Assessment: Final Project Report

The goal of this project, “ORIGEN-based Nuclear Fuel Depletion Module for Fuel Cycle Assessment" is to create a physics-based reactor depletion and decay module for the Cyclus nuclear fuel cycle simulator in order to assess nuclear fuel inventories over a broad space of reactor operating conditions. The overall goal of this approach is to facilitate evaluations of nuclear fuel inventories for a broad space of scenarios, including extended used nuclear fuel storage and cascading impacts on fuel cycle options such as actinide recovery in used nuclear fuel, particularly for multiple recycle scenarios. The advantages of a physics-based approach (compared to a recipe-based approach which has been typically employed for fuel cycle simulators) is in its inherent flexibility; such an approach can more readily accommodate the broad space of potential isotopic vectors that may be encountered under advanced fuel cycle options. In order to develop this flexible reactor analysis capability, we are leveraging the Origen nuclear fuel depletion and decay module from SCALE to produce a standalone “depletion engine” which will serve as the kernel of a Cyclus-based reactor analysis module. The ORIGEN depletion module is a rigorously benchmarked and extensively validated tool for nuclear fuel analysis and thus its incorporation into the Cyclus framework can bring these capabilities to bear on the problem of evaluating long-term impacts of fuel cycle option choices on relevant metrics of interest, including materials inventories and availability (for multiple recycle scenarios), long-term waste management and repository impacts, etc. Developing this Origen-based analysis capability for Cyclus requires the refinement of the Origen analysis sequence to the point where it can reasonably be compiled as a standalone sequence outside of SCALE; i.e., wherein all of the computational aspects of Origen (including reactor cross-section library processing and interpolation, input and output processing, and depletion/decay solvers) can be self-contained into a single executable sequence. Further, to embed this capability into other software environments (such as the Cyclus fuel cycle simulator) requires that Origen’s capabilities be encapsulated into a portable, self-contained library which other codes can then call directly through function calls, thereby directly accessing the solver and data processing capabilities of Origen. Additional components relevant to this work include modernization of the reactor data libraries used by Origen for conducting nuclear fuel depletion calculations. This work has included the development of new fuel assembly lattices not previously available (such as for CANDU heavy-water reactor assemblies) as well as validation of updated lattices for light-water reactors updated to employ modern nuclear data evaluations. The CyBORG reactor analysis module as-developed under this workscope is fully capable of dynamic calculation of depleted fuel compositions from all commercial U.S. reactor assembly types as well as a number of international fuel types, including MOX, VVER, MAGNOX, and PHWR CANDU fuel assemblies. In addition, the Origen-based depletion engine allows for CyBORG to evaluate novel fuel assembly and reactor design types via creation of Origen reactor data libraries via SCALE. The establishment of this new modeling capability affords fuel cycle modelers a substantially improved ability to model dynamically-changing fuel cycle and reactor conditions, including recycled fuel compositions from fuel cycle scenarios involving material recycle into thermal-spectrum systems.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Numerical evaluation of the bispectrum in multiple field inflation—the transport approach with code

We present a complete framework for numerical calculation of the power spectrum and bispectrum in canonical inflation with an arbitrary number of light or heavy fields. Our method includes all relevant effects at tree-level in the loop expansion, including (i) interference between growing and decaying modes near horizon exit; (ii) correlation and coupling between species near horizon exit and on superhorizon scales; (iii) contributions from mass terms; and (iv) all contributions from coupling to gravity. We track the evolution of each correlation function from the vacuum state through horizon exit and the superhorizon regime, with no need to match quantum and classical parts of the calculation; when integrated, our approach corresponds exactly with the tree-level Schwinger or 'in-in' formulation of quantum field theory. In this paper we give the equations necessary to evolve all two- and three-point correlation functions together with suitable initial conditions. The final formalism is suitable to compute the amplitude, shape, and scale dependence of the bispectrum in models with | f {sub NL}| of order unity or less, which are a target for future galaxy surveys such as Euclid, DESI and LSST. As an illustration we apply our framework to a number of examples, obtaining quantitatively accurate predictions for their bispectra for the first time. Two accompanying reports describe publicly-available software packages that implement the method.

79 ASTRONOMY AND ASTROPHYSICS↗

SeqMask: Behavior Extraction Over Cyber Threat Intelligence Via Multi-Instance Learning

Abstract Identification and extraction of Tactics, Techniques and Procedures (TTPs) for Cyber Threat Intelligence (CTI) restore the full picture of cyber attacks and guide the analysts to assess the system risk. Existing frameworks can hardly provide uniform and complete processing mechanisms for TTPs information extraction without adequate knowledge background. A multi-instance learning approach named SeqMask is proposed in this paper as a solution. SeqMask extracts behavior keywords from CTI evaluated by the semantic impact, and predicts TTPs labels by conditional probabilities. Still, the framework has two mechanisms to determine the validity of keywords. One using expert experience verification. The other verifies the distortion of the classification effect by blocking existing keywords. In the experiments, SeqMask reached 86.07% and 73.99% in F1 scores for TTPs classifications. For the top 20% of keywords, the expert approval rating is 92.20%, where the average repetition of keywords whose scores between 100% and 90% is 60.02%. Particularly, when the top 65% of the keywords were blocked, the F1 decreased to about 50%; when removing the top 50%, the F1 was under 31%. Further, we also validate the possibility of extracting TTPs from full-size CTI and malware whose F1 are improved by 2.16% and 0.81%.

Ge, Wenhan↗

Model Test Bed for Evaluating Unstructured-Grid Wave Models for Resource Assessment and Characterization

Recent development of unstructured-grid wave models demonstrates the advantage of using an unstructured-grid modeling framework in wave modeling, especially in coastal areas that have complex coastlines, islands, shallow-water bays, and estuaries. This study evaluated the performance of unstructured-grid models in comparison to structured-grid models for predicting the IEC wave energy resource parameters at a common test bed site. Model skill was assessed using a set of model performace error statistics. Sensitivity of open boundary conditions and ]wind forcing on model results, computational efficiency, and the advantage and limitation of the unstructured-grid model are summarized in the report.

16 TIDAL AND WAVE POWER↗

Considerations regarding the Use of Computer Vision Machine Learning in Safety-Related or Risk-Significant Applications in Nuclear Power Plants

With the advancements made to date in the field of artificial intelligence (AI), significant potential exists to utilize AI capabilities for nuclear power plant (NPP) applications. AI can replicate human decision making and it is usually faster and more accurate than humans. For implementations that impact critical NPP applications (e.g., safety-related or non-safety systems that potentially affect overall plant risk), a deeper safety analysis of the AI methods is necessary. AI applied to NPP operations could resemble the use of digital I&C (DI&C) because such applications involve digital computer hardware and custom-designed software that input plant data, execute complex software algorithms, and output the results to a system or licensed human operator to potentially provoke an action. For AI methods to be compliant with current safety requirements for DI&C, AI compatibility must be evaluated, and AI-related gaps may exist that prevent the prompt deployment of AI in NPPs. This effort aims to evaluate how example AI technologies align with the DI&C safety framework, and discusses how they could be analyzed, modeled, tested, and validated in a manner similar to typical DI&C technologies. Because AI is a broad field that encompasses areas such as machine learning (ML), natural language processing, and computer vision, this research focused on a subset of methods categorized as the computer vision ML (CVML) methods. This report explores two CVML use cases, gauge reading and fire watch, considered relevant to the DI&C standards, as they could play a safety-critical role. For the gauge reading use case, a CVML-enabled technology that can read gauges at oblique angles is utilized. For the fire watch use case, a CVML-enabled technology is utilized that migrates fire watch from a manual (human) approach to automated fire detection. These use cases are mainly intended to give context to the CVML system discussion. This effort assumes the worst-case scenario, with the CVML system being used to replace a safety-related or risk-significant system, thus requiring evaluation. Evaluating CVML against most of the relevant safety requirements for DI&C yielded several CVML-specific considerations due to the uniqueness of its characteristics in comparison with typical DI&C systems. For example, CVML models often employ commonly used (open-source) datasets, and it is not always possible to determine the level of overlap among open-source datasets. Therefore, the independence of the developed CVML models when demonstrating diversity is questionable, therefore creating vulnerability to common cause failure (CCF). The design verification process is also impacted since the data overlap could result in overestimation of the software validation and verification (V&V) performance results. Section 2 of this report evaluates a list of the identified CVML-specific characteristics and discusses the resulting considerations and potential solutions in the context of each referenced requirement. A summation is provided in Section 3. This report is not to be used as a guideline. It was developed to identify and consider issues in the implementation of ML technologies used to augment activities that may have a bearing on plant operation. The report draws parallels to the use of DI&C technologies, for which many standards are available to guide their use in nuclear plant operation. It considers the technologies and some of the potential implications of their use in safety-related applications but is not intended to address regulatory or licensing related issues.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Disentangling atmospheric, hydrological, and coupling uncertainties in compound flood modeling within a coupled Earth system model

Compound riverine and coastal flooding is usually driven by complex interactions among meteorological, hydrological, and ocean extremes. However, existing efforts to model this phenomenon often do not integrate hydrological processes across atmosphere–land–river–ocean systems, leading to substantial uncertainties that have not been fully examined. To bridge this gap, we leverage the new capabilities of the Energy Exascale Earth System Model (E3SM) that enable a multi-component framework that integrates coastally refined atmospheric, terrestrial, and oceanic components. We evaluate compound uncertainties arising from two-way land–river–ocean coupling in E3SM and track the cascading meteorological and hydrological uncertainties through ensemble simulations over the Delaware River basin and estuary during Hurricane Irene (2011). Our findings highlight the importance of two-way river–ocean coupling to compound flood modeling and demonstrate E3SM's capability in capturing compound flood extent near the coast, with a hit rate over 0.75. Our study shows the growing uncertainties that transition from atmospheric forcings to flood distribution and severity. Furthermore, an analysis based on artificial neural networks is used to assess the roles of hydrological drivers, such as infiltration and soil moisture, in the generation of compound flooding. The response of compound floods to tropical cyclones (TCs) is found to be susceptible to these often overlooked drivers. For instance, the flooded area could increase more than 2-fold (∼2.4) if Hurricane Irene were preceded by an extreme antecedent soil moisture condition (AMC). The results not only support the use of a multi-component framework for interactive flooding processes, but also underscore the necessity of broader definitions of compound flooding that encompass the simultaneous occurrence of intense precipitation, storm surge, and high AMC during TCs.

Feng, Dongyu [Pacific Northwest National Laborator↗

Markov Chain Monte Carlo Parameter Estimation of Deflagration Losses in a Rotating Detonation Engine

One of the practical challenges of the studies of rotating detonation engines (RDEs) is the direct estimation of losses from experimental measurements. This study attempts at resolving this limitation by combining a reduced order model (ROM) of the detonation wave characteristics with a Markov chain Monte Carlo parameter estimation framework. The model considers simple deflagration losses and the overall impact of deflagration on RDE performance. To evaluate this model, a Markov Chain Monte Carlo (MCMC) sampling technique was applied to estimate the loss parameters within the model for a set of conditions operated in hydrogen-air over a range of mass flow rates and equivalence ratios. The MCMC parameter estimation framework allowed for the determination of a posterior distribution of the loss parameters for each test condition, an examination of the correlation between the loss parameters and measured performance metrics of the RDE, and an uncertainty propagation of these parameters. The predicted model loss parameters were then compared to indirect experimental measurements of the deflagration combustion fractions to evaluating the validity of the approach, and shed light on the benefits and drawbacks of the model, measurement techniques, and the estimation framework.

33 ADVANCED PROPULSION SYSTEMS↗

Development of Automated Atom Probe Tomography capability to study the influence of applied voltage and laser power on the final apparent composition of the analyzed specimen

This study presents the development and implementation of an autonomous Bayesian optimization (BO) framework for controlling and optimizing experimental parameters in Atom Probe Tomography (APT). Using commercial silicon needle samples as a benchmark system, we demonstrate that BO can efficiently navigate the complex parameter space of voltage and laser power to achieve target charge state ratios (specifically Si + /(Si + +Si 2+ )) with minimal experimental evaluations. Our implementation integrates Gaussian Process modeling with the CAMECA atom probe control framework, enabling autonomous adjustment of experimental conditions in real-time. Results show that the algorithm successfully converges to target ratios under different scenarios: maintaining a reference ratio, increasing the ratio (favoring Si 1+ ), and decreasing the ratio (favoring Si 2+ ). The system adapts to specimen evolution during analysis, compensating for changes in apex geometry while maintaining optimization targets. This work establishes a proof of concept for AI-driven optimization in APT, addressing the traditional challenges of manual parameter tuning and paving the way for applications to more complex materials where compositional accuracy is critical.

36 MATERIALS SCIENCE↗

SHMEMGraph: Efficient and Balanced Graph Processing Using One-Sided Communication

State-of-the-art synchronous graph processing frameworks face both inefficiency and imbalance issues that cause their performance to be suboptimal. These issues include the inefficiency of communication and the imbalanced graph computation/communication costs in an iteration. We propose to replace their conventional two-sided communication model with the one-sided counterpart. Accordingly, we design SHMEMGraph, an efficient and balanced graph processing framework that is formulated across a global memory space and takes advantage of the flexibility and efficiency of one-sided communication for graph processing. Through an efficient one-sided communication channel, SHMEMGraph utilizes the high-performance operations with RDMA while minimizing the resource contention within a computer node. In addition, SHMEMGraph synthesizes a number of optimizations to address both computation imbalance and communication imbalance. By using a graph of 1 billion edges, our evaluation shows that compared to the state-of-the-art Gemini framework, SHMEMGraph achieves an average improvement of 35.5% in terms of job completion time for five representative graph algorithms.

Fu, Huansong↗