SEARCH · Engineering Papers

Results for “low fidelity”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

All sources NASA NTRS DOE OSTI

NASA NTRSDate not supplied

End-To-End Uncertainty Quantification with Analytical Derivatives for Design Under Uncertainty

Uncertainty quantification (UQ) is a rapidly growing and evolving discipline, especially within the aerospace community. Performing analysis with UQ can provide decision makers with a wealth of information about a candidate design. However, the value of UQ is fully realized when the information gained during UQ analysis is leveraged in a feedback loop of a design optimization process, often referred to as design under uncertainty. Although design under uncertainty can be a powerful risk mitigation technique, there are a number of roadblocks that prevent its implementation. Two primary factors are computational costs and added complexity of the analysis. High fidelity simulations on the order tens of uncertain variables quickly become computationally infeasible. Also, implementing UQ into an existing multidisciplinary design and optimization (MDO) process often requires extensive knowledge of the UQ methods and careful treatment of the problem formulation. The objective of this work is to address these two primary roadblocks and enable practitioners to efficiently perform design under uncertainty with limited knowledge of the UQ discipline. Methods outlined in this paper demonstrate MDO incorporating UQ into the design process, leveraging an analytic derivative tool chain through the entire optimization. The proposed approach leverages machine learning techniques to generate a differentiable confidence interval output from polynomial chaos models. This technique, coupled with the incorporation of analytical derivatives through the Polynomial Chaos Expansion (PCE) process, eliminates the need to estimate derivatives which are usually obtained from finite difference, complex step, or similar methods. Developing a differentiable confidence interval allows mixed uncertainty problems (both epistemic and aleatory) to be modeled. Without such modeling, these problems cannot accurately predict objective functions containing statistical quantities such as mean and variance. The addition of analytic derivatives to a polynomial chaos-based UQ method decreases the computational costs of performing design under uncertainty by orders of magnitude in comparison with methods such as complex step. The method and codes developed are modular in nature and are a drop-in solution for design under uncertainty within existing MDO problems. A low-fidelity analytical multidisciplinary optimization under uncertainty for a wing design in OpenMDAO is detailed in this paper. This demonstration case will include both objective functions and constraints which are influenced by uncertain parameters.

Ben Phillips↗

NASA NTRSDate not supplied

RApid Blade and Blade-Vortex InTeraction (RABBIT) GUI implementation via MATLAB Application Designer

Rotorcraft noise source identification and reduction is crucial to the emergence of the Urban Air Mobility (UAM) market. One key rotor noise source is Blade Vortex Interaction (BVI), caused by the rotor wake interacting with the rotor blades. The low-fidelity RApid Blade and Blade-Vortex InTeraction (RABBIT) tool was created to predict the location and characteristics of Blade-Vortex Interaction (BVI) noise. RABBIT enables engineers to quickly locate and understand designs or configurations that cause significant BVI noise. Furthermore, RABBIT can identify the time and location of blade overlap for coaxial rotors. RABBIT is not a general acoustic prediction tool, but instead utilizes vortex and wake parameters to visualize and predict only BVI noise. To date, RABBIT has been validated against CAMRAD II and ANOPP2/AARON for three NASA Revolutionary Vertical Lift Technology (RVLT) concept vehicles: the Quiet Single Main Rotor (QSMR), Side-by-Side, and Quadrotor. RABBIT has been shown to be a useful BVI prediction tool capable of visualizing BVI as a function of various parameters. Impulse factor and time rate of change of loading allows for a unique BVI prediction technique that provides information on impulse strength, location, and frequency. This presentation will highlight how MATLAB’s App Designer was utilized to enable users to navigate RABBIT easily and efficiently without prior experience with MATLAB or rotorcraft acoustics. RABBIT users can build and view rotors in real time with options to control blade chord, radius, phasing, and airfoil geometry. Furthermore, multiple rotors can be combined to create state-of-the-art multi-rotor vehicles or fleets. Editable tables, sliding bars, and push buttons were implemented within MATLAB’s App Designer for RABBIT’s GUI to allow engineers and designers to predict low order acoustic impact of BVI occurrences.

Natasha Lydia Schatzman↗

DOE OSTI2023

Screening and Discovery of Metal Compound Active Sites for Strong and Selective Adsorption of N 2 in Air

Photocatalytic nitrogen fixation has the potential to provide a greener route for producing nitrogen-based fertilizers under ambient conditions. Computational screening is a promising route to discover new materials for the nitrogen fixation process, but requires identifying “descriptors” that can be efficiently computed. In this work, we argue that selectivity toward the adsorption of molecular nitrogen and oxygen can act as a key descriptor. A catalyst that can selectively adsorb nitrogen and resist poisoning of oxygen and other molecules present in air has the potential to facilitate the nitrogen fixation process under ambient conditions. Here we provide a framework for active site screening based on multifidelity density functional theory (DFT) calculations for a range of metal oxides, oxyborides, and oxyphosphides. The screening methodology consists of initial low-fidelity fixed geometry calculations and a second screening in which more expensive geometry optimizations were performed. The approach identifies promising active sites on several TiO 2 polymorph surfaces and a VBO 4 surface, and the full nitrogen reduction pathway is studied with the BEEF-vdW and HSE06 functionals on two active sites. The findings suggest that metastable TiO 2 polymorphs may play a role in photocatalytic nitrogen fixation, and that VBO 4 may be an interesting material for further studies.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2022

Deep learning to estimate permeability using geophysical data

Time-lapse electrical resistivity tomography (ERT) is a popular geophysical method to estimate three-dimensional (3D) permeability fields from electrical potential difference measurements. Traditional inversion and data assimilation methods are used to ingest this ERT data into hydrogeophysical models to estimate permeability. Due to ill-posedness and the curse of dimensionality, existing inversion strategies provide poor estimates and low resolution of the 3D permeability field. Recent advances in deep learning provide us with powerful algorithms to overcome this challenge. This paper presents a deep learning (DL) framework to estimate the 3D subsurface permeability from time-lapse ERT data. To test the feasibility of the proposed framework, we train DL-enabled inverse models on simulation data. Each measurement in both synthetic and field data is standardized by removing the mean and scaling the time-series to unit variance. This pre-processing step is necessary to bring simulation data closer to field observations. Subsurface process models based on hydrogeophysics are used to generate this synthetic data. Training performed on limited simulation data resulted in the DL model over-fitting. An advanced data augmentation based on mixup is implemented to generate additional training samples to overcome this issue. This mixup technique creates weakly labeled (low-fidelity) samples from strongly labeled (high-fidelity) data. The weakly labeled training data is then used to develop DL-enabled inverse models and reduce over-fitting. As both time-lapse ERT (1133048 features/realization) and 3D permeability (585453 features/realization) data samples are from a high-dimensional space, principal component analysis (PCA) is employed to reduce dimensionality. Encoded ERT and encoded permeability are generated using the trained PCA estimators. A deep neural network is then trained to map the encoded ERT to encoded permeability. This mixup training and unsupervised learning allowed us to build a fast and reasonably accurate DL-based inverse model under limited simulation data. Results show that proposed weak supervised learning can capture salient spatial features in the 3D permeability field. Quantitatively, the average mean squared error (in terms of the natural log) on the strongly labeled training, validation, and test datasets is less than 0.5. The R 2 -score (global metric) is greater than 0.75, and the percent error in each cell (local metric) is less than 10%. Finally, an added benefit in terms of computational cost is that the proposed DL-based inverse model is at least O(10 4 ) times faster than running a forward model once it is trained. Data generation, DL model training, and hyperparameter tuning to identify optimal neural network architectures utilized high-performance computing resources while the DL inference is performed on a standard laptop. Approximately, O(10 5 ) processor hours are used for generating data and DL tuning and training. We acknowledge that the data generation and DL model development are expensive. But once a DL model is trained, it can be re-used for inversion rapidly for the given system, with set physics and domain. Note that traditional inversion may require multiple forward model simulations (e.g., in the order of 10 to 1000), which are very expensive. This computational savings ≈ O(10 5 ) – O(10 7 )) makes the proposed DL-based inverse model attractive for subsurface imaging and real-time ERT monitoring applications due to fast and yet reasonably accurate estimations of permeability field.

58 GEOSCIENCES↗

DOE OSTI2023

Multi-output multilevel best linear unbiased estimators via semidefinite programming

Multifidelity forward uncertainty quantification (UQ) problems often involve multiple quantities of interest and heterogeneous models (e.g., different grids, equations, dimensions, physics, surrogate and reduced-order models). While computational efficiency is key in this context, multi-output strategies in multilevel/multifidelity methods are either sub-optimal or non-existent. In this paper we extend multilevel best linear unbiased estimators (MLBLUE) to multi-output forward UQ problems and we present new semidefinite programming formulations for their optimal setup. Not only do these formulations yield the optimal number of samples required, but also the optimal selection of low-fidelity models to use. While existing MLBLUE approaches are single-output only and require a non-trivial nonlinear optimization procedure, the new multi-output formulations can be solved reliably and efficiently. Here, we demonstrate the efficacy of the new methods and formulations in practical UQ problems with model heterogeneity.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Computationally efficient CFD prediction of bubbly flow using physics-guided deep learning

To realize efficient computational fluid dynamics (CFD) prediction of two-phase flow, a multi-scale framework was proposed in this paper by applying a physics-guided data-driven approach. Instrumental to this framework, Feature Similarity Measurement (FSM) technique was developed for error estimation in two-phase flow simulation using coarse-mesh CFD, to achieve a comparable accuracy as fine-mesh simulations with fast-running feature. In this work, by defining physics-guided parameters and variable gradients as physical features, FSM has the capability to capture the underlying local patterns in the coarse-mesh CFD simulation. Massive low-fidelity data and respective high-fidelity data are used to explore the underlying information relevant to the main simulation errors and the effects of phenomenological scaling. By learning from previous simulation data, a surrogate model using deep feedforward neural network (DFNN) can be developed and trained to estimate the simulation error of coarse-mesh CFD. In a demonstration case of two-phase bubbly flow, the DFNN model well captured and corrected the unphysical “peaks” in the velocity and void fraction profiles near the wall in the coarse-mesh configuration, even for extrapolative predictions. The research documented supports the feasibility of the physics-guided deep learning methods for coarse mesh CFD simulations which has a potential for the efficient industrial design.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2020

Deep learning interfacial momentum closures in coarse-mesh CFD two-phase flow simulation using validation data

Multiphase flow phenomena have been widely observed in the industrial applications while it remains a challenging yet unsolved problems. Three-dimensional computational fluid dynamics (CFD) approaches resolve the flow fields on a finer special and temporal scales which can complement the dedicated experimental study. However, closures have to be introduced to reflect the underlying physics in multiphase flow. Among them, the interfacial forces, including drag, lift, turbulent dispersion and wall lubrication forces, play in important role on the bubble’s distribution and migration in liquid-vapor two-phase flow. Development of those closures traditionally rely on the experimental data and analytical derivation with simplified assumptions which usually cannot deliver a universal solution across wide range of flow conditions. In this paper, a data-driven approach, named as Feature Similarity Measurement (FSM), is developed and applied to improve the simulation capability of two-phase flow with coarse-mesh CFD approach. Interfacial momentum transfer in adiabatic bubbly flow serves as the focus of the present study. Both a mature and a simplified set of interfacial closures are taken as the low fidelity data. Experimental data and fine mesh CFD simulations results are adopted as high-fidelity data. Qualitative and quantitative analysis are performed in this paper which reveals that FSM can substantially improve the prediction of coarse mesh CFD model regardless of the choice of interfacial closures and it provides scalability and consistency across discontinuous flow regimes. Furthermore, it demonstrates that data-driven method can aid the multiphase flow modeling by exploring the connections between local physical features and simulation errors.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Accelerated statistical failure analysis of multifidelity TRISO fuel models

Statistical nuclear fuel failure analysis is critical for the design and development of advanced reactor technologies. Although Monte Carlo Sampling (MCS) is a standard method of statistical failure analysis for fuels, the low failure probabilities of some advanced fuel forms and the correspondingly large number of required model evaluations limit its application to low-fidelity (e.g., 1-D) fuel models. In this paper, we present four other statistical methods for fuel failure analysis in Bison, considering tri-structural isotropic (TRISO)-coated particle fuel as a case study. The statistical methods considered are Latin hypercube sampling (LHS), adaptive importance sampling (AIS), subset simulation (SS), and the Weibull theory. Using these methods, we analyzed both 1-D and 2-D representations of TRISO models to compute failure probabilities and the distributions of fuel properties that result in failures. The results of these methods compare well across all TRISO models considered. Overall, SS and the Weibull theory were deemed the most efficient, and can be applied to both 1-D and 2-D TRISO models to compute failure probabilities. Moreover, since SS also characterizes the distribution of parameters that cause TRISO failures, and can consider failure modes not described by the Weibull criterion, it may be preferred over the other methods. Finally, a discussion on the efficacy of different statistical methods of assessing nuclear fuel safety is provided.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2020

Thermochemical Data Fusion Using Graph Representation Learning

Large databases are required for “Big Data” applications in catalysis and materials science. Thermochemical databases can be created by combining data from various sources and by correcting low-fidelity datasets to higher accuracy with minimal computation. To achieve this “data fusion”, thermochemical quantities of interest, calculated at various levels of density functional theory (DFT), need to be mapped to the same, high levels of theory. In this work, a graph theoretical, statistical framework is proposed for such tasks. Subgraph frequencies are shown to provide a natural representation for learning these fusion maps. The maps are linear and are learnt with automated descriptor selection. Using a dataset of as few as ~1% from the QM9 database of 133,885 molecules, these models can predict multiple thermochemical quantities at a higher level of theory with an accuracy of 1 kcal/mol. Here, the method is explainable, generalizable, and provides a diagnostic tool for outlier identification

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2023

Comparison of multifidelity machine learning models for potential energy surfaces

Multifidelity modeling is a technique for fusing the information from two or more datasets into one model. It is particularly advantageous when one dataset contains few accurate results and the other contains many less accurate results. Within the context of modeling potential energy surfaces, the low-fidelity dataset can be made up of a large number of inexpensive energy computations that provide adequate coverage of the N-dimensional space spanned by the molecular internal coordinates. The high-fidelity dataset can provide fewer but more accurate electronic energies for the molecule in question. Here, we compare the performance of several neural network-based approaches to multifidelity modeling. We show that the four methods (dual, Δ-learning, weight transfer, and Meng–Karniadakis neural networks) outperform a traditional implementation of a neural network, given the same amount of training data. We also show that the Δ-learning approach is the most practical and tends to provide the most accurate model.

Chemistry↗

DOE OSTI2025

Efficient wind farm layout optimization with the FLOWERS AEP model and analytic gradients

Wind farm layout optimization (WFLO) studies often aim to maximize the annual energy production (AEP) of a wind farm by choosing an arrangement of turbines that minimizes wake interactions. One way to reduce the cost of WFLO studies is by using more computationally efficient AEP models. The cost of standard AEP modeling approaches, based on the numerical integration of low-fidelity engineering wake models, scales poorly with the number of simulated discrete wind conditions. A second way to reduce cost when using a gradient-based algorithm is to supply exact gradient information instead of finite-difference estimates. However, analytical functions for the derivatives of AEP with respect to turbine positions are not always available in the conventional modeling approach. FLOWERS is a computationally inexpensive, analytical model for wind farm AEP that is specifically developed for WFLO applications. In this paper, we analyze the performance of the FLOWERS AEP model with analytic gradients in a layout optimization study compared with a reference optimization framework across three wind farm case studies. We find that the FLOWERS-based approach reduces computation time by a factor of 50–4000 and improves optimal AEP by about 0.3% with less than half of the variability in AEP across instances with randomized initial conditions. We also find the optimal layouts to be insensitive to model parameter tuning, making FLOWERS-based layout optimization a streamlined, user-friendly approach.

17 WIND ENERGY↗

DOE OSTI2025

Magnetic mesh generation and field line reconstruction for scrape-off layer and divertor modeling in stellarators

The design of divertor targets and baffles for optimal heat and particle exhaust from magnetically confined fusion plasmas requires a combination of fast, low-fidelity models (such as EMC3-Lite (Feng 2022 Plasma Phys. Control. Fusion 64 125012)) for scoping studies and high-fidelity ones (such as EMC3-EIRENE (Feng et al 2014 Contrib. Plasma Phys. 54 426–31)) for verification. Both of those approaches benefit from a magnetic flux tube mesh for fast interpolation and mapping of field line segments (Feng et al 2005 Phys. Plasmas 12 052505). A new automated mesh generator for unstructured quadrilateral flux tubes with adaptive refinement is presented and integrated into FLARE (Frerichs 2024 Nucl. Fusion 64 106034). For HSX with an extended first wall, it is found that several layers of flux tubes can span the entire half field period before splitting is required. This is an advantage over the traditional setup of the EMC3-EIRENE mesh where careful construction of several sub- domains is required already for the much tighter present first wall. In particular, there is no longer the need to manually construct a suitable outer boundary for the mesh. The divide and conquer paradigm with unstructured mesh layout offers a powerful alternative for fast head load approximation that is suitable for integration into optimization workflows. Further examples for W7-X and CTH demonstrate the versatile application range.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2023

Open-Loop Control Co-Design of Semisubmersible Floating Offshore Wind Turbines Using Linear Parameter-Varying Models

Here, this paper discusses a framework to design elements of the plant and control systems for floating offshore wind turbines in an integrated manner using linear parameter-varying models. Multiple linearized models derived from aero-elastic simulation software in different operating regions characterized by the incoming wind speed are combined to construct an approximate low-fidelity model of the system. The combined model is then used to generate open-loop, optimal control trajectories as part of a nested control co-design strategy that explores the system’s power production and stability using the platform pitch tilt as a proxy in the context of crucial plant and control design decisions. The radial distance between the central and outer columns and the diameter of the outer columns of the semisubmersible platform are the plant design variables. The platform stability and power production are studied for different plant design decisions. The effect of plant decisions on subsequent power production and stability response of the floating wind turbine is quantified in terms of the levelized cost of energy. The results show that the inner-loop constraints and the plant design decisions affect the turbine’s power and, subsequently, the cost of the system.

17 WIND ENERGY↗

DOE OSTI2024

Effects of Aluminum Plate Initial Residual Stress on Machined-Part Distortion

Dimensional tolerances for high-speed-machined aluminum products continue to tighten due to the demand for automated assembly of complex monolithic parts in aerospace and other industries. Understanding the contribution of inherent residual stress in wrought Al 7050-T7451 plate, common in aircraft manufacture, to distortion of high-aspect-ratio machined parts is critical but remains problematic due to the alloy's low residual stress magnitude over large geometries. Prior investigations into residual stress effects on machined part distortion suffer inadequate characterizations of the wrought material stress field, either because of low fidelity due to “slitting” methods, confounding effects in machined-layer removal methods, or small sample size when using neutron diffraction (ND). In this work, inherent residual stress is measured via ND at 860 locations in a 90.5 mm thick Al 7050-T7451 plate having dimensions 399 mm in the rolling direction and 335 mm in the transverse direction. Unlike prior studies, the ND residual stress is reconstructed using an iterative algorithm to ensure fully compatible, equilibrated 3D field prior to examining its effect on distortion. Further, the findings from simulations and experiments show that inherent residual stress alone could distort a high-aspect-ratio part beyond aerospace industry requirements, that slitting measurements may not sufficiently characterize residual stress for predicted distortion, and that parts machined from different plate thickness locations could exhibit reversed distortion patterns. Thus, research into distortion prediction that considers machining should carefully characterize and reconstruct inherent residual stress so that the coupled machining effects are accurately modeled.

36 MATERIALS SCIENCE↗

DOE OSTI2025

Data-Driven Closures and Assimilation for Stiff Multiscale Random Dynamics

Here, we introduce a data-driven and physics-informed framework for propagating uncertainty in stiff, multiscale random ordinary differential equations (RODEs) driven by correlated (colored) noise. Unlike systems subjected to Gaussian white noise, a deterministic equation for the joint probability density function (PDF) of RODE state variables does not exist in closed form. Moreover, such an equation would require as many phase-space variables as there are states in the RODE system. To alleviate this curse of dimensionality, we instead derive exact, albeit unclosed, reduced-order PDF (RoPDF) equations for low-dimensional observables/quantities of interest. The unclosed terms take the form of state-dependent conditional expectations, which are directly estimated from data at sparse observation times. However, for systems exhibiting stiff, multiscale dynamics, data sparsity introduces regression discrepancies that compound during RoPDF evolution. This is overcome by introducing a kinetic-like defect term to the RoPDF equation, which is learned by assimilating in sparse, low-fidelity RoPDF estimates. Two assimilation methods are considered, namely nudging and deep neural networks, which are successfully tested against Monte Carlo simulations.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Unified many-worlds browsing of arbitrary physics-based animations

Manually tuning physics-based animation parameters to explore a simulation outcome space or achieve desired motion outcomes can be notoriously tedious. This problem has motivated many sophisticated and specialized optimization-based methods for fine-grained (keyframe) control, each of which are typically limited to specific animation phenomena, usually complicated, and, unfortunately, not widely used. In this paper, we propose Unified Many-Worlds Browsing (UMWB), a practical method for sample-level control and exploration of physics-based animations. Our approach supports browsing of large simulation ensembles of arbitrary animation phenomena by using a unified volumetric WORLDPACK representation based on spatiotemporally compressed voxel data associated with geometric occupancy and other low-fidelity animation state. Beyond memory reduction, the WORLDPACK representation also enables unified query support for interactive browsing: it provides fast evaluation of approximate spatiotemporal queries, such as occupancy tests that find ensemble samples ("worlds") where material is either IN or NOT IN a user-specified spacetime region. WORLDPACKS also support real-time hardware-accelerated voxel rendering by exploiting the spatially hierarchical and temporal RLE raster data structure. Our UMWB implementation supports interactive browsing (and offline refinement) of ensembles containing thousands of simulation samples, and fast spatiotemporal queries and ranking. We show UMWB results using a wide variety of physics-based animation phenomena---not just JELL-O ® .

Computer Science↗

DOE OSTI2023

Dancing the Quantum Waltz: Compiling Three-Qubit Gates on Four Level Architectures

Superconducting quantum devices are a leading technology for quantum computation, but they face several challenges. Gate errors, coherence errors and a lack of connectivity all contribute to low fidelity results. In particular, connectivity restrictions enforce a gate set that requires three-qubit gates to be decomposed into one- or two-qubit gates. This substantially increases the number of two-qubit gates that need to be executed. However, many quantum devices have access to higher energy levels. We can expand the qubit abstraction of |0〉 and |1〉 to a ququart which has access to the |2〉 and |3〉 state, but with shorter coherence times. This allows for two qubits to be encoded in one ququart, enabling increased virtual connectivity between physical units from two adjacent qubits to four fully connected qubits. This connectivity scheme allows us to more efficiently execute three-qubit gates natively between two physical devices.We present direct-to-pulse implementations of several three-qubit gates, synthesized via optimal control, for compilation of three-qubit gates onto a superconducting-based architecture with access to four-level devices with the first experimental demonstration of four-level ququart gates designed through optimal control. We demonstrate strategies that temporarily use higher level states to perform Toffoli gates and always use higher level states to improve fidelities for quantum circuits. We find that these methods improve expected fidelities with increases of 2x across circuit sizes using intermediate encoding, and increases of 3x for fully-encoded ququart compilation.

Litteken, Andrew↗

DOE OSTI2022

Time-Based CAN IDS Paper Results Code

Modern vehicles are complex cyber-physical systems made of hundreds of electronic control units (ECUs) that communicate over controller area networks (CANs). This inherited complexity has expanded the CAN attack surface which is vulnerable to message injection attacks. These injections change the overall timing characteristics of messages on the bus, and thus, to detect these malicious messages, time-based intrusion detection systems (IDSs) have been proposed. However, time-based IDSs are usually trained and tested on low-fidelity datasets with unrealistic, labeled attacks. This makes difficult the task of evaluating, comparing, and validating IDSs. Here we detail and benchmark four time-based IDSs against the newly published ROAD dataset, the first open CAN IDS dataset with real (non-simulated) stealthy attacks with physically verified effects. We found that methods that perform hypothesis testing by explicitly estimating message timing distributions have lower performance than methods that seek anomalies in a distribution related statistic. In particular, these “distribution-agnostic” based methods outperform “distribution-based” methods by at least 55% in area under the precision-recall curve (AUC-PR). Our results expand the body of knowledge of CAN time-based IDSs by providing details of these methods and reporting their results when tested on datasets with real advanced attacks. Finally, we develop an after-market plug-in detector using lightweight hardware, which can be used to deploy the best performing IDS method on nearly any vehicle.

Moriano, Pablo [Oak Ridge National Lab. (ORNL), Oa↗