Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Netflow”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

NetFlow Dynamics

NetFlow Dynamics is a web-accessible analysis environment for simulating dynamic flows of materials on model networks. Performing a simulation requires both the NetFlow Dynamics application and a network model which is a description of the structure of the nodes and edges of a network including the flow capacity of each edge and the storage capacity of each node, and the sources and sinks of the material flowing on the network. NetFlow Dynamics consists of databases for storing network models, algorithms to calculate flows on networks, and a GIS-based graphical interface for performing simulations and viewing simulation results. Simulated flows are dynamic in the sense that flows on each edge of the network and inventories at each node change with time and can be out of equilibrium with boundary conditions. Any number of network models could be simulated using Net Flow Dynamics. To date, the models simulated have been models of petroleum infrastructure. The main model has been the National Transportation Fuels Model (NTFM), a network of U.S. oil fields, transmission pipelines, rail lines, refineries, tank farms, and distribution terminals. NetFlow Dynamics supports two different flow algorithms, the Gradient Flow algorithm and the Inventory Control algorithm, that were developed specifically for the NetFlow Dynamics application. The intent is to add additional algorithms in the future as needed. The ability to select from multiple algorithms is desirable because a single algorithm never covers all analysis needs. The current algorithms use a demand-driven capacity-constrained formulation which means that the algorithms strive to use all available capacity and stored inventory to meet desired flows to sinks, subject to the capacity constraints of each network component. The current flow algorithms are best suited for problems in which a material flows on a capacity-constrained network representing a supply chain in which the material supplied can be stored at each node of the network. In the petroleum models, the flowing materials are crude oil and refined products that can be stored at tank farms, refineries, or terminals (i.e. the nodes of the network). Examples of other network models that could be simulated are currency flowing in a financial network, agricultural products moving to market, or natural gas flowing on a pipeline network.

Corbet Jr., Thomas F↗

Inferring Netflow Data

Netflow data is a specific format for looking at network traffic. With this format, many security tests are easy to run. Cloud Service Providers produce logs that do not contain all of the content that Netflow data contains. There is a possibility that the fields that are in Netflow data but absent in Cloud Service Provider logs can be inferred from the fields that are present in the Cloud Service Provider logs. By inferring the missing data fields, the same test that can be run with Netflow data would be possible to run on the logs produced by Cloud Service Providers. Cloud Service Providers include Amazon Web Services and Box.com. There are multiple different types of Cloud Services, and each provider handles them differently and produces different logs. There are IaaS (meaning infrastructure as a service), Saas (meaning software as a service), and Paas (meaning platform as a service). Each provides a different use for the user.

97 MATHEMATICS AND COMPUTING↗

Netflow Theory Manual

The purpose of this report is to document the theoretical models utilized by the computer code NETFLOW. This report will focus on the theoretical models used to analyze high Mach number fully compressible transonic flows in piping networks.

97 MATHEMATICS AND COMPUTING↗

Calculation of natural convection test at Phenix using the NETFLOW++ code

The present paper describes modeling and analyses of a natural convection of the pool-type fast breeder reactor Phenix. The natural convection test was carried out as one of the End of Life Tests of the Phenix. Objective of the present study is to assess the applicability of the NETFLOW++ code which has been verified thus far using various water facilities and validated using the plant data of the loop-type FBR 'Monju' and the loop-type experimental fast reactor 'Joyo'. The Phenix primary heat transport system is modeled based on the benchmark documents available from IAEA. The calculational model consists of only the primary heat transport system with boundary conditions on the secondary-side of IHX. The coolant temperature at the primary pump inlet, the primary coolant temperature at the IHX inlet and outlet, the secondary coolant temperatures and other parameters are calculated by the code where the heat transfer between the hot and cold pools is explicitly taken into account. A model including the secondary and tertiary systems was prepared, and the calculated results also agree well with the measured data in general. (authors)

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Discrete Mathematical Approaches to Graph-Based Traffic Analysis

Modern cyber defense and anlaytics requires general, formal models of cyber systems. Multi-scale network models are prime candidates for such formalisms, using discrete mathematical methods based in hierarchically-structured directed multigraphs which also include rich sets of labels. An exemplar of an application of such an approach is traffic analysis, that is, observing and analyzing connections between clients, servers, hosts, and actors within IP networks, over time, to identify characteristic or suspicious patterns. Towards that end, NetFlow (or more generically, IPFLOW) data are available from routers and servers which summarize coherent groups of IP packets flowing through the network. In this paper, we consider traffic analysis of Netflow using both basic graph statistics and two new mathematical measures involving labeled degree distributions and time interval overlap measures. We do all of this over the VAST test data set of 96M synthetic Netflow graph edges, against which we can identify characteristic patterns of simulated ground-truth network attacks.

Joslyn, Cliff A.↗

A Network Management System for Handling Scientific Data Flows

Large scientific data transfers often occur at high rates causing increased burstiness in Internet traffic. To limit the adverse effects of these high-rate large-sized flows, which are referred to as $\alpha $ flows, on delay-sensitive audio/video flows, a network management system called Alpha Flow Traffic Engineering System (AFTES) is proposed for intra-domain traffic engineering. An offline approach is used in which AFTES analyzes NetFlow records collected by routers, extracts source–destination address prefixes of $\alpha $ flows, and uses these prefixes to configure firewall filters at ingress routers of a provider’s network to redirect future $\alpha $ flows to traffic-engineered paths and isolated queues. The effectiveness of this scheme was evaluated through an analysis of 7 months of NetFlow data obtained from an ESnet router. For this data set, 91 % of bytes generated by $\alpha $ flows during high-rate intervals would have been directed had AFTES been deployed. The negative aspect of using address prefixes in firewall filters, i.e., the redirection of $\beta $ flows to $\alpha $-flow paths/queues, was also quantified.

97 MATHEMATICS AND COMPUTING↗

Automating Network Node Behavior Characterization by Mining Communication Patterns

Enterprise networks of scale are complex, dynamic computing environments that respond to evolv- ing business objectives and requirements. Characteriz- ing system behaviors in these environments is essential for network management and cyber security operations. Characterization of system’s communication is typical and is supported using network flow information (NetFlow). Related work has characterized behavior using theoretical graph metrics; results are often difficult to interpret by enterprise staff. We propose a different approach, where flow information is mapped to sets of tags that contextualize the data in terms of network principals and enterprise concepts. Frequent patterns are then extracted and are expressed as behaviors. Behaviors can be com- pared, identifying systems expressing similar behaviors. We evaluate the approach using flow information collected by a third party.

cyber system modeling↗

Massive Scale Cyber Traffic Analysis: A Driver for Graph Database Research

We describe the significance and prominence of network traffic analysis (TA) as a graph- and network-theoretical domain for advancing research in graph database systems. TA involves observing and analyzing the connections between clients, servers, hosts, and actors within IP networks, both at particular times and as extended over times. Towards that end, NetFlow (or more generically, IPFLOW) data are available from routers and servers which summarize coherent groups of IP packets flowing through the network. IPFLOW databases are routinely interrogated statistically and visualized for suspicious patterns. But the ability to cast IPFLOW data as a massive graph and query it interactively, in order to e.g.\ identify connectivity patterns, is less well advanced, due to a number of factors including scaling, and their hybrid nature combining graph connectivity and quantitative attributes. In this paper, we outline requirements and opportunities for graph-structured IPFLOW analytics based on our experience with real IPFLOW databases. Specifically, we describe real use cases from the security domain, cast them as graph patterns, show how to express them in two graph-oriented query languages SPARQL and Datalog, and use these examples to motivate a new class of "hybrid" graph-relational systems.

Joslyn, Cliff A.↗

Diventi

Diventi provides a write optimized index of network logs such as netflow or zeek. The program will ingest various network traffic logs (zeek connection logs or NetflowV5, v9 or IPFix logs) and index each IP address and time of its activity. This database can then be queried for timely tracking of network activity. It currently uses the ft-index library from Percona but is adaptable to any key-value store that supports the Berkeley DB api.SAND2019-15037 M Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Kroeger, Thomas↗

Hylia: Time-Series Library for Network Operations (Hylia) v1

The ability to analyze time-series data in network operations is critical to ensure optimal operations and management in wide area networks, sensor networks, and cloud networks. Hylia is a collection of multiple time-series prediction algorithms designed specifically to work with network monitoring data sets such as SNMP, NETFLOW and other monitoring data. It is designed to recognize these features and perform the extrapolation of the data.

Kiran, Mariam↗

Comparison of High Pressure Transient PVT Measurements and Model Predictions – Part I

A series of experiments consisting of vessel-to-vessel transfers of pressurized gas using Transient PVT methodology have been conducted to provide a data set for optimizing heat transfer correlations in high pressure flow systems. In rapid expansions such as these, the heat transfer conditions are neither adiabatic nor isothermal. Compressible flow tools exist, such as NETFLOW that can accurately calculate the pressure and other dynamical mechanical properties of such a system as a function of time. However to properly evaluate the mass that has transferred as a function of time these computational tools rely on heat transfer correlations that must be confirmed experimentally. In this work new data sets using helium gas are used to evaluate the accuracy of these correlations for receiver vessel sizes ranging from 0.090 L to 13 L and initial supply pressures ranging from 2 MPa to 40 MPa. The comparisons show that the correlations developed in the 1980s from sparse data sets perform well for the supply vessels but are not accurate for the receivers, particularly at early time during the transfers. This report focuses on the experiments used to obtain high quality data sets that can be used to validate computational models. Part II of this report discusses how these data were used to gain insight into the physics of gas transfer and to improve vessel heat transfer correlations. Network flow modeling and CFD modeling is also discussed.

99 GENERAL AND MISCELLANEOUS↗

Analysis of a Full Scale Blowdown Due to a Mechanical Failure of a Pressure Relief Device in a Natural Gas Vehicle Maintenance Facility

A computational fluid dynamics (CFD) analysis of a natural gas vehicle experiencing a mechanical failure of a pressure relief device on a full CNG cylinder was completed to determine the resulting amount and location of flammable gas. The resulting overpressure if it were to ignite was also calculated. This study completes what is discussed in Ekoto et al. which covers other related leak scenarios. We are not determining whether or not this is a credible release, rather just showing the result of a possible worst case scenario. The Sandia National Laboratories computational tool Netflow was used to calculate the leak velocity and temperature. The in - house CFD code Fuego was used to determine the flow of the leak into the maintenance garage. A maximum flammable mass of 35 kg collected along the roof of the garage. This would result in an overpressure that could do considerable damage if it were to ignite at the time of this maximum volume. It is up to the code committees to decide whet her this would be a credible leak, but if it were, there should be preventions to keep the flammable mass from igniting.

03 NATURAL GAS↗

NFD Methodology Report

The NetFlow Dynamics (NFD) model was developed for estimating the availability of a commodity supplied by a national- or regional-scale infrastructure following unexpected disruption of one or more of its components. The large scope of the disruptions of interest produce changes in availability lasting days to weeks. Consequently, the model does not resolve daily variations in system state and does not include the long-term processes that cause infrastructures to evolve as assets are added and removed according to owners ’planning decisions. NFD simulates fluid flow, including petroleum and other incompressible fluids, as well as natural gas and other compressible fluids, through pipeline networks characterized by limits on transmission capacity and storage. It was designed to enable efficient exploration of possible transmission system responses to large-scale disruptions lasting for days or longer. The model formulation reflects constraints on transmission and storage capacity imposed by the physical system assets. Those capacity limits are input parameters and are not derived from more basic system properties such as pipeline diameters and compressor power. A system’s response to a large disruption is controlled by operational decisions as well as damage to physical assets. The NFD model formulation allows users to efficiently consider alternative scenarios about the way remaining capacity might be used so that the analysis result appropriately reflects uncertainties about operator response.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Transient PVT measurements and model predictions for vessel heat transfer. Part II.

Part I of this report focused on the acquisition and presentation of transient PVT data sets that can be used to validate gas transfer models. Here in Part II we focus primarily on describing models and validating these models using the data sets. Our models are intended to describe the high speed transport of compressible gases in arbitrary arrangements of vessels, tubing, valving and flow branches. Our models fall into three categories: (1) network flow models in which flow paths are modeled as one-dimensional flow and vessels are modeled as single control volumes, (2) CFD (Computational Fluid Dynamics) models in which flow in and between vessels is modeled in three dimensions and (3) coupled network/CFD models in which vessels are modeled using CFD and flows between vessels are modeled using a network flow code. In our work we utilized NETFLOW as our network flow code and FUEGO for our CFD code. Since network flow models lack three-dimensional resolution, correlations for heat transfer and tube frictional pressure drop are required to resolve important physics not being captured by the model. Here we describe how vessel heat transfer correlations were improved using the data and present direct model-data comparisons for all tests documented in Part I. Our results show that our network flow models have been substantially improved. The CFD modeling presented here describes the complex nature of vessel heat transfer and for the first time demonstrates that flow and heat transfer in vessels can be modeled directly without the need for correlations.

42 ENGINEERING↗

Utilizing Genetic Programming to Identify Failures in ICS Networks

Previously, researchers have attempted to apply machine learning techniques to network anomaly detection problems. Due to the staggering amount of variety that can occur in normal networks, the results have often been underwhelming. These challenges are far less pronounced when considering industrial control system (ICS) networks. The recurrent nature of these networks results in less noise and more consistent patterns for a machine learning algorithm to recognize. Here, we propose a method of evolving decision trees through genetic programming (GP) in order to detect network anomalies, such as device outages. Our approach extracts over a dozen features from network packet captures and netflows, normalizes them, and relates them in decision trees using fuzzy logic operators. Furthermore, the trees were used to detect three specific network events from three different points on the network across a statistically significant number of runs and achieved 100% accuracy on five of the nine experiments. When the trees attempted to detect more challenging events at points of presence further from the occurrence, the accuracy averaged to above 98%. Finally, using our method, all of the evolutionary cycles of the GP algorithm are computed a-priori, allowing the best resultant trees to be deployed as semi-real-time sensors with little overhead.

97 MATHEMATICS AND COMPUTING↗

polishCLR: A Nextflow Workflow for Polishing PacBio CLR Genome Assemblies

Long-read sequencing has revolutionized genome assembly, yielding highly contiguous, chromosome-level contigs. However, assemblies from some third generation long read technologies, such as Pacific Biosciences (PacBio) continuous long reads (CLR), have a high error rate. Such errors can be corrected with short reads through a process called polishing. Although best practices for polishing non-model de novo genome assemblies were recently described by the Vertebrate Genome Project (VGP) Assembly community, there is a need for a publicly available, reproducible workflow that can be easily implemented and run on a conventional high performance computing environment. Here, we describe polishCLR (https://github.com/isugifNF/polishCLR), a reproducible Nextflow workflow that implements best practices for polishing assemblies made from CLR data. PolishCLR can be initiated from several input options that extend best practices to suboptimal cases. It also provides re-entry points throughout several key processes, including identifying duplicate haplotypes in purge_dups, allowing a break for scaffolding if data are available, and throughout multiple rounds of polishing and evaluation with Arrow and FreeBayes. PolishCLR is containerized and publicly available for the greater assembly community as a tool to complete assemblies from existing, error-prone long-read data.

59 BASIC BIOLOGICAL SCIENCES↗