Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “DATA NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

The Network Completion Problem: Inferring Missing Nodes and Edges in Networks

Network structures, such as social networks, web graphs and networks from systems biology, play important roles in many areas of science and our everyday lives. In order to study the networks one needs to first collect reliable large scale network data. While the social and information networks have become ubiquitous, the challenge of collecting complete network data still persists. Many times the collected network data is incomplete with nodes and edges missing. Commonly, only a part of the network can be observed and we would like to infer the unobserved part of the network. We address this issue by studying the Network Completion Problem: Given a network with missing nodes and edges, can we complete the missing part? We cast the problem in the Expectation Maximization (EM) framework where we use the observed part of the network to fit a model of network structure, and then we estimate the missing part of the network using the model, re-estimate the parameters and so on. We combine the EM with the Kronecker graphs model and design a scalable Metropolized Gibbs sampling approach that allows for the estimation of the model parameters as well as the inference about missing nodes and edges of the network. Experiments on synthetic and several real-world networks show that our approach can effectively recover the network even when about half of the nodes in the network are missing. Our algorithm outperforms not only classical link-prediction approaches but also the state of the art Stochastic block modeling approach. Furthermore, our algorithm easily scales to networks with tens of thousands of nodes.

97 MATHEMATICS AND COMPUTING↗

Minimally buffered data transfers between nodes in a data communications network

Methods, apparatus, and products for minimally buffered data transfers between nodes in a data communications network are disclosed that include: receiving, by a messaging module on an origin node, a storage identifier, a origin data type, and a target data type, the storage identifier specifying application storage containing data, the origin data type describing a data subset contained in the origin application storage, the target data type describing an arrangement of the data subset in application storage on a target node; creating, by the messaging module, origin metadata describing the origin data type; selecting, by the messaging module from the origin application storage in dependence upon the origin metadata and the storage identifier, the data subset; and transmitting, by the messaging module to the target node, the selected data subset for storing in the target application storage in dependence upon the target data type without temporarily buffering the data subset.

97 MATHEMATICS AND COMPUTING↗

AmeriFlux BASE data pipeline to support network growth and data sharing

Abstract AmeriFlux is a network of research sites that measure carbon, water, and energy fluxes between ecosystems and the atmosphere using the eddy covariance technique to study a variety of Earth science questions. AmeriFlux’s diversity of ecosystems, instruments, and data-processing routines create challenges for data standardization, quality assurance, and sharing across the network. To address these challenges, the AmeriFlux Management Project (AMP) designed and implemented the BASE data-processing pipeline. The pipeline begins with data uploaded by the site teams, followed by the AMP team’s quality assurance and quality control (QA/QC), ingestion of site metadata, and publication of the BASE data product. The semi-automated pipeline enables us to keep pace with the rapid growth of the network. As of 2022, the AmeriFlux BASE data product contains 3,130 site years of data from 444 sites, with standardized units and variable names of more than 60 common variables, representing the largest long-term data repository for flux-met data in the world. The standardized, quality-ensured data product facilitates multisite comparisons, model evaluations, and data syntheses.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Detection and Characterization of Intrusions to Network Parameter Data in Electric Power Systems

Combating cyberattacks is an emerging challenge in maintaining the reliable and economic operation of electric power systems. Possible cyberattacks include intrusions to the parameter data at a control center. In this class of attacks, algorithms at the control center are correctly executed, but the attacker's modification of the associated parameter data yields improper results. This paper proposes an algorithm for detecting and characterizing cyberattacks to network parameter data, with specific application to optimal power flow problems. The proposed algorithm evaluates whether historical operating point data are consistent with the network parameters. Inconsistencies indicating potential cyberattacks are characterized using historical operational data (power injections and voltage phasors) along with network parameter data. Simulated test cases illustrate the proposed algorithm's detection and characterization capabilities.

cybersecurity↗

Anonymization of Network Traces Data through Condensation-based Differential Privacy

Network traces are considered a primary source of information to researchers, who use them to investigate research problems such as identifying user behavior, analyzing network hierarchy, maintaining network security, classifying packet flows, and much more. However, most organizations are reluctant to share their data with a third party or the public due to privacy concerns. Therefore, data anonymization prior to sharing becomes a convenient solution to both organizations and researchers. Although several anonymization algorithms are available, few of them allow sufficient privacy (organization need), acceptable data utility (researcher need), and efficient data analysis at the same time. This article introduces a condensation-based differential privacy anonymization approach that achieves an improved tradeoff between privacy and utility compared to existing techniques and produces anonymized network trace data that can be shared publicly without lowering its utility value. Our solution also does not incur extra computation overhead for the data analyzer. A prototype system has been implemented, and experiments have shown that the proposed approach preserves privacy and allows data analysis without revealing the original data even when injection attacks are launched against it. When anonymized datasets are given as input to graph-based intrusion detection techniques, they yield almost identical intrusion detection rates as the original datasets with only a negligible impact.

97 MATHEMATICS AND COMPUTING↗

A Deep Learning Approach for In-Network Synchrophasor Missing Data Recovery Using Programmable Network Switches

Phasor measurement unit (PMU) networks deliver accurate and timely measurements, which is essential for managing today’s electric power systems. To ensure data quality and enhance the cyber-resilience of PMU networks against malicious attacks and data errors, this study presents an online PMU missing data recovery scheme by leveraging P4 programmable switches. The data plane incorporates a customized PMU protocol parser that abstracts the necessary payload data for recovery. Recovery processes are executed in the control plane using a pre-trained machine learning model. Both traditional and advanced ML models, such as transformer and TimeGPT, are explicitly employed for data prediction. This approach ensures rapid and precise data recovery. Performance evaluations focus on recovery speed and accuracy, using a real dataset from a campus microgrid. With 20% missing PMU data, the mean absolute percentage error for voltage magnitude is 0.0384%, and the phase angle error discrepancy is approximately 0.4064%.

Phasor Measurement Unit, Machine Learning, Program↗

Toward higher-radix switches with co-packaged optics for improved network locality in data center and HPC networks [Invited]

In this work, we study the network locality improvements that can be achieved by using co-packaged optics in data center and high-performance computing (HPC) networks. The increased escape bandwidth offered by co-packaged optics can enable switches with speeds of 51.2 Tb/s and beyond. From a network architecture perspective, the key advantages of introducing co-packaged optics at the switch points include the implementation of large-scale topologies of >12,000 end points with 4× higher bisection bandwidth and the reduction of the required number of switches by >40% compared with state-of-the-art approaches. From a network operation perspective, improved network locality and faster operation can be achieved since the higher-radix switches can mitigate the impact of network contention. Placing applications under fewer leaf switches reduces the number of packets that cross the spine switches in a leaf-spine topology. The proposed scheme is evaluated via discrete-event simulations: we initially evaluate the network locality properties of the system by using virtual-machine traces from a production data center, and we subsequently quantify the performance improvements by simulating an all-to-all pattern for a variety of message sizes over a number of nodes. The results suggest that co-packaged optics form a promising solution for keeping up with bandwidth scaling in future networks. The virtual-machine analysis shows that large-scale applications can be placed under up to 50% fewer first-level switches, while the network analysis shows speedups of up to 7.1, which translates to execution time reductions of up to 26% and 42.7% for applications with communication ratios of 0.3 and 0.5, respectively.

99 GENERAL AND MISCELLANEOUS↗

Emulation automation and model checking

A method of automating emulations is provided. The method comprising collecting publicly available network data over a predefined time interval, wherein the collected network data might comprise structured and unstructured data. Any unstructured data is converted into structured data. The original and converted structured data is stored in a database and compared to known network vulnerabilities. An emulated network is created according to the collected network data and the comparison of the structured data with known vulnerabilities. Virtual machines are created to run on the emulated network. Director programs and guest actor programs are run on the virtual machines, wherein the actor programs imitate real user behavior on the emulated network. The director programs deliver task commands to the guest actor programs to imitate real user behavior. The imitated behavior is presented to a user via an interface.

Urias, Vincent↗

Scaling of Floods With Geomorphologic Characteristics and Precipitation Variability Across the Conterminous United States

Abstract Accurate flood risk assessment requires a comprehensive understanding of flood sensitivity to regional drivers and climate factors. This paper presents the scaling of floods (duration, peak, volume) with geomorphologic characteristics of the basin (i.e., drainage area, slope, elevation) and precipitation patterns (rainfall accumulation, variability). Long‐term daily streamflow observations over the 20th and early 21st centuries from Hydro‐Climatic Data Network streamgages across the conterminous United States are used to create a flood event database based on their flood stage information. Antecedent daily rainfall accumulation and variability corresponding to these floods are computed using Global Historical Climatology Network daily data set. Two Bayesian scaling models are developed, and the spatial organization of scaling exponents is investigated. The baseline model quantifies the scaling of floods to geomorphologic characteristics. The dynamic model quantifies the scaling of floods to antecedent precipitation distribution which is further conditioned on geomorphologic characteristics. Results show that small and low‐elevation basins have a stronger response to antecedent rainfall distribution in amplifying flood peaks, while high‐elevation steeper basins have a lower response for flood duration and volume. The dynamic models demonstrate that there are significant variations in the flood scaling rates, with the largest rates up to 40% and 4.5% for flood duration, 64% and 44% for peak, and 98% and 40% for volume found across the Northeast, Coastal Southeast, and Northwest with intensifying rainfall accumulation and variability, respectively. This study advances flood predictions by better informing the flood attributes in the context of dynamical land‐atmosphere perturbations.

54 ENVIRONMENTAL SCIENCES↗

AUTOPERF

AutoPerf is a collection of modules for low-overhead performance monitoring on HPC systems, the modules collect monitoring data such as MPI data, network counter data etc. Autoperf modules interoperate with the Open source Darshan software (xgitlab.cels.anl.gov/darshan/) as submodules and leverages the Darshan's log recording and analysis framework.

CHUNDURI, DEVISUDHEER KUMAR↗

Line-plane broadcasting in a data communications network of a parallel computer

Methods, apparatus, and products are disclosed for line-plane broadcasting in a data communications network of a parallel computer, the parallel computer comprising a plurality of compute nodes connected together through the network, the network optimized for point to point data communications and characterized by at least a first dimension, a second dimension, and a third dimension, that include: initiating, by a broadcasting compute node, a broadcast operation, including sending a message to all of the compute nodes along an axis of the first dimension for the network; sending, by each compute node along the axis of the first dimension, the message to all of the compute nodes along an axis of the second dimension for the network; and sending, by each compute node along the axis of the second dimension, the message to all of the compute nodes along an axis of the third dimension for the network.

Archer, Charles J↗

Line-plane broadcasting in a data communications network of a parallel computer

Methods, apparatus, and products are disclosed for line-plane broadcasting in a data communications network of a parallel computer, the parallel computer comprising a plurality of compute nodes connected together through the network, the network optimized for point to point data communications and characterized by at least a first dimension, a second dimension, and a third dimension, that include: initiating, by a broadcasting compute node, a broadcast operation, including sending a message to all of the compute nodes along an axis of the first dimension for the network; sending, by each compute node along the axis of the first dimension, the message to all of the compute nodes along an axis of the second dimension for the network; and sending, by each compute node along the axis of the second dimension, the message to all of the compute nodes along an axis of the third dimension for the network.

97 MATHEMATICS AND COMPUTING↗

Machine Learning Based Network Parameter Estimation Using AMI Data

The expansion of distribution power system and the growing penetration of distributed energy resources present new challenges for situational awareness. Calibrating the extended system model with sensor measurements and maintaining the usability is critical for utilities. This paper presents a distribution network parameter estimation (DNPE) approach using machine learning (ML) and metering data that improve the quality of extended distribution power system modeling. The reliability model can improve the ability of endpoint data to be translated into network-level situational awareness in real time and help distribution system operators (DSOs) solve branch flow and voltage problems. In addition, a data analytic and automate processing scheme is proposed to improve the sensor data quality and prevent misleading information. The effectiveness of the proposed method is verified with actual advanced metering infrastructure (AMI) data on a real utility feeder model, while considering the higher penetration of photovoltaic power generation. The test of DNPE and study results are demonstrated in this paper.

Parameter estimation, machine learning, power dist↗

Utah FORGE: 2024 Discrete Fracture Network Model Data

The Utah FORGE 2024 Discrete Fracture Network (DFN) Model dataset provides a set of files representing discrete fracture network modeling for the FORGE site near Milford, Utah. The dataset includes four distinct DFN model file sets, each corresponding to different time frames and modeling approaches in 2024. These models characterize both natural and induced fractures in the geothermal reservoir, which consists of crystalline granitic and metamorphic rock approximately 8,000 feet below the ground surface. The dataset includes a reference DFN model from February 2024 that incorporates planar fractures and well trajectories, as well as upscaled permeability, porosity, compressibility, and storage values on specified grids. Additionally, there are models based on new microseismic (MEQ) data from May and July 2024, including fracture planes fitted to the latest MEQ catalog datasets, tensile fractures from hydraulic stimulation, and an alternative connected DFN for modeling purposes. Coordinate data is provided in both global and local frames, with detailed instructions on the transformations used to align with principal stress orientations. The dataset also includes notes and calculation files for estimating fracture sizes and differences between various fracture sets. There are subfolders for Global Coordinates and Local Coordinates. To move from the global to the local coordinate frame, fractures and wells were a) rotated 20 degrees counterclockwise looking down about the global point (335376.400482041, 4263189.99998761, 250.093546450195) to better align with the principal stresses; and b) translated by (-335408.68, -4263010.9, 1150). Upscaled permeability values using the _XYZ suffix show directions with respect to the global XYZ coordinate frame, while those using the _IJK suffix are aligned with local coordinate frame.

15 GEOTHERMAL ENERGY↗

Non-harmful insertion of data mimicking computer network attacks

Non-harmful data mimicking computer network attacks may be inserted in a computer network. Anomalous real network connections may be generated between a plurality of computing systems in the network. Data mimicking an attack may also be generated. The generated data may be transmitted between the plurality of computing systems using the real network connections and measured to determine whether an attack is detected.

97 MATHEMATICS AND COMPUTING↗

Non-harmful insertion of data mimicking computer network attacks

Non-harmful data mimicking computer network attacks may be inserted in a computer network. Anomalous real network connections may be generated between a plurality of computing systems in the network. Data mimicking an attack may also be generated. The generated data may be transmitted between the plurality of computing systems using the real network connections and measured to determine whether an attack is detected.

97 MATHEMATICS AND COMPUTING↗

Computing Bottleneck Structures at Scale for High-Precision Network Performance Analysis

The Theory of Bottleneck Structures is a recently-developed framework for studying the performance of data networks. It describes how local perturbations in one part of the network propagate and interact with others. This framework is a powerful analytical tool that allows network operators to make accurate predictions about network behavior and thereby optimize performance. Previous work implemented a software package for bottleneck structure analysis, but applied it only to toy examples. In this work, we introduce the first software package capable of scaling bottleneck structure analysis to production-size networks. Here, we benchmark our system using logs from ESnet, the Department of Energy's high-performance data network that connects research institutions in the U.S. Using the previously published tool as a baseline, we demonstrate that our system achieves vastly improved performance, constructing the bottleneck structure graphs in 0.21 s and calculating link derivatives in 0.09 s on average. We also study the asymptotic complexity of our core algorithms, demonstrating good scaling properties and strong agreement with theoretical bounds. These results indicate that our new software package can maintain its fast performance when applied to even larger networks. They also show that our software is efficient enough to analyze rapidly changing networks in real time. Overall, we demonstrate the feasibility of applying bottleneck structure analysis to solve practical problems in large, real-world data networks.

benchmark↗

GNET2: an R package for constructing gene regulatory networks from transcriptomic data

Abstract Motivation The Gene Network Estimation Tool (GNET) is designed to build gene regulatory networks (GRNs) from transcriptomic gene expression data with a probabilistic graphical model. The data preprocessing, model construction and visualization modules of the original GNET software were developed on different programming platforms, which were inconvenient for users to deploy and use. Results Here, we present GNET2, an improved implementation of GNET as an integrated R package. GNET2 provides more flexibility for parameter initialization and regulatory module construction based on the core iterative modeling process of the original algorithm. The data exchange interface of GNET2 is handled within an R session automatically. Given the growing demand for regulatory network reconstruction from transcriptomic data, GNET2 offers a convenient option for GRN inference on large datasets. Availability and implementation The source code of GNET2 is available at https://github.com/jianlin-cheng/GNET2. Supplementary information Supplementary data are available at Bioinformatics online.

59 BASIC BIOLOGICAL SCIENCES↗