Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “DATA NETWORKS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 613 records · Page 34

Machine Learning-based False Data Injection Attack Detection and Localization in Power Grids

Cyberattacks on critical infrastructures can be catastrophic and bring nations to their knees. Therefore, detecting these attacks is crucial and challenging. This paper presents a novel approach for detecting and locating cyberattacks affecting an electrical power system. The adversary employs a man-in-the-middle technique to inject false data into the communication between distributed energy resources (DER) and Microgrid Controller (MGC) with the goal of disrupting power delivery. The approach for detection and localization is based on integrating multiple machine learning-based anomaly detection models that combine network traffic data and grid measurements. Experiments are performed to assess the method's performance using a hardware-in-the-loop real-time simulation testbed which includes Modbus TCP/IP communication. Power system topology and operating conditions are based on actual topology and real-world data provided by the Holy Cross Energy utility network. Results confirm that the method can be successfully employed for detecting and localizing cyberattacks.

Leao, Bruno P.↗

Maintenance of Maywood Laboratory Operations Support During Contractor Transition and Replacement of the Project Databases - 16329

In December 2013, Cabrera Services (Cabrera) executed transition of the FUSRAP Maywood Superfund Site remediation project operations, including an on-site accredited environmental lab, from a large business that had established technology infrastructure in place for over a decade which required updating to support the expected remediation scope and client needs. As of November 2015, Cabrera has shipped over 116,000 yd{sup 3} of contaminated soil to disposal facilities. During the transition, the USACE FUSRAP Maywood Laboratory (UFML) continued to support project operations, providing the expected timely and reliable results and maintaining their New Jersey Department of Environmental Protection certification for all parameters of concern, while having to cope with new challenges and additional tasks having the potential to disrupt the laboratory's ability to support project operations. This case study examines the challenges with maintaining a certified radiochemistry laboratory and an aggressive remediation schedule during a period of prime contractor transition which included a complete IT infrastructure overhaul, and conversion of the existing Microsoft Access laboratory database and project Electronic Data Management System (EDMS) to a customized commercial solution. Challenges: I. Procurement of a new database subcontractor and working closely with them to achieve the following: a. Migration of all environmental (including radiological, chemical, and quality control), health and safety, and spatial data as well as reports, queries, and analytical instrument software processes from a MS Access data management system to a SQL server system; and b. Verification that the functionality and data integrity of the databases remained intact. II. The onsite laboratory was changed over to a new onsite network and the following tasks were completed with little or no interruption to site operations by January 10, 2014: i. Installation of new instrument personal computers (PCs) and making them fully operational; ii. Configuration of the lab domain and server to host the LIMS (replaced all servers/domain/workstations with Windows 7 and Server 2012 machines). iii. Networking of the laboratory building into the rest of the project site via fiber optic cabling across the site. iv. Upgrade of the wide area network (WAN) data connection to high-speed Fiber Optic service; v. Upgrade of the telephones into voice over internet protocol (VOIP) phones. vi. Splitting of the database into front ends and a centralized back-end hosted on the new server to allow simultaneous access for fixes and improvements. III. Phone service interruption; phone systems were operational by January 10, 2014. Additional Tasks: I. Revision of the USACE FUSRAP Maywood Laboratory (UFML) Quality Systems Manual Rev. 1 Submitted 01/16/14; II. Writing of the Uniform Federal Policy Quality Assurance Project Plan Rev. 1 (UFP-QAPP) submitted 1/21/14. This document replaced the prior project data quality plan, the Chemical Data Quality Management Plan. III. Support of USACE communications to the New Jersey Department of Environmental Protection (NJDEP) on lab management and personnel transition to Cabrera. This work was important since it ensured a smooth transition and shows how critical project records infrastructure such as databases, can be upgraded with minimal interruption of project support. Such work may provide benefit to other similar projects, especially long term projects where database infrastructure upgrades may be beneficial in terms of improved efficiencies in data recording and transfer, and improved security in data storage. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Resource Management for Processing Wide Area Data Streams on Supercomputers

Modern scientific instruments generate enormous amount of data. Typically, the data collected from the instruments are stored in one or more files that are then moved to a distant supercomputer for processing. The final results are sent back to the user. In order to make effective use of the time on expensive instruments, experimenters want to process the data as they are generated. They want to stream the data from instruments' memory directly to a supercomputer's memory for analysis. Since the compute nodes in a supercomputer are not connected directly to the wide area network, the data streams need to be passed through intermediate gateway nodes. As opposed to the best effort file transfers, data streaming applications require resources at a specific time for a specific period. In this paper, we present a system model for enabling data streaming through gateway nodes and an algorithm to efficiently allocate gateway node resources along with compute nodes. We evaluate the algorithm using real-world traces on the Chameleon Cloud. The results show that our system can schedule compute and gateway resources efficiently for streaming analysis.

data streaming↗

Impact of transportation network companies on urban congestion: Evidence from large-scale trajectory data

We collect vehicle trajectory data from major transportation network companies (TNCs) in New York City (NYC) in 2017 and 2019, and we use the trajectory data to understand how the growth of TNCs has impacted traffic congestion and emission in urban areas. By mining the large-scale trajectory data and conduct the case study in NYC, we confirm that the rise of TNC is the major contributing factor that makes urban traffic congestion worse. From 2017 to 2019, the number of for-hire vehicles (FHV) has increased by over 48% and served 90% more daily trips. These resulted in an average citywide speed reduction of 22.5% on weekdays, and the average speed in Manhattan decreased from 11.76 km/h in April 2017 to 9.56 km/h in March 2019. The heavier traffic congestion may have led to 136% more NOx, 152% more CO and 157% more HC emission per kilometer traveled by the FHV sector. Our results show that the traffic condition is consistently worse across different times of the day and at different locations in NYC. And we build the connection between the number of available FHVs and the reduction in travel speed between the two years of data and explain how the rise of TNC may impact traffic congestion in terms of moving speed and congestion time. Our findings provide valuable insights for different stakeholders and decision-makers in framing regulation and operation policies towards more effective and sustainable urban mobility.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Field surveying data of low-cost networked flood sensors in southeast Texas

Floods are common natural disasters worldwide and pose substantial risks to life, property, food production, and natural resources. Effective measures for flood mitigation and warning are essential. Southeast Texas is still at significant risk of flooding, and Lamar University is assisting the region with asset management of a flood sensor network for flooding events. This network provides real-time water stage information. Lamar University developed a survey program to measure elevation and coordinates at each sensor site location to make this data more useful for flood monitoring and mapping. This paper overviews the measurement of the elevation and coordinates of 74 networked flood sensors and various flood stage thresholds at critical points that flood decision-makers can use for reference at each site. In the first phase of this program, these sensors were deployed throughout a 7-county region spanning nearly 6,000 square miles in Southeast Texas. The latitude and longitude of the sensors and their elevations were determined using survey-grade Global Navigation Satellite System (GNSS) technology. Various Continually Operating Reference Stations (CORS) were utilized for post-processing to achieve sub-inch resolution. The flood stage thresholds, water level sensors elevation, and the elevations and positions of other critical surrounding points are viewable to the public through two online repositories and a web-based sensor management dashboard. The data is used to aid with decisions related to road closures or modeling efforts by mitigation decision-makers, emergency managers, and the public, including the Texas Department of Transportation, Houston Transtar, the National Weather Service, and the Sabine River Authority of Texas (SRA).

54 ENVIRONMENTAL SCIENCES↗

Sivers extraction with Neural Network

Pseudo-data with simulated experimental errors can be generated to train an ensemble of Artificial Neural Networks (ANN) implemented on a regression to extract Transverse Momentum-dependent Distributions (TMDs). A preliminary analysis is presented on the reliability in extraction of the Sivers function imposed in the pseudo-data given the bounds on the experimental errors, data sparsity, and complexity of phase-space.

Fernando, Ishara↗

Propagating synthetic populations with dynamic Bayesian networks: a framework for long-horizon demographic forecasting

This study presents a dynamic demographic microsimulator using dynamic Bayesian networks to forecast long–term changes in household and individual life events. Leveraging longitudinal Panel Study of Income Dynamics (PSID) data, two networks for individuals and households were modeled to simulate transitions in employment, income, education, marriage, childbirth, leaving the parental home, home ownership, mortality, and household formation or dissolution. Across 1,000 simulation runs spanning 24 years, household–level outcomes remain highly accurate and individual–level predictions reasonable. Although accuracy naturally declines with projection horizon, performance remains promising at both levels. This study addresses a key limitation of existing population synthesis models, which typically generate only a single static snapshot of the population. In conclusion, by introducing a framework that propagates cross-sectional outputs into the future, the microsimulator enables the tracking of demographic evolution over time, enhances realism in population-based simulations, and supplies credible inputs to agent-based travel demand models.

Demographic modeling↗

Driver Identification Midyear Report

First, we create a profile for each authorized driver based on their existing driving data. We then train a machine learning model on the driving data from this profile, yielding an individualized model for each driver. Finally during a drive, we pass the Controller Area Network (CAN) data to the model and authen ticate the driver’s identity in real-time. This verification or lack thereof could be used to alert supervisors of threats to their drivers or transported materials. Deviations from their normal driving behavior could indicate high-risk situations, medical events, or even insider threats.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

Neural network based emulation of galaxy power spectrum covariances: A reanalysis of BOSS DR12 data

We train neural networks to quickly generate redshift-space galaxy power spectrum covariances from a given parameter set (cosmology and galaxy bias). This covariance emulator utilizes a combination of traditional fully connected network layers and transformer architecture to accurately predict covariance matrices for the high redshift, north galactic cap sample of the BOSS DR12 galaxy catalog. We run simulated likelihood analyses with emulated and brute-force computed covariances, and we quantify the network’s performance via two different metrics: (1) difference in Χ 2 and (2) likelihood contours for simulated BOSS DR 12 analyses. We find that the emulator returns excellent results over a large parameter range. We then use our emulator to perform a reanalysis of the BOSS HighZ NGC galaxy power spectrum, and find that varying covariance with cosmology along with the model vector produces Ω m = $0.27⁢6$$^{+0.013}_{–0.015}$, H 0 = 70.2 ± 1.9 km/s/Mpc, and σ 8 = $0.67⁢4$$^{+0.058}_{–0.077}$. These constraints represent an average 0.46⁢σ shift in best-fit values and a 5% increase in constraining power compared to fixing the covariance matrix (Ω m = 0.293 ± 0.017, H 0 = 70.3 ± 2.0 km/s/Mpc, σ 8 = $0.70⁢2$$^{+0.063}_{–0.075}$). As a result, this work demonstrates that emulators for more complex cosmological quantities than second-order statistics can be trained over a wide parameter range at sufficiently high accuracy to be implemented in realistic likelihood analyses.

79 ASTRONOMY AND ASTROPHYSICS↗

A Gaussian graphical model approach to climate networks

Distinguishing between direct and indirect connections is essential when interpreting network structures in terms of dynamical interactions and stability. When constructing networks from climate data the nodes are usually defined on a spatial grid. The edges are usually derived from a bivariate dependency measure, such as Pearson correlation coefficients or mutual information. Thus, the edges indistinguishably represent direct and indirect dependencies. Interpreting climate data fields as realizations of Gaussian Random Fields (GRFs), we have constructed networks according to the Gaussian Graphical Model (GGM) approach. In contrast to the widely used method, the edges of GGM networks are based on partial correlations denoting direct dependencies. Furthermore, GRFs can be represented not only on points in space, but also by expansion coefficients of orthogonal basis functions, such as spherical harmonics. This leads to a modified definition of network nodes and edges in spectral space, which is motivated from an atmospheric dynamics perspective. We construct and analyze networks from climate data in grid point space as well as in spectral space, and derive the edges from both Pearson and partial correlations. Network characteristics, such as mean degree, average shortest path length, and clustering coefficient, reveal that the networks posses an ordered and strongly locally interconnected structure rather than small-world properties. Despite this, the network structures differ strongly depending on the construction method. Straightforward approaches to infer networks from climate data while not regarding any physical processes may contain too strong simplifications to describe the dynamics of the climate system appropriately.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A Multi-Branch Decoder Network Approach to Adaptive Temporal Data Selection and Reconstruction for Big Scientific Simulation Data

A key challenge in scientific simulation is that the simulation outputs often require intensive I/O and storage space to store the results for effective post hoc analysis. This article focuses on a quality-aware adaptive temporal data selection and reconstruction problem where the goal is to adaptively select simulation data samples at certain key timesteps in situ and reconstruct the discarded samples with quality assurance during post hoc analysis. This problem is motivated by the limitation of current solutions that a significant amount of simulation data samples are either discarded or aggregated during the sampling process, leading to inaccurate modeling of the simulated phenomena. Two unique challenges exist: 1) the sampling decisions have to be made in situ and adapted to the dynamics of the complex scientific simulation data; 2) the reconstruction error must be strictly bounded to meet the application requirement. To address the above challenges, we develop DeepSample , an error-controlled convolutional neural network framework, that jointly integrates a set of coherent multi-branch deep decoders to effectively reconstruct the simulation data with rigorous quality assurance. The results on two real-world scientific simulation applications show that DeepSample significantly outperforms other state-of-the-art methods on both sampling efficiency and reconstructed simulation data quality.

Zhang, Yang↗

Exploring DataVortex Systems for Irregular Applications

Emerging applications for data analytics and knowledge discovery typically have irregular or unpredictable communication patterns that do not scale well on parallel systems designed for traditional bulk-synchronous HPC applications. New network architectures that focus on minimizing (short) message latencies, rather than maximizing (large) transfer bandwidths, are emerging as possible alternatives to better support those applications with irregular communication patterns. We explore a system based upon one such novel network architecture, the Data Vortex interconnection network, and examine how this system performs by running benchmark code written for the Data Vortex network, as well as a reference MPI-over- Infiniband implementation, on the same cluster.

Data Vortex, High-Performance Computing, Networkin↗

Malicious Cyber Activity Detection using Zigzag Persistence

In this study we synthesize zigzag persistence from topological data analysis with autoencoder-based approaches to detect malicious cyber activity, and derive analytic insights. Cybersecurity aims to safeguard computers, networks, and servers from various forms of malicious attacks, including network damage, data theft, and activity monitoring. We focus on the cybersecurity domain and investigate the detection of malicious activity using log data. We consider the dynamics of the log data and explore the changing topology of a hypergraph representation of this data to gain insights into the underlying activity. These hypergraphs capture complex interactions between processes, together with their temporal information. To study the changing topology we use zigzag persistence, which captures how topological features persist at multiple dimensions over time. We observe that this detects malicious activity in a cyber data set. To automate this detection we implement an autoencoder trained on a vectorization of the resulting zigzag persistence barcodes. Our experimental results demonstrate the effectiveness of the autoencoder in detecting malicious activity. Overall, this study highlights the potential of zigzag persistence and its combination with temporal hypergraphs for analyzing cybersecurity log data and detecting malicious behavior.

hypergraphs, temporal hypergraph, topological data↗

SODAs: sparse optimization for the discovery of differential and algebraic equations

Differential-algebraic equations (DAEs) integrate ordinary differential equations (ODEs) with algebraic constraints, providing a fundamental framework for developing models of dynamical systems characterized by time-scale separation, conservation laws and physical constraints. While sparse optimization has revolutionized model development by allowing data-driven discovery of parsimonious models from a library of possible equations, existing approaches for dynamical systems assume DAEs can be reduced to ODEs by eliminating variables before model discovery. This assumption limits the applicability of such methods for DAE systems with unknown constraints and time scales. We introduce sparse optimization for differential-algebraic systems (SODAs), a data-driven method for the identification of DAEs in their explicit form. By discovering the algebraic and dynamic components sequentially without prior identification of the algebraic variables, this approach leads to a sequence of convex optimization problems. It has the advantage of discovering interpretable models that preserve the structure of the underlying physical system. To this end, SODAs improves since SODAs is singular numerical stability when handling high correlations between library terms, caused by near-perfect algebraic relationships, by iteratively refining the conditioning of the candidate library. We demonstrate the performance of our method on biological, mechanical and electrical systems, showcasing its robustness to noise in both simulated time series and real-time experimental data.

DAE↗

Let’s get our hands dirty: a comprehensive evaluation of DAQDB, key-value store for petascale hot storage

Data acquisition systems are a key component for successful data taking in any experiment. The DAQ is a complex distributed computing system and coordinates all operations, from the data selection stage of interesting events to storage elements. For the High Luminosity upgrade of the Large Hadron Collider, the experiments at CERN need to meet challenging requirements to record data with a much higher occupancy in the detectors. The DAQ system will receive and deliver data with a significantly increased trigger rate, one million events per second, and capacity, terabytes of data per second. An effective way to meet these requirements is to decouple real-time data acquisition from event selection. Data fragments can be temporarily stored in a large distributed key-value store. Fragments belonging to the same event can be then queried on demand, by the data selection processes. Implementing such a model relies on a proper combination of emerging technologies, such as persistent memory, NVMe SSDs, scalable networking, and data structures, as well as high performance, scalable software. In this paper, we present DAQDB (Data Acquisition Database) — an open source implementation of this design that was presented earlier, with an extensive evaluation of this approach, from the single node to the distributed performance. Furthermore, we complement our study with a description of the challenges faced and the lessons learned while integrating DAQDB with the existing software framework of the ATLAS experiment.

Abed Abud, Adam↗

Seismicity Monitoring in North-Central New Mexico by the Los Alamos Seismic Network

The initial seismic stations of the Los Alamos Seismic Network (LASN) were installed in 1973, as a part of research on monitoring of nuclear testing. The extent of the network rapidly expanded by the late 1970s. By the middle 1980s, spatial coverage of the network was drastically reduced, due to a loss of funding. Since then, however, it has been possible to expand the coverage of the network with additional stations and to slowly upgrade network equipment to match contemporary instrument standards. These improvements will make it possible to keep recording the network’s data and to initiate real-time exchange of data with other institutions to improve earthquake monitoring throughout New Mexico and neighboring states. During more than 40 yr of operation, the network has provided a slow but steady increase in the volume of earthquake seismograms available to study seismicity and tectonics in north-central New Mexico. The current network covers an area from the Valles Caldera on the west, to within the Rio Grande rift on the east. LASN has yielded locations for about 900 earthquakes in north-central New Mexico between 1973 and 2013 (the most recent locations available). Epicenters of these have a complex pattern, with some that can be attributed to the deformation of the rift, though most are spread in areas west of the rift. A lack of seismicity in and near the Valles Caldera reinforces an earlier observation, based on far fewer earthquakes, which first called attention to this paucity of seismicity and attributed it to ductile deformation resulting from elevated crustal temperatures.

58 GEOSCIENCES↗

Predicting Drug Effects from High-dimensional Asymmetric Drug Data Sets using Graph Neural Networks: A Comprehensive Analysis of Multi-target Drug Effect Prediction

Graph neural networks (GNNs) have emerged as one of the most effective Machine learning (ML) techniques for drug effect prediction from drug molecular graphs. Despite having immense potential, GNN models lack performance when using data sets that contain high dimensional asymmetrically co-occurrent drug effects as targets with complex correlations between them. Training individual learning models for each drug effect and incorporating every prediction result for a wide spectrum of drug effects is beyond practicality. Such an implication provides a testbed to address this challenge as multi-target prediction problems, aiming to predict all drug effects at a time. We develop standard and hybrid graph neural networks (GNNs)to perform two separate tasks that are multi-regression for continuous values and multi-label classification for categorical values contained in our data sets. Since this step makes the target data even more sparse and introduces asymmetric label co-occurrence, the learning of multi-label classification models becomes difficult and heavily impacts the GNN's performance. To address these challenges, we propose a new data oversampling technique to improve multi-label classification performances on all the given imbalanced molecular graph data sets. Using the technique, we improve the data imbalance ratio of the drug effects better than before while protecting the data set's integrity. Finally, we evaluate multi-label classification performance using the best-performant hybrid GNN model on all the oversampled data sets obtained from the proposed oversampling technique. These results outperform those of other ML models including GNN models when they are trained on the original data sets or oversampled data sets using MLSMOTE (a well-known oversampling technique) in all evaluation metrics precision, recall, and F1 score by a significant margin.

Bose, Avishek [ORNL]↗

Cyber Infrastructure for the Smart Electric Grid

As electric power systems undergo a transformative upgrade with the integration of advanced technologies to enable the smarter electric grid, professionals who work in the area require a new understanding of the evolving complexity of the grid. Cyber Infrastructure for the Smart Electric Grid delivers a comprehensive overview of the fundamental principles of smart grid operation and control, smart grid technologies, including sensors, communication networks, computation, data management, and cyber security, and the interdependencies between the component technologies on which a smart grid's security depends. The book offers readers the opportunity to critically analyze the smart grid infrastructure needed to sense, communicate, compute, and control in a secure way.

communication networks↗