Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “large scale network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
NASA NTRS2017

Hyper-Spectral Networking Concept of Operations and Future Air Traffic Management Simulations

The NASA sponsored Hyper-Spectral Communications and Networking for Air Traffic Management (ATM) (HSCNA) project is conducting research to improve the operational efficiency of the future National Airspace System (NAS) through diverse and secure multi-band, multi-mode, and millimeter-wave (mmWave) wireless links. Worldwide growth of air transportation and the coming of unmanned aircraft systems (UAS) will increase air traffic density and complexity. Safe coordination of aircraft will require more capable technologies for communications, navigation, and surveillance (CNS). The HSCNA project will provide a foundation for technology and operational concepts to accommodate a significantly greater number of networked aircraft. This paper describes two of the HSCNA projects technical challenges. The first technical challenge is to develop a multi-band networking concept of operations (ConOps) for use in multiple phases of flight and all communication link types. This ConOps will integrate the advanced technologies explored by the HSCNA project and future operational concepts into a harmonized vision of future NAS communications and networking. The second technical challenge discussed is to conduct simulations of future ATM operations using multi-bandmulti-mode networking and technologies. Large-scale simulations will assess the impact, compared to todays system, of the new and integrated networks and technologies under future air traffic demand.

communication link modeling↗
NASA NTRS1996

Distributed Large Data-Object Environments: End-to-End Performance Analysis of High Speed Distributed Storage Systems in Wide Area ATM Networks

We have developed and deployed a distributed-parallel storage system (DPSS) in several high speed asynchronous transfer mode (ATM) wide area networks (WAN) testbeds to support several different types of data-intensive applications. Architecturally, the DPSS is a network striped disk array, but is fairly unique in that its implementation allows applications complete freedom to determine optimal data layout, replication and/or coding redundancy strategy, security policy, and dynamic reconfiguration. In conjunction with the DPSS, we have developed a 'top-to-bottom, end-to-end' performance monitoring and analysis methodology that has allowed us to characterize all aspects of the DPSS operating in high speed ATM networks. In particular, we have run a variety of performance monitoring experiments involving the DPSS in the MAGIC testbed, which is a large scale, high speed, ATM network and we describe our experience using the monitoring methodology to identify and correct problems that limit the performance of high speed distributed applications. Finally, the DPSS is part of an overall architecture for using high speed, WAN's for enabling the routine, location independent use of large data-objects. Since this is part of the motivation for a distributed storage system, we describe this architecture.

Johnston, William↗
DOE OSTI2022

Peer-to-Peer Energy Trading under Network Constraints Based on Generalized Fast Dual Ascent

We report the wide deployment of renewable energy resources, combined with a more proactive demand-side management, is inducing a new paradigm in both power system operation and electricity market trading, which especially boosts the emergence of the peer-to-peer (P2P) market. A more flexible local market mechanism is highly desirable in response to fast changes in renewable power generation at the distribution network level. Moreover, large-scale implementation of P2P energy trading inevitably affects the secure and economic operation of the distribution network. This paper presents a new P2P electricity trading framework with distribution network security constraints considered using the generalized fast dual ascent method. First, an event-driven local P2P market framework is presented to facilitate short-term or immediate local energy transactions. Then, the sensitivity analysis of nodal voltage and network loss with respect to nodal power injections is used to evaluate the impacts of P2P transactions on the distribution network, which ensures the secure operation of the distribution system. Thereby, the external operational constraints are internalized, and the cost of P2P energy trading can be appropriately allocated in an endogenous way. Moreover, a generalized fast dual ascent method is employed to implement distributed market-clearing efficiently. Finally, numerical results indicate that the proposed model could guarantee secure operation of the distribution system with P2P energy trading, and the solution method enjoys good convergence performance.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Study of interconnect errors, network congestion, and applications characteristics for throttle prediction on a large scale HPC system

Today’s High Performance Computing (HPC) systems contain thousand of nodes which work together to provide performance in the order of petaflops. The performance of these systems depends on various components like processors, memory, and interconnect. Among all, interconnect plays a major role as it glues together all the hardware components in an HPC system. A slow interconnect can impact a scientific application running on multiple processes severely as they rely on fast network messages to communicate and synchronize frequently. Unfortunately, the HPC community lacks a study that explores different interconnect errors, congestion events and applications characteristics on a large-scale HPC system. In our previous work, we process and analyze interconnect data of the Titan supercomputer to develop a thorough understanding of interconnects faults, errors, and congestion events. In this work, we first show how congestion events can impact application performance. We then investigate application characteristics interaction with interconnect errors and network congestion to predict applications encountering congestion with more than 90% accuracy.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1982

The diurnal interaction between convection and peninsular-scale forcing over South Florida

One of the outstanding problems in modern meterology is that of describing in detail the manner in which larger scales of motion interact with, influence and are influenced by successively smaller scales of motion. The present investigation is concerned with a study of the diurnal evolution of convection, the interaction between the peninsular-scale convergence and convection, and the role of the feedback produced by the cloud-scale downdrafts in the maintenance of the convection. Attention is given to the analysis, the diurnal cycle of the network area-averaged divergence, convective-scale divergence, convective mass transports, and the peninsular scale divergence. The links established in the investigation between the large scale (peninsular), the mesoscale (network), and the convective scale (cloud) are found to be of fundamental importance to the understanding of the initiation, maintenance, and decay of deep precipitating convection and to its theoretical parameterization.

Cooper, H. J.↗
DOE OSTI2026

Synthetic Atmospheric River Ensembles Generated by Deep-AR

This dataset contains 35,850 synthetic landfalling atmospheric river (AR) realizations generated by the Deep-AR two-stage deep-learning framework over the Northeast Pacific and U.S. West Coast. The archive contains 25 stochastic ensemble members for each of 1,434 held-out observed seed events. Each synthetic realization is initialized from conditions 48 hours before the corresponding observed AR landfall and is generated autoregressively at 6-hour intervals over a 144-hour period. Deep-AR combines a deterministic residual network (ResNet) that advances the large-scale atmospheric state with a Wasserstein generative adversarial network (WGAN) that produces stochastic, high-resolution fields. Each HDF5 file contains 0.25° gridded synthetic integrated vapor transport components (qu, qv), 10 m wind components (u10, v10), and 6-hour accumulated precipitation on a common 200 × 480 grid. The files also include coordinate and datetime arrays. This dataset supports AR hazard analysis, ensemble-based uncertainty characterization, precipitation-extremes research, and regional stress testing. Synthetic files follow the naming convention deepar.model.YYYYMMDD.HHMMSS.vNN.h5. YYYYMMDD.HHMMSS identifies the UTC initial-condition timestamp, which occurs 48 hours before the diagnosed observed landfall, and vNN identifies the zero-padded ensemble member, ranging from v01 through v25. Each synthetic file can be paired with its corresponding observed file by matching the initial-condition timestamp. The paired observed file follows the naming convention deepar.obs.YYYYMMDD.HHMMSS.h5 and is available in the separately registered oracle/deepar.obs dataset at https://wdh.energy.gov/ds/oracle/deepar.obs (DOI: https://doi.org/10.21947/3377671).

17 WIND ENERGY↗
NASA NTRS2000

From Coexpression to Coregulation: An Approach to Inferring Transcriptional Regulation Among Gene Classes from Large-Scale Expression Data

We provide preliminary evidence that existing algorithms for inferring small-scale gene regulation networks from gene expression data can be adapted to large-scale gene expression data coming from hybridization microarrays. The essential steps are (I) clustering many genes by their expression time-course data into a minimal set of clusters of co-expressed genes, (2) theoretically modeling the various conditions under which the time-courses are measured using a continuous-time analog recurrent neural network for the cluster mean time-courses, (3) fitting such a regulatory model to the cluster mean time courses by simulated annealing with weight decay, and (4) analysing several such fits for commonalities in the circuit parameter sets including the connection matrices. This procedure can be used to assess the adequacy of existing and future gene expression time-course data sets for determining transcriptional regulatory relationships such as coregulation.

Mjolsness, Eric↗
DOE OSTI2022

Hierarchical Distributed Optimal Power Flow of HV and MV Distribution Networks With Continuous and Discrete Devices

With large-scale distributed photovoltaics (PVs) being integrated into distribution networks (DNs), coordinated optimal power flow (OPF) of high voltage (HV) and medium voltage (MV) DNs should be investigated to optimally dispatch the distributed PVs and other network devices. Here, this paper presents a hierarchical distributed OPF method for HV and MV DNs with on-load tap changers, reactive power compensators, feeder switches and distributed PVs. A hierarchical master-slave control architecture is applied to implement coordinated OPF of two-layer DNs. The HV master problem and MV subproblems are transformed into mixed-integer convex problems respectively with second order cone programming and LinDistFlow approximation. Since there is no efficient distributed algorithm to solve such OPF models with integer subproblems, a novel distributed algorithm is proposed in this paper to efficiently solve the hierarchical coordinated OPF model with integer subproblems in a distributed manner. In the proposed algorithm, the coordinated OPF model is solved in a branch-and-bound framework, where in each branch node generalized Benders decomposition (GBD) algorithm is applied to decompose the coordinated OPF model into a master problem and relaxed subproblems and solves them iteratively to get optimal solution. The GBD optimal and feasible cutting planes generated in a branch node are proved to be valid for its descendants. Moreover, three acceleration techniques are introduced into the proposed algorithm to improve computational efficiency. Finally, the effectiveness and accuracy of the proposed method are verified via simulation tests in Jinzhai DNs of China.

42 ENGINEERING↗
NASA NTRS2014

Network Monitor and Control of Disruption-Tolerant Networks

For nearly a decade, NASA and many researchers in the international community have been developing Internet-like protocols that allow for automated network operations in networks where the individual links between nodes are only sporadically connected. A family of Disruption-Tolerant Networking (DTN) protocols has been developed, and many are reaching CCSDS Blue Book status. A NASA version of DTN known as the Interplanetary Overlay Network (ION) has been flight-tested on the EPOXI spacecraft and ION is currently being tested on the International Space Station. Experience has shown that in order for a DTN service-provider to set up a large scale multi-node network, a number of network monitor and control technologies need to be fielded as well as the basic DTN protocols. The NASA DTN program is developing a standardized means of querying a DTN node to ascertain its operational status, known as the DTN Management Protocol (DTNMP), and the program has developed some prototypes of DTNMP software. While DTNMP is a necessary component, it is not sufficient to accomplish Network Monitor and Control of a DTN network. JPL is developing a suite of tools that provide for network visualization, performance monitoring and ION node control software. This suite of network monitor and control tools complements the GSFC and APL-developed DTN MP software, and the combined package can form the basis for flight operations using DTN.

network monitor↗
DOE OSTI2020

Local structure graph models with higher-order dependence

Local structure graph models (LSGMs) describe random graphs and networks as a Markov random field (MRF)—each graph edge has a specified conditional distribution dependent on explicit neighbourhoods of other graph edges. Centered parameterizations of LSGMs allow for direct control and interpretation of parameters for large- and small-scale structures (e.g., marginal means vs. dependence). Here, we extend this parameterization to account for triples of dependent edges and illustrate the importance of centered parameterizations for incorporating covariates and interpreting parameters. Using a MRF framework, common exponential random graph models are also shown to induce conditional distributions without centered parameterizations and thereby have undesirable features. This work attempts to advance graph models through conditional model specifications with modern parameterizations, covariates and higher-order dependencies.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

GIS Visualization of Transportation Energy Consumption

Transportation is witnessing unprecedented transformation, where emerging technologies are disrupting the way we travel. Be it sharing economy, or micro-mobility, the landscape of urban transportation is undergoing a much-needed paradigm shift. In order to capture and model these shifts, researchers need to be agile in their studies of the current and future transportation landscape. The need for agility in turn calls for sophisticated, intuitive, and reliable means to visualize and share transportation data. Travel is inherently spatial, so methods of visualizing transportation information should also be rooted in spatial analysis. Geographic Information Systems (GIS) are the premier technological framework to analyze and display spatial data. GIS tools have been used in the past to depict flow of vehicles, and display of network conditions (speed, congestion, etc.). However, in an ever-changing technological landscape, spanning advancements in vehicle as well as information systems, depicting vehicle flows falls short of providing a comprehensive picture of the impact these advancements have on travel related energy consumption. To address this issue, this research effort presents a web-based mapping application to visualize how travel related energy flows across a city. The application is being developed by National Renewable Energy Laboratory researchers to integrate various transportation energy consumption models within a GIS schema. This has the goal of enabling rapid analysis of energy impacts of the dramatically evolving transportation environment. The application is being developed using a Python framework for ArcGIS Online. Using road network data from the City of Columbus, Ohio and traffic data from CATT Laboratory's Regional Integrated Transportation Information System, transportation energy consumption will be modeled at a macro level. The application aims to provide highly accurate data while still maintaining the flexibility needed to adapt the model when new technologies arise. The methodology presented through this effort is expected to provide insights into how light-duty vehicles use energy on a large scale based on the road network they use. The application will also provide data exports to enable sharing and collaboration. " need for agility in turn calls for sophisticated, intuitive, and reliable means to visualize and share transportation data. Travel is inherently spatial, so methods of visualizing transportation information should also be rooted in spatial analysis. Geographic Information Systems (GIS) are the premier technological framework to analyze and display spatial data. GIS tools have been used in the past to depict flow of vehicles, and display of network conditions (speed, congestion, etc.). However, in an ever-changing technological landscape, spanning advancements in vehicle as well as information systems, depicting vehicle flows falls short of providing a comprehensive picture of the impact these advancements have on travel related energy consumption. To address this issue, this research effort presents a web-based mapping application to visualize how travel related energy flows across a city. The application is being developed by National Renewable Energy Laboratory researchers to integrate various transportation energy consumption models within a GIS schema. This has the goal of enabling rapid analysis of energy impacts of the dramatically evolving transportation environment. The application is being developed using a Python framework for ArcGIS Online. Using road network data from the City of Columbus, Ohio and traffic data from CATT Laboratory's Regional Integrated Transportation Information System, transportation energy consumption will be modeled at a macro level. The application aims to provide highly accurate data while still maintaining the flexibility needed to adapt the model when new technologies arise. The methodology presented through this effort is expected to provide insights into how light-duty vehicles use energy on a large scale based on the road network they use. The application will also provide data exports to enable sharing and collaboration.

33 ADVANCED PROPULSION SYSTEMS↗
NASA NTRS1987

Survey on large scale system control methods

The problem inherent to large scale systems such as power network, communication network and economic or ecological systems were studied. The increase in size and flexibility of future spacecraft has put those dynamical systems into the category of large scale systems, and tools specific to the class of large systems are being sought to design control systems that can guarantee more stability and better performance. Among several survey papers, reference was found to a thorough investigation on decentralized control methods. Especially helpful was the classification made of the different existing approaches to deal with large scale systems. A very similar classification is used, even though the papers surveyed are somehow different from the ones reviewed in other papers. Special attention is brought to the applicability of the existing methods to controlling large mechanical systems like large space structures. Some recent developments are added to this survey.

Mercadal, Mathieu↗
DOE OSTI2023

FPGA Acceleration of GCN in Light of the Symmetry of Graph Adjacency Matrix

Graph Convolutional Neural Networks (GCNs) are widely used to process large-scale graph data. Different from deep neural networks (DNNs), GCNs are sparse, irregular, and unstructured, posing unique challenges to hardware acceleration with regular processing elements (PEs). In particular, the adjacency matrix of a GCN is extremely sparse, leading to frequent but irregular memory access, low spatial/temporal data locality and poor data reuse. Furthermore, a realistic graph usually consists of unstructured data (e.g., unbalanced distributions), creating significantly different processing times and imbalanced workload for each node in GCN acceleration. To overcome these challenges, we propose an end-to-end hardware-software co-design to accelerate GCNs on resource-constrained FPGAs with the features including: (1) A custom dataflow that leverages symmetry along the diagonal of the adjacency matrix to accelerate feature aggregation for undirected graphs. We utilize either the upper or the lower triangular matrix of the adjacency matrix to perform aggregation in GCN to improve data reuse. (2) Unified compute cores for both aggregation and transform phases, with full support to the symmetry-based dataflow. These cores can be dynamically reconfigured to the systolic mode for transformation or as individual accumulators for aggregation in GCN processing. (3) Preprocessing of the graph in software to rearrange the edges and features to match the custom dataflow. This step improves the regularity in memory access and data reuse in the aggregation phase. Moreover, we quantize the GCN precision from FP32 to INT8 to reduce the memory footprint without losing the inference accuracy. We implement our accelerator design in Intel Stratix10 MX FPGA board with HBM2, and demonstrate 1.3x-110.5x improvement in end-to-end GCN latency as compared to the state-of the-art FPGA implementations, on the graph datasets of Cora, Pubmed, Citeseer and Reddit.

Nair, Gopikrishnan R.↗
DOE OSTI2022

A graph neural network (GNN) approach to basin-scale river network learning: the role of physics-based connectivity and data fusion

Abstract. Rivers and river habitats around the world are under sustained pressure from human activities and the changing global environment. Our ability to quantify and manage the river states in a timely manner is critical for protecting the public safety and natural resources. In recent years, vector-based river network models have enabled modeling of large river basins at increasingly fine resolutions, but are computationally demanding. This work presents a multistage, physics-guided, graph neural network (GNN) approach for basin-scale river network learning and streamflow forecasting. During training, we train a GNN model to approximate outputs of a high-resolution vector-based river network model; we then fine-tune the pretrained GNN model with streamflow observations. We further apply a graph-based, data-fusion step to correct prediction biases. The GNN-based framework is first demonstrated over a snow-dominated watershed in the western United States. A series of experiments are performed to test different training and imputation strategies. Results show that the trained GNN model can effectively serve as a surrogate of the process-based model with high accuracy, with median Kling–Gupta efficiency (KGE) greater than 0.97. Application of the graph-based data fusion further reduces mismatch between the GNN model and observations, with as much as 50 % KGE improvement over some cross-validation gages. To improve scalability, a graph-coarsening procedure is introduced and is demonstrated over a much larger basin. Results show that graph coarsening achieves comparable prediction skills at only a fraction of training cost, thus providing important insights into the degree of physical realism needed for developing large-scale GNN-based river network models.

54 ENVIRONMENTAL SCIENCES↗