SEARCH · Engineering Papers

Results for “STORAGE AND NETWORKING SYSTEMS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

All sources NASA NTRS DOE OSTI

DOE OSTI2016

Optimizing End-to-End Big Data Transfers over Terabits Network Infrastructure

While future terabit networks hold the promise of significantly improving big-data motion among geographically distributed data centers, significant challenges must be overcome even on today's 100 gigabit networks to realize end-to-end performance. Multiple bottlenecks exist along the end-to-end path from source to sink, for instance, the data storage infrastructure at both the source and sink and its interplay with the wide-area network are increasingly the bottleneck to achieving high performance. In this study, we identify the issues that lead to congestion on the path of an end-to-end data transfer in the terabit network environment, and we present a new bulk data movement framework for terabit networks, called LADS. LADS exploits the underlying storage layout at each endpoint to maximize throughput without negatively impacting the performance of shared storage resources for other users. LADS also uses the Common Communication Interface (CCI) in lieu of the sockets interface to benefit from hardware-level zero-copy, and operating system bypass capabilities when available. It can further improve data transfer performance under congestion on the end systems using buffering at the source using flash storage. With our evaluations, we show that LADS can avoid congested storage elements within the shared storage resource, improving input/output bandwidth, and data transfer rates across the high speed networks. We also investigate the performance degradation problems of LADS due to I/O contention on the parallel file system (PFS), when multiple LADS tools share the PFS. We design and evaluate a meta-scheduler to coordinate multiple I/O streams while sharing the PFS, to minimize the I/O contention on the PFS. Finally, with our evaluations, we observe that LADS with meta-scheduling can further improve the performance by up to 14 percent relative to LADS without meta-scheduling.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Threat Hunt Guide for BESS Environments

The rapid digitalization of the electric grid - driven by the integration of inverter-based resources (IBRs), battery energy storage systems (BESS), and advanced grid control platforms - has significantly enhanced grid efficiency, visibility, and flexibility. However, this evolution also introduces new cybersecurity risks, particularly through supply chain dependencies and operational blind spots at the grid edge. To address these challenges, Idaho National Laboratory (INL), through the Department of Energy (DOE) Office of Cybersecurity, Energy Security, and Emergency Response (CESER) Rapid Risk initiative, conducted a series of rapid risk assessment engagements with energy organizations across the United States. Drawing on lessons learned from these engagements, INL developed the following threat hunting guide for asset owners and operators (AOOs) to enhance their cybersecurity visibility within BESS and IBR systems. The guide demonstrates how to use passive network monitoring to baseline device behavior, detect adversarial activity, and investigate anomalies without disrupting operations. By implementing these practices, energy sector stakeholders can improve coordination between cybersecurity and operations teams and strengthen the resilience of distributed energy resources (DERs) within the modern power grid. Prior to implementing any network monitoring, packet capture, or threat hunting activity described in this guide, AOOs are strongly advised to review applicable governance frameworks, legal requirements, and organizational policies. This guide is intended for informational and educational purposes only. It does not replace compliance with any federal, state, or local cybersecurity mandates or industry standards. Implementation of described configurations, technologies, or analytic workflows is performed at the discretion and responsibility of the asset owner and operator.

25 - ENERGY STORAGE↗

DOE OSTI2021

Reinforcement Learning for Distribution Grid Optimization (PyCIGAR) v0.1

PyCIGAR is a python software package that merges off-the-shelf reinforcement learning libraries (RLLib and Ray) with electric power distribution system simulation tools (OpenDSS and a custom power flow solver built by LBL). PyCIGAR enables the training of neural networks to optimize the behavior of different components in the electric distribution grid, such as control systems in photovoltaic rooftop solar inverters and electric battery storage systems. The software package has been used to train neural networks to update settings in photovoltaic rooftop solar inverter control systems to mitigate cyber attacks on other solar photovoltaic rooftop devices.

Arnold, Daniel↗

DOE OSTI2025

Recent Experience with the CMS Data Management System

The CMS[1] experiment manages a large-scale data infrastructure, currently handling over 200 PB of disk and 500 PB of tape storage and transferring more than 1 PB of data per day on average between various WLCG[2] sites. Utilizing Rucio[3] for high-level data management, FTS[4] for data transfers, and a variety of storage and network technologies at the sites, CMS confronts inevitable challenges due to the system’s growing scale and evolving nature. Key challenges include managing transfer and storage failures, optimizing data distribution across different storages based on production and analysis needs, implementing necessary technology upgrades and migrations, and efficiently handling user requests. The data management team has established comprehensive monitoring to supervise this system and has successfully addressed many of these challenges. The team’s efforts aim to ensure data availability and protection, minimize failures and manual interventions, maximize transfer throughput and resource utilization, and provide reliable user support. This paper details the operational experience of CMS with its data management system in recent years, focusing on the encountered challenges, the effective strategies employed to overcome them and the ongoing challenges as we prepare for future demands.

Öztürk, Hasan [CERN]↗

DOE OSTI2024

Software defined networking moving target defense honeypot

Disclosed herein are systems, methods, and storage media for thwarting cyber-attacks and data theft. A computing system receives packets and compares with a configuration resource. The computing system determines that the packet does not match the configuration resource and transmits a packet to a decoy environment via an SDN switch. The decoy environment is configured to generate time-out, service unavailable, or restricted access messages. In some embodiments, the computing system determines the packet does match the configuration and transmits the packet to a production network via an SDN switch. The SDN switch is communicatively connected to the decoy environment through a first channel and communicatively connected to a production environment through a separate second channel. The computing system is further configured to create and transmit to the SDN switch, rules to manage transmitting the packet to either the decoy environment or the production environment.

Lyle, Joshua A.↗

DOE OSTI2019

Learning concave-convex profiles of data transport over dedicated connections

Dedicated data transport infrastructures are increasingly being deployed to support distributed big-data and high-performance computing scenarios. These infrastructures employ data transfer nodes that use sophisticated software stacks to support network transport among sites, which often house distributed file and storage systems. Throughput measurements collected over such infrastructures for a range of round trip times (RTTs) reflect the underlying complex end-to-end connections, and have revealed dichotomous throughput profiles as functions of RTT. In particular, concave regions of throughput profiles at lower RTTs indicate near-optimal performance, and convex regions at higher RTTs indicate bottlenecks due to factors such as buffer or credit limits. We present a machine learning method that explicitly infers these concave and convex regions and transitions between them using sigmoid functions. We also provide distribution-free confidence estimates for the generalization error of these concave-convex profile estimates. Throughput profiles for data transfers over 10 Gbps connections with 0–366 ms RTT provide important performance insights, including the near optimality of transfers performed with the XDD tool between XFS filesystems, and the performance limits of wide-area Lustre extensions using LNet routers. A direct application of generic machine learning packages does not adequately highlight these critical performance regions or provide as precise confidence estimates.

Rao, Nageswara↗

DOE OSTI2017

Water NSTF Design, Instrumentation, and Test Planning

The following report serves as a formal introduction to the water-based Natural convection Shutdown heat removal Test Facility (NSTF) program at Argonne. Since 2005, this US Department of Energy (DOE) sponsored program has conducted large scale experimental testing to generate high-quality and traceable validation data for guiding design decisions of the Reactor Cavity Cooling System (RCCS) concept for advanced reactor designs. The most recent facility iteration, and focus of this report, is the operation of a 1/2 scale model of a water-RCCS concept. Several features of the NSTF prototype align with the conceptual design that has been publicly released for the AREVA 625 MWt SC-HTGR. The design of the NSTF also retains all aspects common to a fundamental boiling water thermosiphon, and thus is well poised to provide necessary experimental data to advance basic understanding of natural circulation phenomena and contribute to computer code validation. Overall, the NSTF program operates to support the DOE vision of aiding US vendors in design choices of future reactor concepts, advancing the maturity of codes for licensing, and ultimately developing safe and reliable reactor technologies. In this report, the top-level program objectives, testing requirements, and unique considerations for the water cooled test assembly are discussed, and presented in sufficient depth to support defining the program’s overall scope and purpose. A discussion of the proposed 6-year testing program is then introduced, which outlines the specific strategy and testing plan for facility operations. The proposed testing plan has been developed to meet the toplevel objective of conducting high-quality test operations that span across a broad range of single- and two-phase operating conditions. Details of characterization, baseline test cases, accident scenario, and parametric variations are provided, including discussions of later-stage test cases that examine the influence of geometric variations and off-normal configurations. The facility design follows, including as-built dimensions and specifications of the various mechanical and liquid systems, design choices for the test section, water storage tank, and network piping. Specifications of the instrumentation suite are then presented, along with specific information on performance windows, measurement uncertainties, and installation locations. Finally, descriptions of the control systems and heat removal networks are provided, which have been engineered to support precise quantification of energy balances and facilitate well-controlled test operations.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

DOE OSTI2011

A Virtualized Computing Platform For Fusion Control Systems

The National Ignition Facility (NIF) at the Lawrence Livermore National Laboratory is a stadium-sized facility that contains a 192-beam, 1.8-Megajoule, 500-Terawatt, ultraviolet laser system together with a 10-meter diameter target chamber with room for multiple experimental diagnostics. NIF is the world's largest and most energetic laser experimental system, providing a scientific center to study inertial confinement fusion (ICF) and matter at extreme energy densities and pressures. NIF's laser beams are designed to compress fusion targets to conditions required for thermonuclear burn, liberating more energy than required to initiate the fusion reactions. 2,500 servers, 400 network devices and 700 terabytes of networked attached storage provide the foundation for NIF's Integrated Computer Control System (ICCS) and Experimental Data Archive. This talk discusses the rationale & benefits for server virtualization in the context of an operational experimental facility, the requirements discovery process used by the NIF teams to establish evaluation criteria for virtualization alternatives, the processes and procedures defined to enable virtualization of servers in a timeframe that did not delay the execution of experimental campaigns and the lessons the NIF teams learned along the way. The virtualization architecture ultimately selected for ICCS is based on the Open Source Xen computing platform and 802.1Q open networking standards. The specific server and network configurations needed to ensure performance and high availability of the control system infrastructure will be discussed.

42 ENGINEERING↗

DOE OSTI2013

Performance and scalability evaluation of the Ceph parallel file system. In: PDSW '13 Proceedings of the 8th Parallel Data Storage Workshop

Ceph is an emerging open-source parallel distributed file and storage system. By design, Ceph leverages unreliable commodity storage and network hardware, and provides reliability and fault-tolerance via controlled object placement and data replication. This paper presents our file and block I/O performance and scalability evaluation of Ceph for scientific high-performance computing (HPC) environments. Our work makes two unique contributions. First, our evaluation is performed under a realistic setup for a large-scale capability HPC environment using a commercial high-end storage system. Second, our path of investigation, tuning efforts, and findings made direct contributions to Ceph's development and improved code quality, scalability, and performance. These changes should benefit both Ceph and the HPC community at large.

Wang, Feiyi↗

DOE OSTI2017

Control and Information Systems for the National Ignition Facility

Orchestration of every National Ignition Facility (NIF) shot cycle is managed by the Integrated Computer Control System (ICCS), which uses a scalable software architecture running code on more than 1950 front-end processors, embedded controllers, and supervisory servers. The ICCS operates laser and industrial control hardware containing 66 000 control and monitor points to ensure that all of NIF’s laser beams arrive at the target within 30 ps of each other and are aligned to a pointing accuracy of less than 50 μm root-mean-square, while ensuring that a host of diagnostic instruments record data in a few billionths of a second. NIF’s automated control subsystems are built from a common object-oriented software framework that distributes the software across the computer network and achieves interoperation between different software languages and target architectures. A large suite of business and scientific software tools supports experimental planning, experimental setup, facility configuration, and post-shot analysis. Standard business services using open-source software, commercial workflow tools, and database and messaging technologies have been developed. An information technology infrastructure consisting of servers, network devices, and storage provides the foundation for these systems. Thus, this work is an overview of the control and information systems used to support a wide variety of experiments during the National Ignition Campaign.

42 ENGINEERING↗

DOE OSTI2019

Evaluating Burst Buffer Placement in HPC Systems

Burst buffers (BBs) are increasingly exploited in contemporary supercomputers to bridge the performance gap between compute and storage systems. The design of BBs, particularly the placement of these devices and the underlying network topology, impacts both performance and cost. As the cost of other components such as memory and accelerators is increasing, it is becoming more important that HPC centers provision BBs tailored to their workloads.This work contributes a provisioning system to provide accurate, multi-tenant simulations that model realistic application and storage workloads from HPC systems. The framework aids HPC centers in modeling their workloads against multiple network and BB configurations rapidly. In experiments with our framework, we provide a comparison of representative Oak Ridge Leadership Computing Facility (OLCF) I/O workloads against multiple BB designs. We analyze the impact of these designs on latency, I/O phase lengths, contention for network and storage devices, and choice of network topology.

Khetawat, Harsh↗

DOE OSTI2018

Planning of distributed data production for High Energy and Nuclear Physics

Modern experiments in High Energy and Nuclear Physics heavily rely on distributed computations using multiple computational facilities across the world. One of the essential types of the computations is a distributed data production where petabytes of raw files from a single source has to be processed once (per production campaign) using thousands of CPUs at distant locations and the output has to be transferred back to that source. The data distribution over a large system does not necessary match the distribution of storage, network and CPU capacity. Therefore, bottlenecks may appear and lead to increased latency and degraded performance. In this paper we propose a new scheduling approach for distributed data production which is based on the network flow maximization model. In our approach a central planner defines how much input and output data should be transferred over each network link in order to maximize the computational throughput. Such plans are created periodically for a fixed planning time interval using up-to-date information on network, storage and CPU resources. The centrally created plans are executed in a distributed manner by dedicated services running at participating sites. In conclusion, our simulations based on the log records from the data production framework of the experiment STAR (Solenoid Tracker at RHIC) have shown that the proposed model systematically provides a better performance compared to the simulated traditional techniques.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2023

RADIANCE Executive Summary

Microgrids are gaining attention from organizations and cities because of their potential reliability and resilience benefits, especially in remote geographic areas: Microgrids can provide local power during emergencies, and they can reduce the costs of imported fuel by reducing overall fuel use. But the complexities and novelties of microgrid technologies are often barriers to deployment that require testing and validation to overcome. This report describes the testing, validation, and deployment of microgrid technologies in Cordova, Alaska, completed through the U.S. Department of Energy (DOE) Grid Modernization Laboratory Consortium (GMLC) project Resilient Alaskan Distribution system Improvements using Automation, Network analysis, Control, and Energy storage (RADIANCE).

ARIES↗

DOE OSTI2015

Guest Editorial Introduction to the Special Issue on 'Advanced Signal Processing Techniques and Telecommunications Network Infrastructures for Smart Grid Analysis, Monitoring, and Management'

Electrical power systems are undergoing a radical change in structure, components, and operational paradigms, and are progressively approaching the new concept of smart grids (SGs). Future power distribution systems will be characterized by the simultaneous presence of various distributed resources, such as renewable energy systems (i.e., photovoltaic power plant and wind farms), storage systems, and controllable/non-controllable loads. Control and optimization architectures will enable network-wide coordination of these grid components in order to improve system efficiency and reliability and to limit greenhouse gas emissions. In this context, the energy flows will be bidirectional from large power plants to end users and vice versa; producers and consumers will continuously interact at different voltage levels to determine in advance the requests of loads and to adapt the production and demand for electricity flexibly and efficiently also taking into account the presence of storage systems.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2022

Subsurface Hydrogen and Natural Gas Storage (State of Knowledge and Research Recommendations Report)

Global efforts to decrease carbon dioxide (CO 2 ) emissions will result in increased reliance on renewable energy sources. One of the most significant obstacles to implementing this transition is the need for energy storage to compensate for varying, and sometimes intermittent, production rates and ever-increasing demand. An attractive solution is to use surplus energy to produce hydrogen gas (H 2 ), which can then be used to produce energy on demand without emitting CO 2 . This approach would require a vast H 2 network linking generators with consumers via transmission systems. Additionally, H 2 storage would be needed to support immediate needs, but larger storage reserves also will be required to ensure energy reliability and mitigate the impact of the mismatch between variable production rates and demand. Large-scale storage of H 2 may be achieved by using underground resources similar to how natural gas (NG) has been stored for the past century. While there is much experience in underground NG storage (UGS), significant technical challenges must be addressed to economically and safely store H 2 at the commercial scale. The purpose of this report is to assess the current state of knowledge and to identify technical research needs to facilitate deployment of large-scale underground hydrogen storage (UHS).

03 NATURAL GAS↗

DOE OSTI2025

Thermodynamic Modeling of Complex Solid Solutions in the Lu-H-N System via Graph Neural Network Accelerated Monte Carlo Simulations

Metal hydrides are important across diverse applications, such as hydrogen storage, batteries, gas sensors, nuclear reactions, and high-temperature superconductivity. Previous computational studies of metal hydrides under extreme pressures, e.g., 𝑂⁡(10 2 ) ⁢GPa, usually treat them as stoichiometric compounds without considering interstitial lattice disorder. As pressures become more moderate in the 𝑂⁡(10 0 ) ⁢GPa and below range, hydrogen disorder at interstitial lattice sites becomes prominent, e.g., in the N-doped Lu hydride that was recently claimed superconducting near 1 GPa. Further adding compositional complexity from alloying and/or multielement interstitial occupation makes elucidating pressure- and temperature-dependent observables intractable by first-principles calculations alone. We therefore propose a lattice graph neural-network surrogate modeling approach to predict configuration- and pressure-dependent equation-of-state properties. Their efficiency permits Monte Carlo simulations to calculate Gibbs energies and pressure-dependent phase diagrams, thereby revealing insights into the synthesis conditions required for achieving desired phase equilibria. We demonstrate this concept for the compositionally complex cubic Lu(H,N,Va) 3 system where three constituents (hydrogen, nitrogen and vacancy) have disordered multielement interstitial occupancies and insights into pressure-dependent phase equilibria are critically needed, e.g., N-doping levels can significantly lower dehydrogenation temperatures and provide a new strategy to optimize hydrogen-storage alloys. This work can improve the thermodynamic understanding of the Lu-H-N system and help rational synthesis of N-doped Lu hydrides, but more generally demonstrates an efficient approach to model pressure-dependent thermodynamics of multicomponent solid solutions.

Monte Carlo methods↗

DOE OSTI2014

Coordinated Collaboration between Heterogeneous Distributed Energy Resources

A power distribution feeder, where a heterogeneous set of distributed energy resources is deployed, is examined by simulation. The energy resources include PV, battery storage, natural gas GenSet, fuel cells, and active thermal storage for commercial buildings. The resource scenario considered is one that may exist in a not too distant future. Two cases of interaction between different resources are examined. One interaction involves a GenSet used to partially offset the duty cycle of a smoothing battery connected to a large PV system. The other example involves the coordination of twenty thermal storage devices, each associated with a commercial building. Storage devices are intended to provide maximum benefit to the building, but it is shown that this can have a deleterious effect on the overall system, unless the action of the individual storage devices is coordinated. A network based approach is also introduced to calculate some type of effectiveness metric to all available resources which take part in coordinated operation. The main finding is that it is possible to achieve synergy between DERs on a system; however this required a unified strategy to coordinate the action of all devices in a decentralized way.

Abdollahy, Shahin↗

DOE OSTI2013

Ceph Parallel File System Evaluation Report

We used Data Direct Networks’ (DDN) SFA10K as the storage backend during this evaluation. It consists of 200 SAS drives and 280 SATA drives, organized into various RAID levels by two active-active RAID controllers. The exported RAID groups by these controllers are driven by four hosts. Each host has two InfiniBand (IB) QDR connections to the storage backend. We used a single dualport Mellanox connectX IB card per host. By our calculation, this setup can saturate SFA10K’s maximum theoretical throughput (~12 GB/s). Our Ceph testbed employs a collection of testing nodes. These nodes and their roles are summarized in Table 1. In the following discussion, we use “servers”, “osd servers”, “server hosts” interchangeably. We will emphasize with “client” prefix when we want to distinguish it from above. All hosts (client and servers) were configured with Redhat 6.3 and kernel version 3.5.1 initially, and later upgraded to 3.9 (rhl-ceph image), Glibc 2.12 with syncfs support, locally patched. We used the Ceph 0.48 and 0.55 release in the initial tests, upgraded to 0.64 and then to 0.67RC for a final round of tests.

Wang, Feiyi↗