Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “STORAGE AND NETWORKING SYSTEMS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Software Framework for Federated Science Instruments

There is an unprecedented promise of enhanced capabilities for federations of leadership computing systems and experimental science facilities by leveraging software technologies for fast and efficient operations. These federations seek to unify different science instruments, both computing and experimental, to effectively support science users and operators to execute complex workflows. The FedScI project addresses the software challenges associated with the formation and operation of federated environments by leveraging recent advances in containerization of software and softwarization of hardware. We propose a software framework to streamline the federation usage by science users and it’s provisioning and operations by facility providers. A distinguishing element of our work is the support for improved interaction between experimental devices, such as beam-line instruments, and more traditional high-performance computing resources, including compute, network, storage systems. We present guiding principles for the software framework and highlight portions of a current prototype implementation. We describe our science use case involving neutron imaging beam-lines (SNAP/BL-3, Imaging/CG-1D) at the Spallation Neutron Source and High Flux Isotope Reactor facilities at Oak Ridge National Laboratory. Additionally, we detail plans for a more direct instrument interaction within a federated environment, which could enable more advanced workflows with feedback loops to shorten the time to science.

Naughton III, Thomas↗

Impacts of Control, Penetration, and Distribution of Embedded Storage Network in Bulk Power System

The current shift in generation mix from fossil fuel plants towards variable and intermittent renewable energy sources is poised to create a future grid with reduced physical inertia and mismatch between generation and demand. Embedded storage, which is a concept of a coordinated network of storage units sited at the interface between the transmission and distribution system, is proposed as a mechanism to provide a buffer between generation and demand. This paper proposes an automated framework to model and integrate embedded storage in large-scale power systems with industry-grade grid-following (GFL) and grid-forming (GFM) control technologies. More importantly, the developed framework is used to explore the impacts of embedded storage control, penetration, location, and capacity in providing fast frequency response to the grid under contingency events such as generator trips and faults. The framework and study are conducted using the transient-stability simulation tool PSS/E and a realistic model of the Puerto Rico grid as a chosen test system. The simulation results show that GFL and GFM embedded storage, distributed throughout the system, with sufficient penetration and capacity, can effectively improve primary frequency response of the system under the studied contingency events.

Battery Energy Storage, embedded storage, grid-for↗

AI-Science for Performance Optimization and Diagnosis of Science Instrument Federations

Next generation of science workflows are expected to be executed over complex federations composed of supercomputers, science instruments, storage systems and networks, with new additions of the edge and cloud systems and services. The sheer complexity of these multi-domain federations makes it hard to manage them and optimize their performance, as small impedance mismatches (that can dynamically develop between systems) could drastically degrade the entire federation performance. Recent proliferation of Software Defined Everything (SDX) technologies combined with containerization frameworks provide custom instruments that can monitor and collect critical measurements at various levels to support diagnoses and performance optimization; but their data too enormous for human operators and analysts to process and generate decisions. Machine Learning (ML) methods that extract critical parameters, relationships and trends from the data offer general solutions. Artificial Intelligence (AI) and ML methods must be custom-developed for these problems based on solid, rigorous foundations, since black-box approaches are often ineffective and unsound.We propose to develop comprehensive AI-Science for the performance of science federations to (i) monitor and control storage, networks, experiments, and computing systems across multiple domains via softwarization layers, at speeds and scales orders of magnitude superior to current practice, (ii) optimally realize and orchestrate complex workflows with high performance by using dynamic state and performance estimation methods, and (iii) aggregate measurements across sites and time to develop infrastructure-level profiles, optimizations and diagnoses using AI-Science based on foundational principles from ML, game theory, and information fusion areas.

Rao, Nageswara↗

Representing Carbon Dioxide Transport and Storage Network Investments within Power System Planning Models

Carbon dioxide (CO 2 ) capture and storage (CCS) is frequently identified as a potential component to achieving a decarbonized power system at least cost; however, power system models frequently lack detailed representation of CO 2 transportation, injection, and storage (CTS) infrastructure. In this paper, we present a novel approach to explicitly represent CO 2 storage potential and CTS infrastructure costs and constraints within a continental-scale power system capacity expansion model. In addition, we evaluate the sensitivity of the results to assumptions about the future costs and performance of CTS components and carbon capture technologies. We find that the quantity of CO 2 captured within the power sector is relatively insensitive to the range of CTS costs explored, suggesting that the cost of CO 2 capture retrofits is a more important driver of CCS implementation than the costs of transportation and storage. Finally, we demonstrate that storage and injection costs account for the predominant share of total costs associated with CTS investment and operation, suggesting that pipeline infrastructure costs have limited influence on the competitiveness of CCS.

24 POWER TRANSMISSION AND DISTRIBUTION↗

SWARM: Reimagining scientific workflow management systems in a distributed world

Modern scientific workflows process massive amounts of data from diverse instruments and sensors, leveraging geographically distributed, heterogeneous compute and storage resources—from leadership-class systems to edge devices—connected by high-performance networks. The diversity of resources introduces challenges in harnessing their full potential, with resilience issues arising across applications, system software, networks, storage, and hardware. Today, workflow management systems (WMS) coordinate the execution of computation and data management tasks across target resources. However, WMS’s centralized nature makes them vulnerable to faults and scalability issues that may result in failures of entire computational campaigns. In conclusion, this paper introduces a novel agentic framework for workflow management, fully distributing and decentralizing the WMS functions and modeling them as swarm intelligence agents infused with advanced artificial intelligence solutions and traditional distributed computing algorithms that can make coordinated decisions in the presence of failures of the underlying cyberinfrastructure.

Swarm intelligence↗

OCTOKV: An Agile Network-Based Key-Value Storage System with Robust Load Orchestration

In this paper, we propose OctoKV, an innovative network-based key-value storage system. OctoKV addresses the repetitive address translation overhead associated with traditional key-value stores running on file systems on the client side. To mitigate this overhead, we implemented the key-value store on the server side using NVMe-oF and a user-level NVMe driver. In particular, we employed fine-grained resource monitoring and load balancing based on heuristics to optimize I/O performance. OctoKV is deployed on a Linux cluster with Intel SPDK. The extensive evaluation shows that OctoKV achieves lower I/O response times in comparison to traditional approaches where key-value stores run on the client side. Also, the proposed load balancing strategies efficiently enhance I/O response times by equally distributing the workload from overloaded cores to other cores.

Khan, Awais↗

Viability of S3 Object Storage for the ASC Program at Sandia

Recent efforts at Sandia such as DataSEA are creating search engines that enable analysts to query the institution’s massive archive of simulation and experiment data. The benefit of this work is that analysts will be able to retrieve all historical information about a system component that the institution has amassed over the years and make better-informed decisions in current work. As DataSEA gains momentum, it faces multiple technical challenges relating to capacity storage. From a raw capacity perspective, data producers will rapidly overwhelm the system with massive amounts of data. From an accessibility perspective, analysts will expect to be able to retrieve any portion of the bulk data, from any system on the enterprise network. Sandia’s Institutional Computing is mitigating storage problems at the enterprise level by procuring new capacity storage systems that can be accessed from anywhere on the enterprise network. These systems use the simple storage service, or S3, API for data transfers. While S3 uses objects instead of files, users can access it from their desktops or Sandia’s high-performance computing (HPC) platforms. S3 is particularly well suited for bulk storage in DataSEA, as datasets can be decomposed into object that can be referenced and retrieved individually, as needed by an analyst. In this report we describe our experiences working with S3 storage and provide information about how developers can leverage Sandia’s current systems. We present performance results from two sets of experiments. First, we measure S3 throughput when exchanging data between four different HPC platforms and two different enterprise S3 storage systems on the Sandia Restricted Network (SRN). Second, we measure the performance of S3 when communicating with a custom-built Ceph storage system that was constructed from HPC components. Overall, while S3 storage is significantly slower than traditional HPC storage, it provides significant accessibility benefits that will be valuable for archiving and exploiting historical data. There are multiple opportunities that arise from this work, including enhancing DataSEA to leverage S3 for bulk storage and adding native S3 support to Sandia’s IOSS library.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Software defined grid energy storage

Today, consumer battery installations are isolated, physical devices. Virtual power plants (VPPs) allow consumer devices to aggregate for grid services, but they are are vertically integrated, vendor controlled systems (e.g., Tesla’s VPP). Consumer batteries are therefore unable to participate in energy markets or other grid services outside what their vendor provides. We describe a software system that provides software control of multiple, networked battery energy storage systems in the electric grid. The system introduces two new ideas that enable flexible and dependable management of energy storage. The first is a virtual battery, which can either partition a battery or aggregate multiple batteries. The second is a reservation-based API which allows asynchronous control of batteries to meet contractual guarantees in a safe and dependable manner. Virtual batteries and a reservation-based API address the unique challenges of achieving high and efficient utilization of energy storage systems, including heterogeneity of battery systems such as varying C-rates, participation in energy markets, utility bill management systems, community resource sharing, and reliability. Using a testbed comprised of sonnen Inc. storage units installed in several homes and a lab, we demonstrate that virtualized batteries can seamlessly replace physical batteries, flexibly manage energy storage resources, isolate multiple clients using a shared battery, and create new energy storage applications.

25 ENERGY STORAGE↗

Datum: A Scientific Metadata Catalog

The data catalog market is currently flooded with a myriad of different products, but none serve the scientific community well. There are cloud-native tools like Databricks, Snowflake,to on-premise solutions like Collibra and Datahub. The common failing of all these tools however, is their inability to serve the scientific data community directly. Most catalogs are targeted towards financial, health, or user data - not sensor or scientific domain data. They also prioritize integrations that often don’t exist or are just starting to be used in the scientific realm - all while ignoring common scientific tools and file types. Datum is a catalog which targets the scientific data directly, including the tools and networks in which those tools are used. We work with the producers and consumers of the data where they are, targeting cloud and on-premise with a focus on classified networks. Datum is an Erlang/Elixir application. Technical Features Note: The features listed below are still under development and may change, slightly, upon final delivery of the product. File Formats - Datum has the ability to read additional metadata and provides processing pipelines for the following file formats: Plain Text, PDF, LaTeX, HTML, Open Document Format (.odt), XML, CSV/TSV (and other standard delimiters), OpenDocument Database and Spreadsheets, Geo-Referenced TIFF, Common Data Format, HDF/HDF5, LabView TDMS, Excel, DeltaTables, Parquet, Apache Iceberg, Apache Hudi and many others. Metadata Collection - Scanners for the local and networked file systems and cloud storage providers. Network integration with common databases such as MSSQL and MySQL. User Plugin System - Users are able to provide either file processing, metadata extraction, or sampling plugins in the programming language of their choice. Authentication/Authorization -: OIDC integration, SCIM provisioning and EntraID integration out of the box. Full user and group management system with a “least privilege” operating mode. Governance - Customizable data governance platform; dictate and enforce required metadata, enforce data embargos, and enforce user agreements and NDAs before data access. Ability to create health checks on data, rejecting abandoned or poorly curated data and automatically removing it from the search index. Ability for users to submit corrections. Search - Semantic search is a first class citizen. No licenses to expensive, external software required. Integrated use of vectors and vector-based search allows for AI agent integration at all levels of operation. Metadata Model - Display and control data’s lineage and connections to other data and data directories. Data is modeled after a filesystem - an organization instantly recognizable and navigable by most any user. CLI and SDK - Ships with a Command Line Interface (CLI) tool and with a fully-featured Python SDK. This allows for rapid and programmatic use of Datum by every level of user. Minimal Infrastructure - Datum ships as a single executable file and can be run on any operating system and most CPU architectures. Datum has no reliance on external databases, search indexing tools, or other outside services - and it runs equally well on edge computing devices, cloud services, or in a clustered HPC environment.

darrington, john↗

Optimization-Based Dynamic Voltage Support of Microgrids Using Energy Storage Systems

A microgrid network is characterized by a high R/X ratio, making the voltage more sensitive to active power changes compared to bulk power systems, where the voltage is regulated primarily by reactive power. Due to its sensitivity, voltage control approaches for microgrids should also consider the active power input coupling, making it very different from conventional power systems. Additionally, as the energy costs associated with active and reactive powers are different and the operational conditions of microgrids connected to active distribution systems vary over time, the ideal controller to provide voltage support must be flexible enough to handle these technical and operational constraints. This paper proposes a model predictive control approach to provide dynamic voltage support using energy storage systems. This approach uses a simplified predictive model of the system to solve the model predictive control problem. By proper selection of model predictive control weighting parameters, the quality of service provided can be adjusted to achieve the desired performance. A simulation study in MATLAB/Simulink validates the proposed approach for the Cordova, Alaska microgrid. Results show that the performance of the voltage support can be adjusted depending on the choice of weight and constraints of the controller.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Branching Dueling Q-Network Based Online Scheduling of a Microgrid With Distributed Energy Storage Systems

This letter investigates a Branching Dueling Q-Network (BDQ) based online operation strategy for a microgrid with distributed battery energy storage systems (BESSs) operating under uncertainties. Additionally, the developed deep reinforcement learning (DRL) based microgrid online optimization strategy can achieve a linear increase in the number of neural network outputs with the number of distributed BESSs, which overcomes the curse of dimensionality caused by the charge and discharge decisions of multiple BESSs. Numerical simulations validate the effectiveness of the proposed method.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Thermoeconomic analysis of residential rooftop photovoltaic systems with integrated energy storage and resulting impacts on electrical distribution networks

This paper investigates residential rooftop photovoltaic (PV) systems for long-term thermoeconomic benefits from PV homeowners’ perspectives and for impacts on the electrical distribution network from grid operators’ perspectives. The costs of generating electricity from grid-connected PV systems are studied with and without energy storage at the PV homeowners’ sites. Three selling scenarios for excess PV energy conversion are considered: net metering, wholesale pricing, and no payback. PV systems in Utah are utilized as case studies in this analysis. The presence of PV systems gives homeowners economic benefits such as reduced annual electricity bills. However, the levelized costs of electricity are considerably higher than the weighted electricity price in Utah. Currently, the addition of energy storage only benefits customers in Utah under the no payback policy. The impacts of PV systems toward electrical distribution networks are then studied on a distribution test system. Excess PV generation from residential PV systems causes voltage rise in the electrical distribution network. The results from this paper can educate consumers about the lifetime benefit of integrating solar energy into their homes. For grid operators, residential PV systems with energy storage can reduce the negative impacts on the grid compared with high PV penetration alone.

14 SOLAR ENERGY↗

Software defined networking multiple operating system rotational environment

Systems, methods, and storage media for utilizing software defined networking for a multiple operating system rotational environment, executing on a computing device are disclosed. Some implementations may: receive a request from a user device; modify a packet of the request with a destination address and a port of a first server; forward the modified packet of the request to a controller server; receive a flow modification from the controller server based on the modified packet; and modify further received packets from the user device based on the received flow modification.

97 MATHEMATICS AND COMPUTING↗