Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “distributed services”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Evaluation of Aggregated EV Flexibility With TSO-DSO Coordination

Electric vehicles (EVs) are becoming a promising source of grid ancillary services due to the temporal and spatial charging flexibility, quick response and storage capability. Such advantages are increasing with government policy promotion and technology improvement. However, the exploration of EV flexibility requires the coordination of both transmission system operators (TSOs) and distribution system operators (DSOs), to ensure the safe and reliable operation of power network. In this paper, we propose a coordinated evaluation method that determines the optimal utilization of EV temporal flexibility without compromising EV owners’ usage. At the distribution level, DSOs first evaluate EV aggregators’ operational boundaries to exploit distribution level services. At the transmission level, TSOs then determine EV charging schedules and ancillary service capacity simultaneously, taking into account the requirement from DSOs. Here, we validate the model in a case study using the IEEE 123 node test feeder and EV charging sessions obtained from a transportation simulation tool that uses real-world data.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2021

Knowledge Beacons: Web services for data harvesting of distributed biomedical knowledge

The continually expanding distributed global compendium of biomedical knowledge is diffuse, heterogeneous and huge, posing a serious challenge for biomedical researchers in knowledge harvesting: accessing, compiling, integrating and interpreting data, information and knowledge. In order to accelerate research towards effective medical treatments and optimizing health, it is critical that efficient and automated tools for identifying key research concepts and their experimentally discovered interrelationships are developed. As an activity within the feasibility phase of a project called “Translator” (https://ncats.nih.gov/translator) funded by the National Center for Advancing Translational Sciences (NCATS) to develop a biomedical science knowledge management platform, we designed a Representational State Transfer (REST) web services Application Programming Interface (API) specification, which we call a Knowledge Beacon. Knowledge Beacons provide a standardized basic API for the discovery of concepts, their relationships and associated supporting evidence from distributed online repositories of biomedical knowledge. This specification also enforces the annotation of knowledge concepts and statements to the NCATS endorsed the Biolink Model data model and semantic encoding standards (https://biolink.github.io/biolink-model/). Implementation of this API on top of diverse knowledge sources potentially enables their uniform integration behind client software which will facilitate research access and integration of biomedical knowledge.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2025

Ocelot: An Interactive, Efficient Distributed Compression-As-a-Service Platform With Optimized Data Compression Techniques

Large volumes of data generated by scientific simulations, genome sequencing, and other applications need to be moved among clusters for data collection/analysis. Data compression techniques have effectively reduced data storage and transfer costs. However, users' requirements on interactively controlling both data quality and compression ratios are non-trivial to fulfill. Here, we propose a novel Compression-as-a-Service (CaaS) platform called Ocelot with four important contributions: (1) It offers real-time visualization, interactive compression, and transfer of scientific datasets. (2) It incorporates new strategies for compressing diverse types of datasets more effectively than traditional methods. (3) It provides an effective method for estimating the compression ratio and execution time of compression tasks. (4) Experiments on multiple real-world datasets on geographically distributed computers show that Ocelot can significantly improve data transfer efficiency with a performance gain of more than 10x in computing clusters with relatively slow networks.

compression as a service (CaaS)↗
DOE OSTI2022

funcX: Federated Function as a Service for Science

Here, funcX is a distributed function as a service (FaaS) platform that enables flexible, scalable, and high performance remote function execution. Unlike centralized FaaS systems, funcX decouples the cloud-hosted management functionality from the edge-hosted execution functionality. funcX's endpoint software can be deployed, by users or administrators, on arbitrary laptops, clouds, clusters, and supercomputers, in effect turning them into function serving systems. funcX's cloud-hosted service provides a single location for registering, sharing, and managing both functions and endpoints. It allows for transparent, secure, and reliable function execution across the federated ecosystem of endpoints-enabling users to route functions to endpoints based on specific needs. funcX uses containers (e.g., Docker, Singularity, and Shifter) to provide common execution environments across endpoints. funcX implements various container management strategies to execute functions with high performance and efficiency on diverse funcX endpoints. funcX also integrates with an in-memory data store and Globus for managing data that may span endpoints. We motivate the need for funcX, present our prototype design and implementation, and demonstrate, via experiments on two supercomputers, that funcX can scale to more than 130000 concurrent workers. We show that funcX's container warming-aware routing algorithm can reduce the completion time for 3,000 functions by up to 61% compared to a randomized algorithm and the in-memory data store can speed up data transfers by up to 3x compared to a shared file system.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Accelerated, scalable and reproducible AI-driven gravitational wave detection

The development of reusable artificial intelligence (AI) models for wider use and rigorous validation by the community promises to unlock new opportunities in multi-messenger astrophysics. Here we develop a workflow that connects the Data and Learning Hub for Science, a repository for publishing AI models, with the Hardware-Accelerated Learning (HAL) cluster, using funcX as a universal distributed computing service. Using this workflow, an ensemble of four openly available AI models can be run on HAL to process an entire month's worth (August 2017) of advanced Laser Interferometer Gravitational-Wave Observatory data in just seven minutes, identifying all four binary black hole mergers previously identified in this dataset and reporting no misclassifications. This approach combines advances in AI, distributed computing and scientific data infrastructure to open new pathways to conduct reproducible, accelerated, data-driven discovery. By combining a repository for artificial intelligence models and a supercomputing cluster, an entire month's worth of advanced LIGO data is analysed in just 7 min, finding all binary black hole mergers previously identified in this dataset and reporting no misclassifications.

79 ASTRONOMY AND ASTROPHYSICS↗
NASA NTRS2016

Managing Sustainable Data Infrastructures: The Gestalt of EOSDIS

EOSDIS epitomizes a System of Systems, whose many varied and distributed parts are integrated into a single, highly functional organized science data system. A distributed architecture was adopted to ensure discipline-specific support for the science data, while also leveraging standards and establishing policies and tools to enable interdisciplinary research, and analysis across multiple scientific instruments. The EOSDIS is composed of system elements such as geographically distributed archive centers used to manage the stewardship of data. The infrastructure consists of underlying capabilities connections that enable the primary system elements to function together. For example, one key infrastructure component is the common metadata repository, which enables discovery of all data within the EOSDIS system. EOSDIS employs processes and standards to ensure partners can work together effectively, and provide coherent services to users.

remote sensing↗
DOE OSTI2026

Voltage Service Limits Smart Contract Using Distributed Ledger Technology for Electrical Utility Grid with Customer-Owned Generator

Modern electrical grids face growing stability risks from customer-owned generators, especially at points of common couplings (PCCs). Disruptive behavior from power-electronic sources can cause protective relays to isolate problematic generators, making measurement integrity critical. This article presents a distributed ledger technology (DLT) approach that uses smart contracts to evaluate PCC voltage measurements and trigger backup breaker operations. The approach is framed as a verifiable, multi-organization attestation and audit layer, not as a real-time control security mechanism. In the proposed architecture, voltage measurements from a hardware protective relay are anchored on a DLT through the Cyber Grid Guard (CGG) system for attestation by both the grid utility and customer-owned generator. A Voltage Service Limits (VSLs) smart contract evaluates the on-chain measurements against allowable phase-voltage limits derived from the ANSI C84.1 standard. The framework is validated in a hardware-relay-in-the-loop test bed under sustained-undervoltage, sustained-overvoltage, and transient line-to-line fault scenarios. The results show that the VSL smart contract can process these measurements and issue backup breaker actions consistent with the defined service-limit criteria, demonstrating the DLT potential as a verifiable audit layer at the PCC that complements primary protection.

cyber security↗
DOE OSTI2022

Incentivizing Distributed Energy Resource Participation in Grid Services

The bulk power system is experiencing a dramatic shift as renewable generation growth continues to accelerate. Large-scale renewables adoption will help societies transition to a low-carbon, low-cost, and environmentally friendly electrical power system. However, the transition from a paradigm of generation following load to one where load follows generation will require large-scale interconnection and coordinated operation of Distributed Energy Resources (DERs), supported by open communication protocols. In this future grid scenario, DER aggregations will provide critical grid services that enable high penetration levels of renewable generation. This position paper presents an Energy Service Interface (ESI) that defines scope for ensuring secure, trustworthy information exchange between grid service providers and DERs. The goal of the ESI is to encourage large-scale participation of DERs in order to provide grid services through dispatch of DER aggregations. This position paper also presents an open smart energy communications protocol that allows DERs to advertise their characteristics and participate in grid services, within constraints established by the ESI. The paper presents several monetization incentives that grid service providers could use to encourage large-scale DER participation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Layered Coordination Architecture for Resilient Restoration of Power Distribution Systems

The current practices for restoring critical services in the distribution system during a disaster, align with the traditional centralized ideology of distribution systems operations. A central processor evaluates the distribution system after a disruption and attains a restoration plan. However, the centralized operational paradigm is susceptible to single-point failures, requires full situational awareness of the distribution system, and poses scalability challenges for large multifeeder distribution systems. This motivates a distributed decision-making paradigm where multiple agents solve smaller subproblems and jointly coordinate their individual decisions to achieve the global/network-level objective. Toward this goal, we propose a layered architecture for distributed algorithms for resilience and a two-stage distributed algorithm for distribution system restoration. The proposed distributed decision-making framework enables the bottom-up restoration of the distribution system using all available resources, including distributed generation, while only requiring local awareness and limited communications with neighboring connected regions. The proposed framework is robust to single-point failures, enables autonomy using distributed algorithms, and had reduced computational cost compared to centralized optimization solutions.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2011

Making Earth Science Data Records for Use in Research Environments (MEaSUREs) Projects Data and Services at the GES DISC

NASA's Earth Science Program is dedicated to advancing Earth remote sensing and pioneering the scientific use of satellite measurements to improve human understanding of our home planet. Through the MEaSUREs Program, NASA is continuing its commitment to expand understanding of the Earth system using consistent data records. Emphasis is on linking together multiple data sources to form coherent time-series, and facilitating the use of extensive data in the development of comprehensive Earth system models. A primary focus of the MEaSUREs Program is the creation of Earth System Data Records (ESDRs). An ESDR is defined as a unified and coherent set of observations of a given parameter of the Earth system, which is optimized to meet specific requirements for addressing science questions. These records are critical for understanding Earth System processes; for the assessment of variability, long-term trends, and change in the Earth System; and for providing input and validation means to modeling efforts. Seven MEaSUREs projects will be archived and distributed through services at the Goddard Earth Sciences Data and Information Services Center (GES DISC).

Vollmer, Bruce E.↗
NASA NTRS2021

A Distributed Approach to High-Rate Delay Tolerant Networking Within a Virtualized Environment

The High-Rate Delay Tolerant Networking (HDTN) project has taken a distributed service-based approach to the development of a highly efficient delay tolerant networking (DTN) implementation. Through the analysis of many DTN implementations, system and mission requirements as well as the DTN protocol specifications, HDTN has worked to infuse modern computing technologies into the NASA approach to interplanetary networking. The initial use case of the HDTN software runs on a hypervisor representative of the International Space Station (ISS) DTN Gateway. In this scenario, multiple emulated payloads will send science data through HDTN to a mission operations center. HDTN will provide store and forward capability as well as network flow management. This paper discusses the infusion path of cognitive networking technologies in the NASA SCaN networks using the DTN architecture and protocols as the basis for cognitive routing and network management capabilities. HDTN has been developing the Bundle Protocol encoding and decoding mechanisms and messaging framework that can be used as the basis for integrating DTN with various learning and decision-making processes. The concepts of distributed computing, network virtualization, software defined networking and delay tolerant networking are basic building blocks which will further the development of cognitive networking. In addition to discussion of the HDTN software development and testing, this paper examines the role that each of these technologies play in the evolution of the current state of space networking into an intelligent network of networks.

Delay Tolerant Networking↗
NASA NTRSDate not supplied

A Distributed Approach to High-Rate Delay Tolerant Networking Within a Virtualized Environment

The High-Rate Delay Tolerant Networking (HDTN) project has taken a distributed service-based approach to the development of a highly efficient delay tolerant networking (DTN) implementation. Through the analysis of many DTN implementations, system and mission requirements as well as the DTN protocol specifications, HDTN has worked to infuse modern computing technologies into the NASA approach to interplanetary networking. The initial use case of the HDTN software runs on a hypervisor representative of the International Space Station (ISS) DTN Gateway. In this scenario, multiple emulated payloads will send science data through HDTN to a mission operations center. HDTN will provide store and forward capability as well as network flow management. This paper discusses the infusion path of cognitive networking technologies in the NASA Space Communications and Navigation (SCaN) networks using the DTN architecture and protocols as the basis for cognitive routing and network management capabilities. HDTN has been developing the Bundle Protocol encoding and decoding mechanisms and messaging framework that can be used as the basis for integrating DTN with various learning and decision-making processes. The concepts of distributed computing, network virtualization, software defined networking and delay tolerant networking are basic building blocks which will further the development of cognitive networking. In addition to discussion of the HDTN software development and testing, this paper examines the role that each of these technologies play in the evolution of the current state of space networking into an intelligent network of networks.

Rachel Mary Dudukovich↗
DOE OSTI2020

Safe and Private Forward-trading Platform for Transactive Microgrids

Power grids are evolving at an unprecedented pace due to the rapid growth of distributed energy resources (DER) in communities. These resources are very different from traditional power sources, as they are located closer to loads and thus can significantly reduce transmission losses and carbon emissions. However, their intermittent and variable nature often results in spikes in the overall demand on distribution system operators (DSO). To manage these challenges, there has been a surge of interest in building decentralized control schemes, where a pool of DERs combined with energy storage devices can exchange energy locally to smooth fluctuations in net demand. Building a decentralized market for transactive microgrids is challenging, because even though a decentralized system provides resilience, it also must satisfy requirements such as privacy, efficiency, safety, and security, which are often in conflict with each other. As such, existing implementations of decentralized markets often focus on resilience and safety but compromise on privacy. In this article, we describe our platform, called TRANSAX, which enables participants to trade in an energy futures market, which improves efficiency by finding feasible matches for energy trades, enabling DSOs to plan their energy needs better. TRANSAX provides privacy to participants by anonymizing their trading activity using a distributed mixing service, while also enforcing constraints that limit trading activity based on safety requirements, such as keeping planned energy flow below line capacity. We show that TRANSAX can satisfy the seemingly conflicting requirements of efficiency, safety, and privacy. We also provide an analysis of how much trading efficiency is lost. Trading efficiency is improved through the problem formulation, which accounts for temporal flexibility, and system efficiency is improved using a hybrid-solver architecture. Lastly, we describe a testbed to run experiments and demonstrate its performance using simulation results.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2021

Switching Device-Cognizant Sequential Distribution System Restoration

This paper presents an optimization framework for sequential reconfiguration using an assortment of switching devices and repair process in distribution system restoration. Compared to existing studies, this paper considers types, capabilities and operational limits of different switching devices, making it applicable in practice. We develop a novel multi-phase method to find the optimal sequential operation of various switching devices and repair faulted areas. We consider circuit breakers, reclosers, sectionalizers, load breaker switches, and fuses. The switching operation problem is decomposed into two mixed-integer linear programming (MILP) subproblems. The first subproblem determines the optimal network topology and estimates the number of steps to reach that topology, while the second subproblem generates a sequence of switching operations to coordinate the switches. For repairing the faults, we design an MILP model that dispatches repair crews to clear faults and replace melted fuses. After clearing a fault, we update the topology of the network by generating a new sequence of switching operations, and the process continues until all faults are cleared. To improve the computational efficiency, a network reduction algorithm is developed to group line sections, such that only switchable sections are present in the reduced network. The proposed method is validated on the IEEE 123-bus and 8500-bus systems.

distribution system↗
DOE OSTI2020

Challenges and Opportunities in Turning Large U.S. Archives of Analog Seismograms into a Modern Usable Resource

Abstract Quantitative seismology is based firmly on the analysis of actual ground motions, and the transition to digital recording in the 1980s enabled sophisticated new capabilities to extract useful results from waveforms. With some effort, these tools can also be applied to analog records. Focusing on assets available within U.S. institutions, we review the necessary steps and the challenges in enabling “data rescue”—that is, preserving the scientific information latent in large analog seismogram archives and making it usable. They include: determining what assets are available (the analog seismogram archives held by various institutions, with associated metadata on instrument responses, station locations, and timing information); developing a consensus on the top level of a triage process (which analog records most definitely should be rescued?); deciding the level of quality needed in copying original seismograms to media suitable for digitizing; assessing the relative merits of scanning and digitizing; and, the need for a community service in distributing scans and digital records, as they accumulate. The necessary level of effort can benefit from practical experience. For example, specific studies have used digitized versions of analog recordings to model earthquake sources and assess seismic hazard. Other studies have used them to gain experience with nuclear explosion signals recorded at regional distances, noting that regional signals enable explosions to be monitored down to levels much lower than those attainable teleseismically. The opportunities presented by large archives of analog seismograms include the insights they present to current and future seismologists studying earthquakes and explosions, into the practical areas of assessing seismic hazard, monitoring for test ban compliance down to low explosion yields—and prompt characterization of actual explosions should they occur, as well the traditional academic pursuit of a better understanding of earthquake physics.

Geochemistry & Geophysics↗
DOE OSTI2021

Integrated System and Application Continuous Performance Monitoring and Analysis Capability

Scientific applications run on high-performance computing (HPC) systems are critical for many national security missions within Sandia and the NNSA complex. However, these applications often face performance degradation and even failures that are challenging to diagnose. To provide unprecedented insight into these issues, the HPC Development, HPC Systems, Computational Science, and Plasma Theory & Simulation departments at Sandia crafted and completed their FY21 ASC Level 2 milestone entitled "Integrated System and Application Continuous Performance Monitoring and Analysis Capability." The milestone created a novel integrated HPC system and application monitoring and analysis capability by extending Sandia's Kokkos application portability framework, Lightweight Distributed Metric Service (LDMS) monitoring tool, and scalable storage, analysis, and visualization pipeline. The extensions to Kokkos and LDMS enable collection and storage of application data during run time, as it is generated, with negligible overhead. This data is combined with HPC system data within the extended analysis pipeline to present relevant visualizations of derived system and application metrics that can be viewed at run time or post run. This new capability was evaluated using several week-long, 290-node runs of Sandia's ElectroMagnetic Plasma In Realistic Environments ( EMPIRE ) modeling and design tool and resulted in 1TB of application data and 50TB of system data. EMPIRE developers remarked this capability was incredibly helpful for quickly assessing application health and performance alongside system state. In short, this milestone work built the foundation for expansive HPC system and application data collection, storage, analysis, visualization, and feedback framework that will increase total scientific output of Sandia's HPC users.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Integrated System and Application Continuous Performance Monitoring and Analysis Capability (Final)

Scientific applications run on high-performance computing (HPC) systems are critical for many national security missions within Sandia and the NNSA complex. However, these applications often face performance degradation and even failures that are challenging to diagnose. To provide unprecedented insight into these issues, the HPC Development, HPC Systems, Computational Science, and Plasma Theory & Simulation departments at Sandia crafted and completed their FY21 ASC Level 2 milestone entitled "Integrated System and Application Continuous Performance Monitoring and Analysis Capability." The milestone created a novel integrated HPC system and application monitoring and analysis capability by extending Sandia’s Kokkos application portability framework, Lightweight Distributed Metric Service (LDMS) monitoring tool, and scalable storage, analysis, and visualization pipeline. The extensions to Kokkos and LDMS enable collection and storage of application data during run time, as it is generated, with negligible overhead. This data is combined with HPC system data within the extended analysis pipeline to present relevant visualizations of derived system and application metrics that can be viewed at run time or post run. This new capability was evaluated using several week-long, 290-node runs of Sandia’s ElectroMagnetic Plasma In Realistic Environments (EMPIRE) modeling and design tool and resulted in 1TB of application data and 50TB of system data. EMPIRE developers remarked this capability was incredibly helpful for quickly assessing application health and performance alongside system state. In short, this milestone work built the foundation for expansive HPC system and application data collection, storage, analysis, visualization, and feedback framework that will increase total scientific output of Sandia’s HPC users.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Resource Occurrence and Productivity in Existing and Proposed Wind Energy Lease Areas on the Northeast US Shelf

States in the Northeast United States have the ambitious goal of producing more than 22 GW of offshore wind energy in the coming decades. The infrastructure associated with offshore wind energy development is expected to modify marine habitats and potentially alter the ecosystem services. Species distribution models were constructed for a group of fish and macroinvertebrate taxa resident in the Northeast US Continental Shelf marine ecosystem. These models were analyzed to provide baseline context for impact assessment of lease areas in the Middle Atlantic Bight designated for renewable wind energy installations. Using random forest machine learning, models based on occurrence and biomass were constructed for 93 species providing seasonal depictions of their habitat distributions. We developed a scoring index to characterize lease area habitat use for each species. Subsequently, groups of species were identified that reflect varying levels of lease area habitat use ranging across high, moderate, low, and no reliance on the lease area habitats. Among the species with high to moderate reliance were black sea bass ( Centropristis striata ), summer flounder ( Paralichthys dentatus ), and Atlantic menhaden ( Brevoortia tyrannus ), which are important fisheries species in the region. Potential for impact was characterized by the number of species with habitat dependencies associated with lease areas and these varied with a number of continuous gradients. Habitats that support high biomass were distributed more to the northeast, while high occupancy habitats appeared to be further from the coast. There was no obvious effect of the size of the lease area on the importance of associated habitats. Model results indicated that physical drivers and lower trophic level indicators might strongly control the habitat distribution of ecologically and commercially important species in the wind lease areas. Therefore, physical and biological oceanography on the continental shelf proximate to wind energy infrastructure development should be monitored for changes in water column structure and the productivity of phytoplankton and zooplankton and the effects of these changes on the trophic system.

17 WIND ENERGY↗