Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Middleware”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

54 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

MPI Session: External Network Transport Implementation (V.1.0)

The MPI Sessions extensions to the MPI standard have been accepted by the MPI Forum and will be included in the upcoming MPI 4 version of the standard. MPI Sessions has the potential to address several limitations of MPI’s current specification: MPI cannot be initialized within an MPI process from different application components without a priori knowledge or coordination; MPI cannot be initialized more than once; and, MPI cannot be reinitialized after MPI finalization. MPI Sessions also offers the possibility for more flexible ways for individual components of an application to express the capabilities they require from MPI at a finer granularity than is presently possible. A prototype of MPI Sessions, based on the Open MPI implementation of the MPI standard, was developed to facilitate acceptance of the Sessions proposal by the Forum. The initial implementation had some limitations, one of the more significant ones being that it was limited in its ability to fully exploit modern network APIs such as OFI libfabric and OpenUCX and underlying network hardware. This report presents enhancements to the prototype implementation of MPI Sessions that removes this restriction for the networks to be used in the next generation of DOE exa–scale systems. Open MPI was used as the implementation vehicle, but results here are also relevant to other middleware stacks.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Big PanDa Workflow Management on Titan for High Energy and Nuclear Physics and for Future Extreme Scale Scientific Application

Over a three year period, from 2016-2019, this project demonstrated the scientific benefits of integrating the Titan supercomputer at Oak Ridge Leadership Computing Facility into traditional high throughput grid based distributed computing systems managed by PanDA, the workflow management system used for the execution of all distributed computing applications by the ATLAS experiment at the Large Hadron Collider. PanDA manages millions of batch jobs daily at hundreds of clusters worldwide on request by thousands of physicist users, and processes more than an exabyte of data annually using grid middleware. High levels of operational use of Titan was sustained by PanDA in order to meet the physics goals of ATLAS. The success of this project led to the use of other supercomputers worldwide by ATLAS, and to the adoption of PanDA by other experiments and other scientists. Multiple innovative operational and computer science research goals were achieved supporting the use of supercomputers for scientific domains with large scale distributed data and distributed processing needs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Addressing tokens dynamic generation, propagation, storage and renewal to secure the GlideinWMS pilot based jobs and system

GlideinWMS has been one of the first middleware in the WLCG community to transition from X.509 to support also tokens. The first step was to get from the prototype in 2019 to using tokens in production in 2022. This paper will present the challenges introduced by the wider adoption of tokens and the evolution plans for securing the pilot infrastructure of GlideinWMS and supporting the new requirements. In the last couple of years, the GlideinWMS team supported the migration to tokens of experiments and resources. Inadequate support in the current infrastructure, more stringent requirements, and the higher spatial and temporal granularity forced GlideinWMS to revisit once more how credentials are generated, used, and propagated. The new credential modules have been designed to be used in multiple systems (GWMS, HC) and use a model where credentials have type, purpose, and different flows. Credentials are dynamically generated in order to customize the duration and limit the scope to the targeted resource. This allows to enforce the least privilege principle. Finally, we also considered adding credential storage, renewal, and invalidation mechanisms within the GlideinWMS infrastructure to serve better the experiments’ needs.

Mambelli, Marco↗
DOE OSTI2026

EVs-at-RISC: A Secure and Resilient Interoperable SCM Control System Architecture for Electric Vehicle’s-at-Scale (Final Technical Report)

The EVs-at-RISC project was a five-year research, development, and demonstration initiative to create foundational tools for utility-scale fleet aggregation and Smart Charge Management (SCM) of Electric Vehicles (EV), Electric Vehicle Charging Infrastructure (EVCI), and related Distributed Energy Resources (DER). Rather than seeking to develop and demonstrate highly perfected SCM algorithms and control strategies, this project instead focused on creating foundational software solutions that enable unprecedented digital interoperability across the communications technologies and vendor platforms used to manage EV , EVCI, and DER, as well as existing energy management infrastructure operated by utilities, grid operators, and aggregators. This project then extends these novel interoperability capabilities to develop and deploy powerful middleware abstractions across grid edge networks and EVCI/DER fleet aggregations incorporating modern software tools and best practices, such as CI/CD, to bring the immense capabilities of infrastructure-as-code and policy-as-code to modern grid edge network environments. This addresses the foremost systemic issues preventing realization of any net operational benefits from scaled deployment of behind-the-meter EV, EVCI, and DER assets in electric power grids and markets today. The results of this approach and project unlock massive potential for new SCM capabilities to be easily prototyped, evaluated, and deployed at-scale within the existing grid edge network infrastructure and EVCI/DER technology ecosystem. The EVs-at-RISC project achieves this by extending Open Field Message Bus (OpenFMB), a conceptual model for digital interoperability and distributed intelligence in traditional front-of-meter utility SCADA networks, validating our hypothesis that OpenFMB could be similarly used to solve systemic digital interoperability issues in behind-the-meter environments and unlock real-world utility-scale SCM capabilities without requiring any new proprietary vendor solutions or significant infrastructure reconfiguration.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

Bridging Cloud and Edge Computing at NREL Using CONNECT: Cloud Optimized Networking for Next-Gen Edge Computing Technologies [Slides]

CONNECT is an innovative on-premise hardware and software solution that integrates edge and cloud computing infrastructure at NREL. Built on the AWS Greengrass middleware and leveraging the MQTT protocol, CONNECT enables real-time data streaming from IoT devices and gateways to both cloud and local services, empowering researchers to rapidly capture, analyze, and act upon edge-generated data while leveraging cloud capabilities. The platform addresses research infrastructure challenges by providing a pre-approved platform which is already configured with the correct networking and cybersecurity baselines thus eliminating procurement delays and enabling on-demand availability. CONNECT's hybrid architecture efficiently manages burstable workloads, allowing research teams to dynamically scale computational capacity, handle peak data loads, and reduce operational bottlenecks. Advanced capabilities include built-in GPU support for executing machine learning models which enables low-latency inference at the edge from models trained in the cloud. This architecture supports real-time analytics and filtering, providing a mechanism to allow only transmitting and processing high-value data. Cloud-based configuration management permits engineers to manage on-premise systems remotely, optimizing operational efficiency. By bridging edge and cloud computing, CONNECT provides NREL researchers with a flexible, scalable platform that accelerates scientific discovery while maintaining robust security and performance standards.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Generalizable Web User Interface for Scalable and Streamlined Deployment of Building Energy Management Systems in Small and Medium-Sized Commercial Buildings

Small and medium-sized commercial buildings (SMCBs) comprise 94% of US commercial buildings yet face significant barriers to implementing building energy management systems despite advances in smart device technology. Existing solutions present critical limitations: cloud-based API solutions simplify deployment but create vendor lock-in constraints; commercial integrated software solutions ensure compatibility via standardized protocols but require substantial cost and technical expertise; open-source IoT platforms offer cost-effective vendor independence but provide insufficient standardized protocol support for commercial building automation. This research presents a generalizable web user interface framework that bridges the gap between evolving smart device capabilities and lagging software infrastructure for SMCBs. The proposed system integrates VOLTTRON open-source middleware with an automated configuration converter that transforms unified specifications written in YAML, a human-readable data-serialization format, into system-specific files, streamlining manual setup processes. The vendor-agnostic architecture supports industry-standard protocols (BACnet and Modbus) and semantic building models while providing adaptive web interfaces that dynamically adjust to various building configurations. Demonstrations through simulation-based testing and a field deployment show automatic interface adaptation across heterogeneous HVAC systems and multizone monitoring. The automated configuration converter also substantially reduces labor-intensive setup.

Chung, Jihoon [ORNL] (ORCID:0000000184880815)↗
DOE OSTI2021

GASNet-EX Memory Kinds: Support for Device Memory in PGAS Programming Models

There is an emerging need for adaptive, lightweight communication in irregular HPC applications at exascale, where GPU accelerators provide the majority of available compute cycles. To address this need, Lawrence Berkeley National Lab is developing a programming system to support distributed-memory HPC application development using the Partitioned Global Address Space (PGAS) model. This work includes two major components: UPC++ and GASNet-EX. UPC++ is a C++ template library providing Remote Memory Access (RMA) and Remote Procedure Call (RPC) communication interfaces. GASNet-EX is a portable, high-performance communication middleware library, used by the implementations of UPC++ and many other PGAS programming models. We describe recent advances in GASNet-EX to efficiently implement zero-copy Remote Memory Access (RMA) communication to and from memory on accelerator devices such as GPUs. We demonstrate performance improvements via benchmark results from UPC++ (on Summit) and the Legion programming system (on DGX-1), both using GASNet-EX for communication.

Hargrove, Paul H↗
DOE OSTI2026

Simulation-Based Validation of An Open-Source, Scalable Framework for Building Energy Management in Small and Medium-Sized Commercial Buildings

Abstract: Small and medium-sized commercial buildings (SMCBs) represent 94% of U.S. commercial buildings but encounter substantial obstacles in adopting Building Energy Management (BEM) systems. Current approaches exhibit fundamental limitations: vendor-specific API platforms restrict interoperability through proprietary ecosystems; commercial automation software demands extensive technical expertise and licensing costs; open-source IoT solutions lack native support for building automation protocols and semantic models. This paper introduces a configuration-driven web interface framework addressing the gap between smart device advancements and accessible BEM software infrastructure for SMCBs. The framework leverages VOLTTRON middleware integrated with an automated converter that processes unified YAML configurations into heterogeneous system files, reducing required configuration artifacts from six separate files to a single unified specification. The system architecture enables vendor-agnostic operation through BACnet and Modbus protocols while supporting semantic building model integration via automated Brick Schema parsing. Configuration-driven interfaces automatically adapt to diverse HVAC types without custom development. Simulation-based validation using BOPTEST demonstrates automatic interface generation between fan coil and hydronic systems, with the automated converter successfully generating all platform-specific outputs from the single YAML input. The result demonstrates the framework's capability to streamline BEM system deployment through reduced configuration complexity. This work bridges simulation capabilities with operational deployment, demonstrating how virtual testbeds validate generalizable software frameworks for real-world building automation.

Chung, Jihoon [ORNL] (ORCID:0000000184880815)↗
DOE OSTI2022

Development of an Energy Efficient and Cost Effective Autonomous Vehicle Research Platform

Commercialization of autonomous vehicle technology is a major goal of the automotive industry, thus research in this space is rapidly expanding across the world. However, despite this high level of research activity, literature detailing a straightforward and cost-effective approach to the development of an AV research platform is sparse. To address this need, we present the methodology and results regarding the AV instrumentation and controls of a 2019 Kia Niro which was developed for a local AV pilot program. This platform includes a drive-by-wire actuation kit, Aptiv electronically scanning radar, stereo camera, MobilEye computer vision system, LiDAR, inertial measurement unit, two global positioning system receivers to provide heading information, and an in-vehicle computer for driving environment perception and path planning. Robotic Operating System software is used as the system middleware between the instruments and the autonomous application algorithms. After selection, installation, and integration of these components, our results show successful utilization of all sensors, drive-by-wire functionality, a total additional power* consumption of 242.8 Watts (*Typical), and an overall cost of $118,189 USD, which is a significant saving compared to other commercially available systems with similar functionality. This vehicle continues to serve as our primary AV research and development platform.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2025

Addressing tokens dynamic generation, propagation, storage and renewal to secure the GlideinWMS pilot based jobs and system

GlideinWMS has been one of the first middleware in the WLCG community to transition from X.509 to support also tokens. The first step was to get from the prototype in 2019 to using tokens in production in 2022. This paper will present the challenges introduced by the wider adoption of tokens and the evolution plans for securing the pilot infrastructure of GlideinWMS and supporting the new requirements. In the last couple of years, the GlideinWMS team supported the migration of experiments and resources to tokens. Inadequate support in the current infrastructure, more stringent requirements, and the higher spatial and temporal granularity forced GlideinWMS to revisit once more how credentials are generated, used, and propagated. The new credential modules have been designed to be used in multiple systems (GlideinWMS, HEPCloud) and use a model where credentials have type, purpose, and different flows. Credentials are dynamically generated in order to customize the duration and limit the scope to the targeted resource. This allows to enforce the least privilege principle. Finally, we also considered adding credential storage, renewal, and invalidation mechanisms within the GlideinWMS infrastructure to better serve the experiments' needs.

Coimbra, Bruno Moreira [Fermilab] (ORCID:000900022↗
DOE OSTI2024

Myna

The additive manufacturing (AM) community has been developing digital factory tools over the past decade to better leverage the multi-modal process data coming out of the advanced manufacturing process. As a result, numerous databases of additive manufacturing process data exist in the literature and in the archival storage of disparate research groups. While some efforts have been made to create a standard ontology for storing and sharing AM data, in practice a variety of data structures are used to store AM build data, even within a single institution. This causes many problems for maintainability and extensibility when attempting to integrate computational modeling tools with experimental data to either validate models or to provide further insight into results and trends. Myna is a Python-based framework that aims to decrease the effort needed to connect individual computational models to the variety of AM process data that exist in different research groups and institutions. This type of software is sometimes referred to as "middleware" or “glueware,” in that it connects disparate databases and applications into a single computational ecosystem. Instead of maintaining unique interfaces between each application and each database, developers can create a single interface from each application to Myna and thereby gain access to the implemented database connections. Similarly, developing a database connection in Myna provides access to the developed simulation applications. This framework greatly simplifies the maintainability of model applications that rely on experimental data. Using external simulation tools, users will also be able to run pre-configured workflows using the built-in workflow manager. Several examples of input files are provided with Myna for different workflows, including melt pool geometry predictions and detailed melt pool and solidification microstructure predictions.

Knapp, GerryL. [Oak Ridge National Laboratory (ORN↗
DOE OSTI2021

Priority-enabled Load Balancing for Dispersed Computing

Opportunistic managed access to local in-network compute resources can improve the performance of distributed applications and reduce the dependence on shared network resources. Instead of backhauling application data to a centralized cloud data center for processing, networked services may be adaptively and continuously dispersed into shared compute resources that are closer to the source of need. While this approach has several benefits, support for mission-aware access to computation is often an afterthought, and is implemented as a brittle extension over traditional load-balancer solutions.In this work, we investigate the design of two priority-aware resource allocation strategies and two load-balancing dispatching strategies as first class citizens in an open-source dispersed computing middleware. We present a control theoretic analysis of these load-balancing primitives to identify weaknesses and strengths in our design, and recommend future directions. In parallel, we prototype two priority-aware allocation algorithms to validate our priority predictions. In initial experiments our prototype shows substantial gains in processing prioritized load. Finally, we make our source-code and experimental configurations open source.

Paulos, Aaron↗
DOE OSTI2023

$\mathrm{RADICAL}$-Pilot and $\mathrm{PMIx}$/$\mathrm{PRRTE}$: Executing Heterogeneous Workloads at Large Scale on Partitioned $\mathrm{HPC}$ Resources

Execution of heterogeneous workflows on high-performance computing (HPC) platforms present unprecedented resource management and execution coordination challenges for runtime systems. Task heterogeneity increases the complexity of resource and execution management, limiting the scalability and efficiency of workflow execution. Re-source partitioning and distribution of tasks execution over portioned re-sources promises to address those problems but we lack an experimental evaluation of its performance at scale. Here this paper provides a performance evaluation of the Process Management Interface for Exascale (PMIx) and its reference implementation PRRTE on the leadership-class HPC plat-form Summit, when integrated into a pilot-based runtime system called RADICAL-Pilot. We partition resources across multiple PRRTE Distributed Virtual Machine (DVM) environments, responsible for launching tasks via the PMIx interface. We experimentally measure the work-load execution performance in terms of task scheduling/launching rate and distribution of DVM task placement times, DVM startup and termination overheads on the Summit leadership-class HPC platform. Integrated solution with PMIx/PRRTE enables using an abstracted, standardized set of interfaces for orchestrating the launch process, dynamic process management and monitoring capabilities. It extends scaling capabilities allowing to overcome a limitation of other launching mechanisms (e.g., JSM/LSF). Explored different DVM setup configurations provide insights on DVM performance and a layout to leverage it. Our experimental results show that heterogeneous workload of 65,500 tasks on 2048 nodes, and partitioned across 32 DVMs, runs steady with resource utilization not lower than 52%. While having less concurrently executed tasks resource utilization is able to reach up to 85%, based on results of heterogeneous workload of 8200 tasks on 256 nodes and 2 DVMs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Programming Abstractions for Managing Workflows on Tiered Storage Systems

Scientific workflows in High Performance Computing (HPC) environments are processing large amounts of data. The storage hierarchy on HPC systems is getting deeper, driven by new technologies (NVRAMs, SSDs, etc.) There is a need for new programming abstractions that allow users to seamlessly manage data at the workflow level on multi-tiered storage systems, and provide optimal workflow performance and use of storage resources. In previous work, we introduced a software architecture Managing Data on Tiered Storage for Scientific Workflows (MaDaTS) that used a Virtual Data Space (VDS) abstraction to hide the complexities of the underlying storage system while allowing users to control data management strategies. In this article, we detail the data-centric programming abstractions that allow users to manage a workflow around its data on the storage layer. The programming abstractions simplify data management for scientific workflows on multi-tiered storage systems, without affecting workflow performance or storage capacity. We measure the overheads and effectiveness introduced by the programming abstractions of MaDaTS. Our results show that these abstractions can optimally use the storage capacity in lesser capacity storage tiers, and simplify data management without adding any performance overheads.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2025

Final Technical Report

Explore the source record for details and available documents.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Intern Poster Session 08/13: Autonomous Nuclear Robotics: Applications in nuclear waste inspection and hot cell experiments

The nuclear industry is experiencing renewed interest in autonomous robotics, yet most deployed systems remain teleoperated with limited autonomy. This work presents two contributions toward fully autonomous nuclear robotic systems: autonomous waste inspection at the Hanford Site and an autonomous hot cell laboratory framework. Inspections of Hanford's underground waste storage tanks are performed manually at significant cost and personnel exposure. We developed a reinforcement-learning (RL) training pipeline for a custom-built inspection arm. In parallel, we are designing an autonomous laboratory framework for post-irradiation examination in hot cells at the Specimen Preparation Laboratory (SPL) that integrates computer vision, task and motion planning, hardware execution, and operator-in-the-loop control. These systems demonstrate a path toward safer, more efficient nuclear operations by reducing human exposure while maintaining rigorous human oversight at critical decision points.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗