Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software infrastructure”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Using Pilot Jobs and CernVM File System for Simplified Use of Containers and Software Distribution

High Energy Physics (HEP) experiments entail an abundance of computing resources, i.e. sites, to run simulations and analyses by processing data. This requirement is fulfilled by local batch farms, grid sites, private/commercial clouds, and supercomputing centers via High Throughput Computing (HTC). The growing needs of such experiments and resources being prone to trends of heterogeneity make it difficult for physicists to handle these resources directly. Additionally, HEP collaborations heavily rely on data and software releases, typically in the order of tens of gigabytes, while conducting simulations and analyses. Hence, aspects of scalability, reliability, and maintenance become crucial with regards to the distribution of the necessary data and software stack. The GlideinWMS [4] framework helps with the resource management problem by using pilot jobs, aka Glideins, to provision reliable elastic virtual clusters. Glideins are submitted to unreliable heterogeneous resources which are validated and customized by the Glideins to make the worker nodes available for end-user job execution. On the other hand, the CernVM File System (CernVM-FS or CVMFS) [1] helps with data distribution. It is a write-once, read-everywhere filesystem used to deploy scientific software to thousands of nodes on a worldwide distributed computing infrastructure. CVMFS is based on the Hyper Text Transfer Protocol and has been widely used within the particle physics community for (1) distributing experiment software and data such as calibrations, and (2) facilitating containerization by efficiently hosting container images along with providing containerization software, especially Singularity [3] GlideinWMS relies on CVMFS installed locally on the computing resources to satisfy the experiments' software needs. This requires system administrators' effort to install and maintain CVMFS at the sites and limits the use of sites, especially HPC resources, that do not have CVMFS installed. This poster presents a solution, taking advantage of Glideins to provide CVMFS at most sites without the need for a local installation. Doing so expands the pool of resources available for HEP experiments and reduces the effort of system administrators for current resources. Additionally, the proposed solution allows GlideinWMS to also start Singularity [3], a containerization software that can run unprivileged, on sites where neither CVMFS nor Singularity are available, including HPC sites. The benefits provided by this solution are: (1) lower overhead for site administrators in that they have less software to install, (2) an expanded pool of resources that run user jobs with easy access to software and data provided by CVMFS, thus making life easier for the scientists, and (3) improved flexibility to use HPC resources by enabling GlideinWMS pilot jobs to support HPC sites.

Urs, Namratha↗
DOE OSTI2026

Trapped-ion Quantum Network Real-time Control Software (QuantNet Real-time Control) v0.1.0

This real-time control software for the QUANT-NET testbed is built upon the ARTIQ (Advanced Real-Time Infrastructure for Quantum physics) ecosystem. A central contribution of this work is the software's modular and reusable architecture, which facilitates the creation of hierarchical experimental control sequences by combining precomposed sequences and subsequences as building blocks within the framework.

Cheah, You-Wei [Lawrence Berkeley National Laborat↗
DOE OSTI2022

Emulation Framework for Distributed Large-Scale Systems Integration

Recent trends in systems engineering include integration of very large-scale systems, which entails significant challenges when they are geographically dispersed. In these scenarios, intelligent integration of distributed large-scale systems requires significant coordination among hardware elements as well as all software components. The approach of integrated systems (both computing platform and experimental equipment) for end-to-end orchestration is called federation. Virtual frameworks can aid in the testing, assessment, and implementation of a functional system of interconnected resources. We present an emulation framework that replicates the software environments of multi-site federations of computing systems and instruments. Our emulation framework allows systems engineers to reduce developmentcost and avoid disruptions to production infrastructure. Our framework was effectively used to develop and test software modules for various tasks including container orchestration and instrument access. For performance assessment, however, the emulated framework is severely limited in providing accurate network and IO measurements at 10 Gbps and higher data rates. The data transfer performance profiles estimated using these emulated measurements are usually inaccurate for high bandwidth and high latency connections, since emulation does not accurately reflect the critical network transport dynamics.We utilize measurements from a physical testbed with hardware network emulators to obtain data transfer profiles that closely match the expected profiles for the emulated federations. We show the effectiveness of our approach by an illustrative example of integrated (federated) multi-site ultra large-scale systems that are connected via high speed wide area networks.

Imam, Neena↗
DOE OSTI2022

components for the MuMMI software release

The Department of Energy and the National Cancer Institute have developed new software for conducting multi-scale simulations of complex systems. This software, called the Multiscale Machine-Learned Modeling Infrastructure (MuMMI), couples simulations on three spatial scales to study slow, large-scale reorganizations of biomolecular systems with the speed of continuum and coarse-grained models while revealing selected interactions at full atomic precision. In these simulations, coarse-to-fine model conversions are used to spawn relevant fine-scale simulations along chosen order parameters, and fine-to-coarse feedback is used to iteratively improve the accuracy and multi-scale consistency of coarse-scale and continuum simulations. The complete MuMMI framework will be reviewed and released as open-source software by Lawrence Livermore National Laboratory. This review covers a subset of the MuMMI components that were developed exclusively at the Los Alamos National Laboratory.

Neale, Christopher↗
DOE OSTI2021

Science Federation Emulation Testbed: Demonstration of VFSIE Functionalities

The Virtual Federated Science Instrument Environment (VFSIE) is a digital twin of a federated infrastructure of multiple sites with science instruments and computing systems that are geographically dispersed. It emulates the federation using containers and virtual hosts (vhosts) connected over local and wide-area networks for the main purpose of supporting the development and testing of science workflows and software stacks, which will subsequently transition to the physical infrastructure. In this demo, we present VFSIE of a four sites scenario and illustrate the following use cases: (i) execution of federation stack components that expose resources to enable a science user to execute workflows, (ii) execution of remote control commands of beam-line instruments to collect measurements and position the sensor, and (iii) remote execution and access of Jupyter Notebook for tomographic image computations

Al Najjar, Anees↗
DOE OSTI2024

Visualizing a Vulnerability: Its Connections to Hardware and Software

All Hazards Analysis (AHA) is a framework developed by Idaho National Laboratory that provides capabilities to collect, store, analyze, and visualize critical infrastructure information. A core function of AHA is its ability to simulate faults or outages in networks of infrastructure originating from a plethora of causes, ranging from natural disasters to cyberattacks. AHA utilizes Hardware and Software Bills of Material (HBOM and SBOM, respectively) along with Known Exploited Vulnerabilities (KEVs) to document the potential attack vectors for each piece of infrastructure. The objective of this contribution to AHA was to create a visualization tool that could capture the small details held in each individual artifact as well as preserve the large-scale connections that link them together to aid threat modeling.

58 GEOSCIENCES↗
DOE OSTI2022

Strengthening the Security of Operational Technology: Understanding Contemporary Bill of Materials

The evolution of cyber-physical infrastructure has made its security more challenging. The last few years have witnessed a convergence of hardware and software segments in various domains, including operational technology (OT) which is responsible for carrying out critical tasks such as monitoring and controlling power grids, nuclear plants, transportation, and emergency services. Both hardware and software encapsulate numerous open source and proprietary subcomponents, making it crucial for end-users to understand the composition of the products they are using. For example, wind turbines incorporate thousands of lines of code (software) used for the turbine's design, planning, operation, and analytics in addition to the numerous hardware subcomponents that construct it. Due to the highly complex nature of software and hardware, knowledge of the components and subcomponents is required to mitigate cyber vulnerabilities and defend against cyberattacks. There has also been a transformation from a traditional linear supply chain into a global, dynamic, diverse, and interconnected system. The digitization of the supply chain makes it easier to find and exploit vulnerabilities. Critical infrastructures (e.g., power grids, oil, natural gas, water, and wastewater) rely on OT to function, and if the OT is compromised, equipment damage and potential interruption of services could result. A significant security measure to protect OT systems from disruption is to develop a supply chain bill of materials (BoM) corresponding to the software and hardware used in OT, along with attestations amongst vendors and asset owners. A supply chain BoM is a proactive way to understand the inherent vulnerabilities in the system and mitigate them in advance of being exploited. BoMs bolster the trust placed in the digital infrastructure and enhance software supply chain security by sustaining the management of component obsolescence and compliance, along with the seclusion of unsafe segments of a specific product. Adopting BoM tools is becoming increasingly important across various government sectors, as evidenced by the recent U.S. executive order on cybersecurity (NIST 2021). This paper aims to classify BoMs based on structure, functionality, component type, and architecture. The work also discusses case studies to further highlight the benefits of BoMs. In addition, it identifies missing pieces in existing BoM implementations so that future research may identify bounds on where it could expect to make improvements and directly enable researchers to identify promising areas for exploration. Further, the authors provide valuable recommendations to tool developers, researchers, and standardizing organizations (policymakers), additionally benefitting critical infrastructure owners and government executives. This aids in paving a path for future work, thereby, providing suggestions to determine a tool for consumers that best suit their needs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

ELSI — An open infrastructure for electronic structure solvers

Routine applications of electronic structure theory to molecules and periodic systems need to compute the electron density from given Hamiltonian and, in case of non-orthogonal basis sets, overlap matrices. System sizes can range from few to thousands or, in some examples, millions of atoms. Different discretization schemes (basis sets) and different system geometries (finite non-periodic vs. infinite periodic boundary conditions) yield matrices with different structures. The ELectronic Structure Infrastructure (ELSI) project provides an open-source software interface to facilitate the implementation and optimal use of high-performance solver libraries covering cubic scaling eigensolvers, linear scaling density-matrix-based algorithms, and other reduced scaling methods in between. In this paper, we present recent improvements and developments inside ELSI, mainly covering (1) new solvers connected to the interface, (2) matrix layout and communication adapted for parallel calculations of periodic and/or spin-polarized systems, (3) routines for density matrix extrapolation in geometry optimization and molecular dynamics calculations, and (4) general utilities such as parallel matrix I/O and JSON output. The ELSI interface has been integrated into four electronic structure code projects (DFTB+, DGDFT, FHI-aims, SIESTA), allowing us to rigorously benchmark the performance of the solvers on an equal footing. Based on results of a systematic set of large-scale benchmarks performed with Kohn–Sham density-functional theory and density-functional tight-binding theory, we identify factors that strongly affect the efficiency of the solvers, and propose a decision layer that assists with the solver selection process. As a result, we describe a reverse communication interface encoding matrix-free iterative solver strategies that are amenable, e.g., for use with planewave basis sets.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

pystorms: A simulation sandbox for the development and evaluation of stormwater control algorithms

Advances in cyber–physical technologies have enabled real-time sensing and adaptive control of stormwater infrastructure. These smart stormwater systems allow for inexpensive, minimally-invasive stormwater control interventions in lieu of new construction. Importantly, however promising the area of smart stormwater control, there still remain barriers – for experts and novices alike – to access shared tools and methods for investigating, developing, and contributing to it. In an effort to make smart stormwater control research more methodical and accessible, we present pystorms, an open-source Python-based simulation sandbox that facilitates the quantitative evaluation and comparison of control strategies. pystorms consists of a collection of real world-inspired smart stormwater control scenarios on which any number of control strategies can be applied and tested via an accompanying Python programming interface and coupled stormwater simulator. pystorms provides a framework for the rigorous and efficient evaluation of smart stormwater control methodologies across diverse watersheds with only a few lines of code.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Sandia Toolkit Manual (V.5.21.1)

This report provides documentation for the Sandia Toolkit (STK) modules. STK modules are intended to provide infrastructure that assists the development of computational engineering software such as finite-element analysis applications. STK includes modules for unstructured-mesh data structures, reading/writing mesh files, geometric proximity search, transfers, MPMD coupling support, and various other utilities. This document contains a chapter for each module, and each chapter contains overview descriptions and usage examples. Usage examples are primarily code listings which are generated from working test programs that are included in the STK code-base. A goal of this approach is to ensure that the usage examples will not fall out of date.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Sandia Toolkit Manual Version 5.15.6

This report provides documentation for the Sandia Toolkit (STK) modules. STK modules are intended to provide infrastructure that assists the development of computational engineering software such as finite-element analysis applications. STK includes modules for unstructured-mesh data structures, reading/writing mesh files, geometric proximity search, and various utilities. This document contains a chapter for each module, and each chapter contains overview descriptions and usage examples. Usage examples are primarily code listings which are generated from working test programs that are included in the STK code-base. A goal of this approach is to ensure that the usage examples will not fall out of date.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Bridging Cloud and Edge Computing at NREL Using CONNECT: Cloud Optimized Networking for Next-Gen Edge Computing Technologies [Slides]

CONNECT is an innovative on-premise hardware and software solution that integrates edge and cloud computing infrastructure at NREL. Built on the AWS Greengrass middleware and leveraging the MQTT protocol, CONNECT enables real-time data streaming from IoT devices and gateways to both cloud and local services, empowering researchers to rapidly capture, analyze, and act upon edge-generated data while leveraging cloud capabilities. The platform addresses research infrastructure challenges by providing a pre-approved platform which is already configured with the correct networking and cybersecurity baselines thus eliminating procurement delays and enabling on-demand availability. CONNECT's hybrid architecture efficiently manages burstable workloads, allowing research teams to dynamically scale computational capacity, handle peak data loads, and reduce operational bottlenecks. Advanced capabilities include built-in GPU support for executing machine learning models which enables low-latency inference at the edge from models trained in the cloud. This architecture supports real-time analytics and filtering, providing a mechanism to allow only transmitting and processing high-value data. Cloud-based configuration management permits engineers to manage on-premise systems remotely, optimizing operational efficiency. By bridging edge and cloud computing, CONNECT provides NREL researchers with a flexible, scalable platform that accelerates scientific discovery while maintaining robust security and performance standards.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Results of Application Demonstration of Autonomous Offshore Power System Supporting Multiple Mobile and Static Assets

Over the last decade, several technology trends have dominated the innovation cycle within the terrestrial economy. Electrification, cloud computing, decarbonization, digitization, and autonomy are representative examples of macro trends that have had a rapid and fundamental impact; an impact felt not only in user-facing technologies, such as electric vehicles and software-as-a-service, but also in the infrastructure required to deliver those technologies. However, the scale of the impact and rollout has not been felt equally between the terrestrial and ocean economies. Here, the innovation cycle in the ocean economy is significantly slower. What accounts for the difference and are there solutions to this issue?

16 TIDAL AND WAVE POWER↗
DOE OSTI2024

Focused Ion Beam Tomography of Alloy 617 Corroded in Molten Chloride Salt

Materials qualification of reactor structural materials is a critical step in rapid implementation of advanced nuclear reactor technologies, particularly to assess the corrosion performance in these designs. Accelerated qualification of reactor structural materials requires incorporating powerful computational toolsets, such as phase field modelling in the Multiphysics Object-Oriented Simulation Environment (MOOSE) framework, to predict the evolution of structural materials due to corrosion. Accordingly, computational toolsets will require experimental data generated at appropriate length scales to validate accuracy. Focused ion beam (FIB) provides a high degree of control over manipulation of materials for analytical purposes, including capturing data on the evolution in the microstructure and elemental composition of materials at the mesoscale, an appropriate length scale for phase field modelling of intergranular diffusion phenomena using the MOOSE framework. For instance, the FEI Helios G4 UX dual beam plasma FIB microscope at the Irradiated Materials Characterization Laboratory (IMCL) is capable of backscatter diffraction (EBSD) and energy-dispersive x-ray spectroscopy (EDS) documenting the evolution in the microstructure and elemental composition, respectively. The Helios can perform EDS and EBSD three-dimensionally (3D) using tomography, which is then combined using different software packages to visualize 3D volumes correlating elemental composition to microstructural data. The purpose of this investigation was to develop a streamlined characterization and data processing workflow for 3D tomography studies on the FEI Helios G4 plasma FIB. The investigation is segmented into three parts: 1) Optimizing the data collection workflow, 2) identifying appropriate data processing and visualization software (i.e. DREAM.3D, MIPAR, and VGStudioMax), and 3) establishing an infrastructure for public release. The optimization of the data collection workflow is in collaboration with members of the U220 department to setup formal training on the tomography operation of the G4, through ThermoFisher Scientific, and exploring DREAM.3D, MIPAR, and VGStudioMax data processing/visualization software packages. VGStudioMax currently demonstrates the most promise for future use. Optimization of the data collection and processing workflow is still ongoing. A collaboration with INL High Performance Computing (HPC) established an open-source license for expediting the public release of FIB tomography datasets through HPC. FIB tomography data generated by the G4 will provide comprehensive data for validating 3D phase field mesoscale modelling tools within the MOOSE framework for accelerated qualification of reactor structural materials.

Copeland-Johnson, Trishelle↗
DOE OSTI2023

Energy-Efficient and Resilient Infrastructure: Simulation, Validation, and Installation

Advanced, high-performance computing at the National Renewable Energy Laboratory (NREL) has enabled access to vast data resources with cutting-edge software techniques to understand, design, plan for, and maintain energy-efficient and resilient infrastructure. We have focused on cities and airports, but the technology we have developed will easily translate to seaports, inland ports, military installations, or other complex and large-scale energy-intensive systems. We can digitally simulate and explore current and future scenarios to make datadriven decisions for optimizing advanced energy systems, transportation and building operations, infrastructure planning and expansion, and battery storage to guide short- and long-term investments, electrification strategies, and integration of new technologies.

Athena↗
DOE OSTI2025

Advanced Transmission Technologies – GETs and HPCs Session 2: Advanced Power Flow Control and Transmission Topology Optimization

The INL TADA GETs Cohort Session 2, held on November 7, 2025, conducted in collaboration with ScottMadden, focused on two core Advanced Transmission Technologies (ATTs): Advanced Power Flow Control (APFC) and Transmission Topology Optimization (TTO). These technologies are pivotal in enhancing grid flexibility, reliability, and cybersecurity resilience. APFC, particularly through modular FACTS devices like Modular Static Synchronous Series Compensators (M-SSSCs), enables dynamic voltage injection to reroute power flows. The session highlighted the deployment benefits of APFC, such as rapid installation, minimal civil works, and re-deployability. Regulatory drivers like FERC Order 2023 mandate the inclusion of Grid-Enhancing Technologies (GETs) in interconnection studies. Case studies from Central Hudson, CAISO, and National Grid (UK) demonstrated APFC’s effectiveness in congestion relief and cost savings. The session also addressed cybersecurity concerns, including firmware vulnerabilities, SCADA integration risks, and supply chain dependencies. Participants engaged in interactive exercises to rank cybersecurity and supply chain risks, emphasizing the need for robust digital assurance strategies. TTO involves software-based reconfiguration of transmission networks to optimize power flow without new infrastructure. The session showcased its operational value, with examples from SPP, PJM, and MISO showing significant congestion cost reductions. Cybersecurity vulnerabilities were discussed, particularly in API security and software supply chains, referencing incidents like SolarWinds and attacks on Danish utilities. Digital assurance exercises explored worst-case scenarios, attack paths, and mitigation responsibilities between vendors and utilities. Reliability challenges such as algorithm stability, vendor dependency, and operator trust were also examined. Cross-cutting themes emphasized the importance of digital assurance tools, including Software Bills of Materials (SBOMs) and hardware-in-loop testing. Human performance, training, and operational confidence were identified as critical enablers of technology adoption. The session concluded with a preview of Session 3, which will focus on High Performance Conductors (HPCs) and risk-based cybersecurity tools. Session 2 of 3.

24 - POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Future energy infrastructure, energy platform and energy storage

Today fossil energy dominates energy consumption across the world. There has been an increasing momentum to reduce fossil energy consumption and increase renewable energy utilization to more than 70%. Such high penetrations of distributed renewable resources bring large uncertainty and complexity that cannot be easily handled by the current infrastructure. For the last two decades, telecommunication industry has gone through a similar transition from centralized to distributed systems, and more recently to platform approaches, which become the corner stone of today’s vibrant platform economy. Based on telecommunication history, we believe that a platform-based approach, called the energy platform, is a viable solution for addressing the renewable energy challenges. The energy platform consists of an array of computational algorithms, sensing and control technologies for key industry, energy generators and users to jointly manage and control the complex energy infrastructure. It includes the following key components: (1) the hardware and software to generate, store, control and transmit electricity/data (the energy cloud), (2) the digital platforms and tools for key energy producers, digital, service and communication industry to share the assets, participate in and manage the complex infrastructure, and for customers to participate in energy transactions, (3) the transaction platform for new service and business models. Additionally, the energy platform requires breakthroughs in large scale energy storage and many other areas including efficient power electronics, sensors and controls, new mathematical and computational tools, and deep integration of energy technologies and information sciences to control and stabilize such complex chaotic systems.

25 ENERGY STORAGE↗
DOE OSTI2020

The challenges of theory-software translation

Background: Software is now ubiquitous within research. In addition to the general challenges common to all software development projects, research software must also represent, manipulate, and provide data for complex theoretical constructs. Ensuring this process of theory-software translation is robust is essential to maintaining the integrity of the science resulting from it, and yet there has been little formal recognition or exploration of the challenges associated with it. Methods: We thematically analyse the outputs of the discussion sessions at the Theory-Software Translation Workshop 2019, where academic researchers and research software engineers from a variety of domains, and with particular expertise in high performance computing, explored the process of translating between scientific theory and software. Results: We identify a wide range of challenges to implementing scientific theory in research software and using the resulting data and models for the advancement of knowledge. We categorise these within the emergent themes of design, infrastructure, and culture, and map them to associated research questions. Conclusions: Systematically investigating how software is constructed and its outputs used within science has the potential to improve the robustness of research software and accelerate progress in its development. We propose that this issue be examined within a new research area of theory-software translation, which would aim to significantly advance both knowledge and scientific practice.

97 MATHEMATICS AND COMPUTING↗