SEARCH · Engineering Papers

Results for “software infrastructure”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Generating SBoMs Utilizing Structured Threat Information Expression JSON bundles

With cyber-attacks on the rise, information is one of the first lines of defense. Now more than ever, security experts and developers need efficient ways to know if their software may be outdated or vulnerable to emerging threats. By generating a Software Bill of Material based on STIX2/JSON bundles, security experts and developers can ensure that they know where to find what can go wrong with their systems before the hackers do. As part of the Infrastructure Expression project, Idaho National Laboratory (INL) is currently working on ways that can aid in the development of efficient SBOMs using Structured Threat Information Expression (STIX). Doing so can provide an efficient first line of defense for detection and monitoring that can be easily used by cyber personnel from beginners to highly skilled experts.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

EmSense: A High-Resolution Emulated Sensor for Experiments with the Smart Grid and Distributed Ledger Technology

This work involves the development of a device - EmSense (“Emulated Sensor”) - that emulates a high-resolution sensor for a power grid. The device collects raw current and voltage sensor data which derive from ORNL's signature library. This library is a dataset that ORNL curates from many different sources that include power systems from various utilities. The EmSense packages the data from the library in the form of IEC 61850 Sampled Value (SV) packets and then broadcasts these SV packets on the network. In another mode, EmSense can generate artificial sinusoidal data that appears as waveforms for voltage and current signals. EmSense has an internal algorithm for determining the period of a signal based on the data so that the period can be specified as a variable in the IEC 61850 packets. The purpose of EmSense is to allow for experimentation with the Dark Net Infrastructure where a variety of power line sensors must be represented along with their typical communication traffic. The EmSense device was developed in coordination with the software for receiving and processing the packets in the Distributed Ledger Technology (DLT) framework of the DarkNet Project. This receiving software must have a methodology for dealing with information of high velocity, variety, and volume. Experimenting with EmSense facilitates the development of such software. The results showed that the DLT framework and the trust-anchoring approach managed to process a large flow of traffic even with up to six instances of EmSense device broadcasting data. This was achieved without overfilling packet queues in the memory of the actual hardware of the DLT devices or causing the Central Processing Unit (CPU) of the hardware to be overwhelmed. The DLTs were also able to store the data in a compact and useful form for later analysis and archival purposes.

Werth, Aaron↗

DOE OSTI2024

Advanced Precipitation and Boundary Layer Data Products Derived from ARM Radar Wind Profilers

This research project was successful in delivering on four main objectives. First, software was developed to accurately calculate 915-MHz radar wind profiler (RWP) spectrum moments from the recorded Doppler velocity power spectra. Second, software was developed to calibrate the RWP reflectivity factor using collocated surface disdrometer observations. Third, the Python processing code was documented and given to the ARM Infrastructure to produce ARM ‘b level’ calibrated RWP products. Fourth, calibrated RWP products were uploaded to the ARM Archive as PI Products for 10 years of SGP RWP observations and for GoAmazon and TRACER field campaign RWP observations. In addition to working with RWP observations, this research project also worked with KAZR observations to distinguish insects from boundary layer clouds to help improve the ARSCL cloud mask product. The PI worked with senior and early career ARM funded scientists at BNL exploring how to include calibrated RWP moments into future versions of the ARSCL product.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

Lifting and Dropping VMs to Dynamically Transition Between Time- and Space-sharing for Large-Scale HPC Systems

As HPC environments increasingly integrate with edge based systems, system architectures will need to handle a broader class of workloads and scheduling requirements. One result of this shift will be the need to simultaneously support bulk-synchronous parallel (BSP) and on-demand service based applications on the same infrastructure. This in turn will require that future resource management approaches utilize both space-shared as well as time-shared resource scheduling strategies. In this work we introduce the concept of "VM-lifting'' (and its inverse "VM-Dropping'') which allows dynamically switching an HPC workload between space-shared and time-shared scheduling regimes. Our work targets co-kernel based HPC system software environments, in which multiple specialized OS kernels execute natively on dedicated physical resource partitions inside a single compute node. With VM-lifting, a native co-kernel can be migrated at runtime to and from locally hosted Virtual Machine Environments due to changing scheduling requirements of the node. This allows an HPC node to be dynamically (re-)configured as either a time-shared Infrastructure-as-a-Service (IaaS) resource or a dedicated space shared resource based on the current workload demands. We have implemented this approach in the context of the Hobbes Exascale System Software stack and have demonstrated that a node can be reconfigured with minimal impact on the running applications.

Gordon, Nick↗

DOE OSTI2021

Integration of Rucio in Belle II

The Belle II experiment, which started taking physics data in April 2019, will multiply the volume of data currently stored on its nearly 30 storage elements worldwide by one order of magnitude to reach about 340 PB of data (raw and Monte Carlo simulation data) by the end of operations. To tackle this massive increase and to manage the data even after the end of the data taking, it was decided to move the Distributed Data Management software from a homegrown piece of software to a widely used Data Management solution in HEP and beyond : Rucio. This contribution describes the work done to integrate Rucio with Belle II distributed computing infrastructure as well as the migration strategy that was successfully performed to ensure a smooth transition.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

OpenPATH and Itinerum Integration: A Case Study of Merging Open-Source Projects: Preprint

Throughout the transportation sector, multiple mobility tracking platforms have been developed independently, in parallel, with similar use cases and infrastructure. Maintaining such platforms on an individual level can result in duplicated effort. Innovation within the field could be slowed by the limited resource of people working on extremely similar projects. We provide a case study in how such platforms can be combined with low effort and provide an example in the mobility domain of what the resulting integration points, modules and ownership may look like. OpenPATH (formerly e-mission) and Itinerum are two examples of similar independently developed human mobility tracking platforms that extend beyond standard traffic data collection. Due to the open source nature of the projects, members from both teams decided to merge the functionality, thereby only hosting a single data collection process and limiting maintenance of separate intake platforms. On the front-end, specific questionnaire functionality from Itinerum, mainly the ability to display and return questions and results focused on sociodemographic attributes, was integrated into the OpenPATH user interface. On the back-end, an export pipeline stage was created to transport collected data back to the Itinerum database. Upon completion, the OpenPATH application retained its original functionality while adding the ability to export data for future data sharing and creating an avenue to incorporate Itinerum hosted questions and answers. Itinerum benefits from the rich feature set and infrastructure of OpenPATH without losing needed elements such as the questionnaire component. This case study, within the transportation sector, illustrates the various configurations in which open source software can be developed, maintained and deployed.

ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2019

BOSC 2019, the 20th annual Bioinformatics Open Source Conference

The Bioinformatics Open Source Conference is a volunteer-organized meeting that covers open source software development and open science in bioinformatics. Launched in 2000, BOSC has been held every year since. BOSC 2019, the 20th annual BOSC, took place as one of the Communities of Special Interest (COSIs) at the Intelligent Systems for Molecular Biology meeting (ISMB/ECCB 2019). The two-day meeting included a total of 46 talks and 55 posters, as well as eight Birds of a Feather interest groups. The keynote speaker was University of Cape Town professor Dr. Nicola Mulder, who spoke on “Building infrastructure for responsible open science in Africa”. Immediately after BOSC 2019, about 50 people participated in the two-day CollaborationFest (CoFest for short), an open and free community-driven event at which participants work together to contribute to bioinformatics software, documentation, training materials, and use cases.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

AtomAI framework for deep learning analysis of image and spectroscopy data in electron and scanning probe microscopy

Over the past several decades, electron and scanning probe microscopes have become critical components of condensed matter physics, materials science and chemistry research. At the same time, the infrastructure for establishing a connection between microscopy observations and materials behaviour over a broader parameter space is lacking. In this work, we introduce AtomAI, an open-source software package bridging instrument-specific Python libraries, deep learning and simulation tools into a single ecosystem. AtomAI allows direct applications of deep neural networks for atomic and mesoscopic image segmentation converting image and spectroscopy data into class-based local descriptors for downstream tasks such as statistical and graph analysis. For atomically resolved imaging data, the output is types and positions of atomic species, with an option for subsequent refinement. AtomAI further allows the implementation of a broad range of image and spectrum analysis functions, including invariant variational autoencoders for disentangling structural factors of variation and im2spec type of encoder–decoder models for mapping structure–property relationships. Finally, our framework allows seamless connection to the first principles modelling with a Python interface on the inferred atomic positions.

36 MATERIALS SCIENCE↗

DOE OSTI2020

Distributed Many-to-Many Protein Sequence Alignment using Sparse Matrices

Identifying similar protein sequences is a core step in many computational biology pipelines such as detection of homologous protein sequences, generation of similarity protein graphs for downstream analysis, functional annotation, and gene location. Performance and scalability of protein similarity search have proven to be a bottleneck in many bioinformatics pipelines due to increase in cheap and abundant sequencing data. This work presents a new distributed-memory software PASTIS. PASTIS relies on sparse matrix computations for efficient identification of possibly similar proteins. We use distributed sparse matrices for scalability and show that the sparse matrix infrastructure is a great fit for protein similarity search when coupled with a fully-distributed dictionary of sequences that allow remote sequence requests to be fulfilled. Our algorithm incorporates the unique bias in amino acid sequence substitution in search without altering basic sparse matrix model, and in turn, achieves ideal scaling up to millions of protein sequences.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

A Geodatabase Designed to Inform and Support Safe CO2 Transport-Route Planning

The National Energy Technology Laboratory developed the Carbon Capture and Storage (CCS) Pipeline Route Planning Database to inform safe and sustainable CO2 transport planning in support of decarbonization efforts. This comprehensive Esri Geodatabase contains over 90 GBs of data and 60+ spatial layers representing key considerations including natural hazards, infrastructure, energy, and social justice. Leveraging ArcGIS Pro, nationwide raster, and vector datasets containing millions of features were processed to be easily digestible for users and complex modeling software.

Romeo, Lucy↗

DOE OSTI2025

Modernizing Accelerator Responsiveness and Controls in Operations

Accelerators increasingly use artificial intelligence (AI) and machine learning (ML) software and workflows for a variety of tasks, from optimization to fault detection and recovery. Efficient and sustainable application of these technologies necessitates specialized and facility-specific infrastructure commitments. Accelerator facilities also introduce unique radiation and security hazards, placing additional demands on operational infrastructure. These needs further escalate the prioritization of effective collaboration models and associated funding mechanisms and legal frameworks.

43 PARTICLE ACCELERATORS↗

DOE OSTI2023

Development of a hardware-in-the-loop testbed for laboratory performance verification of flexible building equipment in typical commercial buildings

The goals of reducing energy costs, shifting electricity peaks, increasing the use of renewable energy, and enhancing the stability of the electric grid can be met in part by fully exploiting the energy flexibility potential of buildings and building equipment. The development of strategies that exploit these flexibilities could be facilitated by publicly available high-resolution datasets illustrating how control of HVAC systems in commercial buildings can be used in different climate zones to shape the energy use profile of a building for grid needs. This article presents the development and integration of a Hardware-In-the-Loop Flexible load Testbed (HILFT) that integrates physical HVAC systems with a simulated building model and simulated occupants with the goal of generating datasets to verify load flexibility of typical commercial buildings. Compared to simulation-only experiments, the hardware-in-the-loop approach captures the dynamics of the physical systems while also allowing efficient testing of various boundary conditions. The HILFT integration in this article is achieved through the co-simulation among various software environments including LabVIEW, MATLAB, and EnergyPlus. Although theoretically viable, such integration has encountered many real-world challenges, such as: 1) how to design the overall data infrastructure to ensure effective, robust, and efficient integration; 2) how to avoid closed-loop hunting between simulated and emulated variables; 3) how to quantify system response times and minimize system delays; and 4) how to assess the overall integration quality. Lessons-learned using the examples of an AHU-VAV system, an air-source heat pump system, and a water-source heat pump system are presented.

Chen, zhelun↗

DOE OSTI2020

CEEP (Cyber-Energy Emulation Platform) [SWR-20-102]

NREL's Cyber-Energy Emulation Platform (CEEP) provides the capability to realize cyber-energy security and resilience through automation and orchestration of virtualized systems and software defined networks for the electric grid. CEEP enables testing and validation of grid-security and -control methodologies as the grid evolves to include smart technologies/systems, such as virtualization and containerization of grid components, software defined networking, simulation and co-simulation frameworks, and hardware in the loop. CEEP is a modular system that can be distributed and deployed across different hardware infrastructure sizes and network architectures. For example, CEEP can visualize, emulate, and/or coordinate the Smart-Grid Network Visualization, Intrusion Detection, and Network Healing system. Using CEEP, intrusion-detection and network-self-healing solutions can be deployed at grid control centers, within secure private clouds, and in cyber-energy appliances.

Vaughan, Evan↗

DOE OSTI2022

Cyber Energy Emulation Platform (CEEP) [SWR-20-102]

Rivera, Joshua↗

DOE OSTI2026

MFC 5.0: An exascale many-physics flow solver

Many problems of interest in engineering, medicine, and the fundamental sciences rely on high-fidelity flow simulation, making performant computational fluid dynamics solvers a mainstay of the open-source software community. Previous work MFC 3.0 was made a published, documented, and open-source solver via Bryngelson et al. Comp. Phys. Comm. (2021) with numerous physical features, numerical methods, and scalable infrastructure. MFC 5.0 is a significant update to MFC 3.0, featuring a broad set of well-established and novel physical models and numerical methods, as well as the introduction of GPU and APU (or superchip) acceleration. Here, we exhibit state-of-the-art performance and ideal scaling on the first two exascale supercomputers, OLCF Frontier and LLNL El Capitan. Combined with MFC’s single-accelerator performance, MFC achieves exascale computation in practice, and achieved the largest-to-date public CFD simulation at 200 trillion grid points as a 2025 ACM Gordon Bell Prize finalist. New physical features include the immersed boundary method, N-fluid phase change, Euler–Euler and Euler–Lagrange sub-grid bubble models, fluid-structure interaction, hypo- and hyper-elastic materials, chemically reacting flow, two-material surface tension, magnetohydrodynamics (MHD), and more. Numerical techniques now represent the current state-of-the-art, including general relaxation characteristic boundary conditions, WENO variants, Strang splitting for stiff sub-grid flow features, and low Mach number treatments. Weak scaling to tens of thousands of GPUs on OLCF Summit and Frontier and LLNL El Capitan achieves efficiencies within 5% of ideal to over 90% of their respective system sizes. Strong scaling results for a 16-times increase in device count show parallel efficiencies over 90% on OLCF Frontier. MFC’s software stack has undergone further improvements, including continuous integration, which ensures code resilience and correctness through over 300 regression tests; metaprogramming, which reduces code length while maintaining performance portability; and code generation for computing chemical reactions

Computational fluid dynamics↗

DOE OSTI2024

SCA Tools - SCRM Value Add or Lossy Noise Machines

Software supply chain risk management (SCRM) depends upon accurate information regarding the software components that comprise any given software system. The collection of components included in a software package can be organized within a software bill of materials, or SBOM. SBOMs are ideally generated when the software components are put together, such as at compile time, but for many reasons that has not and is not always possible. For example, legacy or proprietary software packages often do not have SBOMs available to downstream consumers of that software. It’s not just end users that are affected, manufacturers themselves also must deal with this problem. To answer these questions, the market has seen the rise of several commercial software composition analysis (SCA) tools. These tools aim to peer into completed software systems, automatically identifying hidden software dependencies and looking up known vulnerabilities associated with those dependencies to enable end-users to enhance their cyber supply chain risk management processes. These tools are potentially a huge boon to end users of legacy and proprietary software – and a potential bane, depending on how accurate they are. This research asks that question – how accurate are currently available binary SCA tools – and provides answers to several other questions: What does it mean to be “accurate”? What limitations do the tools have in identifying common edge cases that take place in modern software development? Can they help you avoid a devastating supply chain attack, or is it all just noise? After researching SCA tools on the market, we identified three vendors that fit our use case and would provide analysis on compiled binaries. Using these tools, we submitted firmware for critical infrastructure devices for analysis and SBOM generation. The SBOM outputs were then cross referenced with SBOMs generated through manual analysis for comparison. In addition to the firmware samples, we also submitted edge case samples based off a popular open-source library that were specifically crafted to evaluate each tools’ ability to accurately identify components. These samples were customized to be consistent with modifications we have seen in modern software development as well as a couple that are representative of supply chain attacks.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Source Term Analysis of Xenon (STAX): An effort focused on differentiating man-made isotope production from nuclear explosions via stack monitoring

An overview of the hardware and software developed for the Source Term Analysis of Xenon (STAX) project is presented which includes the data collection from two stack monitoring systems installed at medical isotope production facilities, infrastructure to transfer data to a central repository, and methods for sharing data from the repository with users. STAX is an experiment to collect radioxenon emission data from industrial nuclear facilities with the goal of developing a better understanding of the global radioxenon background and the effect industrial radioxenon releases have on nuclear explosion monitoring. The final goal of this work is to utilize collected data along with atmospheric transport modeling to calculate the contribution of a peak or set of peaks detected by the International Monitoring System (IMS) to provide desired discriminating information to the International Data Centre (IDC) and National Data Centers (NDCs). Types of data received from the STAX equipment are shown and collected data was used for a case study to predict radioxenon concentrations at two IMS stations closest to the Institute for RadioElements (IRE) in Belgium. The initial evaluation of results indicate that the data is very valuable to the nuclear explosion monitoring community.

07 ISOTOPE AND RADIATION SOURCES↗

DOE OSTI2021

Secure Command Line Solution for Token-based Authentication

The WLCG is modernizing its security infrastructure, replacing X.509 client authentication with the newer industry standard of JSON Web Tokens (JWTs) obtained through the Open ID Connect (OIDC) protocol. There is a wide variety of software available using the standards, but most of it is for Web browser-based applications and doesn’t adapt well to the command line-based software used heavily in High Throughput Computing (HTC). OIDC command line client software did exist, but it did not meet our requirements for security and convenience. This paper discusses a command line solution we have made based on the popular existing secrets management software from Hashicorp called vault. We made a package called htvault-config to easily configure a vault service and another called htgettoken to be the vault client. In addition, we have integrated use of the tools into the HTCondor workload management system, although they also work well independent of HTCondor. All of the software is open source, under active development, and ready for use.

Dykstra, Dave↗