Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “software requirements”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

Integrated System Planning: Emerging Software Requirements in the Power Industry

Power system planning software remains fragmented across organizational boundaries, with specialized tools for capacity expansion, production cost modeling, power flow, and dynamic analysis operating on incompatible data models and assumptions. This article argues that the fragmentation is not merely a technical problem but a predictable consequence of Conway's law: software architectures mirror the departmental structures within which they are developed. Regulatory milestones like Federal Energy Regulatory Commission (FERC) Order 888 formalized these divisions, but the roots trace back to the distinct engineering disciplines-mechanical, chemical, and electrical-that staffed generation and transmission planning departments in vertically integrated utilities. As the industry moves toward integrated system planning (ISP) that coordinates generation, transmission, and distribution investment decisions, the software ecosystem must evolve accordingly. We identify five categories of software requirements to enable this transition: coherent data inputs decoupled from individual applications, unified and extensible data schemas, modular component representations that support multiple abstraction levels, lifecycle management of planning datasets, and well-defined application programming interface (API) contracts that separate data exchange from algorithmic control. We examine how these requirements interact with three common workflow patterns-serial gate clearing, sequential multiapplication, and convergence oriented-and discuss the interface design principles each demands. We then outline a vision for platform-based planning architectures where specialized analytical services compose through standardized interfaces and where artificial intelligence (AI)/machine learning (ML) tools augment decision support within a disciplined software infrastructure. The practices proposed here offer a path from today's siloed tool collections toward collaborative planning ecosystems capable of handling the complexity of modern power system transformation.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

(U)Vendor Sanitization Requirements – SIEMENS Control

This document defines the software requirements for the sanitization of the IPC and or the controller. The system is intended to manage part programs and associated data (e.g. simulations, probing data) by capturing metadata, securely sanitizing files, logging operations, backing up data to a defined path, deploying a user defined program to a controller, and restoring part programs and associated data when required.

42 ENGINEERING↗
DOE OSTI2025

Probabilistic Impact Assessment and Software Tool Requirements

This research project will address the most pressing uncertainties in modeling and measuring the electric power grid effects of geomagnetic disturbances (GMDs) and the E3 portion of nuclear electromagnetic pulse (EMP). The primary goal is to help decision-makers in the electric power sector have the knowledge and tools they need to most effectively mitigate GMD effects on the North American electric grid, with a secondary focus on EMP response. Primary tasks will involve comprehensive modeling,

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2026

A Review of Software for Designing and Operating Quantum Networks

Quantum networks development is crucial to realizing a production-grade network that can support distributed sensing, secure communication, and utility-scale quantum computation. However, the transition from laboratory demonstration to deployable networks requires software implementations of architectures and protocols tailored to the unique constraints of quantum systems. This paper reviews the current state of software implementations for quantum networks, organized around a three-plane abstraction of infrastructure, logical, and control/service planes. We cover software for both designing quantum network protocols (e.g., SeQUeNCe, QuISP, and NetSquid) and operating testbeds, with a focus on essential control/service plane functions such as entanglement, topology, and resource management, in a proposed taxonomy. Our review highlights a persistent gap between theoretical architecture and protocol proposals and their realization in simulators or testbeds, particularly in dynamic topology and network management. We conclude by outlining open challenges and proposing a roadmap for developing scalable software architectures to enable hybrid, large-scale quantum networks.

Network Design↗
DOE OSTI2025

Reducing the Energy Consumption of Magnetic Resonance Imaging and Computed Tomography Scanners: Integrating Ecodesign and Sustainable Operations

This review aims to provide valuable insights into how energy consumption in magnetic resonance imaging (MRI) and computed tomography (CT) scanners can be effectively monitored, managed, and reduced, thereby contributing to more sustainable medical imaging practices. Demand for advanced imaging technologies such as MRI and CT scanners continues to increase, and understanding the resultant impact on greenhouse gas emissions requires a thorough evaluation of their energy consumption. Here, this review examines the energy monitoring and consumption characteristics of MRI and CT scanners, highlighting potential approaches for energy savings. An overview of MRI and CT principles, hardware components, and their associated energy consumption is provided. After addressing the technical aspects, the hardware and software requirements essential for accurate energy metering are detailed. Baseline measurements of energy consumption data are then provided as a foundation to understand current usage patterns and identify areas for improvement. Ongoing efforts to reduce energy consumption are categorized into 3 main strategies: operations, scanner design enhancements, and active scanning techniques, including accelerated MRI protocols. Ultimately, we emphasize that achieving sustainability in medical imaging requires collaboration across disciplines. By incorporating eco-friendly design in new imaging equipment, we can reduce the environmental impact, promote sustainability, and set a health care industry standard for a healthier planet.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2025

Datum: A Scientific Metadata Catalog

The data catalog market is currently flooded with a myriad of different products, but none serve the scientific community well. There are cloud-native tools like Databricks, Snowflake,to on-premise solutions like Collibra and Datahub. The common failing of all these tools however, is their inability to serve the scientific data community directly. Most catalogs are targeted towards financial, health, or user data - not sensor or scientific domain data. They also prioritize integrations that often don’t exist or are just starting to be used in the scientific realm - all while ignoring common scientific tools and file types. Datum is a catalog which targets the scientific data directly, including the tools and networks in which those tools are used. We work with the producers and consumers of the data where they are, targeting cloud and on-premise with a focus on classified networks. Datum is an Erlang/Elixir application. Technical Features Note: The features listed below are still under development and may change, slightly, upon final delivery of the product. File Formats - Datum has the ability to read additional metadata and provides processing pipelines for the following file formats: Plain Text, PDF, LaTeX, HTML, Open Document Format (.odt), XML, CSV/TSV (and other standard delimiters), OpenDocument Database and Spreadsheets, Geo-Referenced TIFF, Common Data Format, HDF/HDF5, LabView TDMS, Excel, DeltaTables, Parquet, Apache Iceberg, Apache Hudi and many others. Metadata Collection - Scanners for the local and networked file systems and cloud storage providers. Network integration with common databases such as MSSQL and MySQL. User Plugin System - Users are able to provide either file processing, metadata extraction, or sampling plugins in the programming language of their choice. Authentication/Authorization -: OIDC integration, SCIM provisioning and EntraID integration out of the box. Full user and group management system with a “least privilege” operating mode. Governance - Customizable data governance platform; dictate and enforce required metadata, enforce data embargos, and enforce user agreements and NDAs before data access. Ability to create health checks on data, rejecting abandoned or poorly curated data and automatically removing it from the search index. Ability for users to submit corrections. Search - Semantic search is a first class citizen. No licenses to expensive, external software required. Integrated use of vectors and vector-based search allows for AI agent integration at all levels of operation. Metadata Model - Display and control data’s lineage and connections to other data and data directories. Data is modeled after a filesystem - an organization instantly recognizable and navigable by most any user. CLI and SDK - Ships with a Command Line Interface (CLI) tool and with a fully-featured Python SDK. This allows for rapid and programmatic use of Datum by every level of user. Minimal Infrastructure - Datum ships as a single executable file and can be run on any operating system and most CPU architectures. Datum has no reliance on external databases, search indexing tools, or other outside services - and it runs equally well on edge computing devices, cloud services, or in a clustered HPC environment.

darrington, john↗
DOE OSTI2024

Modernizing the Legacy Fission Wire Measurement System for the Advanced Test Reactor-Critical Facility

Operational lifetime extensions of existing research reactors have emphasized the need for refurbishment, replacements, and upgrades to supporting equipment and instrumentation. The Advanced Test Reactor (ATR) at Idaho National Laboratory (INL), which entered service in 1967, has recently completed the sixth core internals change-out and has scheduled operations until at least 2040. Reactor maintenance and operational risk management is critically important in the research reactor community, however supporting measurement systems sometimes get overlooked when maintenance is planned. The Fission Wire Measurement System (FWMS) is a custom measurement system designed in the 1960s to measure the beta-particle activity of irradiated uranium-aluminum fission wires. This measurement is conducted to determine the fission rate profile of the Advanced Reactor Test Critical (ATR-C) facility. The ATR-C is an open-pool, low-power test reactor that was purpose driven to resemble ATR and is used to qualify experiment configurations and verify core models prior to full-power experiment irradiations in ATR. A power distribution measurement in ATR-C uses uranium-aluminum wires that are distributed throughout the ATR-C core to validate simulation and modeling results. These measurements require 340 to 1500 wires to be irradiated and measured within a 12-hour window. The activity of the wires is measured in the required time with the FWMS, which was put into service in 1965 at the Radiation Measurements Laboratory (RML). The system consists of 4 measurement channels and one reference channel, each with a 2-pi proportional gas flow detector and the measurement channels each have an automated sample changer. This legacy system is crucial to the continued operations of ATR and has undergone some minor hardware upgrades since 1965, however the system presently relies on custom control boards, custom gas ion chambers, analog amplifiers/discriminators, and a user interface (UI) for the system written in outdated code. Much of the equipment and software is custom with no commercial replacements or support and limited documentation. The existing control software requires an operating system that is no longer supported, creating more vulnerabilities to continued operations. A project is underway with a third-party vendor to design, build, and document a new control and data acquisition system (CDAS) for the FWMS. The new upgrade will replace the control system, computer, UI, sample changer motors, and main power supply while maintaining the interface with existing detector hardware. The upgraded system will be operated in parallel with the current hardware and software to conduct validation testing. This equipment upgrade demonstrates the commitment at ATR to ensuring successful operations and potential future research reactors at INL.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗
DOE OSTI2025

Tutorial: Machine-Learning-Based CREASE-2D Analysis of 2D SAXS Profiles to Characterize Anisotropic Nanostructures in Soft Materials

We present a tutorial to guide users on how to extend the Computational Reverse Engineering Analysis of Scattering Experiments-2D (CREASE-2D) framework to interpret their experimental two-dimensional small-angle scattering (SAS) data from soft materials (e.g., polymers, peptide amphiphiles, biomolecular fibrils). Unlike most traditional SAS analysis approaches, which typically rely on azimuthally averaged onedimensional (1D) profiles, CREASE-2D utilizes the complete 2D scattering profile to reveal information about anisotropy in the structure. In past applications, CREASE has provided insights into complex structural features, including the cross-sectional shapes of assembled nanostructures and dispersity in these features, which are difficult to discern with existing analytical models. While (1D- ) CREASE has been applied to SANS and SAXS data, this tutorial shares the steps for implementing CREASE-2D using an example of a dipeptide solution system, for which we have SAXS data. We present details for these steps involved in using CREASE-2D to interpret SAXS profiles: how to preprocess SAXS data, define relevant structural features, generate three-dimensional real-space structures for specific values of these features, train a machine learning (ML) surrogate model to predict scattering profiles for given structural features, and optimize these features using genetic algorithms (GA). Then, we use these steps to interpret complex 2DSAXS data collected from dipeptide solutions that, in microscopy images, exhibit nanoscale structures that could be elliptical tubes/ flat tapes/cylinders or a combination of these cross sections. Open-source codes, computational hardware, and software requirements, as well as the strengths and limitations of this protocol, are also presented. We expect researchers working with (soft) biomaterials, peptide amphiphiles, amphiphilic polymer solutions, polymer nanocomposites, and blends of particles/polymers will find this CREASE-2D method and this tutorial of use.

CREASE↗
DOE OSTI2026

Data from: Lowland Tropical Forests Remain a Methane Sink Under Warming and Long-Term Hurricane Disturbance Recovery

The repository folder contains spreadsheets and script for soil greenhouse gas (GHG) fluxes, soil moisture, soil temperature, air temperature, and precipitation measurements collected from the Tropical Responses to Altered Climate Experiment (TRACE) at the Sabana Research Field Station, El Yunque National Forest (USDA Forest Service; 18°19′28.74″ N, 65°43′50.09″ W) — an open-air field warming experiment located in a lowland tropical forest in Puerto Rico within the Luquillo Experimental Forest (LEF) — six to seven years after Hurricanes Irma and Maria (2017). All spreadsheets for soil and air microclimate data, as well as soil greenhouse gas data, are included as csv files. Air temperature data are also included as Excel spreadsheets (.xlsx). The script is built in R Studio, which is the only software required to run data analysis. This dataset is associated with the manuscript “Larocca Conte G ; Zuvela L ; Cruz-Pérez R ; Barreto-Vélez T ; Becerra-Santillan N ; Campbell S ; Chu H ; Dam T ; Grullón-Penkova I ; Kleit M ; Ortiz-Iglesias D ; Rubio-Lebrón L ; Cavaleri M ; Reed S ; Sihi D ; Wood T ; O'Connell C., 2026. Lowland Tropical Forests Remain a Methane Sink Under Warming and Long-Term Hurricane Disturbance Recovery. Agricultural and Forest Meteorology. In review". The dataset was used to test the effect of warming on soil CH4 dynamics following long-term legacy effects of hurricane disturbance. The dataset includes: - An overall README file in word and pdf format describing methodology and spreadsheets’ structure. - Continuous measurements of soil temperature and moisture from January 2023 to July 2024 measured with Campbell CS655 probes (“TRACE_soil_temperature_and_moisture_2023_cleaned(in).csv” and “TRACE_soil_temperature_and_moisture_2024_cleaned. csv”). - Air temperature data measured with a HOBO MX23O1A data logger (“Hobo air temperature 2023 Sep 2024” and “Hobo air temperature 2023 Sep 2024” – “CSV FILES folders”). - Precipitation data from a nearby weather tower downloaded from González et al. (2025; “sabana_2020-2025.csv”). - Soil CH4 and CO2 effluxes measured intermittently in two summer campaigns (June – August 2023 and June – July 2024) with a LI-COR 8200-01S Portable Smart Chamber coupled with a LI-COR LI-7810 CH4/ CO2/H2O Trace Gas Analyzer (“23_24COMBO2.0.csv”). - R markdown script for data analysis (“Trace new_PLOTS.Rmd”).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Large-scale Hydrogen Storage Risk Assessment

This project investigated risks involved in deploying a large-scale hydrogen storage system at the Port of Seattle (hereafter, the Port) for its on-terminal and maritime applications in an urban industrial setting. Alongside, the project attempted to address some of the barriers to risk assessment such as need for an exact system design for a systematic investigation, direct access to surrounding communities to gauge their perceptions, and an integrated software required to undertake a full-fledged risk assessment. These barriers were overcome using illustrative reference station designs, engaging with community-facing agencies through Port support, and pooling national laboratory capabilities available for risk assessments. The project identified relevant public safety risk metrics, compared various hydrogen carriers, engaged with community-facing agencies, and explored potential gaps in existing safety codes and standards. The primary impacts of this project include the development of risk assessment guidance for ports and utilities, informing them of the trade-offs in the choice of hydrogen carriers, and the ability to increase public capacity for dialog and engagement. This paves the way toward decarbonization of the Port activities, bringing about awareness around jobs in the market for risk assessments, and the need to ramp up community engagement long before any hydrogen system deployment is undertaken.

08 HYDROGEN↗
DOE OSTI2025

OpenCSP Step-by-Step: Getting Started Guide for Windows (V.1.0)

This document provides a step-by-step tutorial on how to set up OpenCSP for both users and developers. It is meant to support novice users and does not require software engineering experience. It is a detailed extension of the OpenCSP getting started on-line documentation, found here: https://opencsp.readthedocs.io/en/main/contributing.html#getting-started. For an overview of OpenCSP overall, see the OpenCSP website: https://opencsp.sandia.gov. For details on OpenCSP components and algorithm, see the references.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

DeepDiagnostics: A Software Package for Streamlined Posterior Evaluation

Automated prediction techniques like simulation-based inference (SBI) are important tasks for science experiments that produce large amounts of complex, raw data. However, their development remains in its early stages because the uncertainties of these techniques lack sufficient trustworthiness and interpretability. Packages for SBI provide a growing set of diagnostics; however, the software requirements are substantial, as they are tied to the inference technology itself, and the APIs lack adaptability. We introduce the DeepDiagnostics package for diagnosing posteriors from analytic likelihood-based methods and SBI methods, such as neural posterior estimation. DeepDiagnostics produces a comprehensive set of high-quality visualizations and metrics in a highly accessible, easy-to-use, and flexible package. We address all of these goals by providing a command-line inference tool and a Python API that is controlled through a configuration file. The package includes common diagnostics, such as parity plots, corner (covariance) plots, simulation-based calibration (SBC) diagnostics (including posterior coverage and rank histograms), Lemos et al. s PQMass and TARP, Masserano et al. s WALDO, Linhart et al. s LC2ST, as well as credible region diagnostics developed by our group.

Voetberg, Maggie [Fermilab]↗
DOE OSTI2025

Testing convolutional neural network based deep learning systems: a statistical metamorphic approach

Machine learning technology spans many areas and today plays a significant role in addressing a wide range of problems in critical domains,i.e., healthcare, autonomous driving, finance, manufacturing, cybersecurity,etc. Metamorphic testing (MT) is considered a simple but very powerful approach in testing such computationally complex systems for which either an oracle is not available or is available but difficult to apply. Conventional metamorphic testing techniques have certain limitations in verifying deep learning-based models (i.e., convolutional neural networks (CNNs)) that have a stochastic nature (because of randomly initializing the network weights) in their training. In this article, we attempt to address this problem by using a statistical metamorphic testing (SMT) technique that does not require software testers to worry about fixing the random seeds (to get deterministic results) to verify the metamorphic relations (MRs). We propose seven MRs combined with different statistical methods to statistically verify whether the program under test adheres to the relation(s) specified in the MR(s). We further use mutation testing techniques to show the usefulness of the proposed approach in the healthcare space and test two CNN-based deep learning models (used for pneumonia detection among patients). The empirical results show that our proposed approach uncovers 85.71% of the implementation faults in the classifiers under test (CUT). Furthermore, we also propose an MRs minimization algorithm for the CUT, thus saving computational costs and organizational testing resources.

Computer Science↗
DOE OSTI2026

Terrestrial laser scanning data (Levels 0 and 1) for Pasoh, Malaysia, Sep 2024

This data package contains data from terrestrial laser scanning (TLS) at the Pasoh Forest Reserve, Malaysia. The Pasoh Forest Reserve is a facility of the Forest Research Institute Malaysia, and contains evergreen lowland dipterocarp forest. The Next-Generation Ecosystem Experiments Tropics (NGEE-Tropics) study areas at Pasoh were established to study how different species respond to climatic variation and soil water availability. Two study areas were chosen representing different topography and species. The TLS data archived here were collected to provide detailed, three-dimensional information about forest structure. Specifically, data were collected to allow tree-level characterization of woody structure and leaf area for 12 focal trees with FloraPulse and sap flux sensors, facilitating estimation of woody biomass and leaf area to allow upscaling of water content and transpiration data to the tree-level. Scan positions were not selected to provide consistent data for non-focal trees with the study areas. This data package contains the following data: - High-level files document further details of the campaign and data package: 1_CampaignSummary.csv provides details about the campaign and study site, 2_ScanAreasDetail.csv provides details about each separate scan area (groups of scans post-processed into a single point cloud), 3_TerrestrialLidarSensor.csv provides further technical details about the Riegl VZ-400i TLS sensor, TLS_CSV_dd.csv is a CSV Data Dictionary providing information about the fields in CSV files following the ESS-DIVE CSV File Formatting Guidelines Reporting Format, TLS_flmd.csv is a File Level Metadata file providing information about each file in the data package following the ESS-DIVE File Level Metadata Reporting Format, and README.txt is a text file describing the overall project and file structure. - Level 0 data are the raw data (.PROJ folders) as recorded by the Riegl VZ-400i TLS instrument before scan co-registration and post-processing with the Riegl's proprietary RiSCAN PRO software, which requires a license. - Level 1 data contain post-processed, co-registered data from each scan area. The "PointClouds" folder for each scan area contains a .las file with 1 cm resolution point cloud data exported from RiSCAN PRO. These are the main files likely to be of interest to most users and can be further processed with any software capable of manipulating .las files (e.g. Python, R CloudCompare). The "Project Information" folder contains log files from post-processing in RiSCAN PRO that may be of interest to users who want to see detailed records of post-processing, including all PDF reports generated by RiSCAN PRO. The "ScanPositions" folder contains information about the final position of all TLS scans, after post-processing, in multiple formats. The file ScanPositions_*.csv provides final geo-referenced scan positions, and the file SOP_backup_*.csv can be used in RiSCAN PRO to restore the co-registered scan positions if users wish to re-process raw data (Level 0 .PROJ folders) with RiSCAN PRO software (e.g., subsample to a different resolution, exclude a certain scan position, or apply different filters on reflectance or deviation values) without redoing time-consuming co-registration steps.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2026

Terrestrial laser scanning data (Levels 0 and 1) from Urban Biogeochemistry Pilot Project sites, Knoxville, Tennessee, Jul 2024 - Jul 2025

This data package contains data from terrestrial laser scanning (TLS) at five urban park sites in Knoxville, Tennessee, USA. All parks include open-grown and/or closed-canopy trees and mixed nearby land use. These study sites were established as part of the Urban Biogeochemistry Pilot Project, which has an overall goal of better understanding how hydrobiogeochemical cycling is altered within the human environment. These five sites represent a gradient of urbanization, and were instrumented to understand hydrological and biogeochemical cycling (e.g., soil moisture, soil physical properties and biogeochemistry, tree transpiration, species type). The TLS data archived here were collected to provide detailed, three-dimensional information about forest structure. Specifically, data were collected to allow tree- and stand-level characterization of woody structure and leaf area. TLS scans were placed to capture the area around trees with sap flow sensors, and as much of a 50 m radius area around the meteorological station as possible given site property limits. Derived products will allow upscaling of water content and transpiration data. This data package contains the following data: - High-level files document further details of the campaign and data package: 1_CampaignSummary.csv provides details about the campaign and study site, 2_ScanAreasDetail.csv provides details about each separate scan area (groups of scans post-processed into a single point cloud), 3_TerrestrialLidarSensor.csv provides further technical details about the Riegl VZ-400i TLS sensor, TLS_CSV_dd.csv is a CSV Data Dictionary providing information about the fields in CSV files following the ESS-DIVE CSV File Formatting Guidelines Reporting Format, TLS_flmd.csv is a File Level Metadata file providing information about each file in the data package following the ESS-DIVE File Level Metadata Reporting Format, and README.txt is a text file describing the overall project and file structure. - Level 0 data are the raw data (.PROJ folders) as recorded by the Riegl VZ-400i TLS instrument before scan co-registration and post-processing with the Riegl's proprietary RiSCAN PRO software, which requires a license. - Level 1 data contain post-processed, co-registered data from each scan area. The "PointClouds" folder for each scan area contains a .las file with 1 cm resolution point cloud data exported from RiSCAN PRO. These are the main files likely to be of interest to most users and can be further processed with any software capable of manipulating .las files (e.g. Python, R CloudCompare). The "Project Information" folder contains log files from post-processing in RiSCAN PRO that may be of interest to users who want to see detailed records of post-processing, including all PDF reports generated by RiSCAN PRO. The "ScanPositions" folder contains information about the final position of all TLS scans, after post-processing, in multiple formats. The file ScanPositions_*.csv provides final geo-referenced scan positions, and the file SOP_backup_*.csv can be used in RiSCAN PRO to restore the co-registered scan positions if users wish to re-process raw data (Level 0 .PROJ folders) with RiSCAN PRO software (e.g., subsample to a different resolution, exclude a certain scan position, or apply different filters on reflectance or deviation values) without redoing time-consuming co-registration steps.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

omni-engineer-lbl (omni) v0.5

This is a fork of the omni-engineer software developed by Pietro Schirano. My collaborators and I are altering the software to work better with our LBNL infrastructure and requirements. The software is a coding agent wrapper around LLM model APIs, and can use any of the APIs provided by LBNL's CBorg service, or any service that uses an OpenAI endpoint(s).

Fong, Timothy [Lawrence Berkeley National Laborato↗
DOE OSTI2024

ExaWorks software development kit: a robust and scalable collection of interoperable workflows technologies

Scientific discovery increasingly requires executing heterogeneous scientific workflows on high-performance computing (HPC) platforms. Heterogeneous workflows contain different types of tasks (e.g., simulation, analysis, and learning) that need to be mapped, scheduled, and launched on different computing. That requires a software stack that enables users to code their workflows and automate resource management and workflow execution. Currently, there are many workflow technologies with diverse levels of robustness and capabilities, and users face difficult choices of software that can effectively and efficiently support their use cases on HPC machines, especially when considering the latest exascale platforms. We contributed to addressing this issue by developing the ExaWorks Software Development Kit (SDK). The SDK is a curated collection of workflow technologies engineered following current best practices and specifically designed to work on HPC platforms. We present our experience with (1) curating those technologies, (2) integrating them to provide users with new capabilities, (3) developing a continuous integration platform to test the SDK on DOE HPC platforms, (4) designing a dashboard to publish the results of those tests, and (5) devising an innovative documentation platform to help users to use those technologies. Our experience details the requirements and the best practices needed to curate workflow technologies, and it also serves as a blueprint for the capabilities and services that DOE will have to offer to support a variety of scientific heterogeneous workflows on the newly available exascale HPC platforms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Sentinel

Network intrusion detection systems (NIDS) are commonplace in network security but they frequently employ algorithms that are computational demanding requiring hardware and software with significant power requirements. Two examples of such resource-intensive algorithms used for network security are regular expression matching and broader signature pattern matching which are commonly used in deep packet inspection (DPI). Network security algorithms that have large power requirements may be a challenge for low-power internet-of-things (IoT) environments, which generally lack the power resources to implement complex security measures like computationally expensive DPI at the edge. Furthermore, IoT environments incorporating 5G standalone networks have network latency constraints beyond just power that make DPI at the edge even more difficult. Programmable logic is ideally suited for machine learning inference for DPI because of its deep instruction level parallelism and single-cycle memory access. Machine learning approaches for DPI have been explored before using the programmable logic of field programmable gate arrays (FPGA) as a potential solution for NIDS approaches that would be power-suitable for IoT. However, those previous programmable logic NIDS approaches utilize either a supervised or unsupervised learning model. Sentinel utilizes the ensemble of these two machine learning approaches known as a semi-supervised approach which has shown promise in NIDS implementations. Sentinel provides a programmable logic implementation of a semi-supervised approach for DPI which operates at much lower power and latency than a GPU implementation with negligible loss of accuracy due to quantization through a logistic regressor.

Anderson, MatthewW [Idaho National Laboratory (INL↗