Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Python API”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

BioXTAS RAW 2 : new developments for a free open-source program for small-angle scattering data reduction and analysis

BioXTAS RAW is a free open-source program for reduction, analysis and modelling of biological small-angle scattering data. Here, the new developments in RAW version 2 are described. These include improved data reduction using pyFAI ; updated automated Guinier fitting and D max finding algorithms; automated series ( e.g. size-exclusion chromatography coupled small-angle X-ray scattering or SEC-SAXS) buffer- and sample-region finding algorithms; linear and integral baseline correction for series; deconvolution of series data using regularized alternating least squares ( REGALS ); creation of electron-density reconstructions using electron density via solution scattering ( DENSS ); a comparison window showing residuals, ratios and statistical comparisons between profiles; and generation of PDF reports with summary plots and tables for all analysis. Furthermore, there is now a RAW API, which can be used without the graphical user interface (GUI), providing full access to all of the functionality found in the GUI. In addition to these new capabilities, RAW has undergone significant technical updates, such as adding Python 3 compatibility, and has entirely new documentation available both online and in the program.

97 MATHEMATICS AND COMPUTING↗

Striped Data Analysis Framework

A columnar data representation is known to be an efficient way for data storage, specifically in cases when the analysis is often done based only on a small fragment of the available data structures. A data representation like Apache Parquet is a step forward from a columnar representation, which splits data horizontally to allow for easy parallelization of data analysis. Based on the general idea of columnar data storage, working on the [LDRD Project], we have developed a striped data representation, which, we believe, is better suited to the needs of High Energy Physics data analysis. A traditional columnar approach allows for efficient data analysis of complex structures. While keeping all the benefits of columnar data representations, the striped mechanism goes further by enabling easy parallelization of computations without requiring special hardware. We will present an implementation and some performance characteristics of such a data representation mechanism using a distributed no-SQL database or a local file system, unified under the same API and data representation model. The representation is efficient and at the same time simple so that it allows for a common data model and APIs for wide range of underlying storage mechanisms such as distributed no-SQL databases and local file systems. Striped storage adopts Numpy arrays as its basic data representation format, which makes it easy and efficient to use in Python applications. The Striped Data Server is a web service, which allows to hide the server implementation details from the end user, easily exposes data to WAN users, and allows to utilize well known and developed data caching solutions to further increase data access efficiency. We are considering the Striped Data Server as the core of an enterprise scale data analysis platform for High Energy Physics and similar areas of data processing. We have been testing this architecture with a 2TB dataset from a CMS dark matter search and plan to expand it to multiple 100 TB or even PB scale. We will present the striped format, Striped Data Server architecture and performance test results.

Gutsche, Oliver↗

Striped Data Server for Scalable Parallel Data Analysis

A columnar data representation is known to be an efficient way for data storage, specifically in cases when the analysis is often done based only on a small fragment of the available data structures. A data representation like Apache Parquet is a step forward from a columnar representation, which splits data horizontally to allow for easy parallelization of data analysis. Based on the general idea of columnar data storage, working on the [LDRD Project], we have developed a striped data representation, which, we believe, is better suited to the needs of High Energy Physics data analysis. A traditional columnar approach allows for efficient data analysis of complex structures. While keeping all the benefits of columnar data representations, the striped mechanism goes further by enabling easy parallelization of computations without requiring special hardware. We will present an implementation and some performance characteristics of such a data representation mechanism using a distributed no-SQL database or a local file system, unified under the same API and data representation model. The representation is efficient and at the same time simple so that it allows for a common data model and APIs for wide range of underlying storage mechanisms such as distributed no-SQL databases and local file systems. Striped storage adopts Numpy arrays as its basic data representation format, which makes it easy and efficient to use in Python applications. The Striped Data Server is a web service, which allows to hide the server implementation details from the end user, easily exposes data to WAN users, and allows to utilize well known and developed data caching solutions to further increase data access efficiency. We are considering the Striped Data Server as the core of an enterprise scale data analysis platform for High Energy Physics and similar areas of data processing. As a result, we have been testing this architecture with a 2TB dataset from a CMS dark matter search and plan to expand it to multiple 100 TB or even PB scale. We will present the striped format, Striped Data Server architecture and performance test results.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Archparse

Archparse is a Python package that holds the purpose and capability of converting the contents of a text file to a functioning neural network based on the Tensorflow 2.X framework. This is able to be done with minimal written Python code and next to no knowledge of how to build models with Tensorflow directly. More specifically Archparse parses a text file with extension ".arch" which contains neural network architecture information that corresponds to either the Tensorflow 2.X API or custom code written with the Tensorflow 2.X API. Included in the initial version is the capacity to easily produce sequential autoencoders and sequential neural networks.

Vander Wal, MichaelD.↗

BULKI-Store v0.3.2

BULKI-Store is a distributed object storage system optimized for high-performance computing environments. Built with a Rust core and Python bindings, it efficiently manages scientific and machine learning datasets across HPC clusters. The system employs a client-server architecture with MPI integration, enabling seamless scaling on supercomputers like Perlmutter. BULKI-Store's object-oriented approach provides intuitive data organization with rich metadata support, contrasting with traditional file-based solutions. Key optimizations include selective checkpoint loading, unified checkpoint files, and object chunking for large data transfers. For machine learning workloads, BULKI-Store offers advantages through fine-grained access patterns, dynamic data sharing between training instances, and reduced memory pressure. Memory management features include strategic Python GC calls, minimized data copies, and batch processing capabilities. The system leverages Rayon's thread pool for asynchronous data prefetching and supports multiple CPU architectures (ARM64, x86, AMD, RISC-V). By combining performance optimizations with developer-friendly APIs, BULKI-Store addresses the complex data management challenges of modern HPC applications while maintaining compatibility across heterogeneous computing environments.

Zhang, Wei [Lawrence Berkeley National Laboratory ↗

Wind Integration National Dataset (WIND) Toolkit

Wind resource data for North America was produced using the Weather Research and Forecasting Model (WRF). The WRF model was initialized with the European Centre for Medium Range Weather Forecasts Interim Reanalysis (ERA-Interm) data set with an initial grid spacing of 54 km. Three internal nested domains were used to refine the spatial resolution to 18, 6, and finally 2 km. The WRF model was run for years 2007 to 2014. While outputs were extracted from WRF at 5 minute time-steps, due to storage limitations instantaneous hourly time-step are provided for all variables while full 5 min resolution data is provided for wind speed and wind direction only. The following variables were extracted from the WRF model data: - Wind Speed at 10, 40, 60, 80, 100, 120, 140, 160, 200 m - Wind Direction at 10, 40, 60, 80, 100, 120, 140, 160, 200 m - Temperature at 2, 10, 40, 60, 80, 100, 120, 140, 160, 200 m - Pressure at 0, 100, 200 m - Surface Precipitation Rate - Surface Relative Humidity - Inverse Monin Obukhov Length

API↗

DEPRECATED EVOLVE [SWR-20-79]

DEPRECATED This repository was archived by the owner on Jun 30, 2026. It is now read-only. EVOLVE is a combination of a dashboard developed in React, which enables user to visualize the net load evolution with emerging technologies such as PV, energy storage, and electric vehicles, and a Python backend processes data. A user can choose settings for solar, energy storage, and electric vehicles in the dashboard. Modules that are developed to model the impact of these DERs can also be used independently without being integrated into the dashboard. We used a RESTful API to expose these modules to the React front-end.

Duwadi, Kapil↗

BuildingSync® v.2.5.0 2024 [SWR-18-28]

BuildingSync® is a building data exchange schema to better enable integration between software tools and building data workflows. The schema's original use case was focused on commercial building energy audits; however, several additional use cases have been realized including building energy modeling and more high-level generic building data exchange. BuildingSync helps streamline the data exchange process, improving the value of the data, minimizing duplication of effort for subsequent building data collection efforts (including audits), and facilitating the achievement of greater energy efficiency. This in done in part by standardizing on (a) reporting audits in an electronic format, (b) tracking proposed, implemented, and discarded energy conservation measures, and (c) storing building characteristics (at multiple levels) for audits, benchmarking, and building energy analysis. BuildingSync has several documents and tools available to help users understand how to best leverage BuildingSync. The list below are only a subset of the resources available. If new resources are discovered, then feel free to create a new pull request with the additions. Generic BuildingSync information is available on the DOE website and the project website. BuildingSync Examples - These examples are kept up to date and show a wide range of implementations. Any new update to BuildingSync is required to pass validation on these example files. BuildingSync Use Case Validator allows for users to determine if their instance complies with a specific use case for BuildingSync by checking if the required elements are implemented in an uploaded instance. An API is also provided for automated integration into other tools. Also, the website contains an easy way to view the entirety of the schema and how elements relate to the Building Exchange Data Exchange Specification. The Validator is open sourced here Use Case TestSuite provides a Python package for easier generation of BuildingSync use cases. BuildingSync use cases depend on the generation of schematron documents, which is time-consuming and difficult to implement well. The TestSuite allows users to define a use case using a more palatable CSV template, which it then turns into a Schematron document. The source code is available here. BuildingSync to OpenStudio/EnergyPlus. The translator is open sourced here. This project will translate a Level 1 (and partial Level 2) ASHRAE Energy Audit to a fully defined OpenStudio and EnergyPlus model. This project is in early Beta testing and any feedback is welcome!

Long, Nicholas↗

BuildingSync® v.2.7.0 (released 9.11.2025) [SWR-18-28]

BuildingSync® is a building data exchange schema to better enable integration between software tools and building data workflows. The schema's original use case was focused on commercial building energy audits; however, several additional use cases have been realized including building energy modeling and more high-level generic building data exchange. Version 2.7.0 adds new elements for file attachment feature and FederalBuilding, and generalizes usage of Optional Elements (e.g. EquipmentCondition, EquipmentID) to all assets/systems. BuildingSync helps streamline the data exchange process, improving the value of the data, minimizing duplication of effort for subsequent building data collection efforts (including audits), and facilitating the achievement of greater energy efficiency. This in done in part by standardizing on (a) reporting audits in an electronic format, (b) tracking proposed, implemented, and discarded energy conservation measures, and (c) storing building characteristics (at multiple levels) for audits, benchmarking, and building energy analysis. BuildingSync has several documents and tools available to help users understand how to best leverage BuildingSync. The list below are only a subset of the resources available. If new resources are discovered, then feel free to create a new pull request with the additions. Generic BuildingSync information is available on the DOE website and the project website. BuildingSync Examples - These examples are kept up to date and show a wide range of implementations. Any new update to BuildingSync is required to pass validation on these example files. BuildingSync Use Case Validator allows for users to determine if their instance complies with a specific use case for BuildingSync by checking if the required elements are implemented in an uploaded instance. An API is also provided for automated integration into other tools. Also, the website contains an easy way to view the entirety of the schema and how elements relate to the Building Exchange Data Exchange Specification. The Validator is open sourced here Use Case TestSuite provides a Python package for easier generation of BuildingSync use cases. BuildingSync use cases depend on the generation of schematron documents, which is time-consuming and difficult to implement well. The TestSuite allows users to define a use case using a more palatable CSV template, which it then turns into a Schematron document. The source code is available here. BuildingSync to OpenStudio/EnergyPlus. The translator is open sourced here. This project will translate a Level 1 (and partial Level 2) ASHRAE Energy Audit to a fully defined OpenStudio and EnergyPlus model. This project is in early Beta testing and any feedback is welcome!

Long, Nicholas [National Renewable Energy Lab. (NR↗

BASTet (Berkeley Analysis and Storage Toolkit)

BASTet is an advanced software library written in Python. BASTet serves as the analysis and storage library for the OpenMSI project. BASTet is an integrate framework for: i) storage of spectral imaging data, ii) storage of derived analysis data, iii) provenance of analyses, iv) integration and execution of analyses via complex workflows. BASTet implements the API for the HDF5 storage format used by OpenMSI. Analyses that are developed using BASTet benefit from direct integration with storage format, automatic tracking of provenance, and direct integration with command-line and workflow execution tools. BASTet also defines interfaces to enable developers to directly integrate their analysis with OpenMSI's web-based viewing infrastruture without having to know OpenMSI. BASTet also provides numerous helper classes and tools to assist with the conversion of data files, ease parallel implementation of analysis algorithms, ease interaction with web-based functions, description methods for data reduction. BASTet also includes detailed developer documentation, user tutorials, iPython notebooks, and other supporting documents.

Bowen, Benjamin↗

BUTTER-Clarifier [SWR-23-61]

This repository contains a python package of neural network interpretability techniques (interpretability) and a keras callback to easily compute and capture data related to these techniques (values metrics) during training. It was developed to be used with NREL's BUTTER Deep Learning Experimental Framework, but does not depend on this framework and may be useful to projects outside of this framework. The vision for this codebase is to collect algorithms for explainable artificial intelligence (XAI) in a single framework that is easy to use, easy to read, and can be expand upon. Here, we package XAI algorithms into a module called "metrics", which are implemented as python functions. The return type of a metric is typically a dictionary holding data of multiple data types, such as real values and numpy matrices. Callbacks and any other connector code is provided as necessary in a separate module to make these metrics more easily usable. This project depends on Tensorflow's Keras API, although it would be nice to try and support multiple backends one day. Related to: https://github.com/NREL/BUTTER-Empirical-Deep-Learning-Experimental-Framework

Perr-Sauer, Jordan↗

CiteSoft_Py

There is a need to provide a way for dev-users (Scientists, Engineers, and other Software Developers) to get credit (citations) when they contribute to an important software package - particularly a large package that is already established. Various solutions (including cross package) solutions do exist, but a simple standard format that can work between multiple packages and multiple languages in a modular way is not widely available. CiteSoft_py is a python implementation of CiteSoft. CiteSoft is a plain text standard consisting of a format and a protocol that exports the citations for the end-users for whichever softwares they have used. CiteSoft has been designed so that software dev-users can rely upon it regardless of coding language or platform, and even for cases where multiple codes are working in a coupled manner. The CiteSoft_py implementation includes python decorators to make wrappers for facile use. The purpose is that when dev-users (scientists, engineers and professional programmers) contribute to a collaborative software project, that the appropriate citations (e.g., journal article, conference proceeding) are provided to the end-user. The end-user does not need to know how to use a command line interface, API, etc.

Savara, Aditya↗

PNNL-Predictive-Phenomics/ProCaliper

ProCaliper is a Python library that curates, organizes, and computes protein structure features in a way that easily interfaces with user-provided experimental data. It extracts or computes protein binding site, active site, charge, pLDDT (order/disorder), acid dissociation, protonation, solvent accessible surface area, disulfide bond distance, and protein secondary structure data using precomputed protein structures and publicly available databases. It provides a unified API for integrating additional residue-level data and for visualizing residue features in 3D.

Rozum, Jordan [Pacific Northwest National Lab]↗

arco (Assembled Resource-Constrained Optimization) [SWR-26-030]

Arco (Assembled Resource-Constrained Optimization) is a memory-smart optimization DSL and solver for LP and MIP problems on constrained hardware. The software is an optimization framework built around a KDL-based domain-specific language and a CLI compiler/solver. You write optimization models in .kdl files, and the arco CLI compiles, validates, inspects, and solves them. Language bindings (Python today, more planned) provide programmatic access to the same engine. Built for harder optimization problems on constrained resources, Arco is intentional about every allocation, careful with stack and heap behavior, and relentless about minimizing memory usage so more systems can run real workloads. Arco is built primarily for internal use within our organization. You are welcome to try it, but we make no guarantees about API stability or robustness at this stage

Sanchez Perez, Pedro Andres [National Laboratory o↗

Bringing heterogeneity to the CMS software framework

The advent of computing resources with co-processors, for example Graphics Processing Units (GPU) or Field-Programmable Gate Arrays (FPGA), for use cases like the CMS High-Level Trigger (HLT) or data processing at leadership-class supercomputers imposes challenges for the current data processing frameworks. These challenges include developing a model for algorithms to offload their computations on the co-processors as well as keeping the traditional CPU busy doing other work. The CMS data processing framework, CMSSW, implements multithreading using the Intel Threading Building Blocks (TBB) library, that utilizes tasks as concurrent units of work. In this paper we will discuss a generic mechanism to interact effectively with non-CPU resources that has been implemented in CMSSW. In addition, configuring such a heterogeneous system is challenging. In CMSSW an application is configured with a configuration file written in the Python language. The algorithm types are part of the configuration. The challenge therefore is to unify the CPU and co-processor settings while allowing their implementations to be separate. We will explain how we solved these challenges while minimizing the necessary changes to the CMSSW framework. We will also discuss on a concrete example how algorithms would offload work to NVIDIA GPUs using directly the CUDA API.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

tSFM 1.0: tRNA Structure–Function Mapper

Structure-conditioned information statistics have proven useful to predict and visualize tRNA Class-Informative Features (CIFs) and their evolutionary divergences. Although permutation P-values can quantify the significance of CIF divergences between two taxa, their naive Monte Carlo approximation is slow and inaccurate. The Peaks-over-Threshold approach of Knijnenburg et al. (2009) promises improvements to both speed and accuracy of permutation P-values, but has no publicly available API. Here, we present tRNA Structure–Function Mapper (tSFM) v1.0, an open-source, multi-threaded application that efficiently computes, visualizes and assesses significance of single- and paired-site CIFs and their evolutionary divergences for any RNA, protein, gene or genomic element sequence family. Multiple estimators of permutation P-values for CIF evolutionary divergences are provided along with confidence intervals. tSFM is implemented in Python 3 with compiled C extensions and is freely available through GitHub (https://github.com/tlawrence3/tSFM) and PyPI.

59 BASIC BIOLOGICAL SCIENCES↗

Localized Actual Meteorological Year File Creator (LAF): A tool for using locally observed weather data in building energy simulations

The Localized Actual Meteorological Year File Creator (LAF) application provides web-based access to real meteorological data and processes it into a weather file suitable for building energy modeling. Building energy consumption is affected by what is inside the building (such as occupants, appliances, HVAC systems, etc.) and by what is outside: the weather conditions the building is exposed to. However, freely available weather data files are limited to a number of specific locations, usually airports, which are often located away from city centers where buildings are concentrated. The authors have developed a new tool that supports the investigation and quantification of micro-climate conditions on building energy consumption. LAF is built on the Python open-source programming language and has a Graphical User Interface (GUI) that allows users to create custom weather data files for building energy simulations. Many sets of actual meteorological year weather data for long time periods are publicly available online (such as the MesoWest database) for thousands of locations in the US. LAF selects weather data according to user specifications and automatically processes it through an API across multiple weather stations and multiple time periods. The user may easily select a specific location, time frame, and time step that best meets their needs. This article presents a useful tool for energy modelers, building designers and operators to assist with building performance analysis and optimization.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Accelerating the Inference of the Exa.TrkX Pipeline

Recently, graph neural networks (GNNs) have been successfully used for a variety of particle reconstruction problems in high energy physics, including particle tracking. The Exa.TrkX pipeline based on GNNs demonstrated promising performance in reconstructing particle tracks in dense environments. It includes five discrete steps: data encoding, graph building, edge filtering, GNN, and track labeling. All steps were written in Python and run on both GPUs and CPUs. In this work, we accelerate the Python implementation of the pipeline through customized and commercial GPU-enabled software libraries, and develop a C++ implementation for inferencing the pipeline. The implementation features an improved, CUDA-enabled fixed-radius nearest neighbor search for graph building and a weakly connected component graph algorithm for track labeling. GNNs and other trained deep learning models are converted to ONNX and inferenced via the ONNX Runtime C++ API. The complete C++ implementation of the pipeline allows integration with existing tracking software. We report the memory usage and average event latency tracking performance of our implementation applied to the TrackML benchmark dataset.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗