SEARCH · Engineering Papers

Results for “Data Packages”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 811 records · Page 45

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Columnar data analysis with ATLAS analysis formats

Future analysis of ATLAS data will involve new small-sized analysis formats to cope with the increased storage needs. The smallest of these, named DAOD_PHYSLITE, has calibrations already applied to allow fast downstream analysis and avoid the need for further analysis-specific intermediate formats. This allows for application of the "columnar analysis" paradigm where operations are applied on a per-array instead of a per-event basis. We will present methods to read the data into memory, using Uproot, and also discuss I/O aspects of columnar data and alternatives to the ROOT data format. Furthermore, we will show a representation of the event data model using the Awkward Array package and present proof of concept for a simple analysis application.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Information-incorporated gene network construction with FDR control

Abstract Motivation Large-scale gene expression studies allow gene network construction to uncover associations among genes. To study direct associations among genes, partial correlation-based networks are preferred over marginal correlations. However, FDR control for partial correlation-based network construction is not well-studied. In addition, currently available partial correlation-based methods cannot take existing biological knowledge to help network construction while controlling FDR. Results In this paper, we propose a method called Partial Correlation Graph with Information Incorporation (PCGII). PCGII estimates partial correlations between each pair of genes by regularized node-wise regression that can incorporate prior knowledge while controlling the effects of all other genes. It handles high-dimensional data where the number of genes can be much larger than the sample size and controls FDR at the same time. We compare PCGII with several existing approaches through extensive simulation studies and demonstrate that PCGII has better FDR control and higher power. We apply PCGII to a plant gene expression dataset where it recovers confirmed regulatory relationships and a hub node, as well as several direct associations that shed light on potential functional relationships in the system. We also introduce a method to supplement observed data with a pseudogene to apply PCGII when no prior information is available, which also allows checking FDR control and power for real data analysis. Availability and implementation R package is freely available for download at https://cran.r-project.org/package=PCGII.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2020

Multi-modal Free-moving Data Fusion (MFDF) v1.0

Multi-modal Free-moving Data Fusion (MFDF) is a python package that implements low and high-level functionality for performing qualitative and quantitative gamma-ray imaging analyses. Such analyses enable the use of gamma-ray spectrometers and/or imagers to estimate the distribution and quantity of radiological materials in an environment. Free-moving 3D imaging requires additional information about the 3D trajectory and orientation of the system, however all of these methods can be applied to 2D static gamma-ray imaging as well. The package includes methods (MLEM and MAP) for distributed source reconstruction, methods (PSL) for point source reconstruction, the ability to ingest and utilize quantitative detector response functions (for absolute analyses), and the ability to perform 3D estimation of dose-rates from quantitative reconstructions.

Joshi, Tenzing↗

DOE OSTI2023

ITL (InterfaceLimits.jl) [SWR-23-56]

Network flow of electrical power is governed by Kirchhoff's laws which make calculating the power transfer capacity across interfaces connecting groups of nodes complex. This package leverages the capabilities of Sienna\Data to build and solve interface limit calculation problems.

Barrows, Clayton↗

DOE OSTI2025

evmc-supply-curves (Electric Vehicle Managed Charging Supply Curves) [SWR-25-69]

Data and a supporting lightweight Python package that describes possible costs for enabling EV managed charging from 2025 to 2050 for three dispatch mechanisms: Time-of-Use (TOU), Real Time Pricing (RTP), and direct load control (DLC) and four flexibility scenarios (Flat and Low, Mid, and High Flex).

Matsuda-Dunn, Reiko [National Renewable Energy Lab↗

DOE OSTI2025

BATMODS-lite [SWR-25-108]

Battery Analysis and Training Models for Optimization and Design Studies (BATMODS) is a Python package with an API for pre-built battery models. The original purpose of the package was to quickly generate synthetic data for machine learning models to train with. However, the models are generally useful for any battery simulations or analysis. BATMODS-lite includes the following: 1) A library and API for pre-built battery models 2) Kinetic/transport properties for common battery materials

Randall, Corey [National Laboratory of the Rockies↗

DOE OSTI2026

esm_watermasses

A water mass analysis package for gridded ocean and atmospheric data and Earth system model output

Moore-Maley, Ben [@SciDAC-ImPACTS @E3SM-Project @M↗

DOE OSTI2024

Subsurface electrical conductivity across the BEO site inferred using a capacitively coupled resistivity survey in May 2013, Utqiagvik, Alaska

Multiple transects across and around the Barrow Environmental Observatory (BEO) site were surveyed using a capacitively coupled resistivity tool to infer the subsurface electrical conductivity in the top ~6 m in order to improve the understanding of heterogeneity in ice and salinity content. This work was led by the environmental geophysics team from Lawrence Berkeley National Laboratory as part of the Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic). The acquisition was performed in May 2013 using a capacitively coupled resistivity tool (OhmMapper from Geometrics, Inc) mounted behind a sled hooked to a snowmobile. Twenty-eight ~750 m long transects cover the NGEE intensive site (polygon areas A, B, C, D) in the BEO with a measurement (apparent resistivity for 5 different geometries) taken every 1.5 to 2 m. The survey also includes other transects that have been acquired along Electrical Resistivity Tomography (ERT) transects previously acquired at the site (Site 0, AB, and BD), and several long transects (up to 7 km each) around the BEO site crossing several Drained Thawed Lake Basins (DTLB). The various transects acquired during this survey cover a distance of about 100 km. All the datasets are provided with the acquisition geometry and the associated subsurface apparent electrical conductivity. In addition, the 28 transects at the BEO intensive site were inverted to infer the subsurface electrical conductivity. Unprocessed and processed data products are included in this package. This metadata document contains a description of the survey and processing steps and the inferred products.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic) was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy’s Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska, and 2) multiple areas in the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy’s Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2025

Site and endmember spectra of terrestrial vegetation and soils for the Colorado Headwaters Ecological Spectroscopy Study, June-July 2025

This dataset provides site and endmember spectra collected during the 2025 Colorado Headwaters Ecological Spectroscopy Study (CHESS) campaign. The site spectra were collected to help validate airborne hyperspectral data acquired by the National Ecological Observatory Network's aerial observation platform (NEON AOP). Endmember spectra were collected to augment existing spectral libraries with additional samples of bare surfaces and non-photosynthetic vegetation. All measurements were acquired with an Analytical Spectral Devices (ASD) FieldSpec4 Hi-Res NG (Next Generation) spectroradiometer, which records radiance at 1nm (nanometer) intervals from the ultraviolet to the short-wave infrared (350-2500 nm). The dataset includes spectra measured at meadow sites where the CHESS team also collected vegetation samples for trait analyses. The site spectra were collected with the ASD FieldSpec4 palm grip attachment using an 8° field-of-view foreoptic. Site spectra are integrated measurements of the entire surface within the foreoptic’s field of view. For site-level spectra, the sun is the illumination source. A Spectralon panel mounted on a tripod was used for instrument optimization and white reference measurements for all site spectra. Site spectra were acquired within two hours of solar noon and within 48 hours of a NEON AOP overflight. Site spectra are labeled by date, sampling area, and site number according to the naming conventions of the CHESS campaign’s data management plan. The dataset also contains endmember spectra in the following categories: photosynthetic vegetation (PV), non-photosynthetic vegetation (NPV), bare (soil/rock), and flowers. Endmember measurements were acquired using either the contact probe or the leaf clip attachments of the ASD FieldSpec4. In these configurations, the bulb inside the spectrometer provides the light source for the measurements. The spectrometer was optimized and white reference measurements were recorded using the circular white pucks attached to the contact probe and leaf clip. Because they do not rely on solar illumination, contact probe and leaf clip measurements were collected during a broader time frame than the palm grip site spectra. Some endmembers were measured at CHESS meadow sites, while others were collected within the larger sampling area or in nearby locations (e.g. Gothic Townsite) with similar characteristics. Radiance, reflectance, and metadata files are split into three subfolders according to measurement type: proximal/palm grip (prx), contact probe (cp), and leaf clip (lc). Radiance spectra are provided in ASD file format (.asd file extension). All ASD files can be opened using the provided scripts. Metadata is provided in two formats: CSV file format (no geolocation) and GEOJSON file format (includes geolocation for each spectra). The dataset includes a set of pre-processed reflectance spectra as CSV files (yyyymmdd_rfl.csv). The python scripts and jupyter notebook used to calculate reflectance spectra from the ASD radiance data is included here and was previously published at: https://doi.org/10.3334/ORNLDAAC/2446. There is also a folder of JPEG photographs corresponding to selected spectra. We include a protocol document with detailed steps for ASD FieldSpec4 assembly and operations. This data additionally contains a file level metadata (flmd.csv) and data dictionary (dd.csv) file. Geospatial information: Geospatial data for mapping measurement site locations are in the files CHESS_polygons_lai_UTM.geojson, CHESS_polygons_shrub_UTM.geojson, and CHESS_polygons_meadow_UTM.geojson in the companion geospatial package for the 2025 CHESS campaign, ‘CHESS 2025: Location data for field observations and sampling’ (Henderson et al., 2026). CHESS Project Description: The Colorado Headwaters Ecological Spectroscopy Study (CHESS) comprised a multi-week airborne remote sensing and field observation campaign in the Upper Gunnison Basin, Colorado, conducted in June and July of 2025. Airborne remote sensing was conducted by the National Ecological Observatory Network Airborne Observation Platform (NEON AOP), concurrent with a field campaign run by the Rocky Mountain Biological Laboratory (RMBL), the Lawrence Berkeley National Laboratory (LBNL) and SLAC National Accelerator Laboratory Watershed Function Science Focus Area (SFA), and NASA-JPL (Jet Propulsion Laboratory) Earth Surface Mineral Dust Source Investigation (EMIT) program. Between June 10 and July 18, 2025, the NEON AOP flight team collected high-resolution aerial imaging spectroscopy and Light Detection and Ranging (LiDAR) data over three domains: the Upper East River (CRBU), Almont Triangle (ALMO), and the Upper Taylor Basin (UPTA). In coordination with the flights, a field campaign acquired ground-truth observations, including observations of vegetation composition, foliar traits, forest demography, and subsurface properties in 18 core sampling areas within the domains. Additional surface water observations were taken at over 380 point locations. All CHESS campaign datasets can be found within the CHESS ESS-DIVE data portal: https://data.ess-dive.lbl.gov/portals/chess. Funding Acknowledgment: This research was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration (80NM0018D0004) and was funded by EMIT Extended Mission Phase E Science.

2018 NEON and 2025 CHESS Campaigns↗

DOE OSTI2025

Commercialization of a Non-Intrusive Optical (NIO) Technology to Measure Heliostat Optical Errors in Utility-Scale Concentrating Solar Power Plants: Final TCF Report

The drone-based Non-Intrusive Optical (NIO) Technology has been developed at NREL to allow for efficient and automated optical characterization of heliostats in Concentrating Solar Power (CSP) plants. For this project, the technology will be developed into a commercial tool package, including software and user-interface (UI), operations manual, and training and support services. The project team will partner with Tietronix to perform market assessment and stakeholder engagement, develop the tool package and business model, and perform data collection and analysis to demonstrate and refine the capabilities for use at a commercial plant. The team will collaborate with a commercial plant to conduct the data collection operations and provide optical error deliverables. The goal of the project is to advance the commercialization of the technology to a stage where a beta version can be demonstrated at additional commercial plants and developed into a licensable product.

14 SOLAR ENERGY↗

DOE OSTI2023

SPLENDAQ: A Detector-Agnostic Data Acquisition System for Small-Scale Physics Experiments

Many scientific applications from rare-event searches to condensed matter system characterization to high-rate nuclear experiments require time-domain triggering on a raw stream of data, where the triggering is generally threshold-based or randomly acquired. When carrying out detector R &D, there is a need for a general data acquisition (DAQ) system to quickly and efficiently process such data. In the SPLENDOR collaboration, we are developing the Python-based SPLENDAQ package for this exact purpose—it offers two main features for offline analysis of continuous data: a threshold triggering algorithm based on the time-domain optimal filter formalism and an algorithm for randomly choosing nonoverlapping segments for noise measurements. Further, combined with the commercially available Moku platform, developed by Liquid Instruments, we have a full pipeline of event building off raw data with minimal setup. Here, we review the underlying principles of this detector-agnostic DAQ package and give concrete examples of its utility in various applications.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2022

Automated Shift Detection in Sensor-Based PV Power and Irradiance Time Series: Preprint

PV power and irradiance sensor-based measurements are prone to error, resulting in issues such as abrupt time series data shifts. These shifts, which are usually unintentional, may be caused by software or hardware configuration changes on a PV system, and do not reflect an actual change in overall system performance. Locating these shifts and segmenting the associated time series aids in more accurate future PV analysis. In this research, an offline changepoint detection (CPD) algorithm that automatically detects these abrupt data shifts in sensor-based time series is introduced. Data shift periods in 101 daily PV power and irradiance time series were labeled manually by two solar experts. These data streams represent sensor-based measurements, and display a variety of data shift behaviors. A changepoint detection algorithm was tuned using the 101 labeled data streams, with each model configuration's ability to detect labeled changepoints benchmarked using metrics such as F1-score, recall, and Rand Index. Best performing models on seasonality-corrected data streams include the Pruned Exact Linear (PELT) method, the Binary Segmentation method, and the Bottom-Up method, all scoring an average F1-score of 0.76 or greater at detecting labeled changepoints within a 30-day window for the labeled data sets. To promote further research in this space, we are releasing the labeled data shift sets on U.S. Department of Energy's (DOE) DuraMAT Data Hub, and the associated algorithm in the Python PVAnalytics package.

changepoint detection↗

DOE OSTI2022

Automated Shift Detection in Sensor-Based PV Power and Irradiance Time Series

changepoint detection↗

DOE OSTI2026

Datashare

Datashare facilitates communication and data sharing within local networks in potentially dangerous situations such as an explosive ordnance disposal. During such events, there is a need to transmit information rapidly around the incident area. It is a distributed database that does not require an internet connection for operation. In addition, Datashare interfaces with XTK and other software applications, allowing for seamless integration and data management. Datashare supports video calls over the network, enabling real-time communication among users. This software serves to organize, package, and share between responders on location and export data to those off location. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Eldridge, Bryce [Sandia National Lab. (SNL-CA), Li↗

DOE OSTI2022

ESS-DIVE Reporting Format for Dataset Package Metadata

ESS-DIVE’s (Environmental Systems Science Data Infrastructure for a Virtual Ecosystem) dataset metadata reporting format is intended to compile information about a dataset (e.g., title, description, funding sources) that can enable reuse of data submitted to the ESS-DIVE data repository. The files contained in this dataset include instructions (dataset_metadata_guide.md and README.md) that can be used to understand the types of metadata ESS-DIVE collects. The data dictionary (dd.csv) follows ESS-DIVE’s file-level metadata reporting format and includes brief descriptions about each element of the dataset metadata reporting format. This dataset also includes a terminology crosswalk (dataset_metadata_crosswalk.csv) that shows how ESS-DIVE’s metadata reporting format maps onto other existing metadata standards and reporting formats.Data contributors to ESS-DIVE can provide this metadata by manual entry using a web form or programmatically via ESS-DIVE’s API (Application Programming Interface). A metadata template (dataset_metadata_template.docx or dataset_metadata_template.pdf) can be used to collaboratively compile metadata before providing it to ESS-DIVE.Since being incorporated into ESS-DIVE’s data submission user interface, ESS-DIVE’s dataset metadata reporting format, has enabled features like automated metadata quality checks, and dissemination of ESS-DIVE datasets onto other data platforms including Google Dataset Search and DataCite.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2022

Refining perovskite structures to pair distribution function data using collective Glazer modes as a basis

Structural modelling of octahedral tilts in perovskites is typically carried out using the symmetry constraints of the resulting space group. In most cases, this introduces more degrees of freedom than those strictly necessary to describe only the octahedral tilts. It can therefore be a challenge to disentangle the octahedral tilts from other structural distortions such as cation displacements and octahedral distortions. This paper reports the development of constraints for modelling pure octahedral tilts and implementation of the constraints in diffpy-CMI, a powerful package to analyse pair distribution function (PDF) data. The model in the program allows features in the PDF that come from rigid tilts to be separated from non-rigid relaxations, providing an intuitive picture of the tilting. The model has many fewer refinable variables than the unconstrained space group fits and provides robust and stable refinements of the tilt components. It further demonstrates the use of the model on the canonical tilted perovskite CaTiO 3 which has the known Glazer tilt system α + β – β – . The Glazer model fits comparably to the corresponding space-group model Pnma below r = 14 Å and becomes progressively worse than the space-group model at higher r due to non-rigid distortions in the real material.

36 MATERIALS SCIENCE↗

DOE OSTI2023

pvlib iotools—Open-source Python functions for seamless access to solar irradiance data

Access to accurate solar resource data is critical for numerous applications, including estimating the yield of solar energy systems, developing radiation models, and validating irradiance datasets. However, lack of standardization in data formats and access interfaces across providers constitutes a major barrier to entry for new users. pvlib python’s iotools subpackage aims to solve this issue by providing standardized Python functions for reading local files and retrieving data from external providers. All functions follow a uniform pattern and return convenient data outputs, allowing users to seamlessly switch between data providers and explore alternative datasets. The pvlib package is community-developed on GitHub: https://github.com/pvlib/pvlib-python. As of pvlib python version 0.9.5, the iotools subpackage supports 12 different datasets, including ground measurement, reanalysis, and satellite-derived irradiance data. The supported ground measurement networks include the Baseline Surface Radiation Network (BSRN), NREL MIDC, SRML, SOLRAD, SURFRAD, and the US Climate Reference Network (CRN). Additionally, satellite-derived and reanalysis irradiance data from the following sources are supported: PVGIS (SARAH & ERA5), NSRDB PSM3, and CAMS Radiation Service (including McClear clear-sky irradiance).

14 SOLAR ENERGY↗

DOE OSTI2022

FAIRification, Quality Assessment, and Missingness Pattern Discovery for Spatiotemporal Photovoltaic Data

The growth of the photovoltaic market has pushed the demand for power forecasting and performance evaluation for a huge population of PV power plants. Many of these power plants have spatiotemporal coherence that can be utilized for improving model accuracy. We have demonstrated in this paper the FAIRification of spatiotemporal PV time series data. Through the creation of a solar power plant ontology, we propose standards for the naming and structure of metadata used to describe the data from these power plants. Using the structure from this ontology, we have developed both R and Python packages for the automation of the FAIRification process. Going further, we have also developed an R package that automates the analysis of the quality of a data set through the designation of letter grades. To solve the issue of data missingness, we propose the use of St-GNN autoencoders to detect and impute missing values from a data set by utilizing data from power plants nearby.

14 SOLAR ENERGY↗