Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data platform”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 847 records · Page 47

Partnership For Edge Physics Simulation

In this effort, we will extend our prior work as part of CPES (i.e., DART and DataSpaces) to support in-situ tight coupling between application codes that exploits data locality and core-level parallelism to maximize on-chip data exchange and reuse. This will be accomplished by mapping coupled simulations so that the data exchanges are more localized within the nodes. Coupled simulation workflows can more effectively utilize the resources available on emerging HEC platforms if they can be mapped and executed to exploit data locality as well as the communication patterns between application components. Scheduling and running such workflows requires an extended framework that should provide a unified hybrid abstraction to enable coordination and data sharing across computation tasks that run on the heterogeneous multi-core-based systems, and develop a data-locality based dynamic tasks scheduling approach to increase on-chip or intra-node data exchanges and in-situ execution. This effort will extend our prior work as part of CPES (i.e., DART and DataSpaces), which provided a simple virtual shared-space abstraction hosted at the staging nodes, to support application coordination, data sharing and active data processing services. Moreover, it will transparently manage the low-level operations associated with the inter-application data exchange, such as data redistributions, and will enable running coupled simulation workflow on multi-cores computing platforms.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Multi-Temporal Predictive Modelling of Sorghum Biomass Using UAV-Based Hyperspectral and LiDAR Data

High-throughput phenotyping using high spatial, spectral, and temporal resolution remote sensing (RS) data has become a critical part of the plant breeding chain focused on reducing the time and cost of the selection process for the “best” genotypes with respect to the trait(s) of interest. In this paper, the potential of accurate and reliable sorghum biomass prediction using visible and near infrared (VNIR) and short-wave infrared (SWIR) hyperspectral data as well as light detection and ranging (LiDAR) data acquired by sensors mounted on UAV platforms is investigated. Predictive models are developed using classical regression-based machine learning methods for nine experiments conducted during the 2017 and 2018 growing seasons at the Agronomy Center for Research and Education (ACRE) at Purdue University, Indiana, USA. The impact of the regression method, data source, timing of RS and field-based biomass reference data acquisition, and the number of samples on the prediction results are investigated. R2 values for end-of-season biomass ranged from 0.64 to 0.89 for different experiments when features from all the data sources were included. Geometry-based features derived from the LiDAR point cloud to characterize plant structure and chemistry-based features extracted from hyperspectral data provided the most accurate predictions. Evaluation of the impact of the time of data acquisition during the growing season on the prediction results indicated that although the most accurate and reliable predictions of final biomass were achieved using remotely sensed data from mid-season to end-of-season, predictions in mid-season provided adequate results to differentiate between promising varieties for selection. The analysis of variance (ANOVA) of the accuracies of the predictive models showed that both the data source and regression method are important factors for a reliable prediction; however, the data source was more important with 69% significance, versus 28% significance for the regression method.

09 BIOMASS FUELS↗

Development of a Distribution Optimal Power Flow Federate for Open-Source OEDI-SI Platform

Increasing numbers of distributed generators in the electric power distribution networks require developing a control strategy to optimize solutions in real time. Linearized optimal distribution flow development has seen growth and acceptance in the distribution systems literature for efficiently modeling the \glspl{opf} for distribution systems. This paper examines the implementation and integration procedure for linearized optimal distribution flow federate to \gls{oedisi} platform. Specifically, we discuss i) the usage of the \gls{oedisi} platform, ii) obtaining a tractable solution using developed \gls{opf} federate, and iii) validation of solutions and bench-marking the \gls{oedisi} platform with developed \gls{opf} federate using OpenDSS. In brief, we demonstrate how a general linearized optimal distribution flow federate can be developed and integrated with a co-simulation environment to mimic real-world examples. The efficacy of the proposed method is demonstrated using the IEEE 123-bus test system under different scenarios to obtain a tractable solution and compare its results.

Sadnan, Rabayet↗

h5bench: A unified benchmark suite for evaluating HDF5 I/O performance on pre‐exascale platforms

Summary Parallel I/O is a critical technique for moving data between compute and storage subsystems of supercomputers. With massive amounts of data produced or consumed by compute nodes, high‐performant parallel I/O is essential. I/O benchmarks play an important role in this process; however, there is a scarcity of I/O benchmarks representative of current workloads on HPC systems. Toward creating representative I/O kernels from real‐world applications, we have created h5bench , a set of I/O kernels that exercise hierarchical data format version 5 (HDF5) I/O on parallel file systems in numerous dimensions. Our focus on HDF5 is due to the parallel I/O library's heavy usage in various scientific applications running on supercomputing systems. The various tests benchmarked in the h5bench suite include I/O operations (read and write), data locality (arrays of basic data types and arrays of structures), array dimensionality (one‐dimensional arrays, two‐dimensional meshes, three‐dimensional cubes), I/O modes (synchronous and asynchronous). In this paper, we present the observed performance of h5bench executed along several of these dimensions on existing supercomputers (Cori and Summit) and pre‐exascale platforms (Perlmutter, Theta, and Polaris). h5bench measurements can be used to identify performance bottlenecks and their root causes and evaluate I/O optimizations. As the I/O patterns of h5bench are diverse and capture the I/O behaviors of various HPC applications, this study will be helpful to the broader supercomputing and I/O community.

97 MATHEMATICS AND COMPUTING↗

Distributed Ledger Technology Preliminary Performance Assessment for Applications in Utilities Operational Technology

This paper provides descriptions of the key components of different distributed ledger technology platforms. Distributed ledger technology (DLT) allows for distribution of databases among different organizations and devices. The platforms use cryptographically linked “blocks” to store and verify transactional information between these organizations. DLT increases data security, data integrity, trust among its participants. Different organizations are looking to deploy this distributed and decentralized approach to avoid the single-point-of-failure vulnerabilities associated with centralized data repositories. In this study we examine twelve different DLT platforms. There is agreement within the community that of all the platforms considered here, Hyperledger and Ethereum are the most mature when it comes to privacy and permissions. These DLT platforms are being used for applications such as transactive energy, health care, and the food and goods supply chain. However, further development is required to realize the full promise of DLT. Our assessment includes a general description of each DLT and its key characteristics. Such characteristics include consensus protocol and cryptography used, public vs. private, and permissioned or permissionless. The selection and implementation of a DLT architecture depends heavily on the use case and performance requirements. During this research we found key parameters to measure performance and existing tools for assessment. Four different parameters were identified 1) consensus, 2) throughput, 3) latency, and 4) scalability. The architectures of Hyperledger Caliper and Blockbench are described as different performance assessment frameworks. From this preliminary study it is evident that there are dissimilarities on the performance assessments methods developers and users are characterizing DLT architectures. The purpose of this paper is to identify key parameters to test performance, tools that are being used and provide information on results from previous studies.

97 MATHEMATICS AND COMPUTING↗

deadtrees.earth — An open-access and interactive database for centimeter-scale aerial imagery to uncover global tree mortality dynamics

Excessive tree mortality is a global concern and remains poorly understood as it is a complex phenomenon. We lack global and temporally continuous coverage on tree mortality data. Ground-based observations on tree mortality, e.g., derived from national inventories, are very sparse, and may not be standardized or spatially explicit. Earth observation data, combined with supervised machine learning, offer a promising approach to map overstory tree mortality in a consistent manner over space and time. However, global-scale machine learning requires broad training data covering a wide range of environmental settings and forest types. Low altitude observation platforms (e.g., drones or airplanes) provide a cost-effective source of training data by capturing high-resolution orthophotos of overstory tree mortality events at centimeter-scale resolution. Here, we introduce deadtrees.earth, an open-access platform hosting more than two thousand centimeter-resolution orthophotos, covering more than 1,000,000 ha, of which more than 58,000 ha are manually annotated with live/dead tree classifications. This community-sourced and rigorously curated dataset can serve as a comprehensive reference dataset to uncover tree mortality patterns from local to global scales using space-based Earth observation data and machine learning models. This will provide the basis to attribute tree mortality patterns to environmental changes or project tree mortality dynamics to the future. The open nature of deadtrees.earth, together with its curation of high-quality, spatially representative, and ecologically diverse data will continuously increase our capacity to uncover and understand tree mortality dynamics.

Citizen science↗

Impacts of high resolution data on traveler compliance levels in emergency evacuation simulations

In this article, we conducted a comparison study of evacuation assignment based on Traffic Analysis Zones (TAZ) and high resolution LandScan USA Population Cells (LPC) with detailed real world roads network. A platform for evacuation modeling built on high resolution population distribution data and activity-based microscopic traffic simulation was proposed. This platform can be extended to any cities in the world. The results indicated that evacuee compliance behavior affects evacuation efficiency with traditional TAZ assignment, but it did not significantly compromise the performance with high resolution LPC assignment. The TAZ assignment also underestimated the real travel time during evacuation. This suggests that high data resolution can improve the accuracy of traffic modeling and simulation. The evacuation manager should consider more diverse assignment during emergency evacuation to avoid congestions.

97 MATHEMATICS AND COMPUTING↗

RE Data Explorer is Expanding Public Access to High-Quality Data and Analytics

The Renewable Energy (RE) Data Explorer is an innovative and intuitive web platform from the U.S. Agency for International Development (USAID)-National Renewable Energy Laboratory (NREL) Partnership that enables users to easily access high-quality renewable energy resource data and applied analytics.

ENERGY PLANNING, POLICY, AND ECONOMY↗

Parallel k-means Clustering of Geospatial Data Sets Using Manycore CPU Architectures

The increasing availability of high-resolution geospatiotemporal data sets from sources such as observatory networks, remote sensing platforms, and computational Earth system models has opened new possibilities for knowledge discovery and mining of weather, climate, ecological, and other geoscientific data sets fused from disparate sources. Many of the standard tools used on individual workstations are impractical for the analysis and synthesis of data sets of this size; however, new algorithmic approaches that can effectively utilize the complex memory hierarchies and the extremely high levels of parallelism available in state-of-the-art high-performance computing platforms can enable such analysis. Here, we describe pKluster, an open-source tool we have developed for accelerated k-means clustering of geospatial and geospatiotemporal data, and discuss algorithmic modifications and code optimizations we have made to enable it to effectively use parallel machines based on novel CPU architectures—such as the Intel “Knights Landing” Xeon Phi and Skylake Xeon processors—with many cores and hardware threads, and employing significant single instruction, multiple data (SIMD) parallelism. We outline some applications of the code in ecology and climate science contexts and present a detailed discussion of the performance of the code for one such application, LiDAR-derived vertical vegetation structure classification.

Mills, Richard T↗

Persistent DynAMICS

Persistent DynAMICS is a lightweight software platform that enables distributed sensing systems to operate as a unified, responsive network. It integrates with existing sensors, data systems, and processing tools as an overlay, without requiring hardware replacement or increasing user burden. As data is collected, the system identifies situations that require additional attention and directs selected sensors to gather more information. For example, if a sensor detects an unusual condition along a pipeline, the system can task nearby sensors or different sensing modalities to confirm the event and focus data collection where it is needed. To support this, the platform forms temporary groups of sensors that work together to address specific objectives. Once complete, these groups are dissolved and resources are reassigned as needed. Communication is handled through an integrated messaging framework with client and server components, enabling reliable operation and data continuity even in low-bandwidth or intermittent connectivity environments.

97 MATHEMATICS AND COMPUTING↗

SONET: a semantic ontological network graph for managing points of interest data heterogeneity

Scalability, standardization, and management are important issues when working with very large Volunteered Geographic Information (VGI). VGI is a rich and valuable source of Points of Interest (POI) information, but its inherent heterogeneity in content, structure, and scale across sources present major challenges for interlinking data sources for analysis. To be useful at scale, the raw information needs to be transformed into a standardized schema that can be easily and reliably used by data analysts. In this work, we tackle the problem of unifying POI categories (e.g. restaurants, temple, and hotel) across multiple data sources to aid in improving land use maps and population distribution estimation as well as support data analysts wishing to fuse multiple data sources with the OpenStreetMap (OSM) mapping platform or working with projects that are already configured in the OSM schema and wish to add additional sources of information. Graph theory and its implementation through the SONET graph database, provides a programmatic way to organize, store, and retrieve standardized POI categories at multiple levels of abstraction. Additionally, it addresses category heterogeneity across data sources by standardizing and managing categories in a way that makes cross-domain analysis possible.

Palumbo, Rachel↗

CHESS 2025: Leaf Area Index (LAI) for meadow, shrub, tree, and understory vegetation

This dataset contains Leaf Area Index (LAI) measurements made as part of the Colorado Headwaters Ecological Spectroscopy Study (CHESS) during June and July of 2025. Data were collected in the Upper Gunnison Basin, Colorado, across three study domains: the Upper East River (CRBU), Almont Triangle (ALMO), and the Upper Taylor Basin (UPTA). Field observations of LAI were collected within 72 hours of airborne data collection by the National Ecological Observatory Network’s Aerial Observation Platform (NEON AOP). The NEON AOP collected waveform LiDAR (Light Detection and Ranging) and imaging spectrometer data in 426 spectral bands from the visible to shortwave infrared. LAI measurements were collected using the LICOR LAI-2200C Plant Canopy Analyzer following protocols outlined in the instrument manual (LI-COR 2019). Sampling targeted four distinct vegetation types: meadows, shrubs, trees, and aspen forest understory. We have archived data separately by site type because different field methods were used for each. At meadow sites, measurements were made at the four corners of 1m x 1m plots, with the instrument moving inward toward the center of the plot. At shrub sites, we measured the canopies of individual shrubs. At tree sites, we made measurements within a 10m x 10m subplot centered around a focal tree, with 30 observations taken on a regular grid. At aspen understory sites, we measured overstory trees following the tree protocol and understory herbaceous vegetation following the meadow protocol. All measurements included above-canopy (A) and below-canopy (B) readings, with specific protocols for scattering correction measurements in direct-sun conditions. Data were processed using the R package `rlai` (Worsham 2025). This package includes functions to calculate LAI, gap fraction, apparent clumping factor (Ω), scattering correction, and other canopy metrics. Package contents: Full file descriptions appear in ‘flmd.csv’. Files named according to the convention ‘lai_*_summary_data_cleaned.csv’ contain summary values of LAI, apparent clumping factor (Ωapp), and scattering correction factors for each site. These are the analysis-ready products that most data users will work with. Files named ‘lai_*_metadata_cleaned.csv’ contain additional site-level observations made during field collection. We have also archived intermediate and supplementary data for users who wish to check our processing approach or apply alternative methods. ‘raw_lai_2200C.zip’ contains the raw files as read from the LI-COR instrument, with no processing applied, in TXT format. The zip archive contains subdirectories by site type, which are further subdivided by sampling area. Filenames correspond to the sampling site number. ‘intermediate_results.zip’ contains detailed output from the processing routines, in JSON format. The zip archive contains subdirectories by site type; filenames correspond to the sampling site number. ‘scattering_correction_logs.zip’ contains logfiles from the implementation of Kobayashi et al.'s (2013) scattering correction algorithm. The logfiles report values of several parameters at each iteration of the algorithm, as the model converges toward a stable solution. They are intended for users who want to verify scattering correction performance. The zip archive contains subdirectories by site type; filenames correspond to the sampling site number. ‘spot_checks.csv’ reports LAI and other values for a small number of files processed with LI-COR FV2200 software (LI-COR 2013) using the same control parameters as in our R-based approach. Additional metadata are provided in a data dictionary describing column names and definitions (dd.csv), and in a file-level metadata file (flmd.csv). All zip files can be expanded with common archive utilities. TXT, CSV, and JSON files can be ingested into R or Python computing environments or read in common text editor utilities. Geospatial information: Geospatial data for mapping measurement site locations are in the files CHESS_polygons_lai_UTM.geojson, CHESS_polygons_shrub_UTM.geojson, and CHESS_polygons_meadow_UTM.geojson in the companion geospatial package for the 2025 CHESS campaign, ‘CHESS 2025: Location data for field observations and sampling’ (Henderson et al., 2026). CHESS Project Description: The Colorado Headwaters Ecological Spectroscopy Study (CHESS) comprised a multi-week airborne remote sensing and field observation campaign in the Upper Gunnison Basin, Colorado, conducted in June and July of 2025. Airborne remote sensing was conducted by the National Ecological Observatory Network Airborne Observation Platform (NEON AOP), concurrent with a field campaign run by the Rocky Mountain Biological Laboratory (RMBL), the Lawrence Berkeley National Laboratory (LBNL) and SLAC National Accelerator Laboratory Watershed Function Science Focus Area (SFA), and NASA-JPL (Jet Propulsion Laboratory) Earth Surface Mineral Dust Source Investigation (EMIT) program. Between June 10 and July 18, 2025, the NEON AOP flight team collected high-resolution aerial imaging spectroscopy and Light Detection and Ranging (LiDAR) data over three domains: the Upper East River (CRBU), Almont Triangle (ALMO), and the Upper Taylor Basin (UPTA). In coordination with the flights, a field campaign acquired ground-truth observations, including observations of vegetation composition, foliar traits, forest demography, and subsurface properties in 18 core sampling areas within the domains. Additional surface water observations were taken at over 380 point locations. All CHESS campaign datasets can be found within the CHESS ESS-DIVE data portal: https://data.ess-dive.lbl.gov/portals/chess. Funding Acknowledgement: Field and remote-sensing data acquisition was performed under a grant from the National Aeronautics and Space Administration (80NSSC24K1005). This work was also supported by the Watershed Function Science Focus Area at Lawrence Berkeley National Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-05CH11231. * Todorov and Worsham are co–first authors.

2018 NEON and 2025 CHESS Campaigns↗

Development of an Encoding Method on an Co-simulation Platform for Mitigating the Impact of Unreliable Communication

This report presents a hardware-in-the-loop (HIL) based modeling approach for simulating impacts of unreliable communication on the performance of centralized volt-var control and for developing an encoding method to mitigate the impacts. First, an asynchronous real-time HIL simulation platform is introduced to enable multi-rate co-simulation of a distribution system with many inverter-based distributed energy resources (DERs). The distribution system is modeled by milliseconds phasor-based models and the DERs are modeled by micro-seconds power electronic models. Communication connections between a centralized volt-var controller (modeled externally to the HIL testbed) and smart inverters are built by implementing Modbus links and the Long Term Evolution network. On this co-simulation platform, an enhanced, augmented Lagrangian multiplier based encoded data recovery (EALM-EDR) algorithm for mitigating the impact of unreliable communication is developed and validated. Simulation results demonstrate the efficacy of using the HIL-based co-simulation platform as a power grid digital twin for developing algorithms that coordinate a large number of heterogeneous control systems through wired and wireless communication links.

24 POWER TRANSMISSION AND DISTRIBUTION↗

A systematic feature extraction and selection framework for data-driven whole-building automated fault detection and diagnostics in commercial buildings

In data-driven automated fault detection and diagnostics (AFDD) modeling for building energy systems, feature engineering is a critical process of extracting information from high-dimensional and noisy sensor measurement and turning it into informative and representative inputs or features for data-driven modeling. However, few studies specifically discuss the feature engineering, especially the interactions between feature extraction and feature selection in whole-building AFDD. We developed a systematic feature extraction and selection framework for whole-building AFDD. In this framework, features are aggressively extracted from raw sensor data using statistical feature extraction techniques with various window sizes and statistics. With many features extracted, a hybrid feature selection algorithm that combines the filter and wrapper method then selects the best feature set. The framework considers diversity in the duration of fault behavior among fault types in whole-building AFDD, thus achieving high model generalization. We implemented our developed framework in a virtual testbed calibrated with measured data from Oak Ridge National Laboratory's Flexible Research Platform designed to mimic the operation of a typical small commercial building. The AFDD model is trained by the simulation data generated from the virtual testbed. The results show that (1) the developed framework improves the generalization of the AFDD model by 10.7% compared with literature-reported feature extraction and selection methods and (2) features with diverse window sizes and statistics are selected, providing insight into physical systems beyond the current understanding of buildings and faults and improving the detection and diagnostics of multiple fault types.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

GenomeDepot: data management system for microbial comparative genomics

Summary GenomeDepot is an open-source web-based platform for annotation, management, and comparative analysis of microbial genomic sequences and associated data including ortholog families, protein domains, operons, regulatory interactions, strain taxonomy, and sample metadata. GenomeDepot supports rapid creation of websites for user-defined genome collections that include bioinformatic tools for interactive genome browsing, Basic Local Alignment Search Tool (BLAST) search, annotation search, comparative genomic neighborhood visualization, and sequence download. Gene function annotations are generated by a customizable annotation pipeline. The pipeline runs annotation tools in Conda environments and can be easily extended with additional user-specified tools. Availability and implementation GenomeDepot is open source and distributed under the GNU General Public License via GitHub (https://github.com/aekazakov/genome-depot). GenomeDepot is implemented in Python and was tested in Ubuntu Linux. Full installation instructions and documentation are available at https://aekazakov.github.io/genome-depot/. GenomeDepot demo server is freely accessible at https://iseq.lbl.gov/demogd/.

Kazakov, Alexey [Lawrence Berkeley National Labora↗

Lynx web services for annotations and systems analysis of multi-gene disorders

Lynx is a web-based integrated systems biology platform that supports annotation and analysis of experimental data and generation of weighted hypotheses on molecular mechanisms contributing to human phenotypes and disorders of interest. Lynx has integrated multiple classes of biomedical data (genomic, proteomic, pathways, phenotypic, toxicogenomic, contextual and others) from various public databases as well as manually curated data from our group and collaborators (LynxKB). Lynx provides tools for gene list enrichment analysis using multiple functional annotations and network-based gene prioritization. Lynx provides access to the integrated database and the analytical tools via REST based Web Services (http://lynx.ci.uchicago.edu/webservices.html). This comprises data retrieval services for specific functional annotations, services to search across the complete LynxKB (powered by Lucene), and services to access the analytical tools built within the Lynx platform.

59 BASIC BIOLOGICAL SCIENCES↗

Integrated platform for optimized solar PV system design and engineering plan set generation

The Aurora team has developed software that allows users to quickly generate a three-dimensional model for a building, with a corresponding irradiance map, from any two-dimensional image with associated geo-coordinates. The purpose of this project is to build upon that technology by developing and distributing to solar installers a software platform that automatically retrieves engineering, financial and geographic data for a specific site, and quickly generates an optimal customer proposal and corresponding engineering plans for that site. At the end of the project, Aurora’s optimization platform would have been used to make at least one thousand proposals from at least ten unique solar installation companies, two of whom would sign economically viable contracts to use the software. Furthermore, Aurora’s algorithms would be tested to show that in at least seventy percent of cases, Aurora automatically generated a design equivalent to or better than what a human could have done manually. A ‘better’ design is one that generates more energy for the same cost, or that generates a higher return on investment, while complying with all site-specific aesthetic, electrical and spatial requirements.

14 SOLAR ENERGY↗

Mapping and Synthesis of International Biomass Supply Assessments

This report, Mapping and Synthesis of International Biomass Supply Assessments (or Global Biomass Resource Assessment) is the first step in a long-term process to assemble data from around the globe into a virtual repository that can be updated and provide user-friendly access to the data. The Clean Energy Ministerial (CEM) Biofuture Platform Initiative recommended that research be completed to “address the need for internationally accepted benchmarks quantifying sustainable biomass feedstock supplies.” To act upon the CEM Biofuture recommendation, in 2024, the U.S. Department of Energy (DOE) commissioned Oak Ridge National Laboratory (ORNL) to prepare this report as the primary deliverable for a one-year assignment to assemble data into a citable form that could help resolve the persistent question presented related to bioenergy policy, “Is there enough sustainable biomass?” In response to that query, this report includes information received by August 2024 from national CEM representatives, collaborators, and public sources on current and future sustainable biomass supplies in 62 nations,and subsequently documents (a) the approach used by ORNL to analyze and categorize the information received in a manner that enables aggregation and comparability; and (b) recommendations for next steps and guidelines to help others update and harmonize future assessments of global sustainable biomass supplies.

09 BIOMASS FUELS↗