SEARCH · Engineering Papers

Results for “data platform”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

All sources NASA NTRS DOE OSTI

DOE OSTI2024

UAS remote sensing (Autel EVO II platform): Red-green-blue (RGB) imagery and derived products (Level 0-2 data), Seward Peninsula, Alaska, 2022

Airborne remote sensing data collected using a optical red-green-blue (RGB) sensor installed on an Autel EVO II unoccupied aerial system (UAS). This package includes data from 15 flights flown over the NGEE-Arctic Council Mile Marker (MM) 71, Kougarok MM64, and Teller MM27 sites on the Seward Peninsula, Alaska USA, in July 2022. This package provides the Level 0 (raw, unprocessed) data collected by the platform and sensors, and Level 1-2 processed products including digital surface models (DSM), photo orthomosaics, canopy height models (CHM) and digital terrain models (DTM). Ancillary aircraft data, flight mission parameters, and general flight conditions provided by the onboard flight and data collection computers are also included. Data and metadata are provided as image (*.jpg, *.tif), point cloud (*.laz), tabular (*.csv) and *.pdf formats. Raw images are compressed as tar.gz. The metadata documents contain flight campaign, platform, sensor, flight and file metadata, along with a description of the data and file naming scheme.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2024

UAS remote sensing (Autel EVO II platform): Red-green-blue (RGB) imagery and derived products (Level 0-2 data), Seward Peninsula, Alaska, 2023

Airborne remote sensing data collected using a optical red-green-blue (RGB) sensor installed on an Autel EVO II unoccupied aerial system (UAS). This package includes data from 4 flights flown over the NGEE-Arctic Council Mile Marker (MM) 71 and Teller MM27 sites on the Seward Peninsula, Alaska USA, in July 2023. This package provides the Level 0 (raw, unprocessed) data collected by the platform and sensors, and Level 1-2 processed products including digital surface models (DSM), photo orthomosaics, canopy height models (CHM) and digital terrain models (DTM). Ancillary aircraft data, flight mission parameters, and general flight conditions provided by the onboard flight and data collection computers are also included. Data and metadata are provided as image (*.jpg, *.tif), point cloud (*.laz), tabular (*.csv) and *.pdf formats. Raw images are compressed as tar.gz. The metadata documents contain flight campaign, platform, sensor, flight and file metadata, along with a description of the data and file naming scheme.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a 15-year research effort (2012-2027) to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2026

DuraMAT Data Hub

The DuraMAT Data Hub has been supporting the consortium for the past six years. The Data Hub has had success in supporting the projects, providing a platform for sharing data within projects and to the public, and learning how to better leverage the existing software platform and the available Amazon Web Services environment. During this new generation of the Data Hub, we are looking at ways to help improve the data hub architecture, user experience, and improve operations by taking advantage of new technology platforms and software that will be more impactful on the consortium researchers and the broader scientific community. In this poster we will look at the current operational capabilities, data dissemination, and development that will improve the system in the near and far future.

14 SOLAR ENERGY↗

DOE OSTI2015

End-to-end interoperability and workflows from building architecture design to one or more simulations

An end-to-end interoperability and workflows from building architecture design to one or more simulations, in one aspect, may comprise establishing a BIM enablement platform architecture. A data model defines data entities and entity relationships for enabling the interoperability and workflows. A data definition language may be implemented that defines and creates a table schema of a database associated with the data model. Data management services and/or application programming interfaces may be implemented for interacting with the data model. Web services may also be provided for interacting with the data model via the Web. A user interface may be implemented that communicates with users and uses the BIM enablement platform architecture, the data model, the data definition language, data management services and application programming interfaces to provide functions to the users to perform work related to building information management.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2015

End-to-end interoperability and workflows from building architecture design to one or more simulations

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

An interactive machine learning platform for analyzing multi-particle coincidence data from cold target recoil ion momentum spectroscopy

We present SCULPT (Supervised Clustering and Uncovering Latent Patterns with Training), a comprehensive software platform for analyzing tabulated high-dimensional multi-particle coincidence data from Cold Target Recoil Ion Momentum Spectroscopy (COLTRIMS) experiments. The software addresses critical challenges in modern momentum spectroscopy by integrating advanced machine learning techniques with physics-informed analysis in an interactive web-based environment. SCULPT implements uniform manifold approximation and projection for non-linear dimensionality reduction to reveal correlations in high-dimensional data. We also discuss potential extensions to deep autoencoders for feature learning and genetic programming for automated discovery of physically meaningful observables. A novel adaptive confidence scoring system provides quantitative reliability assessments by evaluating user-selected clustering quality metrics with predefined weights that reflect each metric’s robustness. The platform features configurable molecular profiles for different experimental systems, interactive visualization with selection tools, and comprehensive data filtering capabilities. Utilizing a subset of SCULPT’s capabilities, we analyze photo-double-ionization data measured using the COLTRIMS method for three-body dissociation of the D 2 O molecule, revealing distinct fragmentation channels and their correlations with physics parameters. The software’s modular architecture and web-based implementation make it accessible to the broader atomic and molecular physics community, significantly reducing the time required for complex multi-dimensional analyses. This opens the door to finding and isolating rare events exhibiting non-linear correlations on the fly during experimental measurements, which can help steer exploration and improve the efficiency of experiments.

Artificial neural networks↗

DOE OSTI2018

openECA Platform and Analytics Alpha Test Results

The objective of the Open and Extensible Control and Analytics (openECA) Platform for Phasor Data project is to develop an open source software platform that significantly accelerates the production, use, and ongoing development of real-time decision support tools, automated control systems, and off-line planning systems that (1) incorporate high-fidelity synchrophasor data and (2) enhance system reliability while enabling the North American Electric Reliability Corporation (NERC) operating functions of reliability coordinator, transmission operator, and/or balancing authority to be executed more effectively.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2017

openECA Platform and Analytics Beta Demonstration Results

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2017

Python ARM Radar Toolkit (Py-ART) Roadmap

Radar software is key for producing scientific insight from radar data. There are several platforms for interacting with radar data; the open-source variants are well documented in Heistermann et al. 2014. The Python ARM Radar Toolkit (Py-ART; Helmus and Collis, 2016) is one of these. This document is a guiding roadmap for development for Py-ART. It sets priorities for U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) Climate Research Facility-funded work and it advises the community which contributions are desired and will receive the most assistance from ARM-funded software engineers.

Python ARM Radar Toolkit, linear programming, valu↗

DOE OSTI2025

Tracking seasonal variability in plant traits from spaceborne PRISMA and NEON AOP across forest types and ecoregions

Plant traits serve as critical indicators of how plants adapt to environmental changes and influence ecosystem functions. While airborne hyperspectral remote sensing effectively maps plant traits through detailed reflectance properties, it is limited by cost and scale, making large-scale and temporal studies challenging. The recently launched spaceborne hyperspectral imager, PRecursore IperSpettrale della Missione Applicativa (PRISMA), offers frequent, large scale and high-fidelity observations on a spatial resolution of 30 m and a revisit time of around 29 days, making it suitable for large-scale seasonal trait mapping. However, their potential remains largely unexplored. This study developed a multi-stage framework by leveraging the PRISMA spaceborne hyperspectral data and National Ecological Observatory Network (NEON) Airborne Observation Platform (AOP) hyperspectral data to investigate the seasonal dynamics of four key plant traits — chlorophyll content, carotenoid content, equivalent water thickness, and nitrogen content — across eleven NEON sites representing diverse forest types and ecoregions in the contiguous U.S. Our results demonstrated that PRISMA hyperspectral data can reliably track seasonal variability in plant traits, achieving overall R 2 values ranging from 0.78 to 0.88 and normalized root mean square error (NRMSE) values ranging from 5.4% to 8.4% for the four traits. Seasonal patterns revealed bell-shaped trajectories for chlorophyll and carotenoids, while equivalent water thickness decreased steadily across most sites, driven by structural changes during leaf maturation and senescence. Nitrogen content exhibited less pronounced seasonal variation but followed expected nutrient resorption patterns. Analysis of environmental drivers showed that seasonal variability is primarily controlled by solar radiation and day length in northern sites, vapor pressure in semi-arid regions, and temperature in mid-southeastern sites. Spatial variability, meanwhile, was primarily driven by soil properties, particularly during the peak growing season. However, the influence of soil variables slightly declines toward the end of the season at several sites, as climatic factors become more prominent. This study highlights the capability of PRISMA, and potentially other similar spaceborne hyperspectral data for large-scale, time-series plant trait mapping and provides valuable insights into the interactions between plant traits and environmental factors. In conclusion, these findings contribute to advancing our understanding of plant functional ecology and improving predictions of ecosystem responses to environmental changes.

Environmental drivers↗

DOE OSTI2024

rSHUD v2.0: advancing the Simulator for Hydrologic Unstructured Domains and unstructured hydrological modeling in the R environment

Abstract. Hydrological modeling is a crucial component in hydrology research, particularly for projecting future scenarios. However, achieving reproducibility and automation in distributed hydrological modeling research for modeling, simulation, and analysis is challenging. This paper introduces rSHUD v2.0, an innovative, open-source toolkit developed in the R environment to enhance the deployment and analysis of the Simulator for Hydrologic Unstructured Domains (SHUD). The SHUD is an integrated surface–subsurface hydrological model that employs a finite-volume method to simulate hydrological processes at various scales. The rSHUD toolkit includes pre- and post-processing tools, facilitating reproducibility and automation in hydrological modeling. The utility of rSHUD is demonstrated through case studies of the Shale Hills Critical Zone Observatory in the USA and the Waerma watershed in China. The rSHUD toolkit's ability to quickly and automatically deploy models while ensuring reproducibility has facilitated the implementation of the Global Hydrological Data Cloud (https://ghdc.ac.cn, last access: 1 September 2023), a platform for automatic data processing and model deployment. This work represents a significant advancement in hydrological modeling, with implications for future scenario projections and spatial analysis.

Shu, Lele (ORCID:0000000269034466)↗

DOE OSTI2020

The GABLE Report: Garbled Autonomous Bots Leveraging Ethereum

Simple but mission-critical internet-based applications that require extremely high reliability and availability could potentially benefit from running on robust public programmable blockchain platforms such as Ethereum. Unfortunately, program code running on such blockchains is ordinarily publicly viewable, rendering these platforms unsuitable for applications requiring strict privacy of application code, data, and results. However, might it be possible to encode an application's business logic and data for these platforms in such a way that it becomes impossible for unauthorized parties to infer any meaningful information whatsoever about the semantics of the data, and the operations being performed on that data? In this report, we describe GABLE (Garbled Autonomous Bots Leveraging Ethereum), a system concept developed at Sandia that achieves this security goal in a limited, but still useful range of circumstances. GABLE, uses simple but effective algorithms to permit secure private execution of garbled state machines (and more efficient garbled circuits) on public computing resources. We give an example working implementation for garbled state machines, written using the Python and Solidity programming languages, and outline how our methods can be extended to support a more powerful garbled universal circuit model of computation. The capability embodied by the GABLE, system has significant potential applications, a few of which we discuss in this report.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2017

Python and HPC for High Energy Physics Data Analyses

High level abstractions in Python that can utilize computing hardware well seem to be an attractive option for writing data reduction and analysis tasks. In this paper, we explore the features available in Python which are useful and efficient for end user analysis in High Energy Physics (HEP). A typical vertical slice of an HEP data analysis is somewhat fragmented: the state of the reduction/analysis process must be saved at certain stages to allow for selective reprocessing of only parts of a generally time-consuming workflow. Also, algorithms tend to to be modular because of the heterogeneous nature of most detectors and the need to analyze different parts of the detector separately before combining the information. This fragmentation causes difficulties for interactive data analysis, and as data sets increase in size and complexity (O10 TiB for a “small” neutrino experiment to the O10 PiB currently held by the CMS experiment at the LHC), data analysis methods traditional to the field must evolve to make optimum use of emerging HPC technologies and platforms. Mainstream big data tools, while suggesting a direction in terms of what can be done if an entire data set can be available across a system and analysed with high-level programming abstractions, are not designed with either scientific computing generally, or modern HPC platform features in particular, such as data caching levels, in mind. Our example HPC use case is a search for a new elementary particle which might explain the phenomenon known as “Dark Matter”. Here, using data from the CMS detector, we will use HDF5 as our input data format, and MPI with Python to implement our use case.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2018

Meeting the imperative to accelerate environmental bioelectromagnetics research

Highlights: • Concerns mount about adverse health effects of low-level radiofrequency (RF) radiation. • Bioelectronics and environmental RF effects research both need an omics platform. • Standard bioelectromagnetic data models would advance dosimetry and omics work. • Steps toward developing standards and a shared research platform are proposed. In this article, the author draws on his experience in the world of geospatial information technology standards to suggest a path toward acceleration of bioelectromagnetics science. Many studies show biological effects of extremely low frequency (ELF) and radiofrequency (RF) radiation despite that fact that the radiation is too weak to cause temperature changes in biological features. Considered together in worst case scenarios, such effects, many of which appear to have long latencies, could have potentially disastrous consequences for the health and safety of humans and wildlife. Other studies show no such effects, and in both cases, often there are significant research quality deficits that make it difficult to draw firm conclusions from the data. The progress of bioelectromagnetics science is retarded by a lack of standard data models and experimental protocols that could improve the overall quality of research and make it easier for researchers to benefit from omics-related bioinformatics resources. {sup C}ertainty of safety{sup }of wireless devices used in digital communications and remote sensing (radar) is impossible without dosimetry standards that reflect the effects of non-thermal exposures. Electrical signaling in biological systems, a poorly funded research domain, is as biologically important as chemical signaling, a richly funded research domain, and these two types of signaling are inextricably connected. Entreprenuerial scientists pursuing bioelectronic innovations have begun to attract new funding. With appropriate institutional coordination, this new funding could equally benefit those investigating environmental effects of ELF and RF radiation. The author proposes a concerted effort among both bioelectronics technology stakeholders and environmental bioelectromagnetics science researchers to collaborate in developing institutional arrangements and standard data models that would give the science a stronger bioinformatics platform and give researchers better access to omics data. What is proposed here is essentially a bioelectromagnetics omics initiative.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2018

ShareAnalytics

ShareAnalytics is a lightweight extensible platform for collaborative data analytics and facility characterization via Non-intrusive Load Monitoring (NILM). It consists of three core components: a full stack web application, a dashboard tool for analyzing streaming data and a High Performance Computing (HPC) cluster leveraging slurm for resource management to perform real time analysis. The entire framework is implemented in Docker for ease of deployment and to allow the user to test the platform before making a significant investment in hardware if they require additional computing resources. While many other platforms for analyzing streaming data exists, ShareAnalytics is unique in that HPC capabilities and facility characterization via scored analytics are fully integrated into the platform.

Oesch, TimothyS↗

DOE OSTI2019

From Data to Models and Analytics: An Integrated Scientific Computing Platform to Accelerate Environmental Research SBIR Phase I (Final Technical Report)

Phase I research was conducted to investigate the technical feasibility of an integrated open source platform GeoLib to provide turnkey solutions for model-data integration for environmental systems science (ESS), using modern web technologies, advances in HPC and cloud computing and high-performance visualization. A key feature of our platform is that it minimizes and automates data movement between data storage and computing systems. Another critical feature of the design is that it enables users to perform high-fidelity post-processing by performing high-performance visualization server-side and in-situ on the compute cluster close to the data. The investigation, which was performed over a 12-month period ending April 2019, centered on developing prototype software to demonstrate the key enabling technologies required for this system. This included: (i) rapid access to data from multiple remote sources, via a unified data interface; (ii) software for scalable geospatial visualization of raster, vector, and point cloud data on the web; (iii) development of an easy-to-use API for creating simulation and modeling workflows; (iv) remote data visualization and processing on middle-tier servers; and (v) remote data processing on high-performance computing systems. A critical part of this work is that all of these capabilities be accessible to end users through standard desktop tools such as Project Jupyter, which is widely adopted for ESS simulation and modeling.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

Fast Data Processing for Hyperspectral Sensors on Small Platforms

Hyperspectral imaging is a very promising technology for nuclear proliferation detection. However, due to size and weight restrictions, small hyperspectral platforms such as satellites and small drones lack the on-board computing resources for accurate, real-time analysis of the enormous flow of data that a continuously operating hyperspectral sensor generates. This severely limits satellite systems, which can collect far more data than what they can telemeter, and hinders the ability of all platforms to adapt their missions on the fly in response to observations. This program addresses the hyperspectral data processing challenge through development of new, fast and accurate algorithms that produce data products in real time. The algorithms circumvent the major computational bottlenecks in existing processing streams, and would be incorporated in lightweight, power-efficient single-board computer systems. The toolkit of fast algorithms will be immediately useful in current and future hyperspectral systems being built by the Government and by private industry, including drone-based systems and satellite constellations that acquire timely global imagery.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

DOE OSTI2021

Cric searchable image database as a public platform for conventional pap smear cytology data

Amidst the current health crisis and social distancing, telemedicine has become an important part of mainstream of healthcare, and building and deploying computational tools to support screening more efficiently is an increasing medical priority. The early identification of cervical cancer precursor lesions by Pap smear test can identify candidates for subsequent treatment. However, one of the main challenges is the accuracy of the conventional method, often subject to high rates of false negative. While machine learning has been highlighted to reduce the limitations of the test, the absence of high-quality curated datasets has prevented strategies development to improve cervical cancer screening. The Center for Recognition and Inspection of Cells (CRIC) platform enables the creation of CRIC Cervix collection, currently with 400 images (1,376 × 1,020 pixels) curated from conventional Pap smears, with manual classification of 11,534 cells. This collection has the potential to advance current efforts in training and testing machine learning algorithms for the automation of tasks as part of the cytopathological analysis in the routine work of laboratories.

59 BASIC BIOLOGICAL SCIENCES↗