SEARCH · Engineering Papers

Results for “data platform”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20

All sources NASA NTRS DOE OSTI

DOE OSTI2020

Innovation in Radiological Security, Part 2 of 2 – Insights into Developing a Cloud-hosted Security Technology

A Sentry Remote Monitoring System (Sentry-RMS) is a stand-alone security system that provides detection, assessment, and communication of priority alarms as an additional means of thwarting internal and external threats to sites that maintain radiological material. The SEntry-RMS CommUnications and REsponse (Sentry-SECURE) platform is an optional feature of the Sentry-RMS that relays priority alarm information to the identified response stakeholders. Sentry-SECURE is hosted in a cloud environment that abstracts the data owner’s and data consumer’s platforms to allow for greater information sharing. This promotes situational awareness amongst authorized users and enables future innovation among modern response platforms. When securely architecting a cloud solution such as this, the use of design paradigms can be an effective tool to increase the accuracy and reliability of cyber- and information-security-related decisions made throughout the development process. This approach also supports the categorization of design considerations into three levels: industry concepts, project approaches, and data protections for digital processes. Industry concepts consist of the notional underpinnings that guide or motivate a security process, system, or design but often lack any tangible attributes. Project approaches represent decisions made during the design and development process to prioritize a solution, method, or practice above another that may provide a comparable functional output but lacks a desired security benefit. Data protections for digital processes represent the selection, integration, and implementation of specific controls for a given asset. This paper will explore specific examples of how Sentry-SECURE has been designed to account for considerations at each of these three levels, while balancing the operational intent of the platform with the security enhancements necessary to maintain data integrity, availability, and confidentiality.

assessment, RMS, security, physical, cloud, Cyber ↗

DOE OSTI2022

The Astronomy Commons Platform: A Deployable Cloud-based Analysis Platform for Astronomy

Abstract We present a scalable, cloud-based science platform solution designed to enable next-to-the-data analyses of terabyte-scale astronomical tabular data sets. The presented platform is built on Amazon Web Services (over Kubernetes and S3 abstraction layers), utilizes Apache Spark and the Astronomy eXtensions for Spark for parallel data analysis and manipulation, and provides the familiar JupyterHub web-accessible front end for user access. We outline the architecture of the analysis platform, provide implementation details and rationale for (and against) technology choices, verify scalability through strong and weak scaling tests, and demonstrate usability through an example science analysis of data from the Zwicky Transient Facility’s 1Bn+ light-curve catalog. Furthermore, we show how this system enables an end user to iteratively build analyses (in Python) that transparently scale processing with no need for end-user interaction. The system is designed to be deployable by astronomers with moderate cloud engineering knowledge, or (ideally) IT groups. Over the past 3 yr, it has been utilized to build science platforms for the DiRAC Institute, the ZTF partnership, the LSST Solar System Science Collaboration, and the LSST Interdisciplinary Network for Collaboration and Computing, as well as for numerous short-term events (with over 100 simultaneous users). In a live demo instance, the deployment scripts, source code, and cost calculators are accessible. 4 4 http://hub.astronomycommons.org/

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

UNH TDP - Concurrent Measurements of Inflow, Power Performance, and Loads for a Grid-Synchronized Vertical Axis Cross-Flow Turbine Operating in a Tidal Estuary

This data was collected between October 12 and December 15 of 2021 at the University of New Hampshire (UNH) and Atlantic Marine Energy Center (AMEC) turbine deployment platform (TDP). This data set includes over 29 days of grid connected turbine operation during this 65 day time frame. The priority for this measurement campaign was to collect data while the turbine was electrically connected to the grid by means of a rectifier and inverter. The Fall_2021_UNH_Measurement_Timeline.png highlights when each instrument was functioning and the Fall_2021_UNH_Test_Log.jpg indicates the four main regions for analysis available from this measurement campaign. The TDP is a floating structure moored on the Portsmouth facing side of Memorial Bridge pier #2, which spans the Piscataqua River between Portsmouth, NH and Kittery, ME. The Piscataqua River connects the Great Bay Estuary to the Gulf of Maine and the river currents are dominated by tidal forcing with water velocities exceeding 2.5 m/s during spring ebb tides at this site which were previously characterized by Kaelin Chancey (Assessment Of The Localized Flow And Tidal Energy Conversion System At An Estuarine Bridge - UNH MS Thesis 2019). The turbine under test was a modified New Energy Corporation (Calgary, CA) model EVG-025 4-blade H-Darrius type vertical axis cross flow turbine that rotates in the clockwise direction with a rotor diameter of 3.2m and blade length of 1.7m. The hydro-foil profile was a NACA 0021 with a 10 inch chord length and a blade preset pitch angle of +4deg with a positive angle corresponding with the toe in direction. The standard EVG-025 has a rotor diameter of 3.4m and its rated power output is 25kW at 3 m/s. The rotor diameter was reduced to accommodate the size of the existing TDP moon-pool. This project was pursued to quantify device performance for cross flow turbines operating in a marine environment. Accurate physical models, to characterize cross flow turbine performance, require real operational data sets due to the complexity of blade fluid interactions. This data can help support model development which will help predict turbine performance when analyzing perspective project locations in the future. Instrumentation was deployed to measure; water speed/direction, electrical power output, turbine shaft speed, turbine thrust force, and platform motion. Concurrent measurements of these parameters allow for correlations (cause and affect) to be inferred, allowing for characterization of device performance over a range of operating conditions. Water currents were measured using Acoustic Doppler Current Profilers (ADCP's) and Acoustic Doppler Velocimeters (ADV's) directly upstream and downstream of the turbine for inflow, wake and turbulence measurements. Electrical power output was measured using the Voltsys rectifier and the Shark power meter. Shaft speed was calculated based on the Voltsys measurements of the permanent magnet three phase generator AC generation frequency, coupled directly to the cross flow turbine under test (i.e., no gear box). Platform motions were captured using a Yost IMU (inertial measurement unit). Turbine thrust loading was measured using a reaction arm about the turbine deployment platform spanning beam, where two bi-directional load cells were connected to the system via a pinned connection. This submission includes zipped folders for each instrument containing quality controlled (QC'd) data in daily .csv files for the relevant duration specific to each instrument, along with separate .csv file that contains the units for each variable. Some instrument daily files are quite large and can pose a challenge for a visual spreadsheet editor to open. A processing software like MATLAB or Python is recommended. Note the degree of QC varied between each instrument due to time constraints. Particular time and attention was given to perform quality control tests on the acoustic based instruments that are particularly suscep...

16 TIDAL AND WAVE POWER↗

DOE OSTI2015

Microbial Community Analysis with Ribosomal Gene Fragments from Shotgun Metagenomes

Shotgun metagenomic sequencing does not depend on gene-targeted primers or PCR amplification; thus, it is not affected by primer bias or chimeras. However, searching rRNA genes from large shotgun Illumina data sets is computationally expensive, and no approach exists for unsupervised community analysis of small-subunit (SSU) rRNA gene fragments retrieved from shotgun data. We present a pipeline, SSUsearch, to achieve the faster identification of short-subunit rRNA gene fragments and enabled unsupervised community analysis with shotgun data. It also includes classification and copy number correction, and the output can be used by traditional amplicon analysis platforms. Shotgun metagenome data using this pipeline yielded higher diversity estimates than amplicon data but retained the grouping of samples in ordination analyses. We applied this pipeline to soil samples with paired shotgun and amplicon data and confirmed bias against Verrucomicrobia in a commonly used V6-V8 primer set, as well as discovering likely bias against Actinobacteria and for Verrucomicrobia in a commonly used V4 primer set. In conclusion, this pipeline can utilize all variable regions in SSU rRNA and also can be applied to large-subunit (LSU) rRNA genes for confirmation of community structure.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2018

Monitoring Surface Phenomena Created by an Underground Chemical Explosion Using Fully Polarimetric VideoSAR

Sandia National Laboratories flew its Facility for Advanced RF and Algorithm Development X-Band (9.6-GHz center frequency), fully polarimetric synthetic aperture radar (PolSAR) in VideoSAR mode to collect complex-valued SAR imagery before, during, and after the sixth Source Physics Experiment's (SPE-6) underground explosion. The VideoSAR products generated from the data sets include "movies'' of single- and quad-polarization coherence maps, magnitude imagery, and polarimetric decompositions. Residual defocus, due to platform motion during data acquisition, was corrected with a digital elevation model-based autofocus algorithm. We generated and exploited the VideoSAR image products to characterize the surface movement effects caused by the underground explosion. Unlike seismic sensors, which measure local area seismic waves using sparse spacing and subterranean positioning, these VideoSAR products captured high-spatial resolution, 2-D, time-varying surface movement. The results from the fifth SPE (SPE-5) used single-polarimetric VideoSAR data. In this paper, we present single-polarimetric and fully polarimetric VideoSAR results while monitoring the SPE-6 underground chemical explosion. We show that fully polarimetric VideoSAR imaging provides a unique, coherent, time-varying measure of the surface expression of the SPE-6 underground chemical explosion. As a result, we include new surface characterization results from the measured PolSAR SPE-6 data via H/A/α polarimetric decomposition.

47 OTHER INSTRUMENTATION↗

DOE OSTI2021

A Hardware Platform for Studying Naval Power Electronic Power Distribution Systems

Abstract – Future intelligent ship system designs will likely include electric propulsion, numerous highpower sensors, and directed energy weapons. Supply and control of these large nonlinear loads will require a networked, multi-converter power electronic power distribution system. This work presents a hardware platform to emulate a microgrid power system with multiple power converters and a power data communication network. The platform is reconfigurable and can include both AC and DC power distribution zones, representative of shipboard power systems. It also allows for the study of both power control actuation and power data communication delays. Since the platform is based on electric power hardware, spatial and temporal uncertainties are inherently embedded in the system. Specifically, this work examines the control actuation of multiple pulsed power loads in a single microgrid. Several pulse load levels and operating scenarios have been implemented and measured. A framework for control parameter quantification is presented, and various metrics are explored to capture pulse signal characteristics. The sensitivity of pulse load metric parameters is analyzed. Dynamic shipboard, mission-specific load profiles coupled with pulse loads can also be emulated in the hardware platform.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2018

SIRIUS: Enabling Progressive Data Exploration for Extreme-Scale Scientific Data

Scientific simulations on high performance computing (HPC) platforms generate large quantities of data. To bridge the widening gap between compute and I/O, and enable data to be more efficiently stored and analyzed, simulation outputs need to be refactored, reduced, and appropriately mapped to storage tiers. However, a systematic solution to support these steps has been lacking in the current HPC software ecosystem. To that end, this paper develops SIRIUS, a progressive JPEG-like data management scheme for storing and analyzing big scientific data. It co-designs data decimation, compression, and data storage, taking the hardware characteristics of each storage tier into considerations. With reasonably low overhead, our approach refactors simulation data, using either topological or uniform decimation, into a much smaller, reduced-accuracy base dataset, and a series of deltas that is used to augment the accuracy if needed. The base dataset and deltas are compressed and written to multiple storage tiers. Data saved on different tiers can then be selectively retrieved to restore the level of accuracy that satisfies data analytics. Thus, SIRIUS provides a paradigm shift towards elastic data analytics and enables end users to make trade-offs between analysis speed and accuracy on-the-fly. This paper further develops algorithms to preserve statistics for data decimation, a common requirement for reducing data. Here, we assess the impact of SIRIUS on unstructured triangular meshes, a pervasive data model used in scientific simulations. In particular, we evaluate two realistic use cases: the blob detection in fusion and high-pressure area extraction in computational fluid dynamics.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2014

Scaling Graph Community Detection on the Tilera Many-core Architecture

In an era when power constraints and data movement are proving to be significant barriers for the application of high-end computing, the Tilera many-core architecture offers a low-power platform exhibiting many important characteristics of future systems, including a large number of simple cores, a sophisticated network-on-chip, and fine-grained control over memory and caching policies. While this emerging architecture has been previously studied for structured compute-intensive kernels, benchmarking the platform for data-bound, irregular applications present significant challenges that have remained unexplored. Community detection is an advanced prototypical graph-theoretic operation with applications in numerous scientific domains including life sciences, cyber security, and power systems. In this work, we explore multiple design strategies toward developing a scalable tool for community detection on the Tilera platform. Using several memory layout and work scheduling techniques we demonstrate speedups of up to 46x on 36 cores of the Tilera TileGX36 platform over the best serial implementation, and also show results that have comparable quality and performance to mainstream x86 platforms. To the best of our knowledge this is the first work addressing graph algorithms on the Tilera platform. This study demonstrates that through careful design space exploration, low-power many-core platforms like Tilera can be effectively exploited for graph algorithms that that embody all the essential characteristics of an irregular application.

community detection↗

DOE OSTI2020

A CIM Based Data Integration Framework for Distribution Utilities

With the proliferation of distributed energy resources and advanced metering, modern electric power distribution systems are data rich and include advanced capabilities for distribution automation. Distribution utilities need applications for planning and operations that can integrate all the available data from the enterprise applications and may incorporate distributed approaches to operate and control. This paper describes a framework for standardizing and integrating the data available at different vendor specific applications in a utility utilizing the Common Information Model (CIM). Theframework incorporates the conversion of these the standardized CIM models into models compatible with GridLAB-D, an open source distribution system simulator, for developing advanced planning and operation strategies.

data integration platform, object-oriented data mo↗

DOE OSTI2024

DOE Data Days 2023 Report

The DOE Data Days (D3) workshop brings together data managers, developers, researchers, and program managers across the Department of Energy (DOE) and national laboratories to highlight data management successes, identify potential synergies and common problems, and establish channels for collaboration across the DOE data management community. The fourth D3 workshop was held on October 24th to 26th, 2023 held entirely in-person at Lawrence Livermore National Laboratory (LLNL). The workshop was organized by a multi laboratory committee in an effort to bring data management practitioners at the DOE laboratories together to share their work and results, facilitating knowledge transfers and best practices across project teams. Tools and platforms to support data management and analysis are rapidly evolving and provide enormous opportunities. This report summarizes the important discussions and recommendations from the different working sessions and contains the agenda, submitted abstracts, presentations with links to recorded presentations, breakout session summaries, list of registered attendees, and lessons learned for future organizing committee. The report will be distributed to the DOE, each participating institution’s programmatic stakeholders, and attendees. The dedicated D3 website will host presentations, agenda, and report that is accessible by all labs.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

DOE OSTI2021

S0 Uncrewed Aircraft System Measurement Characterization Field Campaign Report

The lower atmosphere contributes directly to the modulation of weather and climate. Understanding the importance of this part of the atmosphere, scientists have worked to improve the representation of the atmospheric boundary layer in numerical prediction tools. Such work depends upon information from observing systems, including remote sensors, weather balloons, and research aircraft. Recent years have seen significant advances in uncrewed aircraft systems (UAS) for atmospheric research. Those efforts have provided new perspectives on atmospheric and surface conditions, particularly from smaller UAS (sUAS) platforms. This field campaign included collection of data using several sUAS, and comparing the sUAS data to those from different observing facilities at the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) user facility’s Southern Great Plains (SGP) observatory in Oklahoma. Over the course of one week, the Black Swift Technologies and University of Nebraska teams conducted flights for the sole purpose of system evaluation and intercomparison. In addition, these teams conducted joint flight operations at the Marshall, Oklahoma Mesonet site alongside teams from the University of Colorado, University of Oklahoma, and Oklahoma State University to collect additional data for side-by-side comparison. Platforms operated by Black Swift Technologies and the University of Nebraska under this project included one fixed-wing and two rotary-wing platforms, as well as an instrumented surface vehicle. These include the Black Swift Technologies S0 UAS, the University of Nebraska M600 UAS, a University of Nebraska-operated Meteomatics Meteodrone UAS, and the University of Nebraska CoMeT (Combined Mesonet and Tracker) vehicle. Additional details on all of these platforms can be found in de Boer et al. (in prep). In total, 95 sUAS flights were conducted for a combined total of 18.6 flight hours. The S0 was operated both to the south and north of the SGP 60 m tower, while the M600 and Meteodrone were operated directly to the east of the tower. Weather conditions were generally good, with moderate winds. Flights were conducted both to follow the extra radiosondes that were launched as part of this campaign, as well as to conduct extended statistical sampling at tower instrument heights. The later flights also provided data for statistical platform intercomparison.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2021

Synchronized Waveforms – A Frontier of Data-Based Power System and Apparatus Monitoring, Protection, and Control

Voltage and current waveforms contain the most authentic and granular information on the behaviors of power systems. In recent years, it has become possible to synchronize waveform data measured from different locations. Thus large-scale coordinated analyses of multiple waveforms over a wide area are within our reach. This development could unleash a set of new concepts, strategies, and tools for monitoring, protecting, and controlling power systems and apparatuses. This paper presents an in-depth review and analysis of the advancements in synchronized waveform data, including measurement devices, data characteristics, use cases, and comparisons with synchrophasor data. Based on the findings, five strategies are proposed to discover and develop synchronized waveform based applications over multiple application areas. The paper also presents three complementary measurement platforms and two data screening algorithms for application implementation. It further discusses committee activities and standard developments useful to explore the full potential of the data.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2024

Position Papers for the 2024 ASCR Workshop on Energy-Efficient Computing for Science

On behalf of the Advanced Scientific Computing Research (ASCR) program in the US Department of Energy (DOE) Office of Science, we are organizing a Workshop on Energy-Efficient Computing for Science (EECS). Energy efficiency involves coordination across all the interoperating components of a computing system—in particular, applications, algorithms, system software, programming models, data management, and the hardware on which they run. Looking 10-15 years into the future, the goal is to dramatically lower the energy costs of the computational platforms (from the data center to the edge) serving DOE science while expanding the capabilities of these systems, broadening their applicability to science challenges of interest to DOE and the nation.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

DOE OSTI2024

An Open-Source Framework for Characterizing Urban Energy Models: Integrating Top-Down and Bottom-Up Methods to Predict Residential Buildings Characteristics: Preprint

Bottom-up urban energy models are crucial for understanding current energy use patterns and informing design strategies. However, accurately characterizing these models to represent different communities remains a challenge due to the extensive data needed for simulating existing energy use behavior. This data includes information related to human activities and building characteristics, all of which correlate with socioeconomic factors. To overcome this challenge, we developed an automated framework that utilizes both top-down and bottom-up data, to predict unknown building and occupant characteristics that are needed for more accurate and equitable modeling and analytics. Our framework, integrated into the URBANopt district energy modeling platform, uses statistical data models from ResStock. URBANopt models co-located buildings and neighborhoods. At this scale there are data gaps in building characteristic data, such as materials, insulation, occupancy, income, and energy usage of the buildings. To address this data gap, we use ResStock data, representative at the census tract scale, and develop machine-learning and deeplearning techniques to disaggregate it to individual buildings. By mapping unique occupant, building and economic properties to URBANopt energy models, we gain detailed insights into the variability of building energy use across different neighborhoods. This insight helps deploy technologies for co-located buildings and supports targeted upgrades for communities with unique economic and demographic characteristics, ensuring energy equity. Accurate characterization of energy models allows us to develop equitable strategies tailored to diverse neighborhoods, whether underserved or affluent. Our automated framework streamlines energy modeling and provides a reliable tool for building energy characterization.

ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATION↗

DOE OSTI2022

CityBES v2021

City Buildings, Energy, and Sustainability (CityBES) is a web-based data and computing platform, focusing on energy modeling and analysis of a city's building stock to support district or city-scale building energy efficiency programs. CityBES uses an international open data standard, CityGML, to represent and exchange 3D city models. CityBES employs EnergyPlus to simulate building energy use and savings from energy efficient retrofits. Other CityBES features include energy benchmarking, district heating and cooling system modeling, rooftop PV analysis, building performance visualization, heat resilience modeling, as well as urban scale mapping of microclimate and heat vulnerability at census tract level. Different from other tools, CityBES uses integrated open and standard 3D city building data and models each individual building using EnergyPlus. CityBES can be used by urban planners, city energy managers, building owners, utilities, energy consultants and researchers.

Hong, Tianzhen↗

DOE OSTI2026

MSD CoP Webinar: "Advances in MSD-LIVE to Support the MSD Community of Practice"

Context: This webinar was hosted by the MultiSector Dynamics Community of Practice (MSD CoP; https://multisectordynamics.org). Advances in MSD-LIVE to Support the MSD Community of Practice Presenters: Casey Burleyson and Zoe Guillen (Pacific Northwest National Laboratory) Abstract: The MultiSector Dynamics Living, Intuitive, Value-adding, Environment (MSD-LIVE; msdlive.org) is a cloud-based data management system and advanced computing platform that enables MSD researchers to document and archive their data, run their models and analysis tools, and share their data, software, and workflows within the MSD Community of Practice. Recently, several high-profile datasets have attracted many new users to MSD-LIVE. This webinar has two goals: 1) To refamiliarize the MSD community and new users with the components of the platform (e.g., the data repository, model training notebooks, and data dashboards) and to highlight examples of how these components are advancing MSD science and 2) To demonstrate new features in v3 of the platform, released in late 2025. The main new feature in v3 is the ability to interactively explore data in MSD-LIVE without downloading it. MSD-LIVE users can now click a button in our data repository and launch a blank Jupyter notebook with access to the underlying data on AWS. Users can use the notebook to write analysis, visualization, or subsetting routines that process the data directly on the AWS cloud. We also added a GitHub integration feature that allows users to share analysis or visualization code they develop with the community of MSD-LIVE users. The webinar will wrap up with a look at what's coming next for MSD-LIVE in 2026. Moderator: Patrick M. Reed (MSD CoP Facilitation Team) This webinar was held on: May 12th, 2026 from 1-2 PM EST.

Open Science↗

DOE OSTI2025

The Persistent Challenge of Data Locality in the Post-Exascale Era

The era of exascale computing, exemplified by systems like Frontier achieving exaflop-level performance, marks a milestone. However, the quest for sheer compute power leads to strong imbalance in system design. Hence, scaling advancements in memory, network bandwidth, and storage are also necessary and pose challenges, with a crucial need to address data locality issues. This article underscores the fundamental importance of data locality as a key abstraction for optimizing application performance. Despite notable software solutions, the growing complexity of parallelism and memory hierarchy demands performance-portable data locality solutions across diverse computing platforms. Additionally, the article revisits data locality aspects, covering hardware considerations, application perspectives, software stack abstractions, and tool support. It concludes with insights into data locality challenges and opportunities, emphasizing the ongoing significance of collaborative research for progress in this critical issue.

Unat, Didem [Koc University, Istanbul (Turkey)] (O↗

DOE OSTI2020

The MolSSI QCArchive project: An open-source platform to compute, organize, and share quantum chemistry data

The Molecular Sciences Software Institute's (MolSSI) Quantum Chemistry Archive (QCArchive) project is an umbrella name that covers both a central server hosted by MolSSI for community data and the Python-based software infrastructure that powers automated computation and storage of quantum chemistry (QC) results. The MolSSI-hosted central server provides the computational molecular sciences community a location to freely access tens of millions of QC computations for machine learning, methodology assessment, force-field fitting, and more through a Python interface. Facile, user-friendly mining of the centrally archived quantum chemical data also can be achieved through web applications found at the website. The software infrastructure can be used as a standalone platform to compute, structure, and distribute hundreds of millions of QC computations for individuals or groups of researchers at any scale. The QCArchiveInfrastructure is open-source (BSD-3C), code repositories can be found at github, and releases can be downloaded via PyPI and Conda. This article is categorized under: Electronic Structure Theory > Ab Initio Electronic Structure Methods Software > Quantum Chemistry Data Science > Computer Algorithms and Programming

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗