Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data platform”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,189 records · Page 66

Estimating radiological background using imaging spectroscopy

Optical imaging spectroscopy is investigated as a method to estimate radiological background by spectral identification of soils, sediments, rocks, minerals and building materials derived from natural materials and assigning tabulated radiological emission values to these materials. Radiological airborne surveys are undertaken by local, state and federal agencies to identify the presence of radiological materials out of regulatory compliance. Detection performance in such surveys is determined by (among other factors) the uncertainty in the radiation background; increased knowledge of the expected radiation background will improve the ability to detect low-activity radiological materials. Radiological background due to naturally occurring radiological materials (NORM) can be estimated by reference to previous survey results, use of global 40K, 238U, and 232Th (KUT) values, reference to existing USGS radiation background maps, or by a moving average of the data as it is acquired. Each of these methods has its drawbacks: previous survey results may not include recent changes, the global average provides only a zero-order estimate, the USGS background radiation map resolutions are coarse and are accurate only to 1 km – 25 km sampling intervals depending on locale, and a moving average may essentially low pass filter the data to obscure small changes in radiation counts. Imaging spectroscopy from airborne or spaceborne platforms can offer higher resolution identification of materials and background, as well as provide imaging context information. AVIRIS hyperspectral image data is analyzed using commercial exploitation software to determine the usefulness of imaging spectroscopy to identify qualitative radiological background emissions when compared to airborne radiological survey data.

background radiation estimation↗

A cross-platform execution engine for the quantum intermediate representation

Hybrid languages like the quantum intermediate representation (QIR) are essential for programming systems that mix quantum and conventional computing models, while execution of these programs is often deferred to a system-specific implementation. Here, we develop the QIR Execution Engine (QIR-EE) for parsing, interpreting, and executing QIR across multiple hardware platforms. QIR-EE uses LLVM to execute hybrid instructions specifying quantum programs and, by design, presents extension points that support customized runtime and hardware environments. We demonstrate an implementation that uses the XACC quantum hardware-accelerator library to dispatch prototypical quantum programs on different commercial quantum platforms and numerical simulators, and we validate execution of QIR-EE on IonQ, Quantinuum, and IBM hardware. Our results highlight the efficiency of hybrid executable architectures for handling mixed instructions, managing mixed data, and integrating with quantum computing frameworks to realize cross-platform execution.

LLVM↗

Enabling the democratization of the genomics revolution with a fully integrated web-based bioinformatics platform, Version 1.5 and 1.x.

EDGE bioinformatics was developed to help biologists process Next Generation Sequencing data (in the form of raw FASTQ files), even if they have little to no bioinformatics expertise. EDGE is a highly integrated and interactive web-based platform that is capable of running many of the standard analyses that biologists require for viral, bacterial/archaeal, and metagenomic samples. EDGE provides the following analytical workflows: quality trimming and host removal, assembly and annotation, comparisons against known references, taxonomy classification of reads and contigs, whole genome SNP-based phylogenetic analysis, and PCR analysis. EDGE provides an intuitive web-based interface for user input, allows users to visualize and interact with selected results (e.g. JBrowse genome browser), and generates a final detailed PDF report. Results in the form of tables, text files, graphic files, and PDFs can be downloaded. A user management system allows tracking of an individual’s EDGE runs, along with the ability to share, post publicly, delete, or archive their results.

Chain, Patrick↗

Enabling the democratization of the genomics revolution with a fully integrated web-based bioinformatics platform, Version 1.5 and 1.x.

EDGE bioinformatics was developed to help biologists process Next Generation Sequencing data (in the form of raw FASTQ files), even if they have little to no bioinformatics expertise. EDGE is a highly integrated and interactive web-based platform that is capable of running many of the standard analyses that biologists require for viral, bacterial/archaeal, and metagenomic samples. EDGE provides the following analytical workflows: quality trimming and host removal, assembly and annotation, comparisons against known references, taxonomy classification of reads and contigs, whole genome SNP-based phylogenetic analysis, and PCR analysis. EDGE provides an intuitive web-based interface for user input, allows users to visualize and interact with selected results (e.g. JBrowse genome browser), and generates a final detailed PDF report. Results in the form of tables, text files, graphic files, and PDFs can be downloaded. A user management system allows tracking of an individual’s EDGE runs, along with the ability to share, post publicly, delete, or archive their results.

Chain, Patrick↗

A Vision for Coupling Operation of US Fusion Facilities with HPC Systems and the Implications for Workflows and Data Management

The operation of large US Department of Energy (DOE) research facilities, like the DIII-D National Fusion Facility, results in the collection of complex multi-dimensional scientific datasets, both experimental and model-generated. In the future, it is envisioned that integrated data analysis coupled with large-scale high performance computing (HPC) simulations will be used to improve experimental planning and operation. Practically, massive data sets from these simulations provide the physics basis for generation of both reduced semi-analytic and machine-learning-based models. Storage of both HPC simulation datasets (generated from US DOE leadership computing facilities) and experimental datasets presents significant challenges. In this paper, we present a vision for a DOE-wide data management workflow that integrates US DOE fusion facilities with leadership computing facilities. Data persistence and long-term availability beyond the length of allocated projects is essential, particularly for verification and recalibration of artificial intelligence and machine learning (AI/ML) models. Because these data sets are often generated and shared among hundreds of users across multiple leadership computing facility centers, they would benefit from cross-platform accessibility, persistent identifiers (e.g. DOI, or digital object identifier), and provenance tracking. Here, the ability to handle different data access patterns suggests that a combination of low cost, high latency (e.g. for storing ML training sets) and high cost, low latency systems (e.g. for real-time, integrated machine control feedback) may be needed.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Phoenix: A Scalable Streaming Hypergraph Analysis Framework

We present Phoenix, a scalable hypergraph analytics framework for data analytics and knowledge discovery that was implemented on the leadership class computing platforms at Oak Ridge National Laboratory (ORNL). Our software framework comprises a distributed implementation of a streaming server architecture which acts as a gateway for various hypergraph generators/external sources to connect. Phoenix has the capability to utilize diverse hypergraph generators, including HyGen, a very large-scale hypergraph generator developed by ORNL. Phoenix incorporates specific algorithms for efficient data representation by exploiting hidden structures of the hypergraphs. Our experimental results demonstrate Phoenix’s scalable and stable performance on massively parallel computing platforms. Phoenix’s superior performance is due to the merging of high-performance computing with data analytic.

Kurte, Kuldeep↗

Evaluation of Use Cases and Types of Databases for Hosting Wind Power Data

The goal of the National Wind Power Production Data Dashboard project is to develop a publicly available platform to model, process, and share wind power with uncertainty quantification for the current and future onshore and offshore wind plants across the continental United States. As of 2021, more than 70,000 utility-scale wind turbines have been installed across the United States and this number is expected to grow in the next few years. A large-scale wind power production database is needed for stakeholders (policy makers, operators, and researchers) to access easily for their decision-making. Currently, meeting that need is a challenge. Most wind power operators and system operators focus on regions of their own interest, and the data are usually inaccessible to the public.

17 WIND ENERGY↗

A Guide to Using GitHub for Developing and Versioning Data Standards and Reporting Formats

Abstract Data standardization combined with descriptive metadata facilitate data reuse, which is the ultimate goal of the Findable, Accessible, Interoperable, and Reusable (FAIR) principles. Community data or metadata standards are increasingly created through an approach that emphasizes collaboration between various stakeholders. Such an approach requires platforms for collaboration on the development process that centers on sharing information and receiving feedback. Our objective in this study was to conduct a systematic review to identify data standards and reporting formats that use version control for developing data standards and to summarize common practices, particularly in earth and environmental sciences. Out of 108 data standards and reporting formats identified in our review, 32 used GitHub as the version control platform, and no other platforms were used. We found no universally accepted methodology for developing and publishing data standards. Many GitHub repositories did not use key features that could help developers to gather user feedback, or to create and revise standards that build on previous work. We provide guidance for community‐driven standard development and associated documentation on GitHub based on a systematic review of existing practices.

54 ENVIRONMENTAL SCIENCES↗

Millions of targets under attack: a macroscopic characterization of the DoS ecosystem

Denial-of-Service attacks have rapidly increased in terms of frequency and intensity, steadily becoming one of the biggest threats to Internet stability and reliability. However, a rigorous comprehensive characterization of this phenomenon, and of countermeasures to mitigate the associated risks, faces many infrastructure and analytic challenges. We make progress toward this goal, by introducing and applying a new framework to enable a macroscopic characterization of attacks, attack targets, and DDoS Protection Services (DPSs). Our analysis leverages data from four independent global Internet measurement infrastructures over the last two years: backscatter traffic to a large network telescope; logs from amplification honeypots; a DNS measurement platform covering 60% of the current namespace; and a DNS-based data set focusing on DPS adoption. Our results reveal the massive scale of the DoS problem, including an eye-opening statistic that one-third of all / 24 networks recently estimated to be active on the Internet have suffered at least one DoS attack over the last two years. We also discovered that often targets are simultaneously hit by different types of attacks. In our data, Web servers were the most prominent attack target; an average of 3% of the Web sites in .com, .net, and .org were involved with attacks, daily. Finally, we shed light on factors influencing migration to a DPS.

Jonker, Mattijs↗

Grain Boundary Plane Measurement Using Transmission Electron Microscopy Automated Crystallographic Orientation Mapping for Atom Probe Tomography Specimens

Abstract Grain boundaries are critical in determining the properties of materials, including mechanical stability, conductivity, and corrosion resistance. The specific properties of materials depend not only on the misorientation of the crystals, the three most commonly characterized parameters, but also on the angle of the grain boundary plane between the two crystals, the final two parameters in the five-parameter macroscopic description of the grain boundary. The method presented here allows for the direct measurement of all five parameters of the grain boundary in a transmission electron microscopy specimen of various morphologies. This is especially applicable to atom probe specimens, where only a single-tilt axis is generally available, allowing the crystallographic description to be matched to the detailed chemical data available in the atom probe tomography. This method provides a platform for efficient grain boundary analysis in unique samples, saving operator time and allowing for ease of acquisition and interpretation in comparison with traditional electron diffraction methods.

Materials Science↗

An Analysis of Abandoned Oil Well Characteristics Affecting Methane Emissions Estimates in the Cherokee Platform in Eastern Oklahoma

Anthropogenic activities increase methane emissions, contributing to greenhouse gas levels and adversely affecting the environment. Abandoned oil and gas wells potentially leak methane, but data are limited. We analyze methane emissions from abandoned wells (n = 179) in the Cherokee Platform in Oklahoma, a previously unaccounted basin, and compare emissions factors (EFs) to those in the Greenhouse Gas Inventory. We compare the contribution of various characteristics to the propensity for leakage. Higher emissions were observed with shallower wells and with unplugged wells. Plugged wells (n = 20) had an average EF of 96 ± 429 g/day and 65 ± 294 g/day for unplugged wells (n = 159). The majority of wells had no detectable leak. We calculated ethane EFs based on geochemical analysis of gas samples, finding higher EFs for unplugged (1.2 ± 5.5 g/day) versus plugged (0.9 ± 4.6 g/day) wells. The data indicate that in addition to the location of abandoned wells, physical characteristics are necessary to consider in estimating methane emissions.

58 GEOSCIENCES↗

Rahasak—Scalable blockchain architecture for enterprise applications

Blockchain-based decentralized infrastructure has been adapted in various industries to handle the sensitive data in a privacy-preserving manner without trusting third parties. However, integrating state-of-the-art blockchain platforms with the scalable, enterprise-level applications result in several challenges. Current blockchain platforms do not support high transaction throughput, lack high scalability, and cannot provide real-time transaction processing and back-pressure operation handling in high transaction throughput applications(e.g Big data, IoT). In this paper, we propose a novel permissioned blockchain platform “Rahasak” for highly scalable, enterprise applications. Rahasak blockchain adopts the Apache Kafka-based consensus on top of a “Validate-Execute-Group” blockchain architecture to handle realtime transaction execution on the blockchain. The architecture is equipped with a functional programming and actor-based smart contract platform that enables concurrent execution of transactions in the blockchain. Rahasak supports high transaction throughput, high scalability, concurrent transaction execution, data analytics features. Finally, with Rahasak, we make blockchain more scalable, secure, structured and meaningful for further data analytics.

97 MATHEMATICS AND COMPUTING↗

Application of a Machine Learning Algorithm in Generating an Evapotranspiration Data Product From Coupled Thermal Infrared and Microwave Satellite Observations

Land surface evapotranspiration (ET) is one of the main energy sources for atmospheric dynamics and a critical component of the local, regional, and global water cycles. Consequently, accurate measurement or estimation of ET is one of the most active topics in hydro-climatology research. With massive and spatially distributed observational data sets of land surface properties and environmental conditions being collected from the ground, airborne or space-borne platforms daily over the past few decades, many research teams have started to use big data science to advance the ET estimation methods. The Geostationary satellite Evapotranspiration and Drought (GET-D) product system was developed at the National Oceanic and Atmospheric Administration (NOAA) in 2016 to generate daily ET and drought maps operationally. The primary inputs of the current GET-D system are the thermal infrared (TIR) observations from NOAA GOES satellite series. Because of the cloud contamination to the TIR observations, the spatial coverage of the daily GET-D ET product has been severely impacted. Based on the most recent advances, we have tested a machine learning algorithm to estimate all-weather land surface temperature (LST) from TIR and microwave (MW) combined satellite observations. With the regression tree machine learning approach, we can combine the high accuracy and high spatial resolution of GOES TIR data with the better spatial coverage of passive microwave observations and LST simulations from a land surface model (LSM). The regression tree model combines the three LST data sources for both clear and cloudy days, which enables the GET-D system to derive an all-weather ET product. This paper reports how the all-weather LST and ET are generated in the upgraded GET-D system and provides an evaluation of these LST and ET estimates with ground measurements. The results demonstrate that the regression tree machine learning method is feasible and effective for generating daily ET under all weather conditions with satisfactory accuracy from the big volume of satellite observations.

54 ENVIRONMENTAL SCIENCES↗

Multi-Reactor Design and Analysis Platform

MRDAP is designed to simplify the creation, transfer and processing of data between computational codes. MRDAP accomplishes these objectives with three parts: First it allows each integrated code, through a plug-in interface, to specify the required input for execution and the required output needed. Second it creates an interface for execution and data transfer. The code provides a Graphical User Interface (GUI) to assist with input preparation and data visualization. This abstract is for the core software and the plug-in interfaces. This abstract does not include the software used by the plug-in interfaces (such as MCNP), which is distributed and licensed separately.

Cogliati, JoshuaJ.↗

Drone flight visible images, digital elevation maps, and geotiffs, Council, Seward Peninsula, Alaska, USA, July 2017

Remote sensing data collected from Lawrence Berkeley National Laboratory’s (LBNL) unmanned aerial system (UAS) quadcopter platform – Inspire-1 – operated by the Ameriflux Management Project (ameriflux.lbl.gov) Technical Team. The Insprire-1 is a visible imagery UAS platform that simultaneously measures very high spatial resolution optical red/green/blue (RGB) at regular intervals along each flight path. This package includes data from four flight paths over the NGEE Arctic Council Road Site near mile marker 71 in the Seward Peninsula, Alaska in July 2017. Derived image products for each flight include ortho-mosaiced RGB, an RGB-based digital surface model (DSM) using the structure from motion (SfM) technique, and a digital terrain model (DTM). Ancillary aircraft data, flight mission parameters, and general flight conditions are also included. Unprocessed and processed data products are included in this package (processing levels 0-2). This metadata document contains flight campaign, instrument and file metadata, along with a description of data processing levels, data products and file naming scheme.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic) was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy’s Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy’s Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

Drone flight visible images, digital elevation maps, and geotiffs, Teller, Seward Peninsula, Alaska, USA, July 2017

Remote sensing data collected from Lawrence Berkeley National Laboratory’s (LBNL) unmanned aerial system (UAS) quadcopter platform – Inspire-1 – operated by the Ameriflux Management Project (ameriflux.lbl.gov) Technical Team. The Insprire-1 is a visible imagery UAS platform that simultaneously measures very high spatial resolution optical red/green/blue (RGB) at regular intervals along the flight path. This package includes data from a flight path over the NGEE Arctic Teller Site, west of mile marker 27 Bob Blodgett Highway in the Seward Peninsula, Alaska in July 2017. Derived image products include ortho-mosaiced RGB, an RGB-based digital surface model (DSM) using the structure from motion (SfM) technique, and a digital terrain model (DTM). Ancillary aircraft data, flight mission parameters, and general flight conditions are also included. Unprocessed and processed data products are included in this package (processing levels 0-2). This metadata document contains flight campaign, instrument and file metadata, along with a description of data processing levels, data products and file naming scheme.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic) was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy’s Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy’s Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

Drone flight visible images, digital elevation maps, and geotiffs, Kougarok, Seward Peninsula, Alaska, USA, July 2017

Remote sensing data collected from Lawrence Berkeley National Laboratory’s (LBNL) unmanned aerial system (UAS) quadcopter platform – Inspire-1 – operated by the Ameriflux Management Project (ameriflux.lbl.gov) Technical Team. The Insprire-1 is a visible imagery UAS platform that simultaneously measures very high spatial resolution optical red/green/blue (RGB) at regular intervals along each flight path. This package includes data from four flight paths over the NGEE Arctic Kougarok Site, west of mile marker 64 on Kuzitrin Road in the Seward Peninsula, Alaska in July 2017. Derived image products include ortho-mosaiced RGB, an RGB-based digital surface model (DSM) using the structure from motion (SfM) technique, and a digital terrain model (DTM). Ancillary aircraft data, flight mission parameters, and general flight conditions are also included. Unprocessed and processed data products are included in this package (processing levels 0-2). This metadata document contains flight campaign, instrument and file metadata, along with a description of data processing levels, data products and file naming scheme.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic) was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy’s Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy’s Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗