Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “big data visualization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

Knowledge Discovery for Smart Grid Operation, Control, and Situation Awareness -- A Big Data Visualization Platform

In this paper, a big data visualization platform is designed to discover the hidden useful knowledge for smart grid (SG) operation, control and situation awareness. The spawn of smart sensors at both grid side and customer side can provide large volume of heterogeneous data that collect information in all time spectrums. Extracting useful knowledge from this big-data poll is still challenging. In this paper, the Apache Spark, an open source cluster computing framework, is used to process the big-data to effectively discover the hidden knowledge. A high-speed communication architecture utilizing the Open System Interconnection (OSI) model is designed to transmit the data to a visualization platform. This visualization platform uses Google Earth, a global geographic information system (GIS) to link the geological information with the SG knowledge and visualize the information in user defined fashion. The University of Denver's campus grid is used as a SG test bench and several demonstrations are presented for the proposed platform.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2013

Big Data Visual Analytics for Exploratory Earth System Simulation Analysis

Rapid increases in high performance computing are feeding the development of larger and more complex data sets in climate research, which sets the stage for so-called big data analysis challenges. However, conventional climate analysis techniques are inadequate in dealing with the complexities of today s data. In this paper, we describe and demonstrate a visual analytics system, called the Exploratory Data analysis ENvironment (EDEN), with specific application to the analysis of complex earth system simulation data sets. EDEN represents the type of interactive visual analysis tools that are necessary to transform data into insight, thereby improving critical comprehension of earth system processes. In addition to providing an overview of EDEN, we describe real-world studies using both point ensembles and global Community Land Model Version 4 (CLM4) simulations.

big data↗
NASA NTRS2017

Big lunar data visualization and analysis

NASA's earth and planetary spacecraft return large amounts of remote sensing data, such as imagery and raw science measurements, in support of remarkable research. Not only does the data lead to new scientific discoveries about our planet and the solar system, it provides a wealth of information to educate, inspire, and engage the public at large. To leverage this rich data for mission planning, scientific research, public outreach and education, it is essential to make it accessible and understandable, analyzable, all while appealing to their interests. This presentation will highlight web-based capabilities that showcase NASA's large volume of lunar data collected from past and current Moon missions. It is particularly relevant as the new Administration has more plans for the Moon. We will illustrate big data visualization and analysis in easy-touse and interactive mediums for diverse use.

Malhotra, Shan↗
DOE OSTI2012

VisIt: An End-User Tool for Visualizing and Analyzing Very Large Data

VisIt is a popular open source tool for visualizing and analyzing big data. It owes its success to its foci of increasing data understanding, large data support, and providing a robust and usable product, as well as its underlying design that fits today's supercomputing landscape. This report, which draws heavily from an earlier publication at the SciDAC Conference in 2011 describes the VisIt project and its accomplishments.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Web-based visual data exploration for improved radiological source detection

Radiation detection can provide a reliable means of detecting radiological material. Such capabilities can help to prevent nuclear and/or radiological attacks, but reliable detection in uncontrolled surroundings requires algorithms that account for environmental background radiation. The Berkeley Data Cloud (BDC) facilitates the development of such methods by providing a framework to capture, store, analyze, and share data sets. In the era of big data, both the size and variety of data make it difficult to explore and find data sets of interest and manage the data. Thus, in the context of big data, visualization is critical for checking data consistency and validity, identifying gaps in data coverage, searching for data relevant to an analyst's use cases, and choosing input parameters for analysis. Downloading the data and exploring it on an analyst's desktop using traditional tools are no longer feasible due to the size of the data. This study describes the design and implementation of a visualization system that addresses the problems associated with data exploration within the context of the BDC. Finally, the visualization system is based on a JavaScript front end communicating via REST with a back end web server.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
NASA NTRS2016

IN13B-1660: Analytics and Visualization Pipelines for Big Data on the NASA Earth Exchange (NEX) and OpenNEX

We are developing capabilities for an integrated petabyte-scale Earth science collaborative analysis and visualization environment. The ultimate goal is to deploy this environment within the NASA Earth Exchange (NEX) and OpenNEX in order to enhance existing science data production pipelines in both high-performance computing (HPC) and cloud environments. Bridging of HPC and cloud is a fairly new concept under active research and this system significantly enhances the ability of the scientific community to accelerate analysis and visualization of Earth science data from NASA missions, model outputs and other sources. We have developed a web-based system that seamlessly interfaces with both high-performance computing (HPC) and cloud environments, providing tools that enable science teams to develop and deploy large-scale analysis, visualization and QA pipelines of both the production process and the data products, and enable sharing results with the community. Our project is developed in several stages each addressing separate challenge - workflow integration, parallel execution in either cloud or HPC environments and big-data analytics or visualization. This work benefits a number of existing and upcoming projects supported by NEX, such as the Web Enabled Landsat Data (WELD), where we are developing a new QA pipeline for the 25PB system.

visualization↗
DOE OSTI2014

Genomic Island Identification Software v 1.0

Genomic islands are key mobile DNA elements in bacterial evolution, that can distinguish pathogenic strains from each other, or distinguish pathogenic strains from non-pathogenic strains. Their detection in genomes is a challenging problem. We present 3 main software components that attack the island detection problem on two different bases: 1) the preference of islands to insert in chromosomal tRNA or tmRNA genes (islander.pl), and 2) islands' sporadic occurrence among closely related strains. The latter principle is employed in both an algorithm (learnedPhyloblocks.pl) and a visualization method (panGenome.pl). Component islander.pl finds islands based on their preference for a particular target gene type. We annotate each tRNA and tmRNA gene, find fragments of each such gene as candidates for the distal ends of islands, and filter candidates to remove false positives. Component learnedPhyloblocks.pl uses islands found by islander.pl and other methods as a training set to find new islands. Reference genomes are aligned using mugsy, then the "phylotypes" or patterns of occurrence in the reference set are determined for each position in the target genome, and those phylotypes most enriched in the training set of islands are followed to detect yet more islands. Component panGenome.pl produces a big-data visualization of the chromosomally-ordered "pan-genome", that includes every gene of every reference genome (x-axis, pan-genome order; y-axis, reference genomes; color-coding, gene presence/absence etc.), islands appearing as dark patches.

Williams, KellyPorter↗
DOE OSTI2014

Genomic Island Identification Software v 1.0

Genomic islands are key mobile DNA elements in bacterial evolution, that can distinguish pathogenic strains from each other, or distinguish pathogenic strains from non-pathogenic strains. Their detection in genomes is a challenging problem. We present 3 main software components that attack the island detection problem on two different bases: 1) the preference of islands to insert in chromosomal tRNA or tmRNA genes (islander.pl), and 2) islands' sporadic occurrence among closely related strains. The latter principle is employed in both an algorithm (learnedPhyloblocks.pl) and a visualization method (panGenome.pl). Component islander.pl finds islands based on their preference for a particular target gene type. We annotate each tRNA and tmRNA gene, find fragments of each such gene as candidates for the distal ends of islands, and filter candidates to remove false positives. Component learnedPhyloblocks.pl uses islands found by islander.pl and other methods as a training set to find new islands. Reference genomes are aligned using mugsy, then the "phylotypes" or patterns of occurrence in the reference set are determined for each position in the target genome, and those phylotypes most enriched in the training set of islands are followed to detect yet more islands. Component panGenome.pl produces a big-data visualization of the chromosomally-ordered "pan-genome", that includes every gene of every reference genome (x-axis, pan-genome order; y-axis, reference genomes; color-coding, gene presence/absence etc.), islands appearing as dark patches.

Source record↗
DOE OSTI2023

Scalable Volume Visualization for Big Scientific Data Modeled by Functional Approximation

Considering the challenges posed by the space and time complexities in handling extensive scientific volumetric data, various data representations have been developed for the analysis of large-scale scientific data. Multivariate functional approximation (MFA) is an innovative data model designed to tackle substantial challenges in scientific data analysis. It computes values and derivatives with high-order accuracy throughout the spatial domain, mitigating artifacts associated with zero- or first-order interpolation. However, the slow query time through MFA makes it less suitable for interactively visualizing a large MFA model. In this work, we develop the first scalable interactive volume visualization pipeline, MFA-DVV, for the MFA model encoded from large-scale datasets. Our method achieves low input latency through distributed architecture, and its performance can be further enhanced by utilizing a compressed MFA model while still maintaining a high-quality rendering result for scientific datasets. We conduct comprehensive experiments to show that MFA-DVV can decrease the input latency and achieve superior visualization results for big scientific data compared with existing approaches.

big scientific dataset↗
DOE OSTI2013

Oak Ridge Bio-surveillance Toolkit (ORBiT): Integrating Big-Data Analytics with Visual Analysis for Public Health Dynamics

In this position paper, we describe the design and implementation of the Oak Ridge Bio-surveillance Toolkit (ORBiT): a collection of novel statistical and machine learning tools implemented for (1) integrating heterogeneous traditional (e.g. emergency room visits, prescription sales data, etc.) and non-traditional (social media such as Twitter and Instagram) data sources, (2) analyzing large-scale datasets and (3) presenting the results from the analytics as a visual interface for the end-user to interact and provide feedback. We present examples of how ORBiT can be used to summarize ex- tremely large-scale datasets effectively and how user interactions can translate into the data analytics process for bio-surveillance. We also present a strategy to estimate parameters relevant to dis- ease spread models from near real time data feeds and show how these estimates can be integrated with disease spread models for large-scale populations. We conclude with a perspective on how integrating data and visual analytics could lead to better forecasting and prediction of disease spread as well as improved awareness of disease susceptible regions.

Ramanathan, Arvind↗
DOE OSTI2012

Big Data Analaysis and Visualization: What Do LINACS and Tropical Storms Have In Common?

While there is wisdom in the old adage "the two constants in life are death and taxes,'' there are unavoidable truths facing modern experimental and computational science. First is the growing "impedance mismatch'' between our ability to collect and generate data, and our ability to store, manage, and gain understanding from it. The second is the fact that we cannot continue to rely on the same software technologies that have worked well for the past couple of decades for data management, analysis, and visualization. A third is that these complementary activities must be considered in a holistic, rather than Balkanized way. The inseparable interplay between data management, analysis, visualization, and high performance computational infrastructure, are best viewed through the lens of case studies from multiple scientific domains, where teams of computer and scientists combine forces to tackle challenging data understanding problems.

97 MATHEMATICS AND COMPUTING↗