Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “big data visualization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

Knowledge Discovery for Smart Grid Operation, Control, and Situation Awareness -- A Big Data Visualization Platform

In this paper, a big data visualization platform is designed to discover the hidden useful knowledge for smart grid (SG) operation, control and situation awareness. The spawn of smart sensors at both grid side and customer side can provide large volume of heterogeneous data that collect information in all time spectrums. Extracting useful knowledge from this big-data poll is still challenging. In this paper, the Apache Spark, an open source cluster computing framework, is used to process the big-data to effectively discover the hidden knowledge. A high-speed communication architecture utilizing the Open System Interconnection (OSI) model is designed to transmit the data to a visualization platform. This visualization platform uses Google Earth, a global geographic information system (GIS) to link the geological information with the SG knowledge and visualize the information in user defined fashion. The University of Denver's campus grid is used as a SG test bench and several demonstrations are presented for the proposed platform.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2013

Big Data Visual Analytics for Exploratory Earth System Simulation Analysis

Rapid increases in high performance computing are feeding the development of larger and more complex data sets in climate research, which sets the stage for so-called big data analysis challenges. However, conventional climate analysis techniques are inadequate in dealing with the complexities of today s data. In this paper, we describe and demonstrate a visual analytics system, called the Exploratory Data analysis ENvironment (EDEN), with specific application to the analysis of complex earth system simulation data sets. EDEN represents the type of interactive visual analysis tools that are necessary to transform data into insight, thereby improving critical comprehension of earth system processes. In addition to providing an overview of EDEN, we describe real-world studies using both point ensembles and global Community Land Model Version 4 (CLM4) simulations.

big data↗
DOE OSTI2012

VisIt: An End-User Tool for Visualizing and Analyzing Very Large Data

VisIt is a popular open source tool for visualizing and analyzing big data. It owes its success to its foci of increasing data understanding, large data support, and providing a robust and usable product, as well as its underlying design that fits today's supercomputing landscape. This report, which draws heavily from an earlier publication at the SciDAC Conference in 2011 describes the VisIt project and its accomplishments.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Web-based visual data exploration for improved radiological source detection

Radiation detection can provide a reliable means of detecting radiological material. Such capabilities can help to prevent nuclear and/or radiological attacks, but reliable detection in uncontrolled surroundings requires algorithms that account for environmental background radiation. The Berkeley Data Cloud (BDC) facilitates the development of such methods by providing a framework to capture, store, analyze, and share data sets. In the era of big data, both the size and variety of data make it difficult to explore and find data sets of interest and manage the data. Thus, in the context of big data, visualization is critical for checking data consistency and validity, identifying gaps in data coverage, searching for data relevant to an analyst's use cases, and choosing input parameters for analysis. Downloading the data and exploring it on an analyst's desktop using traditional tools are no longer feasible due to the size of the data. This study describes the design and implementation of a visualization system that addresses the problems associated with data exploration within the context of the BDC. Finally, the visualization system is based on a JavaScript front end communicating via REST with a back end web server.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2014

Genomic Island Identification Software v 1.0

Genomic islands are key mobile DNA elements in bacterial evolution, that can distinguish pathogenic strains from each other, or distinguish pathogenic strains from non-pathogenic strains. Their detection in genomes is a challenging problem. We present 3 main software components that attack the island detection problem on two different bases: 1) the preference of islands to insert in chromosomal tRNA or tmRNA genes (islander.pl), and 2) islands' sporadic occurrence among closely related strains. The latter principle is employed in both an algorithm (learnedPhyloblocks.pl) and a visualization method (panGenome.pl). Component islander.pl finds islands based on their preference for a particular target gene type. We annotate each tRNA and tmRNA gene, find fragments of each such gene as candidates for the distal ends of islands, and filter candidates to remove false positives. Component learnedPhyloblocks.pl uses islands found by islander.pl and other methods as a training set to find new islands. Reference genomes are aligned using mugsy, then the "phylotypes" or patterns of occurrence in the reference set are determined for each position in the target genome, and those phylotypes most enriched in the training set of islands are followed to detect yet more islands. Component panGenome.pl produces a big-data visualization of the chromosomally-ordered "pan-genome", that includes every gene of every reference genome (x-axis, pan-genome order; y-axis, reference genomes; color-coding, gene presence/absence etc.), islands appearing as dark patches.

Williams, KellyPorter↗
DOE OSTI2014

Genomic Island Identification Software v 1.0

Genomic islands are key mobile DNA elements in bacterial evolution, that can distinguish pathogenic strains from each other, or distinguish pathogenic strains from non-pathogenic strains. Their detection in genomes is a challenging problem. We present 3 main software components that attack the island detection problem on two different bases: 1) the preference of islands to insert in chromosomal tRNA or tmRNA genes (islander.pl), and 2) islands' sporadic occurrence among closely related strains. The latter principle is employed in both an algorithm (learnedPhyloblocks.pl) and a visualization method (panGenome.pl). Component islander.pl finds islands based on their preference for a particular target gene type. We annotate each tRNA and tmRNA gene, find fragments of each such gene as candidates for the distal ends of islands, and filter candidates to remove false positives. Component learnedPhyloblocks.pl uses islands found by islander.pl and other methods as a training set to find new islands. Reference genomes are aligned using mugsy, then the "phylotypes" or patterns of occurrence in the reference set are determined for each position in the target genome, and those phylotypes most enriched in the training set of islands are followed to detect yet more islands. Component panGenome.pl produces a big-data visualization of the chromosomally-ordered "pan-genome", that includes every gene of every reference genome (x-axis, pan-genome order; y-axis, reference genomes; color-coding, gene presence/absence etc.), islands appearing as dark patches.

Source record↗
DOE OSTI2023

Scalable Volume Visualization for Big Scientific Data Modeled by Functional Approximation

Considering the challenges posed by the space and time complexities in handling extensive scientific volumetric data, various data representations have been developed for the analysis of large-scale scientific data. Multivariate functional approximation (MFA) is an innovative data model designed to tackle substantial challenges in scientific data analysis. It computes values and derivatives with high-order accuracy throughout the spatial domain, mitigating artifacts associated with zero- or first-order interpolation. However, the slow query time through MFA makes it less suitable for interactively visualizing a large MFA model. In this work, we develop the first scalable interactive volume visualization pipeline, MFA-DVV, for the MFA model encoded from large-scale datasets. Our method achieves low input latency through distributed architecture, and its performance can be further enhanced by utilizing a compressed MFA model while still maintaining a high-quality rendering result for scientific datasets. We conduct comprehensive experiments to show that MFA-DVV can decrease the input latency and achieve superior visualization results for big scientific data compared with existing approaches.

big scientific dataset↗
DOE OSTI2013

Oak Ridge Bio-surveillance Toolkit (ORBiT): Integrating Big-Data Analytics with Visual Analysis for Public Health Dynamics

In this position paper, we describe the design and implementation of the Oak Ridge Bio-surveillance Toolkit (ORBiT): a collection of novel statistical and machine learning tools implemented for (1) integrating heterogeneous traditional (e.g. emergency room visits, prescription sales data, etc.) and non-traditional (social media such as Twitter and Instagram) data sources, (2) analyzing large-scale datasets and (3) presenting the results from the analytics as a visual interface for the end-user to interact and provide feedback. We present examples of how ORBiT can be used to summarize ex- tremely large-scale datasets effectively and how user interactions can translate into the data analytics process for bio-surveillance. We also present a strategy to estimate parameters relevant to dis- ease spread models from near real time data feeds and show how these estimates can be integrated with disease spread models for large-scale populations. We conclude with a perspective on how integrating data and visual analytics could lead to better forecasting and prediction of disease spread as well as improved awareness of disease susceptible regions.

Ramanathan, Arvind↗
DOE OSTI2012

Big Data Analaysis and Visualization: What Do LINACS and Tropical Storms Have In Common?

While there is wisdom in the old adage "the two constants in life are death and taxes,'' there are unavoidable truths facing modern experimental and computational science. First is the growing "impedance mismatch'' between our ability to collect and generate data, and our ability to store, manage, and gain understanding from it. The second is the fact that we cannot continue to rely on the same software technologies that have worked well for the past couple of decades for data management, analysis, and visualization. A third is that these complementary activities must be considered in a holistic, rather than Balkanized way. The inseparable interplay between data management, analysis, visualization, and high performance computational infrastructure, are best viewed through the lens of case studies from multiple scientific domains, where teams of computer and scientists combine forces to tackle challenging data understanding problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2013

Feature Tracking and Visualization of Madden-Julian Osciallation in Climate Simulation

Madden-Julian Oscillation (MJO) is one of the less understood aspects of tropical meteorology, which plays a significant role in tropical intra-seasonal variations in rain, temperature and winds over the Indian and Pacific Oceans. In this paper, we present an integrated analysis and visualization framework for MJO episodes simulated by a high resolution regional model. To distinguish MJOs from other weather phenomena, our framework utilizes domain knowledge to track MJOs as finding the globally optimized properties in the data. In addition to enhancing the animation with feature tracking, our visualization system also integrates different visualization components such as Virtual Globe and Hovmoller Diagrams to visualize large scale events both in space and time. By linking all of these visualization components on a web-based interface, scientists can identify cloud and environmental processes associated with the initiation and eastward propagation of MJO more easily.

big-data visualization↗
DOE OSTI2017

Characterizing Time Series Data Diversity for Wind Forecasting

Wind forecasting plays an important role in integrating variable and uncertain wind power into the power grid. Various forecasting models have been developed to improve the forecasting accuracy. However, it is challenging to accurately compare the true forecasting performances from different methods and forecasters due to the lack of diversity in forecasting test datasets. This paper proposes a time series characteristic analysis approach to visualize and quantify wind time series diversity. The developed method first calculates six time series characteristic indices from various perspectives. Then the principal component analysis is performed to reduce the data dimension while preserving the important information. The diversity of the time series dataset is visualized by the geometric distribution of the newly constructed principal component space. The volume of the 3-dimensional (3D) convex polytope (or the length of 1D number axis, or the area of the 2D convex polygon) is used to quantify the time series data diversity. The method is tested with five datasets with various degrees of diversity.

big data visualization↗
DOE OSTI2018

Characterizing Time Series Data Diversity for Wind Forecasting: Preprint

Wind forecasting plays an important role in integrating variable and uncertain wind power into the power grid. Various forecasting models have been developed to improve the forecasting accuracy. However, it is challenging to accurately compare the true forecasting performances from different methods and forecasters due to the lack of diversity in forecasting test datasets. This paper proposes a time series characteristic analysis approach to visualize and quantify wind time series diversity. The developed method first calculates six time series characteristic indices from various perspectives. Then the principal component analysis is performed to reduce the data dimension while preserving the important information. The diversity of the time series dataset is visualized by the geometric distribution of the newly constructed principal component space. The volume of the 3-dimensional (3D) convex polytope (or the length of 1D number axis, or the area of the 2D convex polygon) is used to quantify the time series data diversity. The method is tested with five datasets with various degrees of diversity.

17 WIND ENERGY↗