Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Visualization and Analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

Scalable Computation of Topological Abstractions for Scalar Data

Topological data analysis has become an important tool for large scale scalar data analysis and visualization, efficiently extracting the inherent structure and features of interest of the data. However, with growing dataset sizes and complexity, it is increasingly becoming infeasible to compute topological abstractions of interest in serial and on single machines. This paper presents the state of the art in the scalable computation of topological abstractions on scalar data, in shared memory parallel on single machines, and in distributed memory parallel on multiple machines. We highlight results for set‐based, graph‐based and complex‐based abstractions and organize the state of the art based on this taxonomy. The paper identifies parallelization and distribution techniques common in topological algorithms and highlights further areas of interest with underdeveloped efforts.

97 MATHEMATICS AND COMPUTING↗

A General Framework for Progressive Data Compression and Retrieval

In scientific simulations, observations, and experiments, the transfer of data to and from disk and across networks has become a major bottleneck for data analysis and visualization. Compression techniques have been employed to tackle this challenge, but traditional lossy methods often demand conservative error tolerances to meet the numerical accuracy requirements of both anticipated and unknown data analysis tasks. Progressive data compression and retrieval has emerged as a promising solution, where each analysis task dictates its own accuracy needs. However, few analysis algorithms inherently support progressive data processing, and adapting compression techniques, file formats, client/server frameworks, and APIs to support progressivity can be challenging. Here, this paper presents a framework that enables progressive-precision data queries for any data compressor or numerical representation. Our strategy hinges on a multi-component representation that successively reduces the error between the original and compressed field, allowing each field in the progressive sequence to be expressed as a partial sum of components. We have implemented this approach with four established scientific data compressors and assessed its effectiveness using real-world data sets from the SDRBench collection. The results show that our framework competes in accuracy with the standalone compressors it is based upon. Additionally, (de)compression time is proportional to the number of components requested by the user. Finally, our framework allows for fully lossless compression using lossy compressors when a sufficient number of components are employed.

97 MATHEMATICS AND COMPUTING↗

Parallel visualization and analysis with paraview on a Cray XT4

Scientific data sets produced by modern supercomputers like ORNL's Cray XT 4, Jaguar, can be extremely large, making visualization and analysis more difficult as moving large resultant data to dedicated analysis systems can be prohibitively expensive. We share our continuing work of integrating a parallel visualization system, ParaView, on ORNL's Jaguar system and our efforts to enable extreme scale interactive data visualization and analysis. We will discuss porting challenges and present performance numbers. Jaguar is a Cray XT4 with 7,832 compute nodes each with 4 cores and 8GB of memory, located in Oak Ridge, Tennessee. Jaguar is currently ranked No.8 in the Top 500. Jaguar, and other machines like it, enable increasingly larger data sets. Whether scientists are producing extremely massive data or small data they still have a requirement for analysis. Analysis is often difficult for larger data and it is further complicated when the user is distant to the data. Data sets that are too large to ship and/or analyze on a single desktop require a parallel analysis tool and a high performance computer to run on. Running ParaView interactively on Jaguar is a positive improvement for both local and remote users. We have shown that it is feasible for our user at LANL to visualize his simulation data created on jaguar. Local users may face fewer obstacles and get improved performance, due to their higher bandwidth to Jaguar. For future work, we would like to see higher priority queues that enable interactive visualization. A more interactive user-friendly qsub that estimates time to allocation would be very useful. Hopefully, our work will lead to more users attempting interactive visualization on Jaguar, resulting in quicker turnaround times for visualization and analysis. As users migrate to using supercomputing platforms for visualization, we expect future research possibilities in this area.

00↗

A browser-based tool for visualization and analysis of diffusion MRI data

Human neuroscience research faces several challenges with regards to reproducibility. While scientists are generally aware that data sharing is important, it is not always clear how to share data in a manner that allows other labs to understand and reproduce published findings. Here we report a new open source tool, AFQ-Browser, that builds an interactive website as a companion to a diffusion MRI study. Because AFQ-Browser is portable—it runs in any web-browser—it can facilitate transparency and data sharing. Moreover, by leveraging new web-visualization technologies to create linked views between different dimensions of the dataset (anatomy, diffusion metrics, subject metadata), AFQ-Browser facilitates exploratory data analysis, fueling new discoveries based on previously published datasets. In an era where Big Data is playing an increasingly prominent role in scientific discovery, so will browser-based tools for exploring high-dimensional datasets, communicating scientific discoveries, aggregating data across labs, and publishing data alongside manuscripts.

60 APPLIED LIFE SCIENCES↗

Profiling of Histone Post-Translational Modifications in Mouse Brain with High-Resolution Top-Down Mass Spectrometry

Histones play central roles in most chromosomal functions and both their basic biology and roles in disease have been the subject of intense study. Since multiple PTMs along the entire protein sequence are potential regulators of histones, a top-down approach, where intact proteins are analyzed, is ultimately required for complete characterization of proteoforms. However, significant challenges remain for top-down histone analysis primarily because of deficiencies in separation/resolving power and effective identification algorithms. Here, we used state of the art mass spectrometry and a bioinformatics workflow for targeted data analysis and visualization. The workflow uses ProMex for intact mass deconvolution, MSPathFinder as search engine, and LcMsSpectator as a data visualization tool. ProMex sums across retention time to maximize sensitivity and accuracy for low abundance species in MS1deconvolution. MSPathFinder searches the MS2 data against protein sequence databases with user-defined modifications. LcMsSpectator presents the results from ProMex and MSPathFinder in a format that allows quick manual evaluation of critical attributes for high-confidence identifications. When complemented with the open-modification tool TopPIC, this workflow enabled identification of novel histone PTMs including tyrosine bromination on histone H4 and H2A, H3 glutathionylation, and mapping of conventional PTMs along the entire protein for many histone subunits.

Environmental Molecular Sciences Laboratory↗

Interactive Exploration and Analysis of Large-Scale Simulations Using Topology-Based Data Segmentation

Large-scale simulations are increasingly being used to study complex scientific and engineering phenomena. As a result, advanced visualization and data analysis are also becoming an integral part of the scientific process. Often, a key step in extracting insight from these large simulations involves the definition, extraction, and evaluation of features in the space and time coordinates of the solution. However, in many applications, these features involve a range of parameters and decisions that will affect the quality and direction of the analysis. Examples include particular level sets of a specific scalar field, or local inequalities between derived quantities. A critical step in the analysis is to understand how these arbitrary parameters/decisions impact the statistical properties of the features, since such a characterization will help to evaluate the conclusions of the analysis as a whole. We present a new topological framework that in a single-pass extracts and encodes entire families of possible features definitions as well as their statistical properties. For each time step we construct a hierarchical merge tree a highly compact, yet flexible feature representation. While this data structure is more than two orders of magnitude smaller than the raw simulation data it allows us to extract a set of features for any given parameter selection in a postprocessing step. Furthermore, we augment the trees with additional attributes making it possible to gather a large number of useful global, local, as well as conditional statistic that would otherwise be extremely difficult to compile. We also use this representation to create tracking graphs that describe the temporal evolution of the features over time. Our system provides a linked-view interface to explore the time-evolution of the graph interactively alongside the segmentation, thus making it possible to perform extensive data analysis in a very efficient manner. We demonstrate our framework by extracting and analyzing burning cells from a large-scale turbulent combustion simulation. In particular, we show how the statistical analysis enabled by our techniques provides new insight into the combustion process.

97 MATHEMATICS AND COMPUTING↗

Integrated Genome-Based Studies of Shewanella Echophysiology (Final Scientific/Technical Report)

Shewanella oneidensis MR-1 is a motile, facultative γ-Proteobacterium with remarkable respiratory versatility; it can utilize a range of organic and inorganic compounds as terminal electronacceptors for anaerobic metabolism. The ability to effectively reduce nitrate, S0, polyvalent metals andradionuclides has established MR-1 as an important model dissimilatory metal-reducing microorganism for genome-based investigations of biogeochemical transformation of metals and radionuclides that are of concern to the U.S. Department of Energy (DOE) sites nationwide. Metal-reducing bacteria such as Shewanella also have a highly developed capacity for extracellular transfer of respiratory electrons to solid phase Fe and Mn oxides as well as directly to anode surfaces in microbial fuel cells. More broadly, Shewanellae are recognized free-living microorganisms and members of microbial communities involved in the decomposition of organic matter and the cycling of elements in aquatic and sedimentary systems. To function and compete in environments that are subject to spatial and temporal environmental change, Shewanella must be able to sense and respond to such changes and therefore require relatively robust sensing and regulation systems. The overall goal of this project is to apply the tools of genomics, leveraging the availability of genome sequence for 18 additional strains of Shewanella , to better understand the ecophysiology and speciation of respiratory-versatile members of this important genus. To understand these systems we propose to use genome-based approaches to investigate Shewanella as a system of integrated networks; first describing key cellular subsystems - those involved in signal transduction, regulation, and metabolism - then building towards understanding the function of whole cells and, eventually, cells within populations. As a general approach, this project will employ complimentary "top-down" - bioinformatics-based genome functional predictions, high-throughput expression analyses, and functional genomics approaches to uncover key genes as well as metabolic and regulatory networks. The "bottom-up" component employs more traditional approaches including genetics, physiology and biochemistry to test or verify predictions. This information will ultimately be linked to analyses of signal transduction and transcriptional regulatory systems and used to develop a linked model that will contribute to understanding the ecophysiology of Shewanella in redox stratified environments. A central component of this effort is the development of a data and knowledge integration environment that will allow investigators to query across the individual research domains, link to analysis applications, visualize data in a cell systems context, and produce new knowledge, while minimizing the effort, time and complexity to participating institutions.

59 BASIC BIOLOGICAL SCIENCES↗

Transforming Energy Through Computational Excellence: A View From NREL

At the National Renewable Energy Laboratory (NREL)—a U.S. Department of Energy laboratory—computational science, high-performance computing, applied mathematics, advanced computer science, visualization, and data play a pivotal role in advancing energy abundance, affordability, security, and reliability. From fundamental scientifc discovery to systems engineering and analysis, NREL researchers tackle market-relevant challenges to develop solutions for an independent energy system that is reliable, resilient and secure. Collaborative partnerships with industry, government, and academia ensure that our research remains cutting edge, impactful, applicable, and aligned with real-world energy needs. This special issue of Computing in Science & Engineering highlights exemplary NREL projects where computational tools and methodologies drive discovery and accelerate innovation in scalable and integrated energy systems. The featured articles explore the role of computational modeling, high-performance computing, generative AI, and adaptive computing in advancing independent energy solutions, optimizing sustainability research, and enhancing decision-making for energy solutions using a broad mix of energy technologies. Here, these contributions demonstrate how NREL’s computational research bridges the gap between theoretical advancements and practical implementation, emphasizing interdisciplinary collaboration and a commitment to innovation, with a focus on translating computational excellence into real-world impact, thus accelerate progress toward national energy goals. By showcasing cutting-edge research at the intersection of computational science and energy systems, this issue aims to inspire and inform researchers, practitioners, and policymakers dedicated to shaping a more reliable energy future.

97 MATHEMATICS AND COMPUTING↗

PIPER: Performance Insights for Programmers and Exascale Runtimes

During the lifetime of the project, the PIPER at the University of Utah worked in collaboration with the team from the partner institutions to establish a set of software tools for analyzing and improving the performance parallel computing tools. These activities complement and complete the work performed by the other X-Stack 2 projects with the definition of a common data models and computing infrastructures, which have been a central element in the targeted performance analysis pipeline and new programming models. The University of Utah team have focused mainly on developing data analysis and visualization systems to facilitate exploration, comparison and evaluation of the behavior and performance of complex algorithms in modern HPC infrastructures. This is involved several demonstrations of the performance achieved and presentation of the results in a way that facilitates the users understand bottlenecks and ways to overcome them. One major focus of the University of Utah activities has been the analysis of the performance characteristics of Fat-Tree and Dragonfly topology networks. The work involved the development of practical visualization systems especially in collaboration with the team at LLNL, and published several papers and gave invited talks at prestigious venues such as the Salishan Conference on High-Speed Computing or the International Conference on High Performance Computing, Data, and Analytics. In addition, the university of Utah team has advanced the core activities on analyzing the performance characteristics of I/O and other data movement systems and their practical use for scaling Uintah and other simulation codes poised for use in the exascale computing environments of the future.

42 ENGINEERING↗

ISAVS: Interactive Scalable Analysis and Visualization System

Modern science is inundated with ever increasing data sizes as computational capabilities and image acquisition techniques continue to improve. For example, simulations are tackling ever larger domains with higher fidelity, and high-throughput microscopy techniques generate larger data that are fundamental to gather biologically and medically relevant insights. As the image sizes exceed memory, and even sometimes local disk space, each step in a scientific workflow is impacted. Current software solutions enable data exploration with limited interactivity for visualization and analytic tasks. Furthermore analysis on HPC systems often require complex hand-written parallel implementations of algorithms that suffer from poor portability and maintainability. We present a software infrastructure that simplifies end-to-end visualization and analysis of massive data. First, a hierarchical streaming data access layer enables interactive exploration of remote data, with fast data fetching to test analytics on subsets of the data. Second, a library simplifies the process of developing new analytics algorithms, allowing users to rapidly prototype new approaches and deploy them in an HPC setting. Third, a scalable runtime system automates mapping analysis algorithms to whatever computational hardware is available, reducing the complexity of developing scaling algorithms. Here, we demonstrate the usability and performance of our system using a use case from neuroscience: filtering, registration, and visualization of tera-scale microscopy data. We evaluate the performance of our system using a leadership-class supercomputer, Shaheen II.

97 MATHEMATICS AND COMPUTING↗

Introducing Molecular Hypernetworks for Discovery in Multidimensional Metabolomics Data

Orthogonal separations of data from high-resolution mass spectrometry can provide insight into sample composition and address challenges of complete annotation of molecules in untargeted metabolomics. “Molecular networks” (MNs), as used in the Global Natural Products Social Molecular Networking platform, are a prominent strategy for exploring and visualizing molecular relationships and improving annotation. MNs are mathematical graphs showing the relationships between measured multidimensional data features. MNs also show promise for using network science algorithms to automatically identify targets for annotation candidates and to dereplicate features associated with a single molecular identity. Here, this paper introduces “molecular hypernetworks” (MHNs) as more complex MN models able to natively represent multiway relationships among observations. Compared to MNs, MHNs can more parsimoniously represent the inherent complexity present among groups of observations, initially supporting improved exploratory data analysis and visualization. MHNs also promise to increase confidence in annotation propagation, for both human and analytical processing. We first illustrate MHNs with simple examples, and build them from liquid chromatography- and ion mobility spectrometry-separated MS data. We then describe a method to construct MHNs directly from existing MNs as their “clique reconstructions”, demonstrating their utility by comparing examples of previously published graph-based MNs to their respective MHNs.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Atlas Multimedia Educational Lab for Interactive Analysis

AMELIA is an application with focus on particle physics processes in ATLAS. This will allow students and other users to decode the collision events that unfold after the head-on collisions of protons at the Large hadron Collider. AMELIA uses the Irrlicht engine for the 3D graphics and wxWidgets for the interface. It uses the best aspects of technical animation and allows users to control 3D representations of collision events and to manipulate 3D models of the detector and see how particles are detected as they pass through. It allows the user to rotate, zoom and select virtual pieces of the ATLAS detector and events. The characteristics of the events (momentum etc.) can also be read, and one can select tracks for analysis, activate context-oriented media, etc. This framework intends to integrate different types of media into a single product. This way, videos, animations, sound, interactive visualization and data analysis will be bound together in the same package.-

Pequenao, Joao↗

Analyst Tools and Quality Control Software for the ARM Data System

Mission Research develops analyst tools and automated quality control software in order to assist the Atmospheric Radiation Measurement (ARM) Data Quality Office with their data inspection tasks. We have developed web-based data analysis and visualization tools such as the interactive plotting program NCVweb, various diagnostic plot browsers, and a datastream processing status application. These tools allow even novice ARM researchers to be productive with ARM data with only minimal effort. We also contribute to the ARM Data Quality Office by analyzing ARM data streams, developing new quality control metrics, new diagnostic plots, and integrating this information into DQ HandS - the Data Quality Health and Status web-based explorer. We have developed several ways to detect outliers in ARM data streams and have written software to run in an automated fashion to flag these outliers. We have also embarked on a system to comprehensively generate long time-series plots, frequency distributions, and other relevant statistics for scientific and engineering data in most high-level, publicly available ARM data streams. Furthermore, frequency distributions categorized by month or by season are made available to help define valid data ranges specific to those time domains. These statistics can be used to set limits that when checked, will improve upon the reporting of suspicious data and the early detection of instrument malfunction. The statistics and proposed limits are stored in a database for easy reporting, refining, and for use by other processes. Web-based applications to view the results are also available.

54 ENVIRONMENTAL SCIENCES↗

XCMS Online: A Web-Based Platform to Process Untargeted Metabolomic Data

Recently, interest in untargeted metabolomics has become prevalent in the general scientific community among an increasing number of investigators. The majority of these investigators, however, do not have the bioinformatic expertise that has been required to process metabolomic data by using command-line driven software programs. Here we introduce a novel platform to process untargeted metabolomic data that uses an intuitive graphical interface and does not require installation or technical expertise. This platform, called XCMS Online, is a web-based version of the widely used XCMS software that allows users to easily upload and process liquid chromatography/mass spectrometry data with only a few mouse clicks. XCMS Online provides a solution for the complete untargeted metabolomic workflow including feature detection, retention time correction, alignment, annotation, statistical analysis, and data visualization. Results can be browsed online in an interactive, customizable table showing statistics, chromatograms, and putative METLIN identities for each metabolite. Additionally, all results and images can be downloaded as zip files for offline analysis and publication. XCMS Online is available at https://xcmsonline.scripps.edu.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Visually Relating Gene Expression and in vivo DNA Binding Data

Gene expression and in vivo DNA binding data provide important information for understanding gene regulatory networks: in vivo DNA binding data indicate genomic regions where transcription factors are bound, and expression data show the output resulting from this binding. Thus, there must be functional relationships between these two types of data. While visualization and data analysis tools exist for each data type alone, there is a lack of tools that can easily explore the relationship between them. We propose an approach that uses the average expression driven by multiple of ciscontrol regions to visually relate gene expression and in vivo DNA binding data. We demonstrate the utility of this tool with examples from the network controlling early Drosophila development. The results obtained support the idea that the level of occupancy of a transcription factor on DNA strongly determines the degree to which the factor regulates a target gene, and in some cases also controls whether the regulation is positive or negative.

60 APPLIED LIFE SCIENCES↗

Multivariate Data EXplorer (MDX)

The MDX toolkit facilitates exploratory data analysis and visualization of multivariate datasets. MDX provides and interactive graphical user interface to load, explore, and modify multivariate datasets stored in tabular forms. MDX uses an extended version of the parallel coordinates plot and scatterplots to represent the data. The user can perform rapid visual queries using mouse gestures in the visualization panels to select rows or columns of interest. The visualization panel provides coordinated multiple views whereby selections made in one plot are propagated to the other plots. Users can also export selected data or reconfigure the visualization panel to explore relationships between columns and rows in the data.

Steed, Chad Allen↗

On the use of graph search techniques for the analysis of extreme-scale combustion simulation data

With the continuous increase in high performance computing capabilities, simulations are becoming ever larger and more complex, using bigger domains, tracking more variables, and producing more time steps. This increase in the ranges of spatial and temporal simulation scales results in data that presents significant challenges to as well as new opportunities for the visualization and data analysis community. For example, highly-localized, intermittent events (such as the formation of ignition kernels in turbulent combustion) may be caused by interactions between multiple variables across a series of time steps, making both their definition and their extraction difficult, particularly at scale. This paper introduces an intuitive framework to support the identification, characterization, and tracking of such complex, multivariate, temporally evolving events in large-scale simulations. In a pre-processing step, we use topological techniques to create a hierarchical family of feature definitions for each variable of interest. Subsequently, we select a particular set of features for analysis and, using overlap-based metrics, we generate an attributed relational graph (ARG) capturing the relationships between different variables both within one and across multiple time steps. Finally, we leverage subgraph-isomorphism search heuristics to identify patterns in the ARG that characterize interesting events. We demonstrate the power of this approach by analyzing a large-scale turbulent combustion simulation.

McLendon, W. C.↗

Ensemble Data Analysis ENvironment (EDEN)

The EDEN toolkit facilitates exploratory data analysis and visualization of global climate model simulation datasets. EDEN provides an interactive graphical user interface (GUI) that helps the user visually construct dynamic queries of the characteristically large climate datasets using temporal ranges, variable selections, and geographic areas of interest. EDEN reads the selected data into a multivariate visualization panel which features an extended implementation of parallel coordinates plots as well as interactive scatterplots. The user can query data in the visualization panel using mouse gestures to analyze different ranges of data. The visualization panel provides coordinated multiple views whereby selections made in one plot are propagated to the other plots.

Steed, ChadAllen↗