Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Visualization and Analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 811 records · Page 45

The NASA NEESPI Data Portal to Support Studies of Climate and Environmental Changes in Non-Boreal Europe

NASA NEESPI (Northern Eurasia Earth Science Partnership Initiative) data portal is a NASA funded project that focuses on collecting satellite remote sensing data, providing tools, information, and services in support of NEESPI scientific objectives (Leptoukh, et al., 2007). The data can be accessed online through anonymous ftp, through an advanced data searching and ordering system Mirador that uses keywords to find data quickly in a Google-like interface, and through the Goddard Interactive Online Visualization ANd aNalysis Infrastructure (Giovanni). The portal provides preprocessed data from different satellite sensors and numerical models to the same spatial and temporal resolution and the same projection so that the data can be used easily to perform inter-comparison or relationship studies. In addition, it provides parameter and spatially subsetted data for regional studies. Studies of regional carbon, hydrology, aerosols in non-boreal Europe and their interactions with global climate are very challenging research topics. The NASA NEESPI data portal makes many satellite data available for such studies, including information on land cover types, fire, vegetation index, aerosols, land surface temperature, soil moisture, precipitation, snow/ice, and other parameters. This paper will introduce the features and products available in the system, focusing on the online data 1 tool, Giovanni NEESPI. An example that explores different data through Giovanni NEESPI in temperate region of non-boreal Europe will be presented.

Shen, Suhung↗

Large Scale Comparative Visualisation of Regulatory Networks with TRNDiff

The advent of Next Generation Sequencing (NGS) technologies has seen explosive growth in genomic datasets, and dense coverage of related organisms, supporting study of subtle, strain-specific variations as a determinant of function. Such data collections present fresh and complex challenges for bioinformatics, those of comparing models of complex relationships across hundreds and even thousands of sequences. Transcriptional Regulatory Network (TRN) structures document the influence of regulatory proteins called Transcription Factors (TFs) on associated Target Genes (TGs). TRNs are routinely inferred from model systems or iterative search, and analysis at these scales requires simultaneous displays of multiple networks well beyond those of existing network visualisation tools [1]. In this paper we describe TRNDiff, an open source system supporting the comparative analysis and visualization of TRNs (and similarly structured data) from many genomes, allowing rapid identification of functional variations within species. The approach is demonstrated through a small scale multiple TRN analysis of the Fur iron-uptake system of Yersinia, suggesting a number of candidate virulence factors; and through a larger study exploiting integration with the RegPrecise database (http://regprecise.lbl.gov; [2]) - a collection of hundreds of manually curated and predicted transcription factor regulons drawn from across the entire spectrum of prokaryotic organisms.

Bioinformatics, Visualisation, Transcription, Regu↗

Why wait? Let's start computing while data is still on the wire

In this era of Big Data, computing useful information from data is becoming increasingly complicated, particularly due to the ever increasing volumes of data that need to travel over the network to data centers to be stored and processed, all highly expensive operations in the long haul. In this paper we suggest that we can do computing and analysis of data "on the wire," i.e., while data is still in transit. The nature of these computations include analysis, visualization, pattern recognition, and prediction, or forecasting, on the streaming data. We follow a service function chaining architecture to implement this, assuming that the data packets arrive within a single network administrative domain. As a demonstration of this new computing paradigm, we present three examples. Firstly, we demonstrate pattern recognition and data visualization on streaming forex data, which can be used for lucrative trading in the forex market. In our second example, we analyze and learn user buying patterns from clickstream data streaming from multiple websites. Finally, we monitor solar sensors for a zero reading while the packets are still on their way to the data center, to schedule any maintenance and requisite repairs with no time delay.

97 MATHEMATICS AND COMPUTING↗

FAST User Guide

The Flow Analysis Software Toolkit, FAST, is a software environment for visualizing data. FAST is a collection of separate programs (modules) that run simultaneously and allow the user to examine the results of numerical and experimental simulations. The user can load data files, perform calculations on the data, visualize the results of these calculations, construct scenes of 3D graphical objects, and plot, animate and record the scenes. Computational Fluid Dynamics (CFD) visualization is the primary intended use of FAST, but FAST can also assist in the analysis of other types of data. FAST combines the capabilities of such programs as PLOT3D, RIP, SURF, and GAS into one environment with modules that share data. Sharing data between modules eliminates the drudgery of transferring data between programs. All the modules in the FAST environment have a consistent, highly interactive graphical user interface. Most commands are entered by pointing and'clicking. The modular construction of FAST makes it flexible and extensible. The environment can be custom configured and new modules can be developed and added as needed. The following modules have been developed for FAST: VIEWER, FILE IO, CALCULATOR, SURFER, TOPOLOGY, PLOTTER, TITLER, TRACER, ARCGRAPH, GQ, SURFERU, SHOTET, and ISOLEVU. A utility is also included to make the inclusion of user defined modules in the FAST environment easy. The VIEWER module is the central control for the FAST environment. From VIEWER, the user can-change object attributes, interactively position objects in three-dimensional space, define and save scenes, create animations, spawn new FAST modules, add additional view windows, and save and execute command scripts. The FAST User Guide uses text and FAST MAPS (graphical representations of the entire user interface) to guide the user through the use of FAST. Chapters include: Maps, Overview, Tips, Getting Started Tutorial, a separate chapter for each module, file formats, and system administration.

Walatka, Pamela P.↗

Discover, Visualize, and Deliver Geospatial Data through OGC Standards-based WebGIS System

Geospatial data are important to understand the Earth - ecosystem dynamics, land cover changes, resource management, and human interactions with the Earth to name a few. One of the biggest difficulties users face is to discover, access, and assemble distributed, large volume, heterogeneous geospatial data to conduct geo-analysis. Traditional methods of geospatial data discovery, visualization, and delivery lack the capabilities of resource sharing and automation across systems or organizational boundaries. They require users to download the data ldquoas-isrdquo in their original file format, projection, and extent. Also, discovering data served by traditional methods requires prior knowledge of data location, and processing requires specialized expertise. These drawbacks of traditional methods create additional burden to users, introduce too much overhead to research, and also reduce the potential usage of the data. At the Oak Ridge National Laboratory (ORNL), researchers working on NASA-sponsored projects: Distributed Active Archive Center (DAAC) and Modeling and Synthesis Thematic Data Center (MAST-DC) have tapped into the benefits of Open Geospatial Consortium (OGC) standards to overcome the drawbacks of traditional methods of geospatial data discovery, visualization, and delivery. The OGC standards-based approach facilitates data sharing and interoperability across network, organizational, and geopolitical boundaries. Tools and services based on OGC standards deliver the data in many user defined formats and allow users to visualize the data prior to download. This paper introduces an approach taken to visualize and deliver ORNL DAAC, MAST-DC, and other relevant geospatial data through OGC standards-based Web Services, including Web Map Service (WMS), Web Coverage Service (WCS), and Web Feature Service (WFS). It also introduces a WebGIS system built on top of OGC services that helps users discover, visualize, and access geospatial data.

99 GENERAL AND MISCELLANEOUS↗

Towards elastic in situ analysis for high-performance computing simulations

In situ analysis and visualization have grown increasingly popular for enabling direct access to data from high-performance computing (HPC) simulations. As a simulation progresses and interesting physical phenomena emerge, however, the data produced may become increasingly complex, and users may need to dynamically change the type and scale of in situ analysis tasks being carried out and consequently adapt the amount of resources allocated to such tasks. To date, none of the production in situ analysis frameworks offer such an elasticity feature, and for good reason: the assumption that the number of processes could vary during run time would force developers to rethink software and algorithms at every level of the in situ analysis stack. In this paper we present Colza, a data staging service with elastic in situ visualization capabilities. We demonstrate the use of Colza with the Deep Water Impact and the AMR-Wind simulations, coupling them with the ParaView Catalyst and Ascent in situ libraries, and show that Colza enables dynamic rescaling of these widely-used frameworks with no interruption to the simulation or staging service. Here, we highlight the challenges of enabling such elasticity, which requires overcoming these frameworks' reliance on MPI, using distinct engineering approaches, namely dependency injection and dependency overload. To the best of our knowledge, this work is the first to enable elastic in situ visualization capabilities for HPC applications on top of existing production analysis tools.

97 MATHEMATICS AND COMPUTING↗

Analyst-Driven Knowledge Enhancement & Analysis version 1.0

AKEA provides tools for the creation, maintenance, visualization, and analysis of information expressed in semantic graph form. It is capable of taking information expressed in unstructured text documents or structured data documents (e.g., spreadsheets, databases), ingesting it and extracting entities and relationships of interest, then presenting said information in graphical form. In addition, it contains tools for applying traditional social network analysis algorithms to the generated graphs in order to identify key players or entities with particular characteristics, geospatial and temporal visualization of said data, and user-controlled storage of information to support future analysis needs.

Thurman, DavidA.↗

Sub-visual Cirrus detection and characterization

Analysis of archived cold optics (COR) radiometer data is yielding useful information on the diurnal, geographic, seasonal and altitude variability of atmospheric background radiance levels in the 11 micron window region. This database is a compilation of Kuiper Infrared Technology Experiment (KITE) and Atmospheric Radiance Study (ARS) observations under a wide variety of conditions. Correlating the measurements from these two studies with the LOWTRAN model code has revealed several important results. First, the 11 micron window appears to be filled-in, i.e., the troughs on either side of the nitric acid peak are shallower than expected. Second, the amplitude of the background radiances measured exceeds the model predictions by a factor of 2 to 3 or more. This is thought to be due to the existence of thin, high altitude cirrus clouds (sub-visual cirrus) above the sensor platform. These high background levels are observed under quiescent conditions in the South Pacific (Marshall Islands), as well as over the continental United States (the West Coast). In the tropics, there appears to be little diurnal variability, a plausible seasonal variation and a linear dependence between 7.2 and 11.4 micron band data, indicating possible multi-spectral approaches to detection of sub-visual cirrus clouds. Theoretical analysis of the magnitude of the effects of a sub-visual cirrus cloud on atmospheric background radiances measured by a near-horizontal sensor is in progress.

Schmidt, E.↗

Supporting Co-Design of Extreme-Scale Systems with In Situ Visual Analysis of Event-Driven Simulations (Final Report)

Extreme-scale systems for compute- and data-centric science are pushing the boundaries of system architectures in order to achieve ambitious goals in terms of computing capability, resilience, and power efficiency. Future systems necessitate emerging designs that incorporate new technologies, system organizations, and computer science techniques, and these changes are not yet well understood. Discrete-event-driven simulation of computing system architectures and subsystems has emerged as a productive and cost-effective means to evaluating potential designs, along with capabilities for executing these simulations of extreme-scale systems. However, techniques for analyzing the behavior of these simulations have not kept pace with our ability to perform the simulations, preventing us from extracting the most value from this promising approach. The objective of this project is to support the codesign of extreme-scale system architectures for compute and data-centric science through research and development of novel methods for analysis and visualization of large-scale event-driven simulations.

97 MATHEMATICS AND COMPUTING↗

MSiReader v1.0: Evolving Open-Source Mass Spectrometry Imaging Software for Targeted and Untargeted Analyses

A major update to the mass spectrometry imaging (MSI) software MSiReader is presented, offering a multitude of newly added features critical to MSI analyses. MSiReader is a free, open-source, and vendor-neutral software written in the MATLAB platform and is capable of analyzing most common MSI data formats. A standalone version of the software, which does not require a MATLAB license, is also distributed. The newly incorporated data analysis features expand the utility of MSiReader beyond simple visualization of molecular distributions. The MSiQuantification tool allows researchers to calculate absolute concentrations from quantification MSI experiments exclusively through MSiReader software, significantly reducing data analysis time. An image overlay feature allows the incorporation of complementary imaging modalities to be displayed with the MSI data. A polarity filter has also been incorporated into the data loading step, allowing the facile analysis of polarity switching experiments without the need for data parsing prior to loading the data file into MSiReader. A quality assurance feature to generate a mass measurement accuracy (MMA) heatmap for an analyte of interest has also been added to allow for the investigation of MMA across the imaging experiment. Most importantly, as new features have been added performance has not degraded, in fact it has been dramatically improved. These new tools and the improvements to the performance in MSiReader v1.0 enable the MSI community to evaluate their data in greater depth and in less time. .

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Earth System Grid Center for Enabling Technologies (ESG-CET): A Data Infrastructure for Data-Intensive Climate Research

For the Earth System Grid Federation (ESGF), the ESG-CET team has led international development and delivered a production environment for managing and accessing ultrascale climate data. This production environment includes multiple national and international climate projects (e.g., Couple Model Intercomparison Project, Community Earth System Model), ocean model data (such as the Parallel Ocean Program), observation data (Carbon Dioxide Information and Analysis Center, Atmospheric Infrared Sounder, and so forth), and analysis and visualization tools, all of which serve a diverse community of users. These data holdings and services are distributed across multiple ESG-CET sites (such as LANL, LBNL, LLNL, NCAR, and ORNL) as well as at unfunded partners sites such as the Australian National University National Computational Infrastructure, the British Atmospheric Data Centre, the National Oceanic and Atmospheric Administration Geophysical Fluid Dynamics Laboratory, the Max Planck Institute for Meteorology, the German Climate Computing Centre, and the National Aeronautics and Space Administration Jet Propulsion Laboratory. More recently, ESG-CET has been extending services beyond data-file access and delivery to develop more detailed information products (scientific graphics, animations, etc.), secure binary data-access services (based upon the OPeNDAP protocol), and server-side analysis capabilities. These will allow users to request data subsets transformed through commonly used analysis and intercomparison procedures. As we transition from development activities to production and operations, the ESG-CET team is tasked with making data available to all users seeking to understand, process, extract value from, visualize, and/or communicate it to others. This ongoing effort, though daunting in scope and complexity, will greatly magnify the value of numerical climate model outputs and climate observations for future national and international climate-assessment reports. Continued ESGF progress will result in a production ultrascale data system for empowering scientists who attempt new and exciting data exchanges that could ultimately lead to breakthrough climate-science discoveries.

54 ENVIRONMENTAL SCIENCES↗

Characterizing Time Series Data Diversity for Wind Forecasting: Preprint

Wind forecasting plays an important role in integrating variable and uncertain wind power into the power grid. Various forecasting models have been developed to improve the forecasting accuracy. However, it is challenging to accurately compare the true forecasting performances from different methods and forecasters due to the lack of diversity in forecasting test datasets. This paper proposes a time series characteristic analysis approach to visualize and quantify wind time series diversity. The developed method first calculates six time series characteristic indices from various perspectives. Then the principal component analysis is performed to reduce the data dimension while preserving the important information. The diversity of the time series dataset is visualized by the geometric distribution of the newly constructed principal component space. The volume of the 3-dimensional (3D) convex polytope (or the length of 1D number axis, or the area of the 2D convex polygon) is used to quantify the time series data diversity. The method is tested with five datasets with various degrees of diversity.

17 WIND ENERGY↗

Visual analysis of code security

To help increase the confidence that software is secure, researchers and vendors have developed different kinds of automated software security analysis tools. These tools analyze software for weaknesses and vulnerabilities, but the individual tools catch different vulnerabilities and produce voluminous data with many false positives. This paper describes a system that brings together the results of disparate software analysis tools into a visual environment to support the triage and exploration of code vulnerabilities. Our system allows software developers to explore vulnerability results to uncover hidden trends, triage the most important code weaknesses, and show who is responsible for introducing software vulnerabilities. By correlating and normalizing multiple software analysis tools' data, the overall vulnerability detection coverage of software is increased. A visual overview and powerful interaction allows the user to focus attention on the most pressing vulnerabilities within huge volumes of data, and streamlines the secure software development workflow through integration with development tools.

99 GENERAL AND MISCELLANEOUS↗

Visualization and Analysis of Multi-scale Land Surface Products via Giovanni Portals

Large volumes of MODIS land data products at multiple spatial resolutions have been integrated into the Giovanni online analysis system to support studies on land cover and land use changes,focused on the Northern Eurasia and Monsoon Asia regions through the LCLUC program. Giovanni (Goddard Interactive Online Visualization ANd aNalysis Infrastructure) is a Web-based application developed by the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), providing a simple and intuitive way to visualize, analyze, and access Earth science remotely-sensed and modeled data.Customized Giovanni Web portals (Giovanni-NEESPI andGiovanni-MAIRS) have been created to integrate land, atmospheric,cryospheric, and societal products, enabling researchers to do quick exploration and basic analyses of land surface changes, and their relationships to climate, at global and regional scales. This presentation shows a sample Giovanni portal page, lists selected data products in the system, and illustrates potential analyses with imagesand time-series at global and regional scales, focusing on climatology and anomaly analysis. More information is available at the GES DISCMAIRS data support project portal: http:disc.sci.gsfc.nasa.govmairs.

Shen, Suhung↗

Performance Analysis of Large Scale HPC Workflows for Earth System Models

Climate assessment reports, such as the Intergovernmental Panel on Climate Change’s 5th assesment report [IPCC AR5; IPCC et al., 2014] or the United States National Climate Assessment [U.S. Global Change Research Program, 2014], increasingly rely on many-ensemble climate projections from fully-coupled Earth system models (ESM) to asses possible future climate states. ESM participate in a large set of Coupled Model Intercomparson Projects (CMIP) to better understand their models and the climate system, so that they may provide credible climate projections. However, fully participating in these intercomparison exercises represents an extremely large computational personnel burden for modeling centers [Eyring et al., 2016, Williams et al., 2016a]. However, there has been little attention within the climate modeling community paid to the in situ performance of ESMs up until now. Recently a computational performance model intercomparison project (CPMIP) for CMIP6 has begun [Balaji et al., 2017], but there has not yet been an effort to gather, process, and analyze the historical performance data that has been saved within archived log files from previous modeling efforts, nor the development of performance analysis system capable of providing detailed statistical analyses or visualizations of this performance data. Therefore, we’ve begun a joint effort to analyze the historical and current performance data on both GFDL and OLCF modeling systems. Here, we have developed a preliminary performance analysis system to captue, analyize, and visualize end-to-end in situ performance data and applied it to NOAA/GFDL’s post-processing system. Through the course of the project a number of performance improvements have been made, with the steps necessary to handle the expected volume of data (Big Data) likely to come out of the CMIP6 exercises (TB to PB) provided. We have also begun developing a statistical workload model that will allow modeling centers to determined their expected scientific throughput in the context of expected resource (computational and personnel) availability and accounting for typical workflow disruptions. This will allow them to develop a CMIP participation strategy and maximize their level of participation in the exercises. We expect this infrastructure, and the data it produces, to ultimately allow us to identify performance issues and workflow errors that exist across Earth system modeling centers, and possibly HPC modeling centers in general. Center-independent solutions to workflow or performance issues would help alleviate the computational burden for CMIP6+ modeling projects and should realize both labor and cost savings benefits throughout the CMIP community.

54 ENVIRONMENTAL SCIENCES↗

Information Fusion and Data Analytics for Human Lunar Exploration (CIF REPORT: Detailed PI Write-up)

The Information Fusion & Data Analytics (IFDA) project commenced in FY20, continued through FY21, and its final platform development phase continues in FY22. The objective remains the fusion and rapid accessibility of large quantities of disparate sourced human spaceflight data. IFDA is a platform tailored for NA (S&MA) to develop highly advanced operational data integration and analysis techniques. IFDA leverages the JSC ER7 modeling, simulation,and data fusion capabilities to collect, warehouse, and augment data human exploration data integration and analysis techniques. The IFDA project’s integrated data visualizations have been demonstrated in two validation scenarios in FY21, and provided the architecture and platform basis for development of a full-scale data analysis suite and storage solution useful to all JSC organizations engaged in real time operations and safety tasks. Scenarioand prototypical development including the construction of a full scale data analysis suite and storage solution, useful to all JSC organizations engaged in real time operations and safety tasks, is central to IFDA Phase 3 and provides a demonstrable pathway for the Digital Transformation Program. IFDA Phase 3 is focused on data provider, data utilizer, and SME hands-on workshops that will conclude the Dem / Valphase and deliver a program-ready data integration tool as a product.

information fusion↗

Information Fusion & Analytics for Human Lunar Exploration

The Information Fusion & Data Analytics (IFDA) project commenced in FY20, continued through FY21, and its final platform development phase continues in FY22. The objective remains the fusion and rapid accessibility of large quantities of disparate sourced human spaceflight data. IFDA is a platform tailored for NA (S&MA) to develop highly advanced operational data integration and analysis techniques. IFDA leverages the JSC ER7 modeling, simulation,and data fusion capabilities to collect, warehouse, and augment data human exploration data integration and analysis techniques. The IFDA project’s integrated data visualizations have been demonstrated in two validation scenarios in FY21, and provided the architecture and platform basis for development of a full-scale data analysis suite and storage solution useful to all JSC organizations engaged in real time operations and safety tasks. Scenarioand prototypical development including the construction of a full scale data analysis suite and storage solution, useful to all JSC organizations engaged in real time operations and safety tasks, is central to IFDA Phase 3 and provides a demonstrable pathway for the Digital Transformation Program. IFDA Phase 3 is focused on data provider, data utilizer, and SME hands-on workshops that will conclude the Dem / Valphase and deliver a program-ready data integration tool as a product.

information fusion↗

RCSB PDB Mobile : iOS and Android mobile apps to provide data access and visualization to the RCSB Protein Data Bank

The Research Collaboratory for Structural Bioinformatics Protein Data Bank (RCSB PDB) resource provides tools for query, analysis and visualization of the 3D structures in the PDB archive. As the mobile Web is starting to surpass desktop and laptop usage, scientists and educators are beginning to integrate mobile devices into their research and teaching. In response, we have developed the RCSB PDB Mobile app for the iOS and Android mobile platforms to enable fast and convenient access to RCSB PDB data and services. Lastly, using the app, users from the general public to expert researchers can quickly search and visualize biomolecules, and add personal annotations via the RCSB PDB's integrated MyPDB service.

59 BASIC BIOLOGICAL SCIENCES↗