SEARCH · Engineering Papers

Results for “Data visualization and analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 739 records · Page 41

All sources NASA NTRS DOE OSTI

NASA NTRS1990

Sub-visual Cirrus detection and characterization

Analysis of archived cold optics (COR) radiometer data is yielding useful information on the diurnal, geographic, seasonal and altitude variability of atmospheric background radiance levels in the 11 micron window region. This database is a compilation of Kuiper Infrared Technology Experiment (KITE) and Atmospheric Radiance Study (ARS) observations under a wide variety of conditions. Correlating the measurements from these two studies with the LOWTRAN model code has revealed several important results. First, the 11 micron window appears to be filled-in, i.e., the troughs on either side of the nitric acid peak are shallower than expected. Second, the amplitude of the background radiances measured exceeds the model predictions by a factor of 2 to 3 or more. This is thought to be due to the existence of thin, high altitude cirrus clouds (sub-visual cirrus) above the sensor platform. These high background levels are observed under quiescent conditions in the South Pacific (Marshall Islands), as well as over the continental United States (the West Coast). In the tropics, there appears to be little diurnal variability, a plausible seasonal variation and a linear dependence between 7.2 and 11.4 micron band data, indicating possible multi-spectral approaches to detection of sub-visual cirrus clouds. Theoretical analysis of the magnitude of the effects of a sub-visual cirrus cloud on atmospheric background radiances measured by a near-horizontal sensor is in progress.

Schmidt, E.↗

DOE OSTI2021

Supporting Co-Design of Extreme-Scale Systems with In Situ Visual Analysis of Event-Driven Simulations (Final Report)

Extreme-scale systems for compute- and data-centric science are pushing the boundaries of system architectures in order to achieve ambitious goals in terms of computing capability, resilience, and power efficiency. Future systems necessitate emerging designs that incorporate new technologies, system organizations, and computer science techniques, and these changes are not yet well understood. Discrete-event-driven simulation of computing system architectures and subsystems has emerged as a productive and cost-effective means to evaluating potential designs, along with capabilities for executing these simulations of extreme-scale systems. However, techniques for analyzing the behavior of these simulations have not kept pace with our ability to perform the simulations, preventing us from extracting the most value from this promising approach. The objective of this project is to support the codesign of extreme-scale system architectures for compute and data-centric science through research and development of novel methods for analysis and visualization of large-scale event-driven simulations.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2018

MSiReader v1.0: Evolving Open-Source Mass Spectrometry Imaging Software for Targeted and Untargeted Analyses

A major update to the mass spectrometry imaging (MSI) software MSiReader is presented, offering a multitude of newly added features critical to MSI analyses. MSiReader is a free, open-source, and vendor-neutral software written in the MATLAB platform and is capable of analyzing most common MSI data formats. A standalone version of the software, which does not require a MATLAB license, is also distributed. The newly incorporated data analysis features expand the utility of MSiReader beyond simple visualization of molecular distributions. The MSiQuantification tool allows researchers to calculate absolute concentrations from quantification MSI experiments exclusively through MSiReader software, significantly reducing data analysis time. An image overlay feature allows the incorporation of complementary imaging modalities to be displayed with the MSI data. A polarity filter has also been incorporated into the data loading step, allowing the facile analysis of polarity switching experiments without the need for data parsing prior to loading the data file into MSiReader. A quality assurance feature to generate a mass measurement accuracy (MMA) heatmap for an analyte of interest has also been added to allow for the investigation of MMA across the imaging experiment. Most importantly, as new features have been added performance has not degraded, in fact it has been dramatically improved. These new tools and the improvements to the performance in MSiReader v1.0 enable the MSI community to evaluate their data in greater depth and in less time. .

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2011

Earth System Grid Center for Enabling Technologies (ESG-CET): A Data Infrastructure for Data-Intensive Climate Research

For the Earth System Grid Federation (ESGF), the ESG-CET team has led international development and delivered a production environment for managing and accessing ultrascale climate data. This production environment includes multiple national and international climate projects (e.g., Couple Model Intercomparison Project, Community Earth System Model), ocean model data (such as the Parallel Ocean Program), observation data (Carbon Dioxide Information and Analysis Center, Atmospheric Infrared Sounder, and so forth), and analysis and visualization tools, all of which serve a diverse community of users. These data holdings and services are distributed across multiple ESG-CET sites (such as LANL, LBNL, LLNL, NCAR, and ORNL) as well as at unfunded partners sites such as the Australian National University National Computational Infrastructure, the British Atmospheric Data Centre, the National Oceanic and Atmospheric Administration Geophysical Fluid Dynamics Laboratory, the Max Planck Institute for Meteorology, the German Climate Computing Centre, and the National Aeronautics and Space Administration Jet Propulsion Laboratory. More recently, ESG-CET has been extending services beyond data-file access and delivery to develop more detailed information products (scientific graphics, animations, etc.), secure binary data-access services (based upon the OPeNDAP protocol), and server-side analysis capabilities. These will allow users to request data subsets transformed through commonly used analysis and intercomparison procedures. As we transition from development activities to production and operations, the ESG-CET team is tasked with making data available to all users seeking to understand, process, extract value from, visualize, and/or communicate it to others. This ongoing effort, though daunting in scope and complexity, will greatly magnify the value of numerical climate model outputs and climate observations for future national and international climate-assessment reports. Continued ESGF progress will result in a production ultrascale data system for empowering scientists who attempt new and exciting data exchanges that could ultimately lead to breakthrough climate-science discoveries.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2018

Characterizing Time Series Data Diversity for Wind Forecasting: Preprint

Wind forecasting plays an important role in integrating variable and uncertain wind power into the power grid. Various forecasting models have been developed to improve the forecasting accuracy. However, it is challenging to accurately compare the true forecasting performances from different methods and forecasters due to the lack of diversity in forecasting test datasets. This paper proposes a time series characteristic analysis approach to visualize and quantify wind time series diversity. The developed method first calculates six time series characteristic indices from various perspectives. Then the principal component analysis is performed to reduce the data dimension while preserving the important information. The diversity of the time series dataset is visualized by the geometric distribution of the newly constructed principal component space. The volume of the 3-dimensional (3D) convex polytope (or the length of 1D number axis, or the area of the 2D convex polygon) is used to quantify the time series data diversity. The method is tested with five datasets with various degrees of diversity.

17 WIND ENERGY↗

NASA NTRS2013

Visualization and Analysis of Multi-scale Land Surface Products via Giovanni Portals

Large volumes of MODIS land data products at multiple spatial resolutions have been integrated into the Giovanni online analysis system to support studies on land cover and land use changes,focused on the Northern Eurasia and Monsoon Asia regions through the LCLUC program. Giovanni (Goddard Interactive Online Visualization ANd aNalysis Infrastructure) is a Web-based application developed by the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC), providing a simple and intuitive way to visualize, analyze, and access Earth science remotely-sensed and modeled data.Customized Giovanni Web portals (Giovanni-NEESPI andGiovanni-MAIRS) have been created to integrate land, atmospheric,cryospheric, and societal products, enabling researchers to do quick exploration and basic analyses of land surface changes, and their relationships to climate, at global and regional scales. This presentation shows a sample Giovanni portal page, lists selected data products in the system, and illustrates potential analyses with imagesand time-series at global and regional scales, focusing on climatology and anomaly analysis. More information is available at the GES DISCMAIRS data support project portal: http:disc.sci.gsfc.nasa.govmairs.

Shen, Suhung↗

DOE OSTI2017

Performance Analysis of Large Scale HPC Workflows for Earth System Models

Climate assessment reports, such as the Intergovernmental Panel on Climate Change’s 5th assesment report [IPCC AR5; IPCC et al., 2014] or the United States National Climate Assessment [U.S. Global Change Research Program, 2014], increasingly rely on many-ensemble climate projections from fully-coupled Earth system models (ESM) to asses possible future climate states. ESM participate in a large set of Coupled Model Intercomparson Projects (CMIP) to better understand their models and the climate system, so that they may provide credible climate projections. However, fully participating in these intercomparison exercises represents an extremely large computational personnel burden for modeling centers [Eyring et al., 2016, Williams et al., 2016a]. However, there has been little attention within the climate modeling community paid to the in situ performance of ESMs up until now. Recently a computational performance model intercomparison project (CPMIP) for CMIP6 has begun [Balaji et al., 2017], but there has not yet been an effort to gather, process, and analyze the historical performance data that has been saved within archived log files from previous modeling efforts, nor the development of performance analysis system capable of providing detailed statistical analyses or visualizations of this performance data. Therefore, we’ve begun a joint effort to analyze the historical and current performance data on both GFDL and OLCF modeling systems. Here, we have developed a preliminary performance analysis system to captue, analyize, and visualize end-to-end in situ performance data and applied it to NOAA/GFDL’s post-processing system. Through the course of the project a number of performance improvements have been made, with the steps necessary to handle the expected volume of data (Big Data) likely to come out of the CMIP6 exercises (TB to PB) provided. We have also begun developing a statistical workload model that will allow modeling centers to determined their expected scientific throughput in the context of expected resource (computational and personnel) availability and accounting for typical workflow disruptions. This will allow them to develop a CMIP participation strategy and maximize their level of participation in the exercises. We expect this infrastructure, and the data it produces, to ultimately allow us to identify performance issues and workflow errors that exist across Earth system modeling centers, and possibly HPC modeling centers in general. Center-independent solutions to workflow or performance issues would help alleviate the computational burden for CMIP6+ modeling projects and should realize both labor and cost savings benefits throughout the CMIP community.

54 ENVIRONMENTAL SCIENCES↗

NASA NTRS2022

Information Fusion and Data Analytics for Human Lunar Exploration (CIF REPORT: Detailed PI Write-up)

The Information Fusion & Data Analytics (IFDA) project commenced in FY20, continued through FY21, and its final platform development phase continues in FY22. The objective remains the fusion and rapid accessibility of large quantities of disparate sourced human spaceflight data. IFDA is a platform tailored for NA (S&MA) to develop highly advanced operational data integration and analysis techniques. IFDA leverages the JSC ER7 modeling, simulation,and data fusion capabilities to collect, warehouse, and augment data human exploration data integration and analysis techniques. The IFDA project’s integrated data visualizations have been demonstrated in two validation scenarios in FY21, and provided the architecture and platform basis for development of a full-scale data analysis suite and storage solution useful to all JSC organizations engaged in real time operations and safety tasks. Scenarioand prototypical development including the construction of a full scale data analysis suite and storage solution, useful to all JSC organizations engaged in real time operations and safety tasks, is central to IFDA Phase 3 and provides a demonstrable pathway for the Digital Transformation Program. IFDA Phase 3 is focused on data provider, data utilizer, and SME hands-on workshops that will conclude the Dem / Valphase and deliver a program-ready data integration tool as a product.

information fusion↗

NASA NTRS2022

Information Fusion & Analytics for Human Lunar Exploration

information fusion↗

DOE OSTI2014

RCSB PDB Mobile : iOS and Android mobile apps to provide data access and visualization to the RCSB Protein Data Bank

The Research Collaboratory for Structural Bioinformatics Protein Data Bank (RCSB PDB) resource provides tools for query, analysis and visualization of the 3D structures in the PDB archive. As the mobile Web is starting to surpass desktop and laptop usage, scientists and educators are beginning to integrate mobile devices into their research and teaching. In response, we have developed the RCSB PDB Mobile app for the iOS and Android mobile platforms to enable fast and convenient access to RCSB PDB data and services. Lastly, using the app, users from the general public to expert researchers can quickly search and visualize biomolecules, and add personal annotations via the RCSB PDB's integrated MyPDB service.

59 BASIC BIOLOGICAL SCIENCES↗

NASA NTRS1994

Interactive Visualization of Computational Fluid Dynamics using Mosaic

The Web provides new Methods for accessing Information world-wide, but the current text-and-pictures approach neither utilizes all the Web's possibilities not provides for its limitations. While the inclusion of pictures and animations in a paper communicates more effectively than text alone, It Is essentially an extension of the concept of "publication." Also, as use of the Web increases putting images and animations online will quickly load even the "Information Superhighway." We need to find forms of communication that take advantage of the special nature of the Web. This paper presents one approach: the use of the Internet and the Mosaic interface for data sharing and collaborative analysis. We will describe (and In the presentation, demonstrate) our approach: using FAST (Flow Analysis Software Toolkit), a scientific visualization package, as a data viewer and interactive tool called from MOSAIC. Our intent is to stimulate the development of other tools that utilize the unique nature of electronic communication.

Clucas, Jean↗

NASA NTRS2020

An Update on Global Satellite-Based Precipitation Products and Services at NASA GES DISC

The NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) is home to major NASA satellite precipitation measurement missions including the Tropical Rainfall Measuring Mission (TRMM) and the Global Precipitation Measurement (GPM) as well as other NASA projects. Accurate and timely available global and regional precipitation products play an important role in research and applications around the world. The GES DISC provides near-real-time, near-global precipitation products (e.g.IMERG) to support a wide variety of interdisciplinary research and operational activities including flood modeling, landslides, crop monitoring and assessment, vector-borne diseases, etc. Climate data record products (e.g.GPCP3) are essential for climate research, assessment, model evaluation and applications. To facilitate data access and exploration, the GES DISC has developed data services such asGiovanni, an online visualization and analysis tool for easy access to over 2000 satellite- and model-based variables. Established in the mid 1980s, the GES DISC also distributes data in other disciplines including hydrology, atmospheric chemistry, atmospheric dynamics, etc. In this presentation, we will present an update on global precipitation products and data services including the new IMERG V06B suite, the latest version of GPCP (Version 3) and value-added data subsetting services (L34RS, L2S).

Liu, Zhong↗

DOE OSTI2016

In Situ Visualization and Production of Extract Databases

Simulations running at high concurrency on HPC systems generate large volumes of data that are impractical to write to disk due to time and storage constraints. Applications often adapt by saving data infrequently, resulting in datasets with poor temporal resolution. This can make datasets difficult to interpret during post hoc visualization and analysis, or worse, it can lead to lost science. In Situ visualization and analysis can enable efficient production of small data products such as rendered images or surface extracts that consist of polygonal geometry plus fields. These data products are far smaller than their source data and can be processed much more economically in a traditional post hoc workflow using far fewer computational resources. We used the SENSEI and Libsim in situ infrastructures to implement rendering workflow and surface data extraction workflows in the AVF-LESLIE combustion code. Finally, these workflows were then demonstrated at high levels of concurrency and showed significant data reductions and limited impact on the simulation runtime.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

A Science Gateway for the Repeatable Analysis of Machine Learning Predicted Gravity Anomalies

In recent years, deep learning has become an increasingly popular alternative for modeling in geoscience applications due to its scalability and efficiency. However, the interpretability, compute, data volume, and hyperparameter tuning requirements of deep learning models make development and monitoring difficult. Furthermore, model explainability and communicating results obtained by these models to users or domain experts is a challenge, as domain experts in geoscience also need to have a deep understanding of how those models function in order to support their scientific works. Here, we describe a science gateway and machine learning pipeline for predicting gravity anomalies from geophysical data. The gateway, built on open-source technologies, provides a holistic view of the pipeline through interactive visualizations aimed at enabling efficient exploratory data analysis. The repeatability, reproducibility, and monitoring capabilities of this overall system allow us to iterate and analyze at scale. Using this pipeline and gateway, we can repeatedly produce accurate high-resolution gravity anomaly datasets. By describing the underlying technologies, implementation, and results, here we provide a foundation for the broader adoption of science gateways into cross-cutting geoscience and machine learning research projects as a means to improve the scientific discovery and collaboration in the geophysics and computational sciences community.

58 GEOSCIENCES↗

NASA NTRS2014

Proto-Examples of Data Access and Visualization Components of a Potential Cloud-Based GEOSS-AI System

Once a research or application problem has been identified, one logical next step is to search for available relevant data products. Thus, an early component of a potential GEOSS-AI system, in the continuum between observations and end point research, applications, and decision making, would be one that enables transparent data discovery and access by users. Such a component might be effected via the systems data agents. Presumably, some kind of data cataloging has already been implemented, e.g., in the GEOSS Common Infrastructure (GCI). Both the agents and cataloging could also leverage existing resources external to the system. The system would have some means to accept and integrate user-contributed agents. The need or desirability for some data format internal to the system should be evaluated. Another early component would be one that facilitates browsing visualization of the data, as well as some basic analyses.Three ongoing projects at the NASA Goddard Earth Sciences Data and Information Services Center (GES DISC) provide possible proto-examples of potential data access and visualization components of a cloud-based GEOSS-AI system. 1. Reorganizing data archived as time-step arrays to point-time series (data rods), as well as leveraging the NASA Simple Subset Wizard (SSW), to significantly increase the number of data products available, at multiple NASA data centers, for production as on-the-fly (virtual) data rods. SSWs data discovery is based on OpenSearch. Both pre-generated and virtual data rods are accessible via Web services. 2. Developing Web Feature Services to publish the metadata, and expose the locations, of pre-generated and virtual data rods in the GEOSS Portal and enable direct access of the data via Web services. SSW is also leveraged to increase the availability of both NASA and non-NASA data.3.Federating NASA Giovanni (Geospatial Interactive Online Visualization and Analysis Interface), for multi-sensor data exploration, that would allow each cooperating data center, currently the NASA Distributed Active Archive Centers (DAACs), to configure its own Giovanni deployment, while also allowing all the deployments to incorporate each others data. A federated Giovanni comprises Giovanni Virtual Machines, which can be run on local servers or in the cloud.

access↗

DOE OSTI2019

Parallel streaming between heterogeneous HPC resources for real-time analysis

Performing analysis or generating visualizations concurrently with high performance simulations can yield great benefits compared to post-processing data. Writing and reading large volumes of data can be reduced or eliminated, thereby producing an I/O cost savings. One such method for concurrent simulation and analysis is in transit - streaming data from the resource running the simulation to a separate resource running the analysis. In transit analysis can be beneficial since computational resources may not have certain resources needed for visualization and analysis (e.g. GPUs) and to reduce the impact of performing analysis tasks to the run time of the simulation. When sending and receiving data in transit, data redistribution mechanisms are needed in order to support heterogeneous data layouts that may be required by the simulation and analysis applications. The work described in this paper compares two mechanisms for on-the-fly data redistribution when streaming data in parallel between two distributed memory applications. Finally, our results show that it is often more advantageous to stream data in the same layout as the sender and redistribute data amongst processes on the receiving end than to stream data in the final layout needed by the receiver.

97 MATHEMATICS AND COMPUTING↗

NASA NTRS2011

Online Time Series Analysis of Land Products over Asia Monsoon Region via Giovanni

Time series analysis is critical to the study of land cover/land use changes and climate. Time series studies at local-to-regional scales require higher spatial resolution, such as 1km or less, data. MODIS land products of 250m to 1km resolution enable such studies. However, such MODIS land data files are distributed in 10ox10o tiles, due to large data volumes. Conducting a time series study requires downloading all tiles that include the study area for the time period of interest, and mosaicking the tiles spatially. This can be an extremely time-consuming process. In support of the Monsoon Asia Integrated Regional Study (MAIRS) program, NASA GES DISC (Goddard Earth Sciences Data and Information Services Center) has processed MODIS land products at 1 km resolution over the Asia monsoon region (0o-60oN, 60o-150oE) with a common data structure and format. The processed data have been integrated into the Giovanni system (Goddard Interactive Online Visualization ANd aNalysis Infrastructure) that enables users to explore, analyze, and download data over an area and time period of interest easily. Currently, the following regional MODIS land products are available in Giovanni: 8-day 1km land surface temperature and active fire, monthly 1km vegetation index, and yearly 0.05o, 500m land cover types. More data will be added in the near future. By combining atmospheric and oceanic data products in the Giovanni system, it is possible to do further analyses of environmental and climate changes associated with the land, ocean, and atmosphere. This presentation demonstrates exploring land products in the Giovanni system with sample case scenarios.

Shen, Suhung↗

NASA NTRS1995

The CCD/Transit Instrument (CTI) data-analysis system

The automated software system for archiving, analyzing, and interrogating data from the CCD/Transit Instrument (CTI) is described. The CTI collects up to 450 Mbytes of image-data each clear night in the form of a narrow strip of sky observed in two colors. The large data-volumes and the scientific aims of the project make it imperative that the data are analyzed within the 24-hour period following the observations. To this end a fully automatic and self evaluating software system has been developed. The data are collected from the telescope in real-time and then transported to Tucson for analysis. Verification is performed by visual inspection of random subsets of the data and obvious cosmic rays are detected and removed before permanent archival is made to the optical disc. The analysis phase is performed by a pair of linked algorithms, one operating on the absolute pixel-values and the other on the spatial derivative of the data. In this way both isolated and merged images are reliably detected in a single pass. In order to isolate the latter algorithm from the effects of noise spikes a 3x3 Hanning filter is applied to the raw data before the analysis is run. The algorithms reduce the input pixel-data to a database of measured parameters for each image which has been found. A contrast filter is applied in order to assign a detection-probability to each image and then x-y calibration and intensity calibration are performed using known reference stars in the strip. These are added to as necessary by secondary standards boot-strapped from the CTI data itself. The final stages involve merging the new data into the CTI Master-list and History-list and the automatic comparison of each new detection with a set of pre-defined templates in parameter-space to find interesting objects such as supernovae, quasars and variable stars. Each stage of the processing from verification to interesting image selection is performed under a data-logging system which both controls the pipe-lining of data through the system and records key performance monitor parameters which are built into the software. Furthermore, the data from each stage are stored in databases to facilitate evaluation, and all stages offer the facility to enter keyword-indexed free-format text into the data-logging system. In this way a large measure of certification is built into the system to provide the necessary confidence in the end results.

Cawson, M. G. M.↗