Eigenanalysis of Space Weather Climate Data: Discovering Structure within Large Multivariate Data Sets
No abstract available
Krause, Linda Habash↗
SEARCH · Engineering Papers
Results for “data discover”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Using ADOPT Algorithm and Operational Data to Discover Precursors to Aviation Adverse Events
The US National Airspace System (NAS) is making its transition to the NextGen system and assuring safety is one of the top priorities in NextGen. At present, safety is managed reactively (correct after occurrence of an unsafe event). While this strategy works for current operations, it may soon become ineffective for future airspace designs and high density operations. There is a need for proactive management of safety risks by identifying hidden and "unknown" risks and evaluating the impacts on future operations. To this end, NASA Ames has developed data mining algorithms that finds anomalies and precursors (high-risk states) to safety issues in the NAS. In this paper, we describe a recently developed algorithm called ADOPT that analyzes large volumes of data and automatically identifies precursors from real world data. Precursors help in detecting safety risks early so that the operator can mitigate the risk in time. In addition, precursors also help identify causal factors and help predict the safety incident. The ADOPT algorithm scales well to large data sets and to multidimensional time series, reduce analyst time significantly, quantify multiple safety risks giving a holistic view of safety among other benefits. This paper details the algorithm and includes several case studies to demonstrate its application to discover the "known" and "unknown" safety precursors in aviation operation.
aviation safet↗
Metadata Authoring with Versatility and Extensibility
NASA's Global Change Master Directory (GCMD) assists the scientific community in the discovery of and linkage to Earth science data sets and related services. The GCMD holds over 13,800 data set descriptions in Directory Interchange Format (DIF) and 700 data service descriptions in Service Entry Resource Format (SERF), encompassing the disciplines of geology, hydrology, oceanography, meteorology, and ecology. Data descriptions also contain geographic coverage information and direct links to the data, thus allowing researchers to discover data pertaining to a geographic location of interest, then quickly acquire those data. The GCMD strives to be the preferred data locator for world-wide directory-level metadata. In this vein, scientists and data providers must have access to intuitive and efficient metadata authoring tools. Existing GCMD tools are attracting widespread usage; however, a need for tools that are portable, customizable and versatile still exists. With tool usage directly influencing metadata population, it has become apparent that new tools are needed to fill these voids. As a result, the GCMD has released a new authoring tool allowing for both web-based and stand-alone authoring of descriptions. Furthermore, this tool incorporates the ability to plug-and-play the metadata format of choice, offering users options of DIF, SERF, FGDC, ISO or any other defined standard. Allowing data holders to work with their preferred format, as well as an option of a stand-alone application or web-based environment, docBUlLDER will assist the scientific community in efficiently creating quality data and services metadata.
Pollack, Janine↗
In Interactive, Web-Based Approach to Metadata Authoring
NASA's Global Change Master Directory (GCMD) serves a growing number of users by assisting the scientific community in the discovery of and linkage to Earth science data sets and related services. The GCMD holds over 8000 data set descriptions in Directory Interchange Format (DIF) and 200 data service descriptions in Service Entry Resource Format (SERF), encompassing the disciplines of geology, hydrology, oceanography, meteorology, and ecology. Data descriptions also contain geographic coverage information, thus allowing researchers to discover data pertaining to a particular geographic location, as well as subject of interest. The GCMD strives to be the preeminent data locator for world-wide directory level metadata. In this vein, scientists and data providers must have access to intuitive and efficient metadata authoring tools. Existing GCMD tools are not currently attracting. widespread usage. With usage being the prime indicator of utility, it has become apparent that current tools must be improved. As a result, the GCMD has released a new suite of web-based authoring tools that enable a user to create new data and service entries, as well as modify existing data entries. With these tools, a more interactive approach to metadata authoring is taken, as they feature a visual "checklist" of data/service fields that automatically update when a field is completed. In this way, the user can quickly gauge which of the required and optional fields have not been populated. With the release of these tools, the Earth science community will be further assisted in efficiently creating quality data and services metadata. Keywords: metadata, Earth science, metadata authoring tools
Pollack, Janine↗
Data and Reasoning Fabric (DRF)
There are multiple opportunities to utilize Data & Reasoning Fabric technologies in wildfire response: (1) Supporting autonomous agents to discover data sources and transact with systems, (2) Fusing data in real-time, and (3) Supporting complex operations in a dynamic real-time environment .
Aeronautics↗
Exploring and Visualizing A-Train Instrument Data
The succession of US and international satellites that follow each other in close succession, known as the A-Train, affords an opportunity to atmospheric researchers that no single platform could provide: Increasing the number of observations at any given geographic location.. . a more complete "virtual science platform". However, vertically and horizontally, co-registering and regridding datasets from independently developed missions, Aqua, Calipso, Cloudsat, Parasol, and Aura, so that they can be inter-compared can be daunting to some, and may be repeated by many. Scientists will individually spend much of their time and resources acquiring A-Train datasets of interest residing at various locations, developing algorithms to match up and graph datasets along the A-Train track, and search through large amounts of data for areas and/or phenomena of interest. The aggregate amount of effort that can be expended on repeating pre-science tasks could climb into the tens of millions of dollars. The goal of the A-Train Data Depot (ATDD) is to enable free movement of remotely located A-Train data so that they are combined to create a consolidated vertical view of the Earth's Atmosphere along the A-Train tracks. The innovative approach of analyzing and visualizing atmospheric profiles along the platforms track (i.e., time) is accomplished by through the ATDDs Giovanni data analysis and visualization tool. Giovanni brings together data from Aqua (MODIS, AIRS, AMSR-E), Cloudsat (cloud profiling radar) and Calipso (CALIOP, IIR), as well as the Aura (OMI, MLS, HIRDLS, TES) to create a consolidated vertical view of the Earth's Atmosphere along the A-Train tracks. This easy to learn and use exploration tool will allow users to create vertical profiles of any desired A-Train dataset, for any given time of choice. This presentation shows the power of Giovanni by describing and illustrating how this tool facilitates and aids A-Train science and research. A web based display system Giovanni provides users with the capability of creating co-located profile images of temperature and humidity data from the MODIS, MLS and AIRS instruments for a user specified time and spatial area. In addition, Cloud and Aerosol profiles may also be displayed for the Cloudsat and Caliop instruments. The ability to modify horizontal and vertical axis range, data range and dynamic color range is also provided. Two dimensional strip plots of MODIS, AIRS, OM1 and POLDER parameters, co-located along the Cloudsat reference track, can also be plotted along with the Cloudsat cloud profiling data. Center swath pixels for the same parameters can also be shown as line plots overlaying the Cloudsat or Calipso profile images. Images and subsetted data produced in each analysis run may be downloaded. Users truly can explore and discover data specific to their needs prior to ever transferring data to their analysis tools.
Kempler, S.↗
Restructuring Big Data to Improve Data Access and Performance in Analytic Services Making Research More Efficient for the Study of Extreme Weather Events and Application User Communities
By developing and enhancing various services and tools, the GES DISC provides users with the capability to access and visualize data, and to make comparisons of data from multiple sensor and models via a number of cross-discipline projects. Discovering Data via Faceted Web Interface Web interface to data products and services Search and Download mechanisms Dataset Landing Pages Accessing Data through Interoperable Services: GDS – GrADS Data Server OPeNDAP - Open-source Project for a Network Data Access Protocol WMS – OGC service GIS connector – allowing IS tools to access data easier (coming soon) HTTPS -- direct online access Downloading Data Basics: Subset and egridding Service – Parameter, Spatial, Time, Vertical, Mean averaging, format conversion, and regridding for L3/L4 gridded data Swath Data Subsetter – Parameter, spatial subset of L2 /L1 data. Visualizing Data Online: Giovanni –Visualization and Analysis L3/L4 gridded data AIRS NRT Viewer – AIRS near-real-time DQVis – L2 data quality visualization
data cube↗
Utilization of Geographic Information System in Lunar Mapping
Substantial digital remote sensing, lunar orbital photography, Earth-based remote sensing, and mapping of a variety of surficial lunar phenomena have occurred since the advent of the Space Age. This has led to a bewildering and quite disparate collection of archival sources insofar as this digital data and its cartographic representation can be found within many countries of the world. The importance of this mapping program in support of human expansion onto our nearest planetary neighbor has been recognized. A series of small scale maps of the Moon at 1 km to 1 cm, done with the support of Geographic Information System (GIS), would serve decision makers well in the process of accessing the development of manned occupance of the Moon. Maps and the data that they are derived from are the primary way in which people explore new environments and use previously discovered data to increase the bounties of any exploration. The inherent advantage of GIS is that it would allow immediate online access on the Moon of topographically represented data with analysis either on site or from Earth.
Mardon, A. A.↗
Jupiter: Atmospheric Sounding and Sensing of the Interior (JASSI)
The formation of the giant planets is one of the most fundamental questions in solar system exploration. Understanding the process that led to the creation of Jupiter is essential to understanding the nature of the primordial solar nebula, and the formation of our solar system and others currently being discovered. Data from Galileo combined with HST and Ulysses results validated our basic understanding of Jupiter as a giant planet whose gaseous envelope consists of solar nebula gas enriched in elements heavier than He by in falling icy planetesimals. However, the current Galileo Probe data set does not itself allow firm conclusions about the original planetesimal composition or the process of giant planet formation - we crucially need the O and N abundances that Galileo could not determine. We propose a new and simple concept capable of determining these abundance in Jupiter plus substantial gravity science. Additional information is contained in the original extended abstract.
Bolton, S. J.↗
Smart Hand Offs and Earthdata Search
Often a science user will discover data of interest in a general-purpose discovery tool like Earthdata Search. At that point it might be beneficial for the science user to switch to a more specialized tool. Smart handoffs allow the user to switch from one tool to another with their context (collection, spatial and temporal constraints) intact. This provides an efficient means of traversing tools and services. The idea of smart handoffs could be expanded to commercial search engines such as Google, visualization tools like State of the Ocean and Giovanni and domain-specific search tools like Earthdata Search.
Giovanni↗
DIP Overview
NASA will visit Boeing in Fairfax, VA to share an overview of the Digital Information Platform (DIP) and the digital aviation services that can be enabled from the data discovered on the platform. The goal is to walk though the goals, objectives and concept as well as the problem statements under consideration for a flight deck service to improve airspace operational efficiency for long-haul oceanic flights. Potential digital service capabilities will be discussed with goal to solicit input from Boeing attendees.
Digital Information Platform↗
Data Tips: Learn How to Discover, Access and Analyze NASA GES DISC Data
At the NASA Goddard Earth Sciences Data and Information Services (GES DISC), we strive to simplify data discovery and data access to our wide range of global climate data, concentrated primarily in the areas of atmospheric composition, atmospheric dynamics, global precipitation, solar irradiance, and several modeling data sets related to land surface hydrology. To help meet user needs, we will demonstrate how you can use the GES DISC knowledge-base resources (HowTo's) and we also encourage community contributions.
data tips↗
Automated Discovery and Modeling of Sequential Patterns Preceding Events of Interest
The integration of emerging data manipulation technologies has enabled a paradigm shift in practitioners' abilities to understand and anticipate events of interest in complex systems. Example events of interest include outbreaks of socio-political violence in nation-states. Rather than relying on human-centric modeling efforts that are limited by the availability of SMEs, automated data processing technologies has enabled the development of innovative automated complex system modeling and predictive analysis technologies. We introduce one such emerging modeling technology - the sequential pattern methodology. We have applied the sequential pattern methodology to automatically identify patterns of observed behavior that precede outbreaks of socio-political violence such as riots, rebellions and coups in nation-states. The sequential pattern methodology is a groundbreaking approach to automated complex system model discovery because it generates easily interpretable patterns based on direct observations of sampled factor data for a deeper understanding of societal behaviors that is tolerant of observation noise and missing data. The discovered patterns are simple to interpret and mimic human's identifications of observed trends in temporal data. Discovered patterns also provide an automated forecasting ability: we discuss an example of using discovered patterns coupled with a rich data environment to forecast various types of socio-political violence in nation-states.
Rohloff, Kurt↗
Using Social Media and Mobile Devices to Discover and Share Disaster Data Products Derived From Satellites
Data products derived from Earth observing satellites are difficult to find and share without specialized software and often times a highly paid and specialized staff. For our research effort, we endeavored to prototype a distributed architecture that depends on a standardized communication protocol and applications program interface (API) that makes it easy for anyone to discover and access disaster related data. Providers can easily supply the public with their disaster related products by building an adapter for our API. Users can use the API to browse and find products that relate to the disaster at hand, without a centralized catalogue, for example floods, and then are able to share that data via social media. Furthermore, a longerterm goal for this architecture is to enable other users who see the shared disaster product to be able to generate the same product for other areas of interest via simple point and click actions on the API on their mobile device. Furthermore, the user will be able to edit the data with on the ground local observations and return the updated information to the original repository of this information if configured for this function. This architecture leverages SensorWeb functionality [1] presented at previous IGARSS conferences. The architecture is divided into two pieces, the frontend, which is the GeoSocial API, and the backend, which is a standardized disaster node that knows how to talk to other disaster nodes, and also can communicate with the GeoSocial API. The GeoSocial API, along with the disaster node basic functionality enables crowdsourcing and thus can leverage insitu observations by people external to a group to perform tasks such as improving water reference maps, which are maps of existing water before floods. This can lower the cost of generating precision water maps. Keywords-Data Discovery, Disaster Decision Support, Disaster Management, Interoperability, CEOS WGISS Disaster Architecture
0↗
Data simulation for the Lightning Imaging Sensor (LIS)
This project aims to build a data analysis system that will utilize existing video tape scenes of lightning as viewed from space. The resultant data will be used for the design and development of the Lightning Imaging Sensor (LIS) software and algorithm analysis. The desire for statistically significant metrics implies that a large data set needs to be analyzed. Before 1990 the quality and quantity of video was insufficient to build a usable data set. At this point in time, there is usable data from missions STS-34, STS-32, STS-31, STS-41, STS-37, and STS-39. During the summer of 1990, a manual analysis system was developed to demonstrate that the video analysis is feasible and to identify techniques to deduce information that was not directly available. Because the closed circuit television system used on the space shuttle was intended for documentary TV, the current value of the camera focal length and pointing orientation, which are needed for photoanalysis, are not included in the system data. A large effort was needed to discover ancillary data sources as well as develop indirect methods to estimate the necessary parameters. Any data system coping with full motion video faces an enormous bottleneck produced by the large data production rate and the need to move and store the digitized images. The manual system bypassed the video digitizing bottleneck by using a genlock to superimpose pixel coordinates on full motion video. Because the data set had to be obtained point by point by a human operating a computer mouse, the data output rate was small. The loan and subsequent acquisition of a Abekas digital frame store with a real time digitizer moved the bottleneck from data acquisition to a problem of data transfer and storage. The semi-automated analysis procedure was developed using existing equipment and is described. A fully automated system is described in the hope that the components may come on the market at reasonable prices in the next few years.
Boeck, William L.↗
Engine Icing Data - An Analytics Approach
Engine icing researchers at the NASA Glenn Research Center use the Escort data acquisition system in the Propulsion Systems Laboratory (PSL) to generate and collect a tremendous amount of data every day. Currently these researchers spend countless hours processing and formatting their data, selecting important variables, and plotting relationships between variables, all by hand, generally analyzing data in a spreadsheet-style program (such as Microsoft Excel). Though spreadsheet-style analysis is familiar and intuitive to many, processing data in spreadsheets is often unreproducible and small mistakes are easily overlooked. Spreadsheet-style analysis is also time inefficient. The same formatting, processing, and plotting procedure has to be repeated for every dataset, which leads to researchers performing the same tedious data munging process over and over instead of making discoveries within their data. This paper documents a data analysis tool written in Python hosted in a Jupyter notebook that vastly simplifies the analysis process. From the file path of any folder containing time series datasets, this tool batch loads every dataset in the folder, processes the datasets in parallel, and ingests them into a widget where users can search for and interactively plot subsets of columns in a number of ways with a click of a button, easily and intuitively comparing their data and discovering interesting dynamics. Furthermore, comparing variables across data sets and integrating video data (while extremely difficult with spreadsheet-style programs) is quite simplified in this tool. This tool has also gathered interest outside the engine icing branch, and will be used by researchers across NASA Glenn Research Center. This project exemplifies the enormous benefit of automating data processing, analysis, and visualization, and will help researchers move from raw data to insight in a much smaller time frame.
Engine Icing↗
Data Mining and Knowledge Discover - IBM Cognitive Alternatives for NASA KSC
Skillful tools in cognitive computing to transform industries have been found favorable and profitable for different Directorates at NASA KSC. In this study is shown how cognitive computing systems can be useful for NASA when computers are trained in the same way as humans are to gain knowledge over time. Increasing knowledge through senses, learning and a summation of events is how the applications created by the firm IBM empower the artificial intelligence in a cognitive computing system. NASA has explored and applied for the last decades the artificial intelligence approach specifically with cognitive computing in few projects adopting similar models proposed by IBM Watson. However, the usage of semantic technologies by the dedicated business unit developed by IBM leads these cognitive computing applications to outperform the functionality of the inner tools and present outstanding analysis to facilitate the decision making for managers and leads in a management information system.
Cognitive computing↗
NASA's Global Change Master Directory: Discover and Access Earth Science Data Sets, Related Data Services, and Climate Diagnostics
NASA's Global Change Master Directory provides the scientific community with the ability to discover, access, and use Earth science data, data-related services, and climate diagnostics worldwide. The GCMD offers descriptions of Earth science data sets using the Directory Interchange Format (DIF) metadata standard; Earth science related data services are described using the Service Entry Resource Format (SERF); and climate visualizations are described using the Climate Diagnostic (CD) standard. The DIF, SERF and CD standards each capture data attributes used to determine whether a data set, service, or climate visualization is relevant to a user's needs. Metadata fields include: title, summary, science keywords, service keywords, data center, data set citation, personnel, instrument, platform, quality, related URL, temporal and spatial coverage, data resolution and distribution information. In addition, nine valuable sets of controlled vocabularies have been developed to assist users in normalizing the search for data descriptions. An update to the GCMD's search functionality is planned to further capitalize on the controlled vocabularies during database queries. By implementing a dynamic keyword "tree", users will have the ability to search for data sets by combining keywords in new ways. This will allow users to conduct more relevant and efficient database searches to support the free exchange and re-use of Earth science data. http://gcmd.nasa.gov/
Aleman, Alicia↗