Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “knowledge discovery”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

High-performance analysis of filtered semantic graphs

High performance is a crucial consideration when executing a complex analytic query on a massive semantic graph. In a semantic graph, vertices and edges carry "attributes" of various types. Analytic queries on semantic graphs typically depend on the values of these attributes; thus, the computation must either view the graph through a filter that passes only those individual vertices and edges of interest, or else must first materialize a subgraph or subgraphs consisting of only the vertices and edges of interest. The filtered approach is superior due to its generality, ease of use, and memory efficiency, but may carry a performance cost. In the Knowledge Discovery Toolbox (KDT), a Python library for parallel graph computations, the user writes filters in a high-level language, but those filters result in relatively low performance due to the bottleneck of having to call into the Python interpreter for each edge. In this work, we use the Selective Embedded JIT Specialization (SEJITS) approach to automatically translate filters defined by programmers into a lower-level efficiency language, bypassing the upcall into Python. We evaluate our approach by comparing it with the high-performance C++ /MPI Combinatorial BLAS engine, and show that the productivity gained by using a high-level filtering language comes without sacrificing performance.

97 MATHEMATICS AND COMPUTING↗

Big Data as a Service from an Urban Information System

Big Data has already proven itself as a valuable tool that lets geographers and urban researchers utilize large data re- sources to generate new insights. However, wider adoption of Big Data techniques in these areas is impeded by a number of difficulties in both knowledge discovery and data and sci- ence production. Typically users face such problems as dis- parate and scattered data, data management, spatial search- ing, insufficient computational capacity for data-driven anal- ysis and modelling, and the lack of tools to quickly visual- ize and summarize large data and analysis results. Here we propose an architecture for an Urban Information Sys- tem (UrbIS) that mitigates these problems by utilizing the Big Data as a Service (BDaaS) concept. With technolog- ical roots in High-performance Computing (HPC), BDaaS is based on the idea of outsourcing computations to dif- ferent computing paradigms, scalable to super-computers. UrbIS aims to incorporate federated metadata search, in- tegrated modeling and analysis, and geovisualization into a single seamless workflow. The system is under active devel- opment and is built around various emerging technologies that include hybrid and NoSQL databases, massively par- allel systems, GPGPU computing, and WebGL-based geo- graphic visualization. UrbIS is designed to facilitate using Big Data across multiple cities to better understand how ur- ban areas impact the environment and how climate change and other environmental change impact urban areas.

Sorokine, Alexandre↗

Using Digital Trace Data to Identify Regions and Cities

A greater understanding of human dynamics as they play out in both physical space and through inter-personal communication is vital for the design and development of intelligent and resilient cities. Physical context provides insight in to the space-time distribution of population and their activity patterns, while inter-personal communication can, for the first time, be measured at scale through virtual interactions. The confluence of these two enables spatial social network analysis in the built environment for edge computing, knowledge discovery, and artificial intelligence activities. In this work, we propose a novel method to discover such dynamics. We use a dataset of 72 million tweets to develop a spatially embedded network of communication, and then use community detection algorithms to explore regional and urban delineation in the United States. We find that the broad spatial delineation of communities and community sub-structure is consistent with major United States socio-cultural regions, states, and major metropolitan areas. We describe how these methods could be extended to generate a measure of socially cohesive regions that can be consistently applied anywhere there is a sufficiently rich data source. We hope the proposed methods will generate new perspectives on smart and resilient cities.

Brelsford, Christa↗

Scalable Causal Graph Learning through a Deep Neural Network

Learning the causal graph in a complex system is crucial for knowledge discovery and decision making, yet it remains a challenging problem because of the unknown nonlinear interaction among system components. Most of the existing methods either rely on predefined kernel or data distribution, or they focus simply on the causality between a single target and the remaining system. This work presents a deep neural network for scalable causal graph learning (SCGL) through low-rank approximation. The SCGL model can explore nonlinearity on both temporal and intervariable relationships without any predefined kernel or distribution assumptions. Through low-rank approximation, the noise influence is reduced, and better accuracy and high scalability are achieved. Experiments using synthetic and real-world datasets show that our SCGL algorithm outperforms existing state-of-the-art methods for causal graph learning.

97 MATHEMATICS AND COMPUTING↗

Fragile Earth: AI for Climate Mitigation, Adaptation, and Environmental Justice

The Fragile EarthWorkshop is a recurring event that gathers the research community to find and explore howdata science can measure and progress climate and social issues, following the framework of the United Nations Sustainable Development Goals (SDGs).Fragile Earth 2022: AI for Climate Mitigation, Adaptation, and Environmental Justice is a workshop taking place as part of the ACM's KDD 2022 Conference on research in knowledge discovery and data mining and their applications. The dates for the Conference are August 14-18, 2022.

Abe, Naoki↗

Essential Function Analysis Capability

The Essential Function Analysis Capability (EFAC) is an extension of the All Hazards Knowledge Framework (AHA) and is a dynamic analytical framework that enables critical infrastructure knowledge discovery and decision support across the five mission areas – prevention, protection, mitigation, response and recovery. EFAC provides the ability to store and model infrastructure systems as a linked multigraphs providing an intuitive and natural representation. These infrastructure systems can then be tied to the organizational and essential function breakdown of an entity. This capability provides the foundation to rapidly evaluate and understand the potential consequences of manmade and natural disaster on these infrastructure systems and essential functions.

Hoover, MichaelJ.↗

pnnl/HypoGen

An innovative workflow for automatic knowledge discovery in material science, combining experimental automation with advanced machine learning and artificial intelligence techniques

Yin, Tim [Pacific Northwest National Laboratory (P↗

All Hazards Analysis Core

The All Hazards Knowledge Framework (AHA) is a dynamic analytical framework that enables critical infrastructure knowledge discovery and decision support across the five mission areas – prevention, protection, mitigation, response and recovery. AHA provides the ability to store and model infrastructure systems as a linked multigraphs providing an intuitive and natural representation. This capability provides the foundation to rapidly evaluate and understand the potential consequences of manmade and natural disaster on these systems.

McGillivary, KentE.↗

Augmented cognition tool for rapid military decision making.

This report describes the laboratory directed research and development work to model relevant areas of the brain that associate multi-modal information for long-term storage for the purpose of creating a more effective, and more automated, association mechanism to support rapid decision making. Using the biology and functionality of the hippocampus as an analogy or inspiration, we have developed an artificial neural network architecture to associate k-tuples (paired associates) of multimodal input records. The architecture is composed of coupled unimodal self-organizing neural modules that learn generalizations of unimodal components of the input record. Cross modal associations, stored as a higher-order tensor, are learned incrementally as these generalizations form. Graph algorithms are then applied to the tensor to extract multi-modal association networks formed during learning. Doing so yields a novel approach to data mining for knowledge discovery. This report describes the neurobiological inspiration, architecture, and operational characteristics of our model, and also provides a real world terrorist network example to illustrate the model's functionality.

99 GENERAL AND MISCELLANEOUS↗

US Woody Crop Yield Potential Database Documentation with Referenced Yield Summary Tables

A woody crop yield potential (WCYP) database was created containing yield results with as much associated information as was available concerning the sites, soils, and experimental treatments. The database summarizes a very broad set of old and new standing biomass data from plantation-grown hardwoods and softwoods established under a wide range of conditions across the United States and Canada. The WCYP database, together with this document, is being published to disseminate information on what is available in the literature with respect to yield evaluations and to inform people that not all yield data in the open literature are suitable for evaluation of “potential” regional yields. It has also been presented with the hope that much more information will be collected and added to the WCYP database. The current set of data in the WCYP database is derived from 41 unique reference sources describing 79 experiments in the United States or southern Canada with a total of 473 different treatments. The database contains several worksheets. The worksheet that reports the complete yield curves contains roughly 1,660 rows of data and 81 columns of parameters reporting yields and associated treatment factors. Separate worksheets, linked by the source publication author and year, describe the sites and soils associated with each experiment; another separate worksheet assembles only the final yields deemed appropriate for yield potential analysis. There are additional worksheets with conversion factors relevant to the database and notes pertaining to decisions made regarding organization of the data. This technical manuscript describes how the information was assembled to create the WCYP database that is simultaneously being published on the US Department of Energy (DOE) Bioenergy Knowledge Discovery Framework website maintained by Oak Ridge National Laboratory (ORNL). A summary table with links to all worksheets in the database can be found at the web address https://www.bioenergykdf.net/WCYP .

09 BIOMASS FUELS↗

Analytics for Cyber Network Defense

This report provides a brief survey of analytics tools considered relevant to cyber network defense (CND). Ideas and tools come from elds such as statistics, data mining, and knowledge discovery. Some analytics are considered standard mathematical or statistical techniques, while others re ect current research directions. In all cases the report attempts to explain the relevance to CND with brief examples.

97 MATHEMATICS AND COMPUTING↗

SciDAC Institute for Ultra-Scale Visualization: Activity Recognition for Ultra-Scale Visualization

Understanding the science behind ultra-scale simulations requires extracting meaning from data sets of hundreds of terabytes or more. Developing scalable parallel visualization algorithms is a key step enabling scientists to interact and visualize their data at this scale. However, at extreme scales, the datasets are so huge, there is not even enough time to view the data, let alone explore it with basic visualization methods. Automated tools are necessary for knowledge discovery -- to help sift through the information and isolate characteristic patterns, thereby enabling the scientist to study local interactions, the origin of features and their evolution in large volumes of data. These tools must be able to operate on data of this scale and work with the visualization process. In this project, we developed a framework for activity detection to allow scientists to model and extract spatio-temporal patterns from time-varying data.

97 MATHEMATICS AND COMPUTING↗

Towards Exascale: High Performance Visualization and Analytics – Project Status Report

The LBNL Visualization research program focuses on a key set of problems facing our community as we move towards the exascale regime of computing. First, we aim to better understand how to effectively take advantage of an evolving architectural landscape where there are an increasing number of cores per processor by examining algorithmic reformulation and alternative methods of programming these increasingly complex platforms. Second, as the gap widens between computational and I/O capacity, it is increasingly imperative to perform more visual data exploration and analysis operations concurrent with the simulation to avoid increasingly expensive I/O. Our approach is to examine several interrelated issues aimed at minimizing the memory footprint, data movement, and design/implementation issues related to deploying tools that can have broad applicability across many different simulation codes and science domains. Third, increasing computational power produces larger and more complex scientific data, and our research program focuses on new technologies aimed at enabling scientific data understanding with ever-larger data. Here, we explore how multi-resolution data representation techniques can reduce data movement, and study novel approaches that focus visualization and analysis processing on subsets of data that are “scientifically interesting,” thereby also minimizing data movement while accelerating scientific knowledge discovery.

97 MATHEMATICS AND COMPUTING↗

Board on Research Data and Information

The Board on Research Data and Information (BRDI) has planned and undertaken numerous activities regarding data citation, attribution, management, policy, publishing, centers, access, curation, sharing, and infrastructure; and international collaboration and cooperation. Some of these activities resulted in National Research Council reports (For Attribution: Developing Data Attribution and Citation Practices and Standards (2012), The Case for International Scientific Data Sharing: A Focus on Developing Countries (2012), and The Future of Scientific Knowledge Discovery in Open Networked Environments (2012); and a peer-reviewed paper (Out of Cite, Out of Mind: The Current State of Practice, Policy, and Technology for the Citation of Data, 2013). BRDI held symposia, workshops and sessions in the U.S. and abroad on diverse topics such as global scientific data infrastructures, discovery of data online, privacy in a big data world, and data citation principles, among other timely data-related subjects. In addition, BRDI effects the representation of the United States before the International Council for Science’s International Committee on Data for Science and Technology (CODATA).

99 GENERAL AND MISCELLANEOUS↗

7th Annual Earth System Grid Federation Face-to-Face Conference Report

The Seventh Annual Earth System Grid Federation (ESGF) Face-to-Face (F2F) Conference held December 4–8, 2017, in San Francisco, California, USA, assembled together a collection of independently funded national and international projects comprised of government agencies, institutions, and companies dedicated to the creation, management, analysis, and distribution of extreme-scale scientific data. The purpose of the conference was to discuss sustaining and enhancing the resilient ESGF data infrastructure with friendlier tools for the expanding global scientific community—this year’s emphasis was placed on the preparedness of the Coupled Model Intercomparison Project, phase 6 (CMIP6). It also focused on new tools that fulfill important and strategic capability gaps in scientific data archiving, access, analysis, and knowledge discovery. As the Executive Committee Chair of ESGF, I would like to personally thank each of conference attendees and those who could not attend but contributed to and/or supported the development of the ESGF software stack. It is an exciting time for the ESGF consortium as we continue to grow and adjust, remaining always adaptable, motivated, and responsive to our growing base of community projects. As we move forward, our ESGF organization is confronting and addressing many changes during a time of larger national and international commitment with fewer community resources. That said, our commitment to our sponsors and the community remains strong as we continue to meet the challenges before us and bring inspired developers and the scientific community together through forums like this conference, ensuring our ESGF organization remains robust and at the cutting edge of technology.

58 GEOSCIENCES↗

Advanced Visualization for Scientific Data Analysis and Insight [Slides]

This talk will explore how we have used advanced visualization technologies to support analytical reasoning and knowledge discovery. Specifically, we will present several examples detailing some recent scientific successes using state-of-the-art immersive and high-resolution visualization at the National Renewable Energy Laboratory's Computational Science Center. On multiple occasions, we have observed scientists and engineers discover features in their data using advanced visualization technologies that they had not seen in prior investigations of their data on traditional desktop displays. We have embedded more information into our analytics tools, allowing engineers to explore complex multivariate spaces. We have observed how interactions seem to catalyze understanding.

97 MATHEMATICS AND COMPUTING↗

ETDEWEB versus the World-Wide-Web: a specific database/web comparison

A study was performed comparing user search results from the specialized scientific database on energy-related information, ETDEWEB, with search results from the internet search engines Google and Google Scholar. The primary objective of the study was to determine if ETDEWEB (the Energy Technology Data Exchange – World Energy Base) continues to bring the user search results that are not being found by Google and Google Scholar. As a multilateral information exchange initiative, ETDE’s member countries and partners contribute cost- and task-sharing resources to build the largest database of energy-related information in the world. As of early 2010, the ETDEWEB database has 4.3 million citations to world-wide energy literature. One of ETDEWEB’s strengths is its focused scientific content and direct access to full text for its grey literature (over 300,000 documents in PDF available for viewing from the ETDE site and over a million additional links to where the documents can be found at research organizations and major publishers globally). Google and Google Scholar are well-known for the wide breadth of the information they search, with Google bringing in news, factual and opinion-related information, and Google Scholar also emphasizing scientific content across many disciplines. The analysis compared the results of 15 energy-related queries performed on all three systems using identical words/phrases. A variety of subjects was chosen, although the topics were mostly in renewable energy areas due to broad international interest. Over 40,000 search result records from the three sources were evaluated. The study concluded that ETDEWEB is a significant resource to energy experts for discovering relevant energy information. For the 15 topics in this study, ETDEWEB was shown to bring the user unique results not shown by Google or Google Scholar 86.7% of the time. Much was learned from the study beyond just metric comparisons. Observations about the strengths of each system and factors impacting the search results are also shared along with background information and summary tables of the results. If a user knows a very specific title of a document, all three systems are helpful in finding the user a source for the document. But if the user is looking to discover relevant documents on a specific topic, each of the three systems will bring back a considerable volume of data, but quite different in focus. Google is certainly a highly-used and valuable tool to find significant ‘non-specialist’ information, and Google Scholar does help the user focus on scientific disciplines. But if a user’s interest is scientific and energy-specific, ETDEWEB continues to hold a strong position in the energy research, technology and development (RTD) information field and adds considerable value in knowledge discovery. (auth)

99 GENERAL AND MISCELLANEOUS↗