Accelerating Scientific Data Analytics with SK hynix KV-CSD
Explore the source record for details and available documents.
Computer Science↗
SEARCH · Engineering Papers
Results for “data analytics”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Special Session on Cutting Edge Approaches in Data Analytics for Nonproliferation.
Abstract not provided.
Anderson, Dylan↗
Testing and Evaluation of Data Analytic Approaches for Nonproliferation
Explore the source record for details and available documents.
Anderson, Dylan↗
Creation of the XMAT Database & Leveraging Data Analytics
US-UK Energy Research Collaboration Workshop, Pittsburgh, PA, April 19-20, 2023
Wenzlick, Madison↗
Data Analytics Methods to Measure Plant Outage Resilience
Every 18 or 24 months nuclear power plants (depending on plant configuration, pressurized or boiling water reactor respectively) undergo a period of outage where the plant is taken offline and a large number of maintenance and surveillance activities (that cannot be performed while plant is running) are performed in typically 2–3 weeks. Planning of a plant outage is very challenging since all the activities are required to be performed in the shortest amount of time given available resources (typically contractor crews hired for the duration of the outage). Consequently, plant outages can be costly due the actual loss of power generation and crew costs and, because of it, there is a need to maximize resource usage in the outage planning phase and reduce the risk of outage delays. This paper is addressing these needs by providing a set of analytical methods designed to analyze plant outage schedule and identify critical elements based on available resources (time and crews). These methods are based on natural language processing and optimization algorithms. In this respect, two classes of methods have been developed: one that focuses on the time resource and how variability in the time to complete outage tasks may impact outage delays, and one that minimizes the risk of outage delays by integrating available resources to assess when daily activities should be performed.
97 - MATHEMATICS AND COMPUTING↗
Interactive Data Visualization
Interactive data visualization leverages human visual perception and cognition to improve the accuracy and effectiveness of data analysis. When combined with automated data analytics, data visualization systems orchestrate the strengths of humans with the computational power of machines to solve problems neither approach can manage in isolation. In the intelligent transportation system domain, such systems are necessary to support decision making in large and complex data streams. In this chapter, we provide an introduction to several key topics related to the design of data visualization systems. In addition to an overview of key techniques and strategies, we will describe practical design principles. The chapter is concluded with a detailed case study involving the design of a multivariate visualization tool.
information visualization↗
Sediment Properties: E-Area Completion Project
To accommodate a future need for additional waste disposal facilities at the Savannah River Site, the Solid Waste Management Division (SWMD) designated nine additional plots for development (Kasraii 2007; SRS 2010); these plots are collectively known as the E Area Completion Project (ECP). Subsurface samples were collected from ECP plots 6, 7, 8 and 9 (Figure 1) for chemical and physical property analyses to support Performance Assessment (PA) and Special Analyses (SA) modeling. This document summarizes the sampling and analysis scheme and the resultant data, and provides interpretations of the data particularly in reference to existing soil property data. Analytical data in this document include: gamma log, cone penetrometer log, grain size (sieve and hydrometer), water retention, saturated hydraulic conductivity (falling head permeameter), porosity, dry bulk density, total organic carbon, x-ray diffraction, and x-ray fluorescence data. SRNL provided technical and safety oversight for the fieldwork, which included completion of eight soil borings, four geophysical logs, and the collection of 522 feet of core and 33 Shelby tubes from ECP plots 6, 7, 8, and 9. Boart Longyear provided sonic drilling and logging services. Two soil borings were completed at each location. The first set of boreholes extended into (but did not fully penetrate) the Warley Hill Formation. These boreholes were continuously cored, then geophysically (gamma ray) logged. The recovered core was split, photographed, and described; one half of the core was archived at SRS's Core Lab facilities, and the remaining half was consumed as necessary for testing at SRS and off-site labs. Core descriptions and geophysical data were used to calculate target elevations for Shelby tube samples, which were obtained from the second set of boreholes. Shelby tubes were shipped to MACTEC Engineering and Consulting Inc. (MACTEC) in Atlanta for physical property testing. SRNL deployed their Site Characterization and Analysis Penetrometer System (SCAPS) cone penetrometer test (CPT) truck at ECP plots 6, 7, 8 and 9 to collect inferred lithology data for the vadose zone. Results from this study are used to make recommendations for future modeling efforts involving the ECP plots. The conceptual model of the ECP hydrogeology differs from the conceptual model of the current ELLWF disposal area in that for the ECP plots, the topography (ground surface) is generally lower in elevation; The Upland and top of Tobacco Road lithostratigraphic units are missing (eroded); The water table occurs lower in elevation (i.e., it occurs in lower stratigraphic units); and the Tan Clay Confining Zone (TCCZ) often occurs within the vadose zone (rather than in the saturated zone). Due to the difference in the hydrogeology between the current ELLWF location and the ECP plots, different vadose zone properties are recommended for the ECP plots versus the properties recommended by Phifer et al. (2006) for the current disposal units. Results from this study do not invalidate or conflict with the current PA's use of the Upper and Lower Vadose Zone properties as described by Phifer et al. (2006) for the current ELLWF disposal units. The following modeling recommendations are made for future modeling of the ECP plots where vadose zone properties are required: (1) If a single vadose zone property is preferred, the properties described by Phifer et al. (2006) for the Upper Vadose Zone encompass the general physical properties of the combined sands and clays in the ECP vadose zone sediments despite the differences in hydrostratigraphic units. (2) If a dual zone system is preferred, a combination of the Lower Zone properties and the Clay properties described by Phifer et al. (2006) are appropriate for modeling the physical properties of the ECP vadose zone. The Clay properties would be assigned to the Tan Clay Confining Zone (TCCZ) and any other significant clay layers, while the Lower Zone properties would be assigned for the remainder of the vadose zone. No immediate updates or changes are recommended for the saturated zone properties for modeling of the ECP plots. The hydrogeology and saturated hydraulic conductivity values resulting from the ECP work are consistent with existing data for the area. The following recommendations are provided for consideration in future work: (1) When the database is next updated, we recommend checking/refining hydrostratigraphic interpretations (picks) in addition to revising physical property data for both the vadose zone and saturated zone. (2) Results from this work suggest that separate ECP vadose zone properties may be appropriate for the various ECP plots. Consideration should be given to the fact that the plots toward the northwest (ECP plots 6, 7 and 8) all have the TCCZ within the vadose zone whereas the TCCZ occurs within the saturated zone to the east (ECP plot 9). (3) Further data may be needed at ECP plot 7.
54 ENVIRONMENTAL SCIENCES↗
Frameworks, Algorithms, and Scalable Technologies for Mathematics (FASTMath) SciDAC Institute
As computational models scale to larger computers, the rate at which they produce data has far outstripped the same computers ability to write that data and further the file systems ability to store that data. Almost all of the SciDAC applications, but especially those related to fusion solve very large scale PDEs whose scientific output his impacted by this problem. To gain access to dynamics in an exascale simulation that are not identifiable a priori and to make that dynamical data available to machine learning requires fundamental research in the area of in situ data data analytics. Here data analytics includes compression, visualization, uncertainty quantification, and machine learning. This in situ data analytics will enable on-the-fly spatial and temporal compression of solution dynamics, expose that space-time compressed field to machine learning algorithms that have been specialized to work with dynamically evolving data (existing machine learning algorithms treat data sets as static), greatly improving the opportunity for machine learning to provide feedback to the compression, all within an ongoing simulation, without the need to write data to files. The same concepts are also being applied to uncertainty quantification and multi-fidelity modeling which have similar needs for spatial and temporal compression of the ongoing exascale simulation to perform either without the typical, unacceptable writing of data to files.
97 MATHEMATICS AND COMPUTING↗
Analytics-Driven Lossless Data Compression for Rapid In-situ Indexing, Storing, and Querying
The analysis of scientific simulations is highly data-intensive and is becoming an increasingly important challenge. Peta-scale data sets require the use of light-weight query-driven analysis methods, as opposed to heavy-weight schemes that optimize for speed at the expense of size. This paper is an attempt in the direction of query processing over losslessly compressed scientific data. We propose a co-designed double-precision compression and indexing methodology for range queries by performing unique-value-based binning on the most significant bytes of double precision data (sign, exponent, and most significant mantissa bits), and inverting the resulting metadata to produce an inverted index over a reduced data representation. Without the inverted index, our method matches or improves compression ratios over both general-purpose and floating-point compression utilities. The inverted index is light-weight, and the overall storage requirement for both reduced column and index is less than 135%, whereas existing DBMS technologies can require 200-400%. As a proof-of-concept, we evaluate univariate range queries that additionally return column values, a critical component of data analytics, against state-of-the-art bitmap indexing technology, showing multi-fold query performance improvements.
Jenkins, John↗
2015 HPC Annual Report
Center 9300 is very proud to provide a wide range of computing and networking resources ranging from desktop support to high performance computing that enable Sandia’s national security mission needs. Our goal is to provide these services in the most efficient, secure, user friendly, and agile means necessary. High performance computing is one example of how we are delivering on this goal. Thanks to investments from NNSA’s Advanced Simulation and Computing program, coupled with Sandia’s Institutional Computing program, a wide variety of computing resources are available and used to support national security needs. This year we are expanding our high performance computing infrastructure with the procurement of institutional resources for information computing or analytics. Big data analytics has been the buzzword in the computing industry for over a decade now and industries in finance, food and drug, business, and security organizations have been leading the charge. Much like how scientific computing has changed the way science and engineering are performed, data analytics is changing just about every aspect of our lives and more especially our ability to contribute to the national security mission of Sandia. This third edition of our HPC Annual Report continues our commitment to communicate the details and impact of Sandia’s large-scale computing resources that support missions work across Sandia. The Laboratories’ and NNSA’s strategic investments in high performance computing demonstrate our commitment to keeping mission-related research on the cutting edge of science, at pace with technology advances, and at a scale that meets the toughest demands of the nation we so proudly serve.
97 MATHEMATICS AND COMPUTING↗
RE Data Explorer is Expanding Public Access to High-Quality Data and Analytics
The Renewable Energy (RE) Data Explorer is an innovative and intuitive web platform from the U.S. Agency for International Development (USAID)-National Renewable Energy Laboratory (NREL) Partnership that enables users to easily access high-quality renewable energy resource data and applied analytics.
ENERGY PLANNING, POLICY, AND ECONOMY↗
Clustering-Based Predictive Analytics to Improve Scientific Data Discovery
Given the sheer volume of scientific data archived within the data-intensive projects at the US Department of Energy's Oak Ridge National Laboratory, finding precisely what data we are looking for may not be a trivial task; conversely, we may also miss a more prominent data product. To address such issues, we propose improving the data discovery system and using data analytics methods to comprehend what specific users might be interested in based on their physiological state, search patterns, and past data usage history. This work's primary goal is to prune the complexity, increase the visibility of popular data products, and direct users toward the data that best meet their needs. The proposed algorithm constructs a user profile based on the user's explicit or implicit interactions with the system, such as items they are currently looking at on-site and the key metadata mappings related to the data set. The pattern is then used to build a training data set, which will help find relevant data to recommend to the user.
Devarakonda, Ranjeet↗
Democratizing an electroluminescence imaging apparatus and analytics project for widespread data acquisition in photovoltaic materials
In this work, we present a description of an electroluminescence (EL) apparatus, easily sourced from commercially available components, with a quantitative image processing platform that demonstrates feasibility for the widespread utility of EL imaging as a characterization tool. We validated our system using a Gage R&R analysis to fnd a variance contribution by the measurement system of 80.56%, which is typically unacceptable, but through quantitative image processing and development of correction factors a variance contribution by the measurement system of 2.41% was obtained. We further validated the system by quantifying the signal-to-noise ratio (SNR) and found values consistent with other systems published in the literature, at SNR values of 10-100, albeit at exposure times of greater than 1 s compared to 10 ms for other systems. Finally, this SNR value range is acceptable for image feature recognition, providing the opportunity for widespread data acquisition and large scale data analytics of photovoltaics.
14 SOLAR ENERGY↗
Technical Advances for Sustainable Remediation: In Situ Monitoring and Climate Resiliency - 18436
Recent advances in sustainable remediation techniques show promise as solutions for contaminant removal in soil and groundwater media. Although passive remediation (or natural attenuation) has gained momentum as an acceptable technique, there are still challenges in regulatory frameworks as well as in public acceptance. Leaving contaminants in subsurface requires increased burden of proof to show that plumes are stable and residual contaminants do not pose a significant health risk. Climate changes and frequent extreme climate events contribute another level of complexity and make it difficult to demonstrate system stability and predict future plume conditions. In this study, we show two coupled approaches to tackle this challenge: (1) in situ groundwater monitoring based on advanced sensor and data analytics technologies (i.e., Big Data), and (2) state-of-art modeling for optimizing monitoring and site closure strategies. In the new monitoring approach, in situ sensors are used to continuously measure the key variables that control contaminant plume mobility and its spatial and temporal distribution (such as pH, redox potential, electrical conductivity, and groundwater level). Coupled with a limited number of groundwater sampling, the data analytics methods - data mining and machine learning - allow us to estimate contaminant concentrations, and also to detect any significant changes associated with the plume mobility. In parallel, the state-of-art numerical flow and reactive transport model - ASCEM software under DOE Office of Environmental Management - provides a physical and mechanistic understanding of the system behavior, and predicts the long-term plume migration for optimizing the monitoring and closure strategies. In particular, we focus on the effects of extreme precipitation or drought on the site closure as well as monitoring strategies. We demonstrate this approach at the Savannah River Site (SRS) F-Area, where groundwater is contaminated by various radionuclides, including uranium, tritium and technetium. (authors)
12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
National Geothermal Data System: Transforming the Discovery, Access, and Analytics of Data for Geothermal Exploration
Compendium of Papers from the 38th Workshop on Geothermal Reservoir Engineering Stanford University, Stanford, California February 11-13, 2013 The National Geothermal Data System (NGDS) is a distributed, interoperable network of data collected from state geological surveys across all fifty states and the nation’s leading academic geothermal centers. The system serves as a platform for sharing consistent, reliable, geothermal-relevant technical data with users of all types, while supplying tools relevant for their work. As aggregated data supports new scientific findings, this content-rich linked data ultimately broadens the pool of knowledge available to promote discovery and development of commercial-scale geothermal energy production. Most of the up-front risks associated with geothermal development stem from exploration and characterization of subsurface resources. Wider access to distributed data will, therefore, result in lower costs for geothermal development. NGDS is on track to become fully operational by 2014 and will provide a platform for custom applications for accessing geothermal relevant data in the U.S. and abroad. It is being built on the U.S. Geoscience Information Network (USGIN) data integration framework to promote interoperability across the Earth sciences community. The basic structure of the NGDS employs state-of-the art informatics to advance geothermal knowledge. The following four papers comprising this Open-File Report are a compendium of presentations, from the 38th Annual Workshop on Geothermal Reservoir Engineering, taking place February 11-13, 2013 at Stanford University, Stanford, California. “NGDS Geothermal Data Domain: Assessment of Geothermal Community Data Needs,” outlines the efforts of a set of nationwide data providers to supply data for the NGDS. In particular, data acquisition, delivery, and methodology are discussed. The paper addresses the various types of data and metadata required and why simple links to existing data are insufficient for promoting geothermal exploration. Authors of this paper are Arlene Anderson, US DOE Geothermal Technologies Office, David Blackwell, Southern Methodist University (SMU), Cathy Chickering (SMU), Toni Boyd, Oregon Institute of Technology’s GeoHeat Center, Roland Horne, Stanford University, Matthew MacKenzie, Uberity, Joe Moore, University of Utah, Duane Nickull, Uberity, Stephen Richard, Arizona Geological Survey, and Lisa Shevenell, University of Nevada, Reno. “NGDS User Centered Design: Meeting the Needs of the Geothermal Community,” discusses the user- centered design approach taken in the development of a user interface solution for the NGDS. The development process is research based, highly collaborative, and incorporates state-of-the-art practices to ensure a quality user interface for the widest and greatest utility. Authors of this paper are Harold Blackman, Boise State University, Suzanne Boyd, Anthro-Tech, Kim Patten, Arizona Geological Survey, and Sam Zheng, Siemens Corporate Research. “Fueling Innovation and Adoption by Sharing Data on the DOE Geothermal Data Repository Node on the National Geothermal Data System,” describes the motivation behind the development of the Geothermal Data Repository (GDR) and its role in the NGDS. This includes the benefits of using the GDR to share geothermal data of all types and DOE’s data submission process. Authors of this paper are Jon Weers, National Renewable Energy Laboratory and Arlene Anderson, US DOE Geothermal Technologies Office. Finally, “Developing the NGDS Adoption of CKAN for Domestic & International Data Deployment,” provides an overview of the “Node-In-A-Box” software package designed to provide data consumers with a highly functional interface to access the system, and to ease the burden on data providers who wish to publish data in the system. It is important to note that this software package constitutes a reference implementation and that the NGDS architecture is based on open standards, which means other server software can make resources available, and other client applications can utilize NGDS data. Authors of this paper are Ryan Clark, Arizona Geological Survey (AZGS), Christoph Kuhmuench, Siemens Corporate Research, and Stephen Richard, AZGS.
15 GEOTHERMAL ENERGY↗
Facilitating Staging-based Unstructured Mesh Processing to Support Hybrid In-Situ Workflows
In-situ and in-transit processing alleviate the gap between the computing and I/O capabilities by scheduling data analytics close to the data source. Hybrid in-situ processing splits data analytics into two stages: the data processing that runs in-situ aims to extract regions of interest, which are then transferred to staging services for further in-transit analytics. To facilitate this type of hybrid in-situ processing, the data staging service needs to support complex intermediate data representations generated/consumed by the in-situ tasks. Unstructured (or irregular) mesh is one such derived data representation that is typically used and bridges simulation data and analytics. However, how staging services efficiently support unstructured mesh transfer and processing remains to be explored. This paper investigates design options for transferring and processing unstructured mesh data using staging services. Using polygonal mesh data as an example, we show that hybrid in-situ workflows with staging-based unstructured mesh processing can effectively support hybrid in-situ workflows, and can significantly decrease data movement overheads.
data-driven↗
Phoenix: A Scalable Streaming Hypergraph Analysis Framework
We present Phoenix, a scalable hypergraph analytics framework for data analytics and knowledge discovery that was implemented on the leadership class computing platforms at Oak Ridge National Laboratory (ORNL). Our software framework comprises a distributed implementation of a streaming server architecture which acts as a gateway for various hypergraph generators/external sources to connect. Phoenix has the capability to utilize diverse hypergraph generators, including HyGen, a very large-scale hypergraph generator developed by ORNL. Phoenix incorporates specific algorithms for efficient data representation by exploiting hidden structures of the hypergraphs. Our experimental results demonstrate Phoenix’s scalable and stable performance on massively parallel computing platforms. Phoenix’s superior performance is due to the merging of high-performance computing with data analytic.
Kurte, Kuldeep↗
State-of-the-art of data collection, analytics, and future needs of transmission utilities worldwide to account for the continuous growth of sensing data
Nowadays, transmission system operators require higher degree of observability in real-time to gain situational awareness and improve the decision-making process to guarantee a safe and reliable operation. Digitalization of energy systems allows utilities to monitor the system dynamic performance in real-time at fast time scales. The use of such technologies has unlocked new opportunities to introduce new data driven algorithms for improving the stability assessment and control of the system. Motivated by these challenges, a group of experts have worked together to highlight and establish a baseline set of these common concerns, which can be used as motivation to propose innovative analytics and data-driven solutions. In this document, the results of a survey on 10 transmission system operators around the world are presented and it aims to understand the current practices of the participating companies, in terms of data acquisition, handling, storage, modelling and analytics. The overall objective of this document is to capture the actual needs from the interviewed utilities, thereby laying the groundwork for setting valid assumptions for the development of advanced algorithms in this field.
24 POWER TRANSMISSION AND DISTRIBUTION↗