TAU Analytica: Bringing Advanced Data Analytics to Performance Analysis
Explore the source record for details and available documents.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Explore the source record for details and available documents.
Explore the source record for details and available documents.
Abstract not provided.
Abstract not provided.
Abstract not provided.
Abstract not provided.
Explore the source record for details and available documents.
Abstract not provided.
Explore the source record for details and available documents.
US-UK Energy Research Collaboration Workshop, Pittsburgh, PA, April 19-20, 2023
Every 18 or 24 months nuclear power plants (depending on plant configuration, pressurized or boiling water reactor respectively) undergo a period of outage where the plant is taken offline and a large number of maintenance and surveillance activities (that cannot be performed while plant is running) are performed in typically 2–3 weeks. Planning of a plant outage is very challenging since all the activities are required to be performed in the shortest amount of time given available resources (typically contractor crews hired for the duration of the outage). Consequently, plant outages can be costly due the actual loss of power generation and crew costs and, because of it, there is a need to maximize resource usage in the outage planning phase and reduce the risk of outage delays. This paper is addressing these needs by providing a set of analytical methods designed to analyze plant outage schedule and identify critical elements based on available resources (time and crews). These methods are based on natural language processing and optimization algorithms. In this respect, two classes of methods have been developed: one that focuses on the time resource and how variability in the time to complete outage tasks may impact outage delays, and one that minimizes the risk of outage delays by integrating available resources to assess when daily activities should be performed.
As computational models scale to larger computers, the rate at which they produce data has far outstripped the same computers ability to write that data and further the file systems ability to store that data. Almost all of the SciDAC applications, but especially those related to fusion solve very large scale PDEs whose scientific output his impacted by this problem. To gain access to dynamics in an exascale simulation that are not identifiable a priori and to make that dynamical data available to machine learning requires fundamental research in the area of in situ data data analytics. Here data analytics includes compression, visualization, uncertainty quantification, and machine learning. This in situ data analytics will enable on-the-fly spatial and temporal compression of solution dynamics, expose that space-time compressed field to machine learning algorithms that have been specialized to work with dynamically evolving data (existing machine learning algorithms treat data sets as static), greatly improving the opportunity for machine learning to provide feedback to the compression, all within an ongoing simulation, without the need to write data to files. The same concepts are also being applied to uncertainty quantification and multi-fidelity modeling which have similar needs for spatial and temporal compression of the ongoing exascale simulation to perform either without the typical, unacceptable writing of data to files.
The Renewable Energy (RE) Data Explorer is an innovative and intuitive web platform from the U.S. Agency for International Development (USAID)-National Renewable Energy Laboratory (NREL) Partnership that enables users to easily access high-quality renewable energy resource data and applied analytics.
Given the sheer volume of scientific data archived within the data-intensive projects at the US Department of Energy's Oak Ridge National Laboratory, finding precisely what data we are looking for may not be a trivial task; conversely, we may also miss a more prominent data product. To address such issues, we propose improving the data discovery system and using data analytics methods to comprehend what specific users might be interested in based on their physiological state, search patterns, and past data usage history. This work's primary goal is to prune the complexity, increase the visibility of popular data products, and direct users toward the data that best meet their needs. The proposed algorithm constructs a user profile based on the user's explicit or implicit interactions with the system, such as items they are currently looking at on-site and the key metadata mappings related to the data set. The pattern is then used to build a training data set, which will help find relevant data to recommend to the user.
In-situ and in-transit processing alleviate the gap between the computing and I/O capabilities by scheduling data analytics close to the data source. Hybrid in-situ processing splits data analytics into two stages: the data processing that runs in-situ aims to extract regions of interest, which are then transferred to staging services for further in-transit analytics. To facilitate this type of hybrid in-situ processing, the data staging service needs to support complex intermediate data representations generated/consumed by the in-situ tasks. Unstructured (or irregular) mesh is one such derived data representation that is typically used and bridges simulation data and analytics. However, how staging services efficiently support unstructured mesh transfer and processing remains to be explored. This paper investigates design options for transferring and processing unstructured mesh data using staging services. Using polygonal mesh data as an example, we show that hybrid in-situ workflows with staging-based unstructured mesh processing can effectively support hybrid in-situ workflows, and can significantly decrease data movement overheads.
We present Phoenix, a scalable hypergraph analytics framework for data analytics and knowledge discovery that was implemented on the leadership class computing platforms at Oak Ridge National Laboratory (ORNL). Our software framework comprises a distributed implementation of a streaming server architecture which acts as a gateway for various hypergraph generators/external sources to connect. Phoenix has the capability to utilize diverse hypergraph generators, including HyGen, a very large-scale hypergraph generator developed by ORNL. Phoenix incorporates specific algorithms for efficient data representation by exploiting hidden structures of the hypergraphs. Our experimental results demonstrate Phoenix’s scalable and stable performance on massively parallel computing platforms. Phoenix’s superior performance is due to the merging of high-performance computing with data analytic.
Nowadays, transmission system operators require higher degree of observability in real-time to gain situational awareness and improve the decision-making process to guarantee a safe and reliable operation. Digitalization of energy systems allows utilities to monitor the system dynamic performance in real-time at fast time scales. The use of such technologies has unlocked new opportunities to introduce new data driven algorithms for improving the stability assessment and control of the system. Motivated by these challenges, a group of experts have worked together to highlight and establish a baseline set of these common concerns, which can be used as motivation to propose innovative analytics and data-driven solutions. In this document, the results of a survey on 10 transmission system operators around the world are presented and it aims to understand the current practices of the participating companies, in terms of data acquisition, handling, storage, modelling and analytics. The overall objective of this document is to capture the actual needs from the interviewed utilities, thereby laying the groundwork for setting valid assumptions for the development of advanced algorithms in this field.