Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Data Visualization and Analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

Visualizing Clustering and Uncertainty Analysis with Multivariate Longitudinal Data

Multivariate time-series datasets are intrinsic to the study of dynamic, naturalistic behavior, such as in the applications of finance and motion video analysis. Statistical models provide the ability to identify event patterns in these data under conditions of uncertainty, but researchers must be able to evaluate how well a model uses available information in a dataset for clustering decisions and for uncertainty information. The Hidden Markov Model (HMM) is an established method for clustering time-series data, where the hidden states of the HMM are the clusters. We develop novel methods for quantifying the uncertainty of the performance of and for visualizing the clustering performance and uncertainty of fitting a HMM to multivariate time-series data. We explain the usefulness of uncertainty quantification and visualization with evaluating the performance of clustering models, as well as how information exploitation of time-series datasets can be enhanced. We implement our methods to cluster patterns of scanpaths from raw eye tracking data.

97 MATHEMATICS AND COMPUTING↗

FREDA: A Web Application for the Processing, Analysis, and Visualization of Fourier‐Transform Mass Spectrometry Data

The high-resolution measurement capability of Fourier-transform mass spectrometry (FT-MS) has made it a necessity for exploring the molecular composition of complex organic mixtures, like soil, plant, aquatic, and petroleum samples. This demand has driven a need for informatics tools to explore and analyze FT-MS data in a robust and reproducible manner. FREDA is an interactive web application developed to enable spectrometrists to format, process, and explore their FT-MS data without the need for statistical programming expertise. FREDA was built to explore outputs from a molecular identification tool, like CoreMS, and provide a suite of methods to filter data, compute chemical properties of peaks, statistically compare samples and groups of samples, conduct exploratory data analysis, and download the results with a report detailing all steps conducted. To demonstrate the utility of FREDA, an example analysis was conducted using FT-MS data from a soil microbiology study of samples collected in two different soil depths at the Sphagnum bog forest north of Grand Rapids, Minnesota. Differences between the two depths are observed using Kendrick, Gibbs free energy, and van Krevelen plots. G-tests are used to quantify a significant difference between the groups. All analyses and plotting are conducted using only the FREDA application. FREDA is an open-source and readily available web application that allows users to explore and make statistically valid conclusions about their FT-MS data. The application is available online (https://map.emsl.pnnl.gov/app/freda) with a tutorial web series (https://youtu.be/k5HLE2kNSBY?si=yB6sGoyvzxrFf5MP) and freely accessible code on Github (https://github.com/EMSL-Computing/FREDA).

47 OTHER INSTRUMENTATION↗

Streaming Data in HPC Workflows Using ADIOS

The “IO Wall” problem, in which the gap between computation rate and data access rate grows continuously, poses significant problems to scientific workflows which have traditionally relied upon using the filesystem for intermediate storage between workflow stages. One way to avoid this problem in scientific workflows is to stream data directly from producers to consumers and avoiding storage entirely. However, the manner in which this is accomplished is key to both performance and usability. This paper presents the Sustainable Staging Transport, an approach which allows direct streaming between traditional file writers and readers with few application changes. SST is an ADIOS “engine”, accessible via standard ADIOS APIs, and because ADIOS allows engines to be chosen at run-time, many existing file-oriented ADIOS workflows can utilize SST for direct application-to-application communication without any source code changes. This paper describes the design of SST and presents performance results from various applications that use SST, for feeding model training with simulation data with substantially higher bandwidth than the theoretical limits of Frontier’s file system, for strong coupling of separately developed applications for multiphysics multiscale simulation, or for in situ analysis and visualization of data to complete all data processing shortly after the simulation finishes.

Podhorszki, Norbert [ORNL] (ORCID:000000019647542X↗

pyCroscopy

Suite of Python Libraries that implement structure data format processing, multivariate analysis, and interactive visualization of microscopy data and other spectral imaging modalities, including scanning probe microscopy, electron microscopy, Bragg diffraction microscopy, and chemical imaging.

Jesse, Stephen↗

pyCroscopy

Suite of Python Libraries that implement structure data format processing, multivariate analysis, and interactive visualization of microscopy data and other spectral imaging modalities, including scanning probe microscopy, electron microscopy, Bragg diffraction microscopy, and chemical imaging.

Jesse, Stephen↗

VisIO: enabling interactive visualization of ultra-scale, time-series data via high-bandwidth distributed I/O systems

Petascale simulations compute at resolutions ranging into billions of cells and write terabytes of data for visualization and analysis. Interactive visuaUzation of this time series is a desired step before starting a new run. The I/O subsystem and associated network often are a significant impediment to interactive visualization of time-varying data; as they are not configured or provisioned to provide necessary I/O read rates. In this paper, we propose a new I/O library for visualization applications: VisIO. Visualization applications commonly use N-to-N reads within their parallel enabled readers which provides an incentive for a shared-nothing approach to I/O, similar to other data-intensive approaches such as Hadoop. However, unlike other data-intensive applications, visualization requires: (1) interactive performance for large data volumes, (2) compatibility with MPI and POSIX file system semantics for compatibility with existing infrastructure, and (3) use of existing file formats and their stipulated data partitioning rules. VisIO, provides a mechanism for using a non-POSIX distributed file system to provide linear scaling of 110 bandwidth. In addition, we introduce a novel scheduling algorithm that helps to co-locate visualization processes on nodes with the requested data. Testing using VisIO integrated into Para View was conducted using the Hadoop Distributed File System (HDFS) on TACC's Longhorn cluster. A representative dataset, VPIC, across 128 nodes showed a 64.4% read performance improvement compared to the provided Lustre installation. Also tested, was a dataset representing a global ocean salinity simulation that showed a 51.4% improvement in read performance over Lustre when using our VisIO system. VisIO, provides powerful high-performance I/O services to visualization applications, allowing for interactive performance with ultra-scale, time-series data.

97 MATHEMATICS AND COMPUTING↗

Guiding the choice of informatics software and tools for lipidomics research applications

Progress in mass spectrometry lipidomics has led to a rapid proliferation of studies across biology and biomedicine. These generate extremely large raw datasets requiring sophisticated solutions to support automated data processing. To address this, numerous software tools have been developed and tailored for specific tasks. However, for researchers, deciding which approach best suits their application relies on ad hoc testing, which is inefficient and time consuming. Here we first review the data processing pipeline, summarizing the scope of available tools. Next, to support researchers, LIPID MAPS provides an interactive online portal listing open-access tools with a graphical user interface. This guides users towards appropriate solutions within major areas in data processing, including (1) lipid-oriented databases, (2) mass spectrometry data repositories, (3) analysis of targeted lipidomics datasets, (4) lipid identification and (5) quantification from untargeted lipidomics datasets, (6) statistical analysis and visualization, and (7) data integration solutions. Detailed descriptions of functions and requirements are provided to guide customized data analysis workflows.

59 BASIC BIOLOGICAL SCIENCES↗

Automation of Data Collection Methods for Online Monitoring of Nuclear Power Plants

Maintenance activities in the United States nuclear power industry are mainly centered on preventive and corrective maintenance. Predictive maintenance, which is a variable time basis maintenance strategy that is dependent on the condition of the equipment, is less utilized but potentially more effective. In the current maintenance practices, the nuclear power plants rely on manual processes to supply information to a human decision making process. The plants have several processes that generate information that is not typically used beyond the intended target for collecting that information. This information is a source of data that is manually collected by labor intensive tasks. The data is therefore expensive to collect, yet limited in use. The industry has realized the opportunity present in reducing the labor intensive tasks by automating them and increasing the uses of the data collected by either developing advanced predictive maintenance methods using data-driven decision making that could span from trending to machine learning and artificial intelligence. This shift in maintenance strategy may result in high cost saving impact. In addition to reducing labor requirements and frequency of maintenance activities, this shift in strategy will reduce the materials portion of the operations and maintenance cost as well. The aim of this specific effort is to focus on the automation of data collection process. It is part of a series of planned efforts to target multiple elements of migrating the current preventive maintenance approach in nuclear power plants to a data-driven approach. These elements include data collection, management, analysis, visualization, in addition to value analysis and change enabling. The other five elements of migration were not considered because they are planned to be explicitly studied in multiple parallel efforts. Challenges including regulations requirements or limitations are planned in a future effort that focuses on change enabling. The risk and benefit/cost of the approach shift is planned in another effort targeted towards value. The storage, communication, computational power, data architecture, and protection are part of the information infrastructure and are planned in dedicated data management effort. The best means to present the data for assisted human decision making is planned in a data visualization effort. The data interpretation and patterns recognition for machine decision making is planned through multiple efforts in data analytics over the following years. This work identified seventeen data sources collection methods in nuclear power plants that could be automated for an integrated data platform that enables comprehensive and informed decision making. For each of the identified seventeen data sources, the base, modern, and state of the art states were described. The current state of data collection used by most utilities was identified as the "base state". The current achievable state was identified as the "modern state", a state that has either been achieved by at least some US utilities or in other industries. The future state that requires research and development to reach was identified as the "state of the art" state. The described states used an out-of-the-box thinking approach. This effort found that there is a hug potential for task automation and data fidelity improvements by evolving the current industry data collection methods. The majority of the processes analyzed were found to be manual and labor intensive and fell in the base state. As a result, the modern state often targeted reducing the manual logging efforts by using electronic and semi-intelligent means or complementing the work with sensors and mobile or fixed technologies that can reduce the demand of labor activities. The state of the art state involved applications advanced methods of data correlation, data mining, and machine learning along with increasing the spatial distribution and fidelity of data, coupling multiple data sources, and enabling the use of sophisticated mobile devices and machine comprehension methods.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Energy-water interdependencies across the three major United States electric grids: A multi-sectoral analysis

As water availability and timing of delivery fluctuates and the US electric grid sees rapid transformation and reconfiguration under decarbonization and resource adequacy strategies, there is a critical need for information and data that supports understanding the water-energy interdependency landscape. The United States currently lacks comprehensive data, informative visualizations, and analysis of energy-water interdependencies at scales necessary to support resource and operational decision-making. This article provides US electricity interconnection-level Sankey diagrams that show the relative reliance of water and energy across various economic sectors. A deeper analysis is additionally provided at the county level to illustrate trends and potential opportunities related to resiliency and efficiency in multi-sectoral water and energy flow distributions and intensities. We find that the electricity interconnections in the US vary dramatically in their water and energy interdependencies across applications and economic sectors.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Phage Eco-Locator: a web tool for visualization and analysis of phage genomes in metagenomic data sets

Bacteriophages, viruses that infect bacteria, are the most abundant biological entities on our planet, and their nucleic acids constitute a substantial proportion of total DNA in Earth’s ecosystems. While the advent of metagenomic methods has allowed the rapid and efficient investigation of microbial and viral communities, there has not been a comprehensive comparative analysis of phage genes and genomes present in all sequenced ecosystems. To examine the abundance and distribution of phage genes in environmental metagenomic sequences, we developed a web-based tool, Phage Eco-Locator [http://www.phantome.org/eco-locator] that screens all publicly available sequenced metagenomes for a user-defined phage genome, or all phage genomes within a user-selected metagenomic sample.

59 BASIC BIOLOGICAL SCIENCES↗

Use of the Tool to Support Renewable Energy Auctions Processes (RE Data Explorer)

Renewable energy auctions are now a common competitive approach to procure low-cost renewable power around the world. Ensuring a successful auction process increasingly depends on the capabilities of auction designers and participants to identify actionable and defensible insights from large data sets (on renewable energy resources and complementary data) to both attract potential investors and address stakeholder concerns. The Renewable Energy (RE) Data Explorer is a user-friendly geospatial analysis tool for analyzing renewable energy potential and informing decisions. Developed by the National Renewable Energy Laboratory (NREL) and supported by the U.S. Agency for International Development (USAID), RE Data Explorer performs visualization and analysis of renewable energy potential that can be customized for different scenarios. RE Data Explorer can support prospecting, integrated planning, policymaking, and other decision-making activities to accelerate renewable energy deployment. The broader RE Explorer website provides guidance and information to link the RE Data Explorer geospatial analysis tool to key decision areas. This document provides information on how the RE Data Explorer can be used to support renewable energy auction processes.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Use of the Tool to Support Renewable Energy Project Development (RE Data Explorer)

The Renewable Energy (RE) Data Explorer is a user-friendly, geospatial analysis tool for analyzing renewable energy potential and informing decisions. Developed by the National Renewable Energy Laboratory (NREL) and supported by the U.S. Agency for International Development (USAID), RE Data Explorer performs visualization and analysis of renewable energy potential that can be customized for different scenarios. RE Data Explorer can support prospecting, integrated planning, policymaking, and other decision-making activities to accelerate renewable energy deployment. This document provides information on how the RE Data Explorer can support private sector utility-scale renewable energy project development.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

SSTDR and FDR Detection of Un-Energized and Energized Cable Anomalies Including Thermal Degradation Using Machine Learning

Historically, cables are initially qualified for nuclear power plant use for 40 years. As plants extend their operating license to 60 and 80 years, continued use of these cables must shift to a performance-based approach since it is cost prohibitive to completely replace cables that are likely still capable of performing their design function. A variety of cable tests are available and are commonly applied during outages when the cables can be taken out of service. Frequency domain reflectometry (FDR) is one of these test methods that is being more broadly accepted and used because it not only detects anomalies along the cable with a low-voltage signal that does not stress the cable insulation, but the technique also locates the anomalies. This supports follow-up local inspection and local repair or partial replacement of a damaged cable segment. Currently, FDR testing is only applied to cables that are taken out of service since the test instrument would be damaged by operational voltages. A related technology that has found some acceptance in the aircraft and rail industry is spread spectrum time domain reflectometry (SSTDR). This technology has been implemented with a custom commercial instrument by LiveWire Innovation that is designed to operate on live cables up to 1000 volts and with a bandwidth of 48 MHz. Initial evaluation by the Pacific Northwest National Laboratory (PNNL) of the Live Wire system indicated that a broader bandwidth (BW) SSTDR may be better for many kinds of flaws. This led PNNL to develop an SSTDR laboratory instrument suitable for tests up to 500 MHz bandwidth. Testing on energized cables is also desirable for online monitoring systems so an inductive clamshell coupler was developed that allows energized cables to be tested up to at least 5 kV and likely higher voltage levels. Dielectric spectroscopy and tan delta testing plus various laboratory destructive tests were included in this data acquisition campaign directed to feed a machine learning (ML) study. With these kinds of developments, online energized cable tests may be possible with industrial adoption of such hardware advances but it will be completely impractical to have highly skilled data analysts continually examine these complex signals for indications of damage or compromised conditions. If online testing is to be implemented in new test hardware, it must be accompanied by software that can interpret the signals and alert plant operators of changing or degraded conditions. The thermally aged, shielded cable investigated here was separately treated for ML analysis. Visual analysis of electrical data showed generally increasing peaks where the cable entered and exited the oven. These peaks were not exactly aligned with expected locations, but these differences were attributed to velocity of propagation calibration errors. Only supervised ML was applied to the thermally aged data as this data was only available shortly before the committed publication date of this report. The supervised ML was structured to divide the 0 to 70-day responses as ‘normal’ from 0 to 35 days or ‘anomalous’ from 36 to 70 days, based on cable tensile elongation at break (EAB) insulation characterization. Using 80% of the data for training and 20% for testing, the supervised ML predicted normal versus anomalous was 70% accurate. Important conclusions include: • Accuracy to predict the presence of cable damage is improved from the 2023 effort by more training data. Weighted accuracies for comparisons among the instruments ranged from 67 to 89 % for unsupervised ML and 71 to 99% for supervised ML. • Based on the synthetic data tests, the unsupervised models are more generalizable to unseen anomalies. The Multi-Layer Perceptron classifier (MLP) model reported as high as 99.7% accuracy on the test data, but this dropped to 58.3% when tested on the synthetic data. In contrast, the unsupervised Pointwise model only achieved 89.7% accuracy on the experimental data but reported 78.3% accuracy on the synthetic data. • The best anomaly indicators are higher frequency (400 MHz BW) FDR data. Other tests may be interesting but for this study, this was the best predicter.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Web-Based Visual Analytics for Social Media

Social media provides a rich source of data that reflects current trends and public opinion on a multitude of topics. The data can be harvested from Twitter, Facebook, Blogs, and other social applications. The high rate of adoption of social media has created a domain that has an ever expanding volume of data that make it difficult to use the raw data for analysis. Information visual analytics is key in drawing out features of interest in social media. The Scalable Reasoning System is an application that couples a back end server performing analysis algorithms and an intuitive front end visualization to allow for investigation. We provide a componentized system that can be rapidly adapted to customer needs such that the information they are most interested in is brought to their attention through the application. To this end, we have developed a social media application for use by emergency operations for the city of Seattle to show current weather and traffic trends which is important for their tasks.

99 GENERAL AND MISCELLANEOUS↗