Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Design Visualization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

A Visual Designer of Layer‐wise Relevance Propagation Models

Abstract Layer‐wise Relevance Propagation (LRP) is an emerging and widely‐used method for interpreting the prediction results of convolutional neural networks (CNN). LRP developers often select and employ different relevance backpropagation rules and parameters, to compute relevance scores on input images. However, there exists no obvious solution to define a “best” LRP model. A satisfied model is highly reliant on pertinent images and designers' goals. We develop a visual model designer, named as VisLRPDesigner, to overcome the challenges in the design and use of LRP models. Various LRP rules are unified into an integrated framework with an intuitive workflow of parameter setup. VisLRPDesigner thus allows users to interactively configure and compare LRP models. It also facilitates relevance‐based visual analysis with two important functions: relevance‐based pixel flipping and neuron ablation. Several use cases illustrate the benefits of VisLRPDesigner. The usability and limitation of the visual designer is evaluated by LRP users.

Huang, Xinyi↗
DOE OSTI2021

A Visual Designer of Layer-wise Relevance Propagation Models

Layer-wise Relevance Propagation (LRP) is an emerging and widely-used method for interpreting the prediction results of convolutional neural networks (CNN). LRP developers often select and employ different relevance backpropagation rules and parameters, to compute relevance scores on input images. However, there exists no obvious solution to define a "best" LRP model. A satisfied model is highly reliant on pertinent images and designers' goals. We develop a visual model designer, named as VisLRPDesigner, to overcome the challenges in the design and use of LRP models. Various LRP rules are unified into an integrated framework with an intuitive workflow of parameter setup. VisLRPDesigner thus allows users to interactively configure and compare LRP models. It also facilitates relevance-based visual analysis with two important functions: relevance-based pixel flipping and neuron ablation. Several use cases illustrate the benefits of VisLRPDesigner. The usability and limitation of the visual designer is evaluated by LRP users.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Experts’ Familiarity Versus Optimality of Visualization Design: How Familiarity Affects Perceived and Objective Task Performance

Experts, cutting across domains like biology, climate science, cyber, energy, etc., frequently use visualizations as the principal medium for making analytical judgments or for presenting the results of their analysis. However, scientists are often skeptical about adopting new visualization methods over familiar ones, although the latter might be perceptually sub-optimal. This is due to the use of the familiarity heuristic, where the perceived cognitive ease in processing familiar representations of information leads scientists to undermine the effect of visualization best practices. Recent studies have shown that this often results in a discrepancy between scientists’ perceived and actual performance quality. It has also been shown that in some cases, participatory design sessions and qualitative and quantitative user studies are able to mitigate the effects of such bias. In this paper, we discuss the potential causes and effects of familiarity related biases with examples from recent studies and reflect on the related research questions.

visualization, climate science, cognitive bias, fa↗
DOE OSTI2015

Bridging Theory with Practice: An Exploratory Study of Visualization Use and Design for Climate Model Comparison

Evaluation methodologies in visualization have mostly focused on how well the tools and techniques cater to the analytical needs of the user. While this is important in determining the effectiveness of the tools and advancing the state-of-the-art in visualization research, a key area that has mostly been overlooked is how well established visualization theories and principles are instantiated in practice. This is especially relevant when domain experts, and not visualization researchers, design visualizations for analysis of their data or for broader dissemination of scientific knowledge. There is very little research on exploring the synergistic capabilities of cross-domain collaboration between domain experts and visualization researchers. To fill this gap, in this paper we describe the results of an exploratory study of climate data visualizations conducted in tight collaboration with a pool of climate scientists. The study analyzes a large set of static climate data visualizations for identifying their shortcomings in terms of visualization design. The outcome of the study is a classification scheme that categorizes the design problems in the form of a descriptive taxonomy. The taxonomy is a first attempt for systematically categorizing the types, causes, and consequences of design problems in visualizations created by domain experts. We demonstrate the use of the taxonomy for a number of purposes, such as, improving the existing climate data visualizations, reflecting on the impact of the problems for enabling domain experts in designing better visualizations, and also learning about the gaps and opportunities for future visualization research. We demonstrate the applicability of our taxonomy through a number of examples and discuss the lessons learnt and implications of our findings.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2017

Human Factors in Streaming Data Analysis: Challenges and Opportunities for Information Visualization: Human Factors in Streaming Data Analysis

State-of-the-art visual analytics models and frameworks mostly assume a static snapshot of the data, while in many cases it is a stream with constant updates and changes. Exploration of streaming data poses unique challenges as machine-level computations and abstractions need to be synchronized with the visual representation of the data and the temporally evolving human insights. In the visual analytics literature, we lack a thorough characterization of streaming data and analysis of the challenges associated with task abstraction, visualization design, and adaptation of the role of human-in-the-loop for exploration of data streams. We aim to fill this gap by conducting a survey of the state-of-the-art in visual analytics of streaming data for systematically describing the contributions and shortcomings of current techniques and analyzing the research gaps that need to be addressed in the future. Our contributions are: i) problem characterization for identifying challenges that are unique to streaming data analysis tasks, ii) a survey and analysis of the state-of-the-art in streaming data visualization research with a focus on the visualization design space for dynamic data and the role of the human-in-the-loop, and iii) reflections on the design-trade-offs for streaming visual analytics techniques and their practical applicability in real-world application scenarios.

streaming data↗
DOE OSTI2019

Remote Sensor Design for Visual Recognition with Convolutional Neural Networks

While deep learning technologies for computer vision have developed rapidly since 2012, modeling of remote sensing systems has remained focused around human vision. In particular, remote sensing systems are usually constructed to optimize sensing cost-quality trade-offs with respect to human image interpretability. While some recent studies have explored remote sensing system design as a function of simple computer vision algorithm performance, there has been little work relating this design to the state-of-the-art in computer vision: deep learning with convolutional neural networks. We develop experimental systems to conduct this analysis, showing results with modern deep learning algorithms and recent overhead image data. Our results are compared to standard image quality measurements based on human visual perception, and we conclude not only that machine and human interpretability differ significantly, but that computer vision performance is largely self-consistent across a range of disparate conditions. This repository contains all the code required to replicate the results of our research paper, and the corresponding docker environment in which that code can be executed. All code is written in Python3 and utilizes the PyTorch library (v1.0) for neural network model training and evaluation. Jupyter notebooks are used to visualize images transformed with our code, in addition to plotting experimental results. We also include the specific parameter files used to execute our experiments, with the intent that these experiments can be replicated.

Zelinski, michaelE.↗
DOE OSTI2018

A Heuristic Approach to Value-Driven Evaluation of Visualizations

Recently, an approach for determining the value of a visualization was proposed, one moving beyond simple measurements of task accuracy and speed. The value equation contains components for the time savings a visualization provides, the insights and insightful questions it spurs, the overall essence of the data it conveys, and the confidence about the data and its domain it inspires. This articulation of value is purely descriptive, however, providing no actionable method of assessing a visualization's value. In this work, we create a heuristic-based evaluation methodology to accompany the value equation for assessing interactive visualizations. We refer to the methodology colloquially as ICE-T, based on an anagram of the four value components. Our approach breaks the four components down into guidelines, each of which is made up of a small set of low-level heuristics. Evaluators who have knowledge of visualization design principles then assess the visualization with respect to the heuristics. We conducted an initial trial of the methodology on three interactive visualizations of the same data set, each evaluated by 15 visualization experts. As a result, we found that the methodology showed promise, obtaining consistent ratings across the three visualizations and mirroring judgments of the utility of the visualizations by instructors of the course in which they were developed.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Visualization for Insight and Data Analysis in Energy Research

This talk explores how advanced visualization technologies are transforming analytical reasoning and knowledge discovery in energy research, drawing on recent work at the National Laboratory of the Rockies' Computational Science Center. Through a series of scientific case studies, we demonstrate how immersive and high-resolution visualization environments enable scientists and engineers to identify previously unseen patterns and features - insights that often remain hidden in traditional desktop-based analysis. By embedding richer information into interactive analytics tools, these approaches support the exploration of complex, multivariate parameter spaces, where interaction itself catalyzes understanding. Beyond capability, we emphasize the critical role of visualization design grounded in perception and cognition, showing how visual encodings directly influence analytical outcomes. Spanning applications from materials science to integrated energy systems, these visualization approaches accelerate innovation and improve decision-making by enabling deeper, more reliable insight into increasingly complex energy data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Scalable Comparative Visualization of Ensembles of Call Graphs

Optimizing the performance of large-scale parallel codes is critical for efficient utilization of computing resources. Code developers often explore various execution parameters, such as hardware configurations, system software choices, and application parameters, and are interested in detecting and understanding bottlenecks in different executions. They often collect hierarchical performance profiles represented as call graphs, which combine performance metrics with their execution contexts. The crucial task of exploring multiple call graphs together is tedious and challenging because of the many structural differences in the execution contexts and significant variability in the collected performance metrics (e.g., execution runtime). In this paper, we present Ensemble CallFlow to support the exploration of ensembles of call graphs using new types of visualizations, analysis, graph operations, and features. We introduce ensemble-Sankey , a new visual design that combines the strengths of resource-flow (Sankey) and box-plot visualization techniques. Whereas the resource-flow visualization can easily and intuitively describe the graphical nature of the call graph, the box plots overlaid on the nodes of Sankey convey the performance variability within the ensemble. Our interactive visual interface provides linked views to help explore ensembles of call graphs, e.g., by facilitating the analysis of structural differences, and identifying similar or distinct call graphs. Finally, we demonstrate the effectiveness and usefulness of our design through case studies on large-scale parallel codes.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Trust Your Gut: Comparing Human and Machine Inference from Noisy Visualizations

People commonly utilize visualizations not only to examine a given dataset, but also to draw generalizable conclusions about the underlying models or phenomena. Prior research has compared human visual inference to that of an optimal Bayesian agent, with deviations from rational analysis viewed as problematic. However, human reliance on non-normative heuristics may prove advantageous in certain circumstances. We investigate scenarios where human intuition might surpass idealized statistical rationality. In two experiments, we examine individuals’ accuracy in characterizing the parameters of known data-generating models from bivariate visualizations. Our findings indicate that, although participants generally exhibited lower accuracy compared to statistical models, they frequently outperformed Bayesian agents, particularly when faced with extreme samples. Participants appeared to rely on their internal models to filter out noisy visualizations, thus improving their resilience against spurious data. However, participants displayed overconfidence and struggled with uncertainty estimation. They also exhibited higher variance than statistical machines. Our findings suggest that analyst gut reactions to visualizations may provide an advantage, even when departing from rationality. These results carry implications for designing visual analytics tools, offering new perspectives on how to integrate statistical models and analyst intuition for improved inference and decision-making. The data and materials for this paper are available at https://osf.io/qmfv6

human-machine collaboration↗
DOE OSTI2016

Modeling and evaluating user behavior in exploratory visual analysis

Empirical evaluation methods for visualizations have traditionally focused on assessing the outcome of the visual analytic process as opposed to characterizing how that process unfolds. There are only a handful of methods that can be used to systematically study how people use visualizations, making it difficult for researchers to capture and characterize the subtlety of cognitive and interaction behaviors users exhibit during visual analysis. To validate and improve visualization design, however, it is important for researchers to be able to assess and understand how users interact with visualization systems under realistic scenarios. This paper presents a methodology for modeling and evaluating the behavior of users in exploratory visual analysis. We model visual exploration using a Markov chain process comprising transitions between mental, interaction, and computational states. These states and the transitions between them can be deduced from a variety of sources, including verbal transcripts, videos and audio recordings, and log files. This model enables the evaluator to characterize the cognitive and computational processes that are essential to insight acquisition in exploratory visual analysis, and reconstruct the dynamics of interaction between the user and the visualization system. We illustrate this model with two exemplar user studies, and demonstrate the qualitative and quantitative analytical tools it affords.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Report for the ASCR Workshop on Visualization for Scientific Discovery, Decision-Making, and Communication

Visualization—the use of visual elements to explore data, form hypotheses, or convey conclusions—is an integral part of the scientific process. Starting from an initial exploration of new data to illustrating outcomes for the general public, visualization is one of the most intuitive and powerful modes of communication. With the explosion of new data sources and types, unprecedented volumes of data, and new technologies, such as virtual reality (VR) and artificial intelligence (AI), visualization has become increasingly essential but also ever more challenging. The Department of Energy’s (DOE) Office of Advanced Scientific Computing Research (ASCR) sponsored a Basic Research Needs workshop in January 2022 to understand the major opportunities and grand challenges in visualization tools and technologies for scientific computing as well as for DOE-relevant applications and goals in general. The workshop identified five priority research directions (PRDs) for visualization to support scientific discovery, decision making, and communication. The first three PRDs describe interconnected research themes addressing the need for new techniques to deal with complex data, uncertainty, and interpretability (PRD 1); the need for scalable and interoperable software stacks (PRD 2); and the challenges and opportunities inherent in new technologies, such as VR, cloud, or exascale computing (PRD 3). The remaining two PRDs describe foundational research themes that recognize the potential of visualizations to provide equitable access to information and to strengthen the scientific discourse (PRD 4); and the need to consider human factors when designing visualizations (PRD 5). Collectively, these PRDs form the pillars for a coherent, long-term research and development strategy in Visualization for Scientific Discovery, Decision-Making, and Communication in the context of the Office of Science’s mission scope.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

A Prospective Design Method for Nuclear Power: The Evaluation, Requirements, and Goals Outline for Nuclear (ERGON) Method

Human factors researchers at Idaho National Laboratory (INL) have worked on projects spanning control room modernization, operator support systems, visualization design, and novel system creation. These projects demonstrated the need for an explicit design method for nuclear power. Human factors teams found a high standard in the Human Factors Engineering Program Review Model (NUREG-0711) and needed a design methodology which could be successful in gaining approval. Previous work has been synthesized as the Evaluation, Requirements, and Goals Outline for Nuclear (ERGON) method here. Design tasks are broken into four phases: Context and Orientation, Human Factors Review, Prototyping and Evaluation, Iteration and Improvement. ERGON is intended as a flexible and direct design method for many applications in nuclear power. ERGON has been vetted through collaborative research and development with nuclear utilities and as such, the ERGON method can assist utilities to achieve approval from a NUREG-0711 summative evaluation for HSI implementations.

42 ENGINEERING↗
DOE OSTI2024

Forte: An Interactive Visual Analytic Tool for Trust-Augmented Net-Load Forecasting

Accurate net-load forecasting is vital for energy planning, aiding decisions on trade and load distribution. However, assessing the performance of forecasting models across diverse input variables, like temperature and humidity, remains challenging, particularly for eliciting a high degree of trust in the model outcomes. In this context, there is a growing need for data-driven technological interventions to aid scientists in comprehending how models react to both noisy and clean input variables, thus shedding light on complex behaviors and fostering confidence in the outcomes. In this paper, we present Forte, a visual analytics-based application to explore deep probabilistic net-load forecasting models across various input variables and understand the error rates for different scenarios. With carefully designed visual interventions, this web-based interface empowers scientists to derive insights about model performance by simulating diverse scenarios, facilitating an informed decision-making process. We discuss observations made using Forte and demonstrate the effectiveness of visualization techniques to provide valuable insights into the correlation between weather inputs and net-load forecasts, ultimately advancing grid capabilities by improving trust in forecasting models.

Bhattacharjee, Kaustav↗
DOE OSTI2018

Novel capabilities for examining and summarizing hierarchical and geographic resiliency profiles

The geospatial research and development team in the National and Homeland Security Division at Idaho National Laboratory was tasked with providing tools to derive insight from the substantial amount of data currently available - and continuously being produced - associated with the critical infrastructure of the US. This effort is in support of the Department of Homeland Security, whose mission includes the protection of this infrastructure and the enhancement of its resilience to hazards, both natural and human. We present geovisual-analytics-based approaches for analysis of vulnerabilities and resilience of critical infrastructure, designed so that decision makers, analysts, and infrastructure owners and managers can manage risk, prepare for hazards, and direct resources before and after an incident that might result in an interruption in service. Our designs are based on iterative discussions with DHS leadership and analysts, who in turn will use these tools to explore and communicate data in partnership with utility providers, law enforcement, and emergency response and recovery organizations, among others. In most cases these partners desire summaries of large amounts of data, but increasingly, our users seek the additional capability of focusing on, for example, a specific infrastructure sector, a particular geographic region, or time period, or of examining data in a variety of generalization or aggregation levels. These needs align well with tenets of in-formation-visualization design; in this paper, selected applications among those that we have designed are described and positioned within geovisualization, geovisual analytical, and information visualization frameworks.

97 MATHEMATICS AND COMPUTING↗