Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “data visualization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38

GRACC: GRid ACcounting Collector

The OSG has long maintained a central accounting system called Gratia. It uses small probes on each computing and storage resource in order to collect resource usage. The probes report to a central collector which stores the usage in a database. The database is then queried to generate reports. As the OSG aged, the size of the database grew very large. It became too large for the database technology to efficiently query to generate detailed reports. The design of a replacement requires data storage that could be queried efficiently to generate multi-year reports. Additionally, it requires flexibilityto add new attributes to the collected data.In this paper we will describe updates to the GRACC architecture in the last 18 months. GRACC uses modern web technologies that were designed for large data storage, query, and visualization. That includes the open source database Elasticsearch, message broker software RabbitMQ, and Grafana and Kibana as data visualization platforms. It uses multiple agents that perform operations on the data to transform it for easier querying and summarization.

Weitzel, Derek↗

Advanced Simulation Capability for Environmental Management (ASCEM): Early Site Demonstration

The U.S. Department of Energy Office of Environmental Management, Technology Innovation and Development (EM-32), is supporting development of the Advanced Simulation Capability for Environmental Management (ASCEM). ASCEM is a state-of-the-art scientific tool and approach for understanding and predicting contaminant fate and transport in natural and engineered systems. The modular and open source high performance computing tool will facilitate integrated approaches to modeling and site characterization that enable robust and standardized assessments of performance and risk for EM cleanup and closure activities. As part of the initial development process, a series of demonstrations were defined to test ASCEM components and provide feedback to developers, engage end users in applications, and lead to an outcome that would benefit the sites. The demonstration was implemented for a sub-region of the Savannah River Site General Separations Area that includes the F-Area Seepage Basins. The physical domain included the unsaturated and saturated zones in the vicinity of the seepage basins and Fourmile Branch, using an unstructured mesh fit to the hydrostratigraphy and topography of the site. The calculations modeled variably saturated flow and the resulting flow field was used in simulations of the advection of non-reactive species and the reactive-transport of uranium. As part of the demonstrations, a new set of data management, visualization, and uncertainty quantification tools were developed to analyze simulation results and existing site data. These new tools can be used to provide summary statistics, including information on which simulation parameters were most important in the prediction of uncertainty and to visualize the relationships between model input and output.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Information-Theoretic Exploration of Multivariate Time-Varying Image Databases

Modern scientific simulations produce very large datasets, making interactive exploration of such data computationally prohibitive. An increasingly common data reduction technique is to store visualizations and other data extracts in a database. The Cinema project is one such approach, storing visualizations in an image database for post hoc exploration and interactive image-based analysis. This work focuses on developing efficient algorithms that can quantify various types of multivariate dependencies existing within multi-variable datasets. It applies specific mutual information measures for the quantification of salient regions from multivariate image data. Here, using such information measures, the opacity of the images is modulated so that the salient regions are automatically highlighted and the domain scientists can interactively explore the most relevant regions for scientific discovery.

97 MATHEMATICS AND COMPUTING↗

DOE Computer Graphics Forum 2013 Site Survey

The Data group provides data analysis and visualization support to its customers. This consists primarily of performing data analysis and visualization for customers, assisting customers with the use of visualization and analysis tools, and the development of VisIt, a data analysis and visualization tool.

97 MATHEMATICS AND COMPUTING↗

Tractometry of the Human Connectome Project: resources and insights

The Human Connectome Project (HCP) has become a keystone dataset in human neuroscience, with a plethora of important applications in advancing brain imaging methods and an understanding of the human brain. We focused on tractometry of HCP diffusion-weighted MRI (dMRI) data. We used an open-source software library (pyAFQ; https://yeatmanlab.github.io/pyAFQ) to perform probabilistic tractography and delineate the major white matter pathways in the HCP subjects that have a complete dMRI acquisition (n = 1,041). We used diffusion kurtosis imaging (DKI) to model white matter microstructure in each voxel of the white matter, and extracted tract profiles of DKI-derived tissue properties along the length of the tracts. We explored the empirical properties of the data: first, we assessed the heritability of DKI tissue properties using the known genetic linkage of the large number of twin pairs sampled in HCP. Second, we tested the ability of tractometry to serve as the basis for predictive models of individual characteristics (e.g., age, crystallized/fluid intelligence, reading ability, etc.), compared to local connectome features. To facilitate the exploration of the dataset we created a new web-based visualization tool and use this tool to visualize the data in the HCP tractometry dataset. Finally, we used the HCP dataset as a test-bed for a new technological innovation: the TRX file-format for representation of dMRI-based streamlines. We released the processing outputs and tract profiles as a publicly available data resource through the AWS Open Data program's Open Neurodata repository. We found heritability as high as 0.9 for DKI-based metrics in some brain pathways. We also found that tractometry extracts as much useful information about individual differences as the local connectome method. We released a new web-based visualization tool for tractometry—“Tractoscope” (https://nrdg.github.io/tractoscope). We found that the TRX files require considerably less disk space-a crucial attribute for large datasets like HCP. In addition, TRX incorporates a specification for grouping streamlines, further simplifying tractometry analysis.

59 BASIC BIOLOGICAL SCIENCES↗

Visual Analytics: Building a Vibrant and Resilient National Science

Five years after the science of visual analytics was formally established, we attempt to use two different studies to assess the current state of the community and evaluate the progress the community has made in the past few years. The first study involves a comparison analysis of intellectual and scholastic accomplishments recently made by the visual analytics community. The second one aims to measure the degree of community reach and internet penetration of visual-analytics-related resources. This paper describes our efforts to harvest the study data, conduct analysis, and make interpretations based on parallel comparisons with five other established computer science areas.

97 MATHEMATICS AND COMPUTING↗

On the use of graph search techniques for the analysis of extreme-scale combustion simulation data

With the continuous increase in high performance computing capabilities, simulations are becoming ever larger and more complex, using bigger domains, tracking more variables, and producing more time steps. This increase in the ranges of spatial and temporal simulation scales results in data that presents significant challenges to as well as new opportunities for the visualization and data analysis community. For example, highly-localized, intermittent events (such as the formation of ignition kernels in turbulent combustion) may be caused by interactions between multiple variables across a series of time steps, making both their definition and their extraction difficult, particularly at scale. This paper introduces an intuitive framework to support the identification, characterization, and tracking of such complex, multivariate, temporally evolving events in large-scale simulations. In a pre-processing step, we use topological techniques to create a hierarchical family of feature definitions for each variable of interest. Subsequently, we select a particular set of features for analysis and, using overlap-based metrics, we generate an attributed relational graph (ARG) capturing the relationships between different variables both within one and across multiple time steps. Finally, we leverage subgraph-isomorphism search heuristics to identify patterns in the ARG that characterize interesting events. We demonstrate the power of this approach by analyzing a large-scale turbulent combustion simulation.

McLendon, W. C.↗

Utah FORGE: Well 16A(78)-32 Perforation Images and Raw Data

This archive contains raw data of visual and acoustic mapping of perforations in Utah FORGE well 16A(78)-32 acquired during the August 2024 circulation program. The dataset includes downhole images captured by EV, a downhole visual analytics company, providing visual records of each perforation. Images are organized in two folders: one set with perforation visualization overlays and one without. An included Excel spreadsheet provides the organized raw data.

15 GEOTHERMAL ENERGY↗

PreDatA - Preparatory Data Analytics on Peta-Scale Machines

Peta-scale scientific applications running on High End Computing (HEC) platforms can generate large volumes of data. For high performance storage and in order to be useful to science end users, such data must be organized in its layout, indexed, sorted, and otherwise manipulated for subsequent data presentation, visualization, and detailed analysis. In addition, scientists desire to gain insights into selected data characteristics 'hidden' or 'latent' in the massive datasets while data is being produced by simulations. PreDatA, short for Preparatory Data Analytics, is an approach for preparing and characterizing data while it is being produced by the large scale simulations running on peta-scale machines. By dedicating additional compute nodes on the peta-scale machine as staging nodes and staging simulation's output data through these nodes, PreDatA can exploit their computational power to perform selected data manipulations with lower latency than attainable by first moving data into file systems and storage. Such in-transit manipulations are supported by the PreDatA middleware through RDMA-based data movement to reduce write latency, application-specific operations on streaming data that are able to discover latent data characteristics, and appropriate data reorganization and metadata annotation to speed up subsequent data access. As a result, PreDatA enhances the scalability and flexibility of current I/O stack on HEC platforms and is useful for data pre-processing, runtime data analysis and inspection, as well as for data exchange between concurrently running simulation models. Performance evaluations with several production peta-scale applications on Oak Ridge National Laboratory's Leadership Computing Facility demonstrate the feasibility and advantages of the PreDatA approach.

99 GENERAL AND MISCELLANEOUS↗

A Science Data Gateway for Environmental Management

Science data gateways are effective in providing complex science data collections to the world-wide user Q1 communities. In this paper we describe a gateway for the Advanced Simulation Capability for Environmental Management (ASCEM) framework. Built on top of established web service technologies, the ASCEM data gateway is specifically designed for environmental modeling applications. Its key distinguishing feature sinclude (1) handling of complex spatiotemporal data, (2) offering a variety of selective data access mechanisms, (3) providing state-of-the-art plotting and visualization of spatiotemporal data records, and (4) integrating seamlessly with a distributed workflow system using a RESTful interface. ASCEM projectscientists have been using this data gateway since 2011.

ASCEM, Akuna, Database Management↗

Report of the DOE Workshop on Management, Analysis, and Visualization of Experimental and Observational data – The Convergence of Data and Computing

The Department of Energy (DOE) Office of Science (SC) operates dozens of national science user facilities that span many different disciplines. These facilities include accelerators, colliders, supercomputers, light sources and neutron sources, as well as facilities for studying the nanoworld, the environment, and the atmosphere. In Fiscal Year 2014 over 33,000 researchers from academia, industry, and government laboratories, spanning all fifty states and the District of Columbia, utilized these unique facilities to perform new scientific research. Each of these facilities generates vast amounts of scientific data, and the rate, size, and complexity of this data is rapidly increasing, thanks to advances in technology. A growing concern, which motivates this workshop, is the likely significant adverse impact on science programs that will result without significant advances in the capabilities needed to manage and gain knowledge from these collections of data. The purpose of this workshop, held 29 September 2015 through 1 October 2015 in Bethesda, MD, is to help the Advanced Scientific Computing Research (ASCR) and research community better understand needs related to the management, analysis, and visualization of experimental and observational data (EOD) collected and generated by experimental and observational science projects (EOS) at Office of Science user facilities.

97 MATHEMATICS AND COMPUTING↗

DOECGF 2009 Site Report: Lawrence Livermore National Laboratory

The Data group provides data analysis and visualization support to its customers. This consists primarily of the development and support of VisIt, a data analysis and visualization tool. Support ranges from answering questions about the tool, providing classes on how to use the tool, and performing data analysis and visualization for customers. The Information Management and Graphics Group supports and develops tools that enhance our ability to access, display, and understand large, complex data sets. Activities include applying visualization software for terascale data exploration; running video production labs on two networks; supporting graphics libraries and tools for end users; maintaining PowerWalls and assorted other displays; and developing software for searching, managing, and browsing scientific data. Researchers in the Center for Applied Scientific Computing (CASC) work on various projects including the development of visualization techniques for terascale data exploration that are funded by the ASC program, among others. The researchers also have LDRD projects and collaborations with other lab researchers, academia, and industry.

99 GENERAL AND MISCELLANEOUS↗

Development of a Technology Roadmap for Online Monitoring of Nuclear Power Plants

The nuclear power industry has recently recognized the potential cost saving that can be achieved from migrating the current manual and labor dependent surveillance and preventive maintenance activities to data-driven online monitoring methods. Since then, several efforts have been launched with various degrees of momentum to tackle specific surveillance or preventive maintenance activities. The light water reactor sustainability program (LWRS) program has anticipated this need and launched an effort that aimed to develop a technology roadmap for the nuclear power industry. Because the roadmap has to be focused towards the user or application’s value, and because nuclear power plants have different process efficiencies and deficiencies, the technology roadmap has to be plant independent. Achieving this objective required optimizing the extent of details described by a technology roadmap. The approach followed is for the roadmap to describe the sequence of activities needed to automate a process or equipment activity, but not describe the sequence of processes or equipment to target. For example, the steps required to migrate the inspection of a pump are plant independent. However, defining that the migration of feed water pumps inspections to online monitoring should be performed before inspections of feed water valves is dependent on multiple plant factors such as the plant requirements, labor work capacity, and process or equipment conditions. The technology roadmap was broken down into six elements, four of which have a sequential path of advancement, while the other two are supporting elements to the sequential elements. The sequential part of the roadmap analyzes data collection, analytics, visualization, and management. For these elements of the technology roadmap, the migration of an activity is defined in three states. The base state is defined as the most primitive, manual, and labor dependent process used by some nuclear power plants. The modern state is defined as the state that can be achieved if the activities are augmented or replaced by current technologies that are commercially available or are soon to be available. The state of the art is defined as using technologies of the future, i.e. concept and technologies that are being researched. The data collection part of the technology roadmap defines three states of every process performed in a nuclear power plant. However, the data analytics part is described based on equipment, because automating an activity performed on one equipment required considering multiple data sources. The visualization and data management elements are process or equipment independent. Visualization describes the means to use human factor science and technologies advancement in presenting the data collected and the data analytics results. The data management targets storage, communication, and computational power need, in addition to data integration and sharing. Since storage, communication, and computational power technologies and path for deployment are known to any information technology organization, this effort targeted the cost of data management. This has a direct impact on the feasibility of the automation effort. The supporting elements analyzes the value of automating the activity and the change challenges to tackle. For the support elements, the roadmap targeted describing activities that need to be performed as part of the roadmap development. The value element of the roadmap describes how to analyze the cost saving and the associated risk. The change management identifies multiple factors to consider such as regulation, cybersecurity, resource availability, supporting infrastructure availability, feasibility (power, fitness, environment tolerance, facilities), and cultural change.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Identifying, Visualizing, and Fusing Social Media Data to Support Nonproliferation and Arms Control Treaty Verification: Preliminary Results

While international nonproliferation and arms control verification capabilities have their foundations in physical and chemical sensors, state declarations, and on-site inspections, verification experts are beginning to consider the importance of open source data to complement and support traditional means of verification. One of those new, and increasingly expanding, sources of open source information is social media, which can be ingested and understood through social media analytics (SMA). Pacific Northwest National Laboratory (PNNL) is conducting research to further our ability to identify, visualize, and fuse social media data to support nonproliferation and arms control treaty verification efforts. This paper will describe our preliminary research to examine social media signatures of nonproliferation or arms control proxy events. We will describe the development of our preliminary nonproliferation and arms control proxy events, outline our initial findings, and propose ideas for future work.

social media, arms control, nonproliferation↗

Compression-based integral curve data reuse framework for flow visualization

Currently, by default, integral curves are repeatedly re-computed in different flow visualization applications, such as FTLE field computation, source-destination queries, etc., leading to unnecessary resource cost. We present a compression-based data reuse framework for integral curves, to greatly reduce their retrieval cost, especially in a resource-limited environment. In our design, a hierarchical and hybrid compression scheme is proposed to balance three objectives, including high compression ratio, controllable error, and low decompression cost. Specifically, we use and combine digitized curve sparse representation, floating-point data compression, and octree space partitioning to adaptively achieve the objectives. Results have shown that our data reuse framework could acquire tens of times acceleration in the resource-limited environment compared to on-the-fly particle tracing, and keep controllable information loss. Moreover, our method could provide fast integral curve retrieval for more complex data, such as unstructured mesh data.

97 MATHEMATICS AND COMPUTING↗

Interactive Exploration and Analysis of Large-Scale Simulations Using Topology-Based Data Segmentation

Large-scale simulations are increasingly being used to study complex scientific and engineering phenomena. As a result, advanced visualization and data analysis are also becoming an integral part of the scientific process. Often, a key step in extracting insight from these large simulations involves the definition, extraction, and evaluation of features in the space and time coordinates of the solution. However, in many applications, these features involve a range of parameters and decisions that will affect the quality and direction of the analysis. Examples include particular level sets of a specific scalar field, or local inequalities between derived quantities. A critical step in the analysis is to understand how these arbitrary parameters/decisions impact the statistical properties of the features, since such a characterization will help to evaluate the conclusions of the analysis as a whole. We present a new topological framework that in a single-pass extracts and encodes entire families of possible features definitions as well as their statistical properties. For each time step we construct a hierarchical merge tree a highly compact, yet flexible feature representation. While this data structure is more than two orders of magnitude smaller than the raw simulation data it allows us to extract a set of features for any given parameter selection in a postprocessing step. Furthermore, we augment the trees with additional attributes making it possible to gather a large number of useful global, local, as well as conditional statistic that would otherwise be extremely difficult to compile. We also use this representation to create tracking graphs that describe the temporal evolution of the features over time. Our system provides a linked-view interface to explore the time-evolution of the graph interactively alongside the segmentation, thus making it possible to perform extensive data analysis in a very efficient manner. We demonstrate our framework by extracting and analyzing burning cells from a large-scale turbulent combustion simulation. In particular, we show how the statistical analysis enabled by our techniques provides new insight into the combustion process.

97 MATHEMATICS AND COMPUTING↗

Distributed Data-Flow for In-Situ Visualization and Analysis at Petascale

We conducted a feasibility study to research modifications to data-flow architectures to enable data-flow to be distributed across multiple machines automatically. Distributed data-flow is a crucial technology to ensure that tools like the VisIt visualization application can provide in-situ data analysis and post-processing for simulations on peta-scale machines. We modified a version of VisIt to study load-balancing trade-offs between light-weight kernel compute environments and dedicated post-processing cluster nodes. Our research focused on memory overheads for contouring operations, which involves variable amounts of generated geometry on each node and computation of normal vectors for all generated vertices. Each compute node independently decided whether to send data to dedicated post-processing nodes at each stage of pipeline execution, depending on available memory. We instrumented the code to allow user settable available memory amounts to test extremely low-overhead compute environments. We performed initial testing of this prototype distributed streaming framework, but did not have time to perform scaling studies at and beyond 1000 compute-nodes.

97 MATHEMATICS AND COMPUTING↗

UAE6 - Wind Tunnel Tests Data - UAE6 - Sequence P - Raw Data

Sequence P: Wake Flow Visualization, Upwind (P) This test sequence used an upwind, rigid turbine with a 0° cone angle. The wind speed ranged from 5 m/s to 15 m/s. Yaw angles of 0° to –60° were achieved. The blade tip pitch was 3°. The rotor rotated at 72 RPM. Blade and probe pressure measurements were collected. The teeter dampers were replaced with rigid links, and these two channels were flagged as not applicable by setting the measured values in the data file to –99999.99 Nm. The teeter link load cell was pre-tensioned to 40,000 N. The aluminum blade tip designed to contain a smoke generator was installed, and counterweights were installed in the non-instrumented blade tip to compensate. The turbine was positioned at the appropriate yaw angle, and the smoke generator was ignited remotely. The campaign duration was 3 minutes for all tests except P1000000, which was 2 minutes. The file name convention was the standard format except for P10000A0, which indicated a 3° pitch angle. File P1000000 used a 12° pitch angle. After these two campaigns were collected, it was determined that all subsequent data should be collected with a 3° pitch angle. Pressure data were not acquired during this sequence, so all associated data values were flagged as not applicable by setting the measured values in the data file to 0.000 Pa. Corresponding pressure data are available from Sequence H for the 3° pitch angle test points. Flow visualization data obtained from wall- and ceiling-mounted video cameras were recorded to videotape. The camera locations and calibration procedures are described in Appendix J.

17 WIND ENERGY↗