TOD-Tree: Task-Overlapped Direct Send Tree Image Compositing for Hybrid MPI Parallelism and GPUs
Not provided.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Not provided.
As computing technology continues to advance, computational modeling of scientific and engineering problems produces data of increasing complexity: large in size and unstructured in shape. Volume visualization of such data is a challenging problem. This paper proposes a distributed parallel solution that makes ray-casting volume rendering of unstructured-grid data practical. Both the data and the rendering process are distributed among processors. At each processor, ray-casting of local data is performed independent of the other processors. The global image composing processes, which require inter-processor communication, are overlapped with the local ray-casting processes to achieve maximum parallel efficiency. This algorithm differs from previous ones in four ways: it is completely distributed, less view-dependent, reasonably scalable, and flexible. Without using dynamic load balancing, test results on the Intel Paragon using from two to 128 processors show, on average, about 60% parallel efficiency.
For the past twelve months, we have conducted and completed a joint research entitled "Time- Critical Volume Rendering" with NASA Ames. As expected, High performance volume rendering algorithms have been developed by exploring some new faster rendering techniques, including object presence acceleration, parallel processing, and hierarchical level-of-detail representation. Using our new techniques, initial experiments have achieved real-time rendering rates of more than 10 frames per second of various 3D data sets with highest resolution. A couple of joint papers and technique reports as well as an interactive real-time demo have been compiled as the result of this project.
The analysis of complex multiphysics astrophysical simulations presents a unique and rapidly growing set of challenges: reproducibility, parallelization, and vast increases in data size and complexity chief among them. In order to meet these challenges, and in order to open up new avenues for collaboration between users of multiple simulation platforms, we present yt (available at http://yt.enzotools.org/) an open source, community-developed astrophysical analysis and visualization toolkit. Analysis and visualization with yt are oriented around physically relevant quantities rather than quantities native to astrophysical simulation codes. While originally designed for handling Enzo's structure adaptive mesh refinement data, yt has been extended to work with several different simulation methods and simulation codes including Orion, RAMSES, and FLASH. We report on its methods for reading, handling, and visualizing data, including projections, multivariate volume rendering, multi-dimensional histograms, halo finding, light cone generation, and topologically connected isocontour identification. Furthermore, we discuss the underlying algorithms yt uses for processing and visualizing data, and its mechanisms for parallelization of analysis tasks.
This is a set of slides from a guest lecture for a class at the University of Texas, El Paso on visualization and data analysis for high-performance computing. The topics covered are the following: trends in high-performance computing; scientific visualization, such as OpenGL, ray tracing and volume rendering, VTK, and ParaView; data science at scale, such as in-situ visualization, image databases, distributed memory parallelism, shared memory parallelism, VTK-m, "big data", and then an analysis example.
Not provided.
In the second year, we continued to built upon and improve our scanline-based direct volume renderer that we developed in the first year of this grant. This extremely general rendering approach can handle regular or irregular grids, including overlapping multiple grids, and polygon mesh surfaces. It runs in parallel on multi-processors. It can also be used in conjunction with a k-d tree hierarchy, where approximate models and error terms are stored in the nodes of the tree, and approximate fast renderings can be created. We have extended our software to handle time-varying data where the data changes but the grid does not. We are now working on extending it to handle more general time-varying data. We have also developed a new extension of our direct volume renderer that uses automatic decimation of the 3D grid, as opposed to an explicit hierarchy. We explored this alternative approach as being more appropriate for very large data sets, where the extra expense of a tree may be unacceptable. We also describe a new approach to direct volume rendering using hardware 3D textures and incorporates lighting effects. Volume rendering using hardware 3D textures is extremely fast, and machines capable of using this technique are becoming more moderately priced. While this technique, at present, is limited to use with regular grids, we are pursuing possible algorithms extending the approach to more general grid types. We have also begun to explore a new method for determining the accuracy of approximate models based on the light field method described at ACM SIGGRAPH '96. In our initial implementation, we automatically image the volume from 32 equi-distant positions on the surface of an enclosing tessellated sphere. We then calculate differences between these images under different conditions of volume approximation or decimation. We are studying whether this will give a quantitative measure of the effects of approximation. We have created new tools for exploring the differences between images produced by various rendering methods. Images created by our software can be stored in the SGI RGB format. Our idtools software reads in pair of images and compares them using various metrics. The differences of the images using the RGB, HSV, and HSL color models can be calculated and shown. We can also calculate the auto-correlation function and the Fourier transform of the image and image differences. We will explore how these image differences compare in order to find useful metrics for quantifying the success of various visualization approaches. In general, progress was consistent with our research plan for the second year of the grant.
We improve the interpolation accuracy and efficiency of the Delaunay tessellation field estimator (DTFE) for surface density field reconstruction by proposing an algorithm that takes advantage of the adaptive triangular mesh for line-of-sight integration. The costly computation of an intermediate 3D grid is completely avoided by our method and only optimally chosen interpolation points are computed, thus, the overall computational cost is significantly reduced. The algorithm is implemented as a parallel shared-memory kernel for large-scale grid rendered field reconstructions in our distributed-memory framework designed for N-body gravitational lensing simulations in large volumes. We also introduce a load balancing scheme to optimize the efficiency of processing a large number of field reconstructions. Our results show our kernel outperforms existing software packages for volume weighted density field reconstruction, achieving ~ 10x speedup, and our load balancing algorithm gains an additional ~ 3.6x speedup at scales with ~ 16k processes.
In the Concurrent Computing Laboratory in the Department of Physics and Astronomy at Louisiana State University we have constructed a heterogeneous computing environment that permits us to routinely simulate complicated three-dimensional fluid flows and to readily visualize the results of each simulation via three-dimensional animation sequences. An 8192-node MasPar MP-1 computer with 0.5 GBytes of RAM provides 250 MFlops of execution speed for our fluid flow simulations. Utilizing the parallel virtual machine (PVM) language, at periodic intervals data is automatically transferred from the MP-1 to a cluster of workstations where individual three-dimensional images are rendered for inclusion in a single animation sequence. Work is underway to replace executions on the MP-1 with simulations performed on the 512-node CM-5 at NCSA and to simultaneously gain access to more potent volume rendering workstations.
Hypoxic areas are a common feature of rapidly growing malignant tumors and their metastases, and are typically spatially heterogeneous. Hypoxia has a strong impact on tumor cell biology and contributes to tumor progression in multiple ways. To date, only a few molecular key players in tumor hypoxia, such as for example hypoxia-inducible factor-1 (HIF-1), have been discovered. The distribution of biomolecules is frequently heterogeneous in the tumor volume, and may be driven by hypoxia and HIF-1α. Understanding the spatially heterogeneous hypoxic response of tumors is critical. Mass spectrometric imaging (MSI) provides a unique way of imaging biomolecular distributions in tissue sections with high spectral and spatial resolution. In this paper, breast tumor xenografts grown from MDA-MB-231-HRE-tdTomato cells, with a red fluorescent tdTomato protein construct under the control of a hypoxia response element (HRE)-containing promoter driven by HIF-1α, were used to detect the spatial distribution of hypoxic regions. We elucidated the 3D spatial relationship between hypoxic regions and the localization of small molecules, metabolites, lipids, and proteins by using principal component analysis – linear discriminant analysis (PCA-LDA) on 3D rendered MSI volume data from MDA-MB-231-HRE-tdTomato breast tumor xenografts. In this study we identified hypoxia-regulated proteins active in several distinct pathways such as glucose metabolism, regulation of actin cytoskeleton, protein folding, translation/ribosome, splicesome, the PI3K-Akt signaling pathway, hemoglobin chaperone, protein processing in endoplasmic reticulum, detoxification of reactive oxygen species, aurora B signaling/apoptotic execution phase, the RAS signaling pathway, the FAS signaling pathway/caspase cascade in apoptosis and telomere stress induced senescence. In parallel we also identified co-localization of hypoxic regions and various lipid species such as PC(16:0/18:1), PC(16:0/18:2), PC(18:0/18:1), PC(18:1/18:1), PC(18:1/18:2), PC(16:1/18:4), PC(18:0/20:3), PC(16:0/22:1), among others. Lastly, our findings shed light on the biomolecular composition of hypoxic tumor regions, which may be responsible for a given tumor’s resistance to radiation or chemotherapy.
A new and easy-to-implement method for direct volume rendering that uses 3D texture maps for acceleration, and incorporates directional lighting, is described. The implementation, called Voltx, produces high-quality images at nearly interactive speeds on workstations with hardware support for three-dimensional texture maps. Previously reported methods did not incorporate a light model, and did not address issues of multiple texture maps for large volumes. Our research shows that these extensions impact performance by about a factor of ten. Voltx supports orthographic, perspective, and stereo views. This paper describes the theory and implementation of this technique, and compares it to the shear-warp factorization approach. A rectilinear data set is converted into a three-dimensional texture map containing color and opacity information. Quantized normal vectors and a lookup table provide efficiency. A new tesselation of the sphere is described, which serves as the basis for normal-vector quantization. A new gradient-based shading criterion is described, in which the gradient magnitude is interpreted in the context of the field-data value and the material classification parameters, and not in isolation. In the rendering phase, the texture map is applied to a stack of parallel planes, which effectively cut the texture into many slabs. The slabs are composited to form an image.
A new and easy-to-implement method for direct volume rendering that uses 3D texture maps for acceleration, and incorporates directional lighting, is described. The implementation, called Voltx, produces high-quality images at nearly interactive speeds on workstations with hardware support for three-dimensional texture maps. Previously reported methods did not incorporate a light model, and did not address issues of multiple texture maps for large volumes. Our research shows that these extensions impact performance by about a factor of ten. Voltx supports orthographic, perspective, and stereo views. This paper describes the theory and implementation of this technique, and compares it to the shear-warp factorization approach. A rectilinear data set is converted into a three-dimensional texture map containing color and opacity information. Quantized normal vectors and a lookup table provide efficiency. A new tesselation of the sphere is described, which serves as the basis for normal-vector quantization. A new gradient-based shading criterion is described, in which the gradient magnitude is interpreted in the context of the field-data value and the material classification parameters, and not in isolation. In the rendering phase, the texture map is applied to a stack of parallel planes, which effectively cut the texture into many slabs. The slabs are composited to form an image.
A general volume rendering technique is described that efficiently produces images of excellent quality from data defined over irregular grids having a wide variety of formats. Rendering is done in software, eliminating the need for special graphics hardware, as well as any artifacts associated with graphics hardware. Images of volumes with about one million cells can be produced in one to several minutes on a workstation with a 150 MHz processor. A significant advantage of this method for applications such as computational fluid dynamics is that it can process multiple intersecting grids. Such grids present problems for most current volume rendering techniques. Also, the wide range of cell sizes (by a factor of 10,000 or more), which is typical of such applications, does not present difficulties, as it does for many techniques. A spatial hierarchical organization makes it possible to access data from a restricted region efficiently. The tree has greater depth in regions of greater detail, determined by the number of cells in the region. It also makes it possible to render useful 'preview' images very quickly (about one second for one-million-cell grids) by displaying each region associated with a tree node as one cell. Previews show enough detail to navigate effectively in very large data sets. The algorithmic techniques include use of a kappa-d tree, with prefix-order partitioning of triangles, to reduce the number of primitives that must be processed for one rendering, coarse-grain parallelism for a shared-memory MIMD architecture, a new perspective transformation that achieves greater numerical accuracy, and a scanline algorithm with depth sorting and a new clipping technique.
A test was conducted from November 2008 to January 2009 to help determine the compatibility of an inhibited propylene glycol/water solution with planned Constellation vehicles. Dowfrost(TradeMark) HD was selected as the baseline for Orion, as well as other Constellation systems. Therefore, the same Dowfrost(TradeMark) HD/Water solution planned for Orion was chosen for this test. The fluid was subjected to a thermal fluid loop that had flightlike properties, as compared to Orion. The fluid loop had similar wetted materials, temperatures, flow rates, and aluminum wetted surface area to fluid volume ratio. The test was designed to last for 10 years, the life expectancy of the lunar habitat. However, the test lasted less than two months. System filters became clogged with precipitate, rendering the fluid system inoperable. Upon examination of the precipitate, it was determined that the precipitate composition contained aluminum, which could have only come from materials in the test stand, as aluminum is not part of the Dowfrost(TradeMark HD composition. Also, the fluid pH was determined to have increased from 10.1, at the first test sample, to 12.2, at the completion of the test. This high of a pH is corrosive to aluminum and was certainly a contributing factor to the development of precipitate. Chemical analyses and bench-top tests are currently ongoing to determine the underlying cause for this rapid degradation of the fluid. Hamilton Sundstrand, the contractor developing the Orion thermal fluid loop, is performing a parallel effort to not only understand the cause of fluid degradation in the test, but also to investigate solutions to avoid this problem in the Orion s thermal control system. JSC also consulted with the Hamilton Sundstrand team in the development of this test and the subsequent analysis.
The relationship between our ability to analyze and extract insights from visualization of climate model output and the capability of the available resources to make those visualizations has reached a crisis point. The large volume of data currently produced by climate models is overwhelming the current, decades-old visualization workflow. The traditional methods for visualizing climate output also have not kept pace with changes in the types of grids used, the number of variables involved, and the number of different simulations performed with a climate model or the feature-richness of high-resolution simulations. This project has developed new and faster methods for visualization in order to get the most knowledge out of the new generation of high-resolution climate models. While traditional climate images will continue to be useful, there is need for new approaches to visualization and analysis of climate data if we are to gain all the insights available in ultra-large data sets produced by high-resolution model output and ensemble integrations of climate models such as those produced for the Coupled Model Intercomparison Project. Towards that end, we have developed new visualization techniques for performing correlation analysis. We have also introduced highly scalable, parallel rendering methods for visualizing large-scale 3D data. This project was done jointly with climate scientists and visualization researchers at Argonne National Laboratory and NCAR.
Smoothed-particle hydrodynamics (SPH) is a mesh-free method used to simulate volumetric media in fluids, astrophysics, and solid mechanics. Visualizing these simulations is problematic because these datasets often contain millions, if not billions of particles carrying physical attributes and moving over time. Radial basis functions (RBFs) are used to model particles, and overlapping particles are interpolated to reconstruct a high-quality volumetric field; however, this interpolation process is expensive and makes interactive visualization difficult. Existing RBF interpolation schemes do not account for color-mapped attributes and are instead constrained to visualizing just the density field. To address these challenges, we exploit ray tracing cores in modern GPU architectures to accelerate scalar field reconstruction. We use a novel RBF interpolation scheme to integrate per-particle colors and densities, and leverage GPU-parallel tree construction and refitting to quickly update the tree as the simulation animates over time or when the user manipulates particle radii. We also propose a Hilbert reordering scheme to cluster particles together at the leaves of the tree to reduce tree memory consumption. Finally, we reduce the noise of volumetric shadows by adopting a spatially temporal blue noise sampling scheme. Our method can provide a more detailed and interactive view of these large, volumetric, time-series particle datasets than traditional methods, leading to new insights into these physics simulations.
Purpose: The presence of Calypso Beacon-transponders in patients can cause artifacts during MRI imaging studies. This could be a problem for post-treatment follow up of cancer patients using MRI studies to evaluate metastasis and for functional imaging studies.This work assesses (1) the volume immediately surrounding the transponders that will not be visualized by the MRI due to the beacons, and (2) the dependence of the non-visualized volume on beacon orientation, and scanning techniques. Methods: Two phantoms were used in this study (1) water filled box, (2) and a 2300 cc block of pork meat. Calypso beacons were implanted in the phantoms both in parallel and perpendicular orientations with respect to the MR scanner magnetic field. MR image series of the phantom were obtained with on a 1.0T high field open MR-SIM with multiple pulse sequences, for example, T1-weighted fast field echo and T2-weighted turbo spin echo. Results: On average, a no-signal region with 2 cm radius and 3 cm length was measured. Image artifacts are more significant when beacons are placed parallel to scanner magnetic field; the no-signal area around the beacon was about 0.5 cm larger in orthogonal orientation. The no-signal region surrounding the beacons slightly varies in dimension for the different pulse sequences. Conclusion: The use of Calypso beacons can prohibit the use of MRI studies in post-treatment assessments, especially in the immediate region surrounding the implanted beacon. A characterization of the MR scanner by identifying the no-signal regions due to implanted beacons is essential. This may render the use of Calypso beacons useful for some cases and give the treating physician a chance to identify those patients prior to beacon implantation.
Many clandestine development and production activities can be conducted underground to evade surveillance. The purpose of the study reported here was to develop a technique to detect underground facilities by broad-area search and then to characterize the facilities by inversion of the collected data. This would enable constraints to be placed on the types of activities that would be feasible at each underground site, providing a basis the design of targeted surveillance and analysis for more complete characterization. Excavation of underground cavities causes deformation in the host material and overburden that produces displacements at the ground surface. Such displacements are often measurable by a variety of surveying or geodetic techniques. One measurement technique, Interferometric Synthetic Aperture Radar (InSAR), uses data from satellite-borne (or airborne) synthetic aperture radars (SARs) and so is ideal for detecting and measuring surface displacements in denied access regions. Depending on the radar frequency and the acquisition mode and the surface conditions, displacement maps derived from SAR interferograms can provide millimeter- to centimeter-level measurement accuracy on regional and local scales at spatial resolution of {approx}1-10 m. Relatively low-resolution ({approx}20 m, say) maps covering large regions can be used for broad-area detection, while finer resolutions ({approx}1 m) can be used to image details of displacement fields over targeted small areas. Surface displacements are generally expected to be largest during or a relatively short time after active excavation, but, depending on the material properties, measurable displacement may continue at a decreasing rate for a considerable time after completion. For a given excavated volume in a given geological setting, the amplitude of the surface displacements decreases as the depth of excavation increases, while the area of the discernable displacement pattern increases. Therefore, the ability to detect evidence for an underground facility using InSAR depends on the displacement sensitivity and spatial resolution of the interferogram, as well as on the size and depth of the facility and the time since its completion. The methodology development described in this report focuses on the exploitation of synthetic aperture radar data that are available commercially from a number of satellite missions. Development of the method involves three components: (1) Evaluation of the capability of InSAR to detect and characterize underground facilities ; (2) inversion of InSAR data to infer the location, depth, shape and volume of a subsurface facility; and (3) evaluation and selection of suitable geomechanical forward models to use in the inversion. We adapted LLNL's general-purpose Bayesian Markov Chain-Monte Carlo procedure, the 'Stochastic Engine' (SE), to carry out inversions to characterize subsurface void geometries. The SE performs forward simulations for a large number of trial source models to identify the set of models that are consistent with the observations and prior constraints. The inverse solution produced by this kind of stochastic method is a posterior probability density function (pdf) over alternative models, which forms an appropriate input to risk-based decision analyses to evaluate subsequent response strategies. One major advantage of a stochastic inversion approach is its ability to deal with complex, non-linear forward models employing empirical, analytical or numerical methods. However, while a geomechanical model must incorporate adequate physics to enable sufficiently accurate prediction of surface displacements, it must also be computationally fast enough to render the large number of forward realizations needed in stochastic inversion feasible. This latter requirement prompted us first to investigate computationally efficient empirical relations and closed-form analytical solutions. However, our evaluation revealed severe limitations in the ability of existing empirical and analytical forms to predict deformations from underground cavities with an accuracy consistent with the potential resolution and precision of InSAR data. We followed two approaches to overcoming these limitations. The first was to develop a new analytical solution for a 3D cavity excavated in an elastic half-space. The second was to adapt a fast parallelized finite element method to the SE and evaluate the feasibility of using in the stochastic inversion. To date we have demonstrated the ability of InSAR to detect underground facilities and measure the associated surface displacements by mapping surface deformations that track the excavation of the Los Angeles Metro system. The Stochastic Engine implementation has been completed and undergone functional testing.