Running BLAST on HPC Systems and in Hadoop (2010 JGI/ANL HPC Workshop)
Shane Canon of NERSC presents at the JGI/Argonne HPC Workshop on January 26, 2010.
SEARCH · Engineering Papers
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Shane Canon of NERSC presents at the JGI/Argonne HPC Workshop on January 26, 2010.
Particle acceleration in relativistic collisionless shocks remains an open problem in high-energy astrophysics. Particle-in-cell (PIC) simulations predict that electron acceleration in weakly magnetized shocks proceeds via small-angle scattering, leading to a maximum electron energy significantly below the Bohm limit. This upper bound on electron energy manifests observationally as a characteristic synchrotron cutoff, providing a direct probe of the underlying acceleration physics. Gamma-ray burst (GRB) afterglows offer an exceptional laboratory for testing these predictions. Here, we model the spectral evolution of GRB afterglows during the relativistic deceleration phase, incorporating PIC-motivated acceleration prescriptions and self-consistently computing synchrotron and synchrotron self-Compton emission. We find that low-energy bursts in low-density environments, typical of short GRBs, exhibit a pronounced synchrotron cutoff in the GeV band within minutes to hours after the trigger. Applying our framework to GRB 190114C and GRB 130427A, we find that current observations are insufficient to discriminate between PIC-motivated acceleration and the Bohm limit, primarily due to poor photon statistics in the Fermi-Large Area Telescope band. Nevertheless, future MeV–TeV afterglow observations can break model degeneracies and place substantially tighter constraints on the mechanisms responsible for particle acceleration in relativistic shocks. To this end, we simulate a fiducial nearby short GRB as a promising probe of the cutoff location, for which the two acceleration scenarios are cleanly distinguishable and the detection of such an event in the near future remains feasible.
Low energy cyclotrons for Positron emission tomography (PET) have been wanted for the production of radio-isotopes after 2002. In the low energy cyclotron magnet design, increase of magnetic field between the poles is needed to make a smaller size of magnet and decrease power consumption. The Permanent magnet can support this work without additional electric power consumption in the cyclotron. In this paper the study of cyclotron magnet design using permanent magnet is shown and also the comparison between normal magnet and the magnet which is designed with permanent magnet is shown. Maximum energy of proton is 8 MeV and RF frequency is 79.3 MHz. 3D CAD design was done by CATIA P3 V5 R18 and the All field calculations had been performed by OPERA-3D TOSCA. The self-made beam dynamics program OPTICY is used for making isochronous field and other calculations.
In early 2007, SLAC was faced with a shortage of both electrical power and cooling in the main computer building, at the same time that the BaBar collaboration needed a new cluster of 250 batch machines installed. A number of different options were explored for the expansion. Provision of additional electrical power to the building was estimated to take one to two years, and cost several million dollars; additional cooling was even worse. Space in a Silicon Valley co-location facilities was reasonable on a one-year timescale, but broke even in costs by the end of three years, and were more expensive after that. There were also unresolved questions about the affects of additional latency from an offsite compute cluster to the onsite disk servers. The option of converting existing experimental hall space into computer space was estimated at one year, with uncertain availability. An option to aggressively replace several existing clusters with more power-efficient equipment was studied closely, but was disruptive to continued operations, expensive, and didn't provide any additional headroom. Finally, the installation of a Sun Project Blackbox (PBB) unit was selected as providing the capacity on a timescale of six months for a reasonable cost with minimal disruption to service. SLAC obtained and installed a beta unit and have been running it in production since September 2007. The experiences described are with the Early Access version of the PBB. The production version of the box has engineering changes based in part on our experiences.
Computational design for fission reactor materials is ready to accelerate the development and qualification of nuclear materials. This review is primarily aimed at computational materials scientists that seek to apply ICME to the development of fission reactor materials. We summarize reactor materials and technology, discuss reactor material development and qualification today, show how ICME is being applied to the unique requirements of reactor materials, and provide a future vision.
Communication switches have sometimes been augmented to process collectives (e.g., the IBM BlueGene project and the Mellanox SHArP switch). In this work, we find that there is a great acceleration opportunity through the further augmentation of switches to accelerate more complex functions that combine communication with computation. We consider three types of such functions. The first is fully-fused collectives built by fusing multiple existing collectives like Allreduce with Alltoall. The second is semi-fused collectives built by combining a collective with another computation. The third we refer to as higher-order collectives built by combining multiple computations and communications, such as to perform matrix-matrix multiply (PGEMM). In this work, we propose a framework called SmartFuse to accelerate fused collective functions. The core of SmartFuse is a reconfigurable smart switch to support these operations. The semi/fully fused collectives are implemented with a CGRAlike architecture, while higher-order collectives are implemented with a more specialized computational unit that can also schedule communication. Supporting our framework is software to evaluate and translate relevant parts of the input program, compile them into a control data flow graph, and then map this graph to the switch hardware. The proposed framework, once deployed, has the strong potential to accelerate existing HPC applications transparently by encapsulation within an MPI implementation. Experimental results show that this approach improves the performance of the PGEMM kernel, MINIFE, and AMG by, on average, 94%, 15%, and 13%, respectively.
Numerical models have been developed to investigate and understand responses of biogeochemical cycle to global changes. Steady state, when a system is in dynamic equilibrium, is generally required to initialize these model simulations. However, the spin-up process that is used to achieve steady state pose a great burden to computational resources, limiting the efficiency of global modeling analysis on biogeochemical cycles. This study introduces a new Semi-Analytical Spin-Up (SASU) to tackle this grand challenge. We applied SASU to Community Land Model version 5 and examined its computational efficiency and accuracy. At the Brazil site, SASU is computationally 7 times more efficient than (or saved up to 86% computational cost in comparison with) the traditional native dynamics (ND) spin-up to reach the same steady state. Globally, SASU is computationally 8 times more efficient than the accelerated decomposition spin-up and 50 times more efficient than ND. In summary, SASU achieves the highest computational efficiency for spin-up on site and globally in comparison with other spin-up methods. It is generalizable to wide biogeochemical models and thus makes computationally costly studies (e.g., parameter perturbation ensemble analysis and data assimilation) possible for a better understanding of biogeochemical cycle under climate change.
It is well known that application specific computing systems, optimally designed and configured for a given workload, offer much higher energy-efficiency and throughput than general purpose systems. In modern computing systems, heterogeneous computing systems have emerged that exploit the energy and performance benefits of combining various different domain-specific processor architectures. Application domains such as high-performance computing and machine learning now process terabyte-sized data sets, requiring enormous processing and memory resources. These applications have very high-power consumption due to bottlenecks in the electrical interconnection between processing units. This project aims to reduce both communication energy and latency by integrating universally available accelerators with silicon photonics. It also aims to increase system throughput by exploiting emerging technologies in silicon photonic reconfigurable interconnects; this will allow the system to balance itself in real time to accommodate changes in workloads and data flows.
Large-scale parallel simulations and data analysis drive scientific discovery across many disciplines. To drive larger and more detailed simulations, and deal with larger data volumes, exascale machines capable of 1018 operations per second are expected within the next five to ten years. However, the complexity of developing and adapting modern simulation codes for these architectures is increasing rapidly. Workloads on exascale machines will be billion-way parallel. Exploiting the full capability of the machine will require carefully assigning application tasks to cores, accelerators, deep memories, and other heterogeneous compute resources, while simultaneously optimizing for time to solution, data movement, power, and resilience. Optimizations that improve performance on one machine may slow down another. Worse, applications themselves are dauntingly complex. Production simulations comprise millions of lines of code and use sophisticated, adaptive algorithms whose performance is input-dependent. Complex workflows can couple multi-physics simulation with data preprocessing and post-processing modules.
The interaction of a linearly polarized intense laser pulse with an ultrathin nanometer plasma layer is investigated to understand the physics of the ion acceleration. It is shown by the computer simulation that the plasma response to the laser pulse comprises two steps. First, due to the vxB effect, electrons in the plasma layer are extracted and periodic ultrashort relativistic electron bunches are generated every half of a laser period. Second, strongly asymmetric Coulomb explosion of ions in the foil occurs due to the strong electrostatic charge separation, once the foil is burnt through. Followed by the laser accelerated electron bunch, the ion expansion in the forward direction occurs along the laser beam that is much stronger as compared to the backward direction.
This project developed the hyper-QC multiscale method to make possible the computation of previously inaccessible space and time scales for materials with thermally activated defects. The hyper-QC method combines the spatial coarse-graining feature of a finite temperature extension of the quasicontinuum (QC) method (aka “hot-QC”) with the accelerated dynamics feature of hyperdynamics. The hyper-QC method was developed, optimized, and tested from a rigorous mathematical foundation.
I develop a formulation for Hamiltonian dynamics in an accelerator with magnets whose edges follow a spiral. I demonstrate using this Hamiltonian that a spiral FFAG can be made perfectly 'scaling'. I describe how one computes the RF phase during a rapid acceleration cycle to keep the beam at the appropriate RF phase. I examine the effect of tilting an RF cavity with respect a radial line from the center of the machine, potentially with a different angle than the spiral of the magnets. I discuss partially the effects of the finite energy jumps on the dynamics. This is a status report of work that is still incomplete.
This proposal is to develop the software and algorithmic infrastructure needed for the numerical study of quantum chromodynamics (QCD), and of theories that have been proposed to describe physics beyond the Standard Model (BSM) of high energy physics, on current and future computers. This infrastructure will enable users (1) to improve the accuracy of QCD calculations to the point where they no longer limit what can be learned from high-precision experiments that seek to test the Standard Model, and (2) to determine the predictions of BSM theories in order to understand which of them are consistent with the data that will soon be available from the LHC. Work will include the extension and optimizations of community codes for the next generation of leadership class computers, the IBM Blue Gene/Q and the Cray XE/XK, and for the dedicated hardware funded for our field by the Department of Energy. Members of our collaboration at Brookhaven National Laboratory and Columbia University worked on the design of the Blue Gene/Q, and have begun to develop software for it. Under this grant we will build upon their experience to produce high-efficiency production codes for this machine. Cray XE/XK computers with many thousands of GPU accelerators will soon be available, and the dedicated commodity clusters we obtain with DOE funding include growing numbers of GPUs. We will work with our partners in NVIDIA's Emerging Technology group to scale our existing software to thousands of GPUs, and to produce highly efficient production codes for these machines. Work under this grant will also include the development of new algorithms for the effective use of heterogeneous computers, and their integration into our codes. It will include improvements of Krylov solvers and the development of new multigrid methods in collaboration with members of the FASTMath SciDAC Institute, using their HYPRE framework, as well as work on improved symplectic integrators.
The Multiphysics Object Oriented Simulation Environment (MOOSE) is a widely used open- source finite element software for performing multiphysics multiscale simulations in a massively parallel fashion. Recently, the computational team at Idaho National Laboratory (INL) has implemented Probabilistic Machine Learning (ProbML) capabilities in MOOSE—in a parallelized fashion—and enable active learning with large-scale computational models for tasks such as surrogate model development, scale bridging, forward/inverse uncertainty quantification (UQ), Bayesian optimization, etc. This presentation summarizes these developments in MOOSE along with demonstrations on several real applications relevant to nuclear energy. At the fundamental level, samplers like Monte Carlo/Latin Hypercube, variance reduction, parallelized Markov Chain Monte Carlo (MCMC) support uncertainty propagation in both forward and inverse settings. These samplers can be integrated with the Gaussian processes (GP) suite in MOOSE, which offer several variants like scalar GPs, multi-output GPs, and deep GPs, to enable active learning. These GPs can be tuned using gradient-based optimization methods like Adam and its variants or gradient-free methods like the elliptical slice sampler (a variant of MCMC adept under Gaussian settings) for more complex covariance kernels or likelihoods whose gradient computations can be cumbersome. A variety of batch acquisition functions permit parallelized evaluation of the computational model and support different learning objectives with high efficiency like Bayesian inference, global surrogate development, optimization, etc. Furthermore, libtorch integration supports training, evaluation, and re-training of neural networks and other complex machine learning models in active learning settings. The impacts of these developments are shown on several real applications: (1) nuclear fuel inverse UQ and model inadequacy assessment using the Kennedy O’Hagan framework; (2) uncertainty aware surrogate modeling for additive manufacturing to predict field quantities; (3) nuclear reactor rare events analysis; and (4) complex fluid flow prediction using a global surrogate with quantified prediction uncertainty. Finally, the outlook of MOOSE ProbML is discussed for both outer-loop and inner-loop computations in the broad view to accelerate fuels and materials qualification, address gaps in knowledge and data, and assess new reactor/fuel systems.
Quantum computing research and development is growing worldwide; yet the energy and physical resource demands of future quantum-accelerated data centres are unknown. Planning for quantum computing requires strong collaboration between research communities across engineering, physics, environmental sciences, economics, policy, and energy systems and scenario modelling.
Threading edge dislocations (TEDs) in a 4H-SiC epitaxial layer are investigated using high-resolution synchrotron topography. Six types of TED image are confirmed to correspond to the Burgers vector directions by a comparison of computer simulated images and observed topography images in crystal boundaries. Using a mapping method, a wide spatial distribution of the six types of TED is examined in a quarter section of a 2-inch wafer.
Purpose: To improve the quality of liver stereotactic body radiation therapy (SBRT) treatments, a novel 4π framework was developed with accompanying algorithms to optimize non-coplanar beam orientations and fluences. The dose optimization is performed on a patient-specific deliverable beam geometry solution space, parameterized with patient and linear accelerator gantry orientations. Methods and Materials: Beams causing collision between the gantry and the couch or patient were eliminated by simulating all beam orientations using a precise computer assisted design model of the linear accelerator and a human subject. Integrated beam orientation and fluence map optimizations were performed on remaining beams using a greedy column generation method. Testing of the new method was performed on 10 liver SBRT cases previously treated with 50 to 60 Gy in 5 fractions using volumetric modulated arc therapy (VMAT). For each patient, both 14 and 22 non-coplanar fields were selected and optimized to meet the objective of ≥95% of the planning target volume (PTV) covered by 100% of the prescription dose. Doses to organs at risk, normal liver volumes receiving <15 Gy, integral dose, and 50% dose spillage volumes were compared against the delivered clinical VMAT plans. Results: Compared with the VMAT plans, the 4π plans yielded reduced 50% dose spillage volume and integral dose by 22% (range 10%-40%) and 19% (range 13%-26%), respectively. The mean normal liver volume receiving <15 Gy was increased by 51 cc (range 21-107 cc) with a 31% reduction of the mean normal liver dose. Mean doses to the left kidney and right kidney and maximum doses to the stomach and spinal cord were on average reduced by 70%, 51%, 67%, and 64% (P≤.05). Conclusions: This novel 4π non-coplanar radiation delivery technique significantly improved dose gradient, reduced high dose spillage, and improved organ at risk sparing compared with state of the art VMAT plans.
Computing the set of influential nodes with a given size to ensure maximal spread of influence on a complex network is a challenging problem impacting multiple applications. A rigorous approach to influence maximization involves utilization of optimization routines that comes with a high computational cost. In this work, we propose to exploit the existence of communities in complex networks to accelerate the mining of influential seeds. We provide intuitive reasoning to explain why our approach should be able to provide speedups without significantly degrading the extent of the spread of influence when compared to the case of influence maximization without using the community information. Additionally, we have parallelized the complete workflow by leveraging an existing parallel implementation of the Louvain community detection algorithm. We then conduct a series of experiments on a dataset with three representative graphs to first verify our implementation and then demonstrate the speedups. Our method achieves speedups ranging from 3x - 28x for graphs with small number of communities while nearly matching or even exceeding the activation performance on the entire graph. Complexity analysis reveals that dramatic speedups are possible for larger graphs that contain a correspondingly larger number of communities. In addition to the speedups obtained from the utilization of the community structure, scalability results show up to 6.3x speedup on 20 cores relative to the baseline run on 2 cores. Finally, current limitations of the approach are outlined along with the planned next steps.