SEARCH · Engineering Papers
Results for “Arithmetic”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Sampling-based Sublinear Low-rank Matrix Arithmetic Framework for Dequantizing Quantum Machine Learning
We present an algorithmic framework for quantum-inspired classical algorithms on close-to-low-rank matrices, generalizing the series of results started by Tang’s breakthrough quantum-inspired algorithm for recommendation systems [STOC’19]. Motivated by quantum linear algebra algorithms and the quantum singular value transformation (SVT) framework of Gilyén et al. [STOC’19], we develop classical algorithms for SVT that run in time independent of input dimension, under suitable quantum-inspired sampling assumptions. Our results give compelling evidence that in the corresponding QRAM data structure input model, quantum SVT does not yield exponential quantum speedups. Since the quantum SVT framework generalizes essentially all known techniques for quantum linear algebra, our results, combined with sampling lemmas from previous work, suffice to generalize all prior results about dequantizing quantum machine learning algorithms. In particular, our classical SVT framework recovers and often improves the dequantization results on recommendation systems, principal component analysis, supervised clustering, support vector machines, low-rank regression, and semidefinite program solving. We also give additional dequantization results on low-rank Hamiltonian simulation and discriminant analysis. Our improvements come from identifying the key feature of the quantum-inspired input model that is at the core of all prior quantum-inspired results: ℓ 2 -norm sampling can approximate matrix products in time independent of their dimension. We reduce all our main results to this fact, making our exposition concise, self-contained, and intuitive.
Efficient Floating-Point Arithmetic on Fault-Tolerant Quantum Computers
Explore the source record for details and available documents.
Increasing Accuracy of Iterative Refinement in Limited Floating-Point Arithmetic on Half-Precision Accelerators.
Abstract not provided.
Dynamic variable precision computation
A conversion unit converts operands from a conventional number system that represents each binary number in the operands as one bit to redundant number system (RNS) operands that represent each binary number as a plurality of bits. An arithmetic logic unit performs an arithmetic operation on the RNS operands in a direction from a most significant bit (MSB) to a least significant bit (LSB). The arithmetic logic unit stops performing the arithmetic operation prior to performing the arithmetic operation on a target binary number indicated by a dynamic precision associated with the RNS operands. In some cases, a power supply provides power to bit slices in the arithmetic logic unit and a clock signal generator provides clock signals to the bit slices. Gate logic is configured to gate the power or the clock signals provided to a subset of the bit slices.
Dynamic variable precision computation
A conversion unit converts operands from a conventional number system that represents each binary number in the operands as one bit to redundant number system (RNS) operands that represent each binary number as a plurality of bits. An arithmetic logic unit performs an arithmetic operation on the RNS operands in a direction from a most significant bit (MSB) to a least significant bit (LSB). The arithmetic logic unit stops performing the arithmetic operation prior to performing the arithmetic operation on a target binary number indicated by a dynamic precision associated with the RNS operands. In some cases, a power supply provides power to bit slices in the arithmetic logic unit and a clock signal generator provides clock signals to the bit slices. Gate logic is configured to gate the power or the clock signals provided to a subset of the bit slices.
The use of imprecise processing to improve accuracy in weather and climate prediction
The use of stochastic processing hardware and low precision arithmetic in atmospheric models is investigated. Stochastic processors allow hardware-induced faults in calculations, sacrificing bit-reproducibility and precision in exchange for improvements in performance and potentially accuracy of forecasts, due to a reduction in power consumption that could allow higher resolution. A similar trade-off is achieved using low precision arithmetic, with improvements in computation and communication speed and savings in storage and memory requirements. As high-performance computing becomes more massively parallel and power intensive, these two approaches may be important stepping stones in the pursuit of global cloud-resolving atmospheric modelling. The impact of both hardware induced faults and low precision arithmetic is tested using the Lorenz '96 model and the dynamical core of a global atmosphere model. In the Lorenz '96 model there is a natural scale separation; the spectral discretisation used in the dynamical core also allows large and small scale dynamics to be treated separately within the code. Such scale separation allows the impact of lower-accuracy arithmetic to be restricted to components close to the truncation scales and hence close to the necessarily inexact parametrised representations of unresolved processes. By contrast, the larger scales are calculated using high precision deterministic arithmetic. Hardware faults from stochastic processors are emulated using a bit-flip model with different fault rates. Our simulations show that both approaches to inexact calculations do not substantially affect the large scale behaviour, provided they are restricted to act only on smaller scales. By contrast, results from the Lorenz '96 simulations are superior when small scales are calculated on an emulated stochastic processor than when those small scales are parametrised. This suggests that inexact calculations at the small scale could reduce computation and power costs without adversely affecting the quality of the simulations. This would allow higher resolution models to be run at the same computational cost.
Fine-Grained Exploitation of Mixed Precision for Faster CNN Training
As deep convolutional neural networks (CNNs) have become increasingly popular and successful at an ever-widening number of machine learning tasks specialized hardware has become increasingly available for training and deploying them. NVIDIA's recent Volta architecture includes tensor cores which perform a fused operation reduced and mixed precision (16-bit multiply, 32-bit accumulate). Recent research indicates that, typically, very little is lost (in terms of training accuracy) when half precision is used in place of single precision, and performance gains can be made by doing arithmetic in reduced precision. In this work we demonstrate that making layer-by-layer choices as to the arithmetic/data precision can lead to further performance improvement. In our study of 25,200 CNNs we demonstrate an average speedup (over purely half precision) of 1.27x and speedups as high as 3.64x by appropriately combining single and half precision arithmetic and data types on a layer-by-layer basis.
MRI-based treatment planning with pseudo CT generated through atlas registration
Purpose: To evaluate the feasibility and accuracy of magnetic resonance imaging (MRI)-based treatment planning using pseudo CTs generated through atlas registration. Methods: A pseudo CT, providing electron density information for dose calculation, was generated by deforming atlas CT images previously acquired on other patients. The authors tested 4 schemes of synthesizing a pseudo CT from single or multiple deformed atlas images: use of a single arbitrarily selected atlas, arithmetic mean process using 6 atlases, and pattern recognition with Gaussian process (PRGP) using 6 or 12 atlases. The required deformation for atlas CT images was derived from a nonlinear registration of conjugated atlas MR images to that of the patient of interest. The contrasts of atlas MR images were adjusted by histogram matching to reduce the effect of different sets of acquisition parameters. For comparison, the authors also tested a simple scheme assigning the Hounsfield unit of water to the entire patient volume. All pseudo CT generating schemes were applied to 14 patients with common pediatric brain tumors. The image similarity of real patient-specific CT and pseudo CTs constructed by different schemes was compared. Differences in computation times were also calculated. The real CT in the treatment planning system was replaced with the pseudo CT, and the dose distribution was recalculated to determine the difference. Results: The atlas approach generally performed better than assigning a bulk CT number to the entire patient volume. Comparing atlas-based schemes, those using multiple atlases outperformed the single atlas scheme. For multiple atlas schemes, the pseudo CTs were similar to the real CTs (correlation coefficient, 0.787–0.819). The calculated dose distribution was in close agreement with the original dose. Nearly the entire patient volume (98.3%–98.7%) satisfied the criteria of chi-evaluation (<2% maximum dose and 2 mm range). The dose to 95% of the volume and the percentage of volume receiving at least 95% of the prescription dose in the planning target volume differed from the original values by less than 2% of the prescription dose (root-mean-square, RMS < 1%). The PRGP scheme did not perform better than the arithmetic mean process with the same number of atlases. Increasing the number of atlases from 6 to 12 often resulted in improvements, but statistical significance was not always found. Conclusions: MRI-based treatment planning with pseudo CTs generated through atlas registration is feasible for pediatric brain tumor patients. The doses calculated from pseudo CTs agreed well with those from real CTs, showing dosimetric accuracy within 2% for the PTV when multiple atlases were used. The arithmetic mean process may be a reasonable choice over PRGP for the synthesis scheme considering performance and computational costs.
TabVar: Tabulated Variables
TabVar: A Python library for manipulating datasets in the form of tabulated variables. Tables in tabvar contain many columns representing independent variables, but exactly one distinguished column for the dependent variable. Having a single distinguished column allows a natural lifting of arithmetic operators to tables, much (and in fact fully generalizing) multidimensional array arithmetic. The convenient syntax of whole-table arithmetic, along with the usual operations of filtering and aggregation, and all in the setting of python's interactive REPL allows for rapid exploration of datasets.
Technology Performance Level (TPL) Scoring Tool
Three different ways of combining scores are used in the revised formulation. These are arithmetic mean, geometric mean and multiplication with normalisation. Arithmetic mean is used when combining scores that measure similar attributes, e.g. used for combining costs. The arithmetic mean has the property that it is similar to a logical OR, e.g. when combining costs it does not matter what the individual costs are only what the combined cost is. Geometric mean and Multiplication are used when combining scores that measure disparate attributes. Multiplication is similar to a logical AND, it is used to combine ‘must haves.’ As a result, this method is more punitive than the geometric mean; to get a good score in the combined result it is necessary to have a good score in ALL of the inputs. e.g. the different types of survivability are ‘must haves.’ On balance, the revised TPL is probably less punitive than the previous spreadsheet, multiplication is used sparingly as a method of combining scores. This is in line with the feedback of the Wave Energy Prize judges.
Technical report: Design of the accessor
As we have seen the arithmetic power of hardware architectures growing much faster than the memory bandwidth, there is an urgent need to rethink the way we operate, communicate, and store data. A promising strategy for overcoming the “memory wall” is the decoupling of memory precision from arithmetic precision as proposed in Anzt et al. Such a decoupling of memory precision and arithmetic precision requires an “accessor” that handles the conversion between formats (resp. compression of the data) on the fly in registers. Such an accessor has to be general in the sense that it can handle a wide range of floating point formats and compression strategies, and still allows for further extension. At the same time, is important that the use of the accessor is not a burden for the developer. Therefore, it is important to abstract all implementation details from the user, hide the underlying storage format from the user, and make the accessor cross-platform portable. In this document we present the design of an accessor that aims at fulfilling all these requirements.
Technical Report: Accessor Realization
As we have seen the arithmetic power of hardware architectures growing much faster than the memory bandwidth, there is an urgent need to rethink the way we operate, communicate, and store data. A promising strategy for overcoming the “memory wall” is the decoupling of memory precision from arithmetic precision as proposed in Anzt et al.. Such a decoupling of memory precision and arithmetic precision requires a “memory accessor” that handles the conversion between formats (resp. compression of the data) on the fly in registers. For such a memory accessor to be useful in practice, it has to combine multiple characteristics.
The Implications of 3D Thermal Structure on 1D Atmospheric Retrieval
Using the atmospheric structure from a 3D global radiation-hydrodynamic simulation of HD 189733b and the open-source Bayesian Atmospheric Radiative Transfer (BART) code, we investigate the difference between the secondary-eclipse temperature structure produced with a 3D simulation and the best-fit 1D retrieved model. Synthetic data are generated by integrating the 3D models over the Spitzer , the Hubble Space Telescope ( HST ), and the James Web Space Telescope ( JWST ) bandpasses, covering the wavelength range between 1 and 11 μ m where most spectroscopically active species have pronounced features. Using the data from different observing instruments, we present detailed comparisons between the temperature–pressure profiles recovered by BART and those from the 3D simulations. We calculate several averages of the 3D thermal structure and explore which particular thermal profile matches the retrieved temperature structure. We implement two temperature parameterizations that are commonly used in retrieval to investigate different thermal profile shapes. To assess which part of the thermal structure is best constrained by the data, we generate contribution functions for our theoretical model and each of our retrieved models. Our conclusions are strongly affected by the spectral resolution of the instruments included, their wavelength coverage, and the number of data points combined. We also see some limitations in each of the temperature parametrizations, as they are not able to fully match the complex curvatures that are usually produced in hydrodynamic simulations. The results show that our 1D retrieval is recovering a temperature and pressure profile that most closely matches the arithmetic average of the 3D thermal structure. When we use a higher resolution, more data points, and a parametrized temperature profile that allows more flexibility in the middle part of the atmosphere, we find a better match between the retrieved temperature and pressure profile and the arithmetic average. The Spitzer and HST simulated observations sample deep parts of the planetary atmosphere and provide fewer constraints on the temperature and pressure profile, while the JWST observations sample the middle part of the atmosphere, providing a good match with the middle and most complex part of the arithmetic average of the 3D temperature structure.
Evaluating marginal likelihood with thermodynamic integration method and comparison with several other numerical methods
Evaluating marginal likelihood is the most critical and computationally expensive task, when conducting Bayesian model averaging to quantify parametric and model uncertainties. The evaluation is commonly done by using Laplace approximations to evaluate semianalytical expressions of the marginal likelihood or by using Monte Carlo (MC) methods to evaluate arithmetic or harmonic mean of a joint likelihood function. This study introduces a new MC method, i.e., thermodynamic integration, which has not been attempted in environmental modeling. Instead of using samples only from prior parameter space (as in arithmetic mean evaluation) or posterior parameter space (as in harmonic mean evaluation), the thermodynamic integration method uses samples generated gradually from the prior to posterior parameter space. This is done through a path sampling that conducts Markov chain Monte Carlo simulation with different power coefficient values applied to the joint likelihood function. The thermodynamic integration method is evaluated using three analytical functions by comparing the method with two variants of the Laplace approximation method and three MC methods, including the nested sampling method that is recently introduced into environmental modeling. The thermodynamic integration method outperforms the other methods in terms of their accuracy, convergence, and consistency. The thermodynamic integration method is also applied to a synthetic case of groundwater modeling with four alternative models. The application shows that model probabilities obtained using the thermodynamic integration method improves predictive performance of Bayesian model averaging. As a result, the thermodynamic integration method is mathematically rigorous, and its MC implementation is computationally general for a wide range of environmental problems.
Mixed-precision iterative refinement using tensor cores on GPUs to accelerate solution of linear systems
Double-precision floating-point arithmetic (FP64) has been the de facto standard for engineering and scientific simulations for several decades. Problem complexity and the sheer volume of data coming from various instruments and sensors motivate researchers to mix and match various approaches to optimize compute resources, including different levels of floating-point precision. In recent years, machine learning has motivated hardware support for half-precision floating-point arithmetic. A primary challenge in high-performance computing is to leverage reduced-precision and mixed-precision hardware. We show how the FP16/FP32 Tensor Cores on NVIDIA GPUs can be exploited to accelerate the solution of linear systems of equations Ax = b without sacrificing numerical stability. The techniques we employ include multiprecision LU factorization, the preconditioned generalized minimal residual algorithm (GMRES), and scaling and auto-adaptive rounding to avoid overflow. We also show how to efficiently handle systems with multiple right-hand sides. On the NVIDIA Quadro GV100 (Volta) GPU, we achieve a 4×-5× performance increase and 5× better energy efficiency versus the standard FP64 implementation while maintaining an FP64 level of numerical stability.
An Analytical Time–Domain Expression for the Net Ripple Produced by Parallel Interleaved Converters
We apply modular arithmetic and Fourier series to analyze the superposition of N interleaved triangular waveforms with identical amplitudes and duty-ratios. Here, interleaving refers to the condition when a collection of periodic waveforms with identical periods are each uniformly phase-shifted across one period. The main result is a time-domain expression which provides an exact representation of the summed and interleaved triangular waveforms, where the peak amplitude and parameters of the time-periodic component are all specified in closed-form. Analysis is general and can be used to study various applications in multi-converter systems. This model is unique not only in that it reveals a simple and intuitive expression for the net ripple, but its derivation via modular arithmetic and Fourier series is distinct from prior approaches. The analytical framework is experimentally validated with a system of three parallel converters under time-varying operating conditions.
Improved Nested Sampling and Surrogate-Enabled Comparison With Other Marginal Likelihood Estimators
Estimating marginal likelihood is of central importance to Bayesian model selection and/or model averaging. The nested sampling method has been recently used together with the Metropolis-Hasting (M-H) sampling algorithm for estimating marginal likelihood. This study develops a new implementation of nested sampling by using the DiffeRential Evolution Adaptive Metropolis (DREAMzs) sampling algorithm. The two implementations of nested sampling are evaluated for four models of a synthetic groundwater flow modeling. The DREAMzs-based nested sampling outperforms the M-H-based nested sampling in terms of their accuracy, convergence, efficiency, and stability, which is attributed to the fact that DREAMzs is more robust than M-H for parameter sampling. The nested sampling method is also compared with four other methods (arithmetic mean, harmonic mean, stabilized harmonic mean, and thermodynamic integration) commonly used for estimating marginal likelihood and posterior probability of the four groundwater models. The comparative study requires hundreds of millions of model executions, which would not be possible without using surrogate models to replace the original models. Using the arithmetic mean estimator as the reference, the comparison reveals that thermodynamic integration outperforms nested sampling in terms of accuracy and stability, whereas nested sampling is more computationally efficient to reach to convergence. Here, the harmonic mean and stabilized harmonic mean methods give the worst marginal likelihood estimation. Accurate marginal likelihood estimation is important for accurate estimation of posterior model probability and better predictive performance of Bayesian model averaging.