Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “restricted boltzmann machine”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

36 records · Page 2

Classical versus quantum models in machine learning: insights from a finance application

Although several models have been proposed towards assisting machine learning (ML) tasks with quantum computers, a direct comparison of the expressive power and efficiency of classical versus quantum models for datasets originating from real-world applications is one of the key milestones towards a quantum ready era. Here, we take a first step towards addressing this challenge by performing a comparison of the widely used classical ML models known as restricted Boltzmann machines (RBMs), against a recently proposed quantum model, now known as quantum circuit Born machines (QCBMs). Both models address the same hard tasks in unsupervised generative modeling, with QCBMs exploiting the probabilistic nature of quantum mechanics and a candidate for near-term quantum computers, as experimentally demonstrated in three different quantum hardware architectures to date. To address the question of the performance of the quantum model on real-world classical data sets, we construct scenarios from a probabilistic version out of the well-known portfolio optimization problem in finance, by using time-series pricing data from asset subsets of the S&P500 stock market index. It is remarkable to find that, under the same number of resources in terms of parameters for both classical and quantum models, the quantum models seem to have superior performance on typical instances when compared with the canonical training of the RBMs. Our simulations are grounded on a hardware efficient realization of the QCBMs on ion-trap quantum computers, by using their native gate sets, and therefore readily implementable in near-term quantum devices.

97 MATHEMATICS AND COMPUTING↗

Wildfire Segmentation From Remotely Sensed Data Using Quantum-Compatible Conditional Vector Quantized-Variational Autoencoders

Wildfires represent a critical environmental hazard with multifaceted implications for ecosystems, communities, and public health [1]. The escalating frequency and intensity of wildfires globally have intensified the urgency for robust segmentation methodologies to facilitate effective mitigation, response, and recovery strategies [2]. Accurate wildfire segmentation is pivotal for delineating fire boundaries, assessing progression patterns, and prioritizing resource allocation during emergency scenarios. Furthermore, precise segmentation enables stakeholders, including policymakers, environmental scientists, and emergency responders, to formulate evidence-based strategies, thereby minimizing socio-economic disruptions and ecological degradation. Consequently, advancing wildfire segmentation techniques through innovative technological interventions remains a paramount research imperative. Although foundational in wildfire segmentation, traditional deterministic models exhibit inherent limitations that compromise their efficacy in dynamic and uncertain environments. These models often operate on rigid algorithms prioritizing deterministic classifications, thereby overlooking the inherent complexities and uncertainties associated with wildfire behavior and satellite data variability. Such deterministic frameworks tend to produce oversimplified representations that fail to capture the intricate nuances of evolving fire dynamics, spatial heterogeneity, and environmental interactions [1]. Consequently, the deterministic approach’s propensity for uncertainty collapsing [1, 3] hampers the accuracy, reliability, and applicability of segmentation outcomes in real-world scenarios. Contrastingly, stochastic models offer a more nuanced and adaptable framework for wildfire segmentation. By integrating probabilistic elements into the modeling paradigm, stochastic approaches, particularly probabilistic approaches such as variational auto encoders (VAEs) [4], facilitate comprehensive uncertainty assessment, enabling researchers to quantify and incorporate uncertainties into segmentation outcomes effectively. This probabilistic nature empowers stochastic models to encapsulate variability, account for data inconsistencies, and adapt to evolving environmental conditions, enhancing segmentation accuracy, reliability, and robustness. Embracing stochastic methodologies thus catalyzes advancements in wildfire science by fostering a more holistic, adaptive, and resilient segmentation framework. Despite VAEs demonstrating significant promise in various applications, they come with inherent limitations that have garnered attention within the machine learning community. One of the primary drawbacks lies in their reliance on static priors, which essentially assume a fixed distribution for latent variables, thereby limiting the model’s flexibility to capture complex data structures effectively [5]. This static nature leads to suboptimal representations, especially when dealing with complex and high-dimensional data. Additionally, VAEs often struggle with generating sharp and realistic samples, a phenomenon commonly referred to as mode collapse [5, 7, 6]. Furthermore, the optimization process in VAEs, which involves balancing the reconstruction loss and the regularization term, can sometimes be challenging to fine-tune [7]. In recent efforts to address these shortcomings, alternative approaches like Vector Quantized Variational Auto encoders(VQ-VAEs) [7], address the challenges by incorporating discrete latent variables and leveraging techniques that enhance the quality and diversity of generated samples while maintaining efficient training dynamics. VQ-VAEs propose a dynamic prior distribution generation mechanism that diverges from the static priors commonly associated with traditional VAEs. This dynamic approach allows for more adaptive and context-aware latent variable representations, thereby potentially capturing complex data structures more effectively. Unlike autoregressive prior models such as PixelCNN, which, despite their ability to model dependencies across data dimensions, suffer from significant computational inefficiencies and lack flexibility in handling diverse datasets. In our work, we propose to use a generative quantum-compatible approach to help alleviate the shortcomings of autoregressive prior model in VQ-VAEs. Restricted Boltzmann Machines (RBMs) are a viable alternative prior model that can learn prior distributions in a faster and more flexible manner. In this research endeavor, we meticulously curate a state-of-the-art dataset leveraging satellite MODIS data in conjunction with VIIRS fire masks, derived from Fire Radiative Power (FRP), thereby encapsulating diverse wildfire scenarios and environmental contexts. We developed a conditional VQ-VAE architecture with the RBM prior model that is trained in a supervised manner for segmenting wildfire masks. This innovative approach synergistically harnesses deep learning capabilities, enabling the generation of segmentation maps characterized by heightened precision, granularity, and contextual relevance. Furthermore, replacing the autoregressive prior learning method proposed by the original VQ-VAE with a prior density approximation via quantum-compatible RBM facilitates expedited inference processes, augments flexibility in prior sampling, optimizes computational efficiency and establishes a groundbreaking benchmark in wildfire segmentation methodologies.

quantum machine learning↗

Non-convex Multi-species Hopfield Models

In this work we introduce a multi-species generalization of the Hopfield model for associative memory, where neurons are divided into groups and both inter-groups and intra-groups pair-wise interactions are considered, with different intensities. Thus, this system contains two of the main ingredients of modern deep neural-network architectures: Hebbian interactions to store patterns of information and multiple layers coding different levels of correlations. The model is completely solvable in the low-load regime with a suitable generalization of the Hamilton–Jacobi technique, despite the Hamiltonian can be a non-definite quadratic form of the Mattis magnetizations. The family of multi-species Hopfield model includes, as special cases, the 3-layers Restricted Boltzmann Machine with Gaussian hidden layer and the Bidirectional Associative Memory model.

97 MATHEMATICS AND COMPUTING↗

Quantum computing based hybrid deep learning for fault diagnosis in electrical power systems

Quantum computing (QC) and deep learning have shown promise of supporting transformative advances and have recently gained popularity in a wide range of areas. Here, this paper proposes a hybrid QC-based deep learning framework for fault diagnosis of electrical power systems that combine the feature extraction capabilities of conditional restricted Boltzmann machine with an efficient classification of deep networks. Computational challenges stemming from the complexities of such deep learning models are overcome by QC-based training methodologies that effectively leverage the complementary strengths of quantum assisted learning and classical training techniques. The proposed hybrid QC-based deep learning framework is tested on a simulated electrical power system with 30 buses and wide variations of substation and transmission line faults, to demonstrate the framework’s applicability, efficiency, and generalization capabilities. High computational efficiency is enjoyed by the proposed hybrid approach in terms of computational effort required and quality of diagnosis performance over classical training methods. In addition, superior and reliable fault diagnosis performance with faster response time is achieved over state-of-the-art pattern recognition methods based on artificial neural networks (ANN) and decision trees (DT).

42 ENGINEERING↗

Conditioned quantum-assisted deep generative surrogate for particle-calorimeter interactions

Particle collisions at accelerators like the Large Hadron Collider (LHC), recorded by experiments such as ATLAS and CMS, enable precise standard model measurements and searches for new phenomena. Simulating these collisions significantly influences experiment design and analysis but incurs immense computational costs, projected at millions of CPU-years annually during the high luminosity LHC (HL-LHC) phase. Currently, simulating a single event with Geant4 consumes around 1000 CPU seconds, with calorimeter simulations especially demanding. To address this, we propose a conditioned quantum-assisted generative model, integrating a conditioned variational autoencoder (VAE) and a conditioned restricted Boltzmann machine (RBM). Our RBM architecture is tailored for D-Wave’s Pegasus-structured advantage quantum annealer for sampling, leveraging the flux bias for conditioning. This approach combines classical RBMs as universal approximators for discrete distributions with quantum annealing’s speed and scalability. We also introduce an adaptive method for efficiently estimating effective inverse temperature, and validate our framework on Dataset 2 of CaloChallenge.

97 MATHEMATICS AND COMPUTING↗

Magnetic phases of spatially modulated spin-1 chains in Rydberg excitons: Classical and quantum simulations

Here, in this work, we study the magnetic phases of a spatially modulated chain of spin-1 Rydberg excitons. Using the Density Matrix Renormalization Group (DMRG) technique, we study various magnetic and topologically nontrivial phases using both single-particle properties, such as local magnetization and quantum entropy, and many-body ones, such as pair-wise Néel and long-range string correlations. In particular, we investigate the emergence and robustness of the Haldane phase, a topological phase of anti-ferromagnetic spin-1 chains. Furthermore, we devise a hybrid quantum algorithm employing restricted Boltzmann machine to simulate the ground state of such a system that shows very good agreement with the results of exact diagonalization and DMRG.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Efficient quantum state tomography with mode-assisted training

Neural networks (NNs) representing quantum states are typically trained using Markov chain Monte Carlo based methods. However, unless specifically designed, such samplers only consist of local moves, making the slow-mixing problem prominent even for extremely simple quantum states. Here, we propose to use mode-assisted training that provides global information via the modes of the NN distribution. Further, applied to quantum state tomography using restricted Boltzmann machines, this method improves the quality of reconstructed quantum states by orders of magnitude. The method is applicable to other types of NNs and may efficiently tackle problems previously unmanageable.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Entanglement features of random neural network quantum states

Restricted Boltzmann machines (RBMs) are a class of neural networks that have been successfully employed as a variational ansatz for quantum many-body wave functions. Here, we develop an analytic method to study quantum many-body spin states encoded by random RBMs with independent and identically distributed complex Gaussian weights. By mapping the computation of ensemble-averaged quantities to statistical mechanics models, we are able to investigate the parameter space of the RBM ensemble in the thermodynamic limit. We discover qualitatively distinct wave functions by varying RBM parameters, which correspond to distinct phases in the equivalent statistical mechanics model. Notably, there is a regime in which the typical RBM states have near-maximal entanglement entropy in the thermodynamic limit, similar to that of Haar-random states. However, these states generically exhibit nonergodic behavior in the Ising basis, and do not form quantum state designs, making them distinguishable from Haar-random states.

36 MATERIALS SCIENCE↗

Quantum neural networks to simulate many-body quantum systems

We conduct experimental simulations of many-body quantum systems using a hybrid classical-quantum algorithm. In our setup, the wave function of the transverse field quantum Ising model is represented by a restricted Boltzmann machine. This neural network is then trained using variational Monte Carlo assisted by a D -wave quantum sampler to find the ground-state energy. Our results clearly demonstrate that already the first generation of quantum computers can be harnessed to tackle nontrivial problems concerning physics of many-body quantum systems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Optimizing temperature distributions for training neural quantum states using parallel tempering

Parametrized artificial neural networks (ANNs) can be very expressive ansatzes for variational algorithms, reaching state-of-the-art energies on many quantum many-body Hamiltonians. Nevertheless, the training of the ANN can be slow and stymied by the presence of local minima in the parameter landscape. One approach to mitigate this issue is to use parallel tempering methods, and in this work, we focus on the role played by the temperature distribution of the parallel tempering replicas. Using an adaptive method that adjusts the temperatures in order to equate the exchange probability between neighboring replicas, we show that this temperature optimization can significantly increase the success rate of the variational algorithm with negligible computational cost by eliminating bottlenecks in the replicas' random walk. Furthermore, we demonstrate this using two different neural networks, a restricted Boltzmann machine and a feedforward network, which we use to study a toy problem based on a permutation invariant Hamiltonian with a pernicious local minimum and the 𝐽 1 −𝐽 2 model on a rectangular lattice.

Neural network simulations↗

Autoregressive Neural Network for Simulating Open Quantum Systems via a Probabilistic Formulation

The theory of open quantum systems lays the foundation for a substantial part of modern research in quantum science and engineering. Rooted in the dimensionality of their extended Hilbert spaces, the high computational complexity of simulating open quantum systems calls for the development of strategies to approximate their dynamics. In this Letter, we present an approach for tackling open quantum system dynamics. Using an exact probabilistic formulation of quantum physics based on positive operator-valued measure, we compactly represent quantum states with autoregressive neural networks; such networks bring significant algorithmic flexibility due to efficient exact sampling and tractable density. We further introduce the concept of string states to partially restore the symmetry of the autoregressive neural network and improve the description of local correlations. Efficient algorithms have been developed to simulate the dynamics of the Liouvillian superoperator using a forward-backward trapezoid method and find the steady state via a variational formulation. Our approach is benchmarked on prototypical one-dimensional and two-dimensional systems, finding results which closely track the exact solution and achieve higher accuracy than alternative approaches based on using Markov chain Monte Carlo method to sample restricted Boltzmann machines. Our Letter provides general methods for understanding quantum dynamics in various contexts, as well as techniques for solving high-dimensional probabilistic differential equations in classical setups.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Exact block encoding of imaginary time evolution with universal quantum neural networks

We develop a constructive approach to generate quantum neural networks capable of representing the exact thermal states of all many-body qubit Hamiltonians. The Trotter expansion of the imaginary time propagator is implemented through an exact block encoding by means of a unitary, restricted Boltzmann machine architecture. Marginalization over the hidden-layer neurons (auxiliary qubits) creates the nonunitary action on the visible layer. Then, we introduce a unitary deep Boltzmann machine architecture in which the hidden-layer qubits are allowed to couple laterally to other hidden qubits. We prove that this wave-function is closed under the action of the imaginary time propagator and, more generally, can represent the action of a universal set of quantum gate operations. We provide analytic expressions for the coefficients for both architectures, thus enabling exact network representations of thermal states without stochastic optimization of the network parameters. In the limit of large imaginary time, the yields the ground state of the system. The number of qubits grows linearly with the number of interactions and total imaginary time for a fixed interaction order. Both networks can be readily implemented on quantum hardware via midcircuit measurements of auxiliary qubits. If only one auxiliary qubit is measured and reset, the circuit depth scales linearly with imaginary time and number of interactions, while the width is constant. Alternatively, one can employ a number of auxiliary qubits linearly proportional to the number of interactions, and circuit depth grows linearly with imaginary time only. Every midcircuit measurement has a postselection success probability, and the overall success probability is equal to the product of the probabilities of the midcircuit measurements.

97 MATHEMATICS AND COMPUTING↗

Optimization, Inference and Learning for Advanced Networks

This piece of software aims at providing an implementation of methods to perform efficient optimization, probabilistic inference and model selection and learning for various type of networked systems. The software contains a suite of methods and functions that: 1) perform probabilistic inference and random sampling on networked models 2) select or learn networked models from data 3) solve optimization tasks with a networked structure Network models that are supported by these features include, but are not limited to, factor graphs and Markov random fields such as Ising models, Potts models, p-spins models, Gaussian random fields and restricted Boltzmann machine.

Hijazi, Hassan↗

Quantum-inspired tempering for ground state approximation using artificial neural networks

A large body of work has demonstrated that parameterized artificial neural networks (ANNs) can efficiently describe ground states of numerous interesting quantum many-body Hamiltonians. However, the standard variational algorithms used to update or train the ANN parameters can get trapped in local minima, especially for frustrated systems and even if the representation is sufficiently expressive. We propose a parallel tempering method that facilitates escape from such local minima. This methods involves training multiple ANNs independently, with each simulation governed by a Hamiltonian with a different "driver" strength, in analogy to quantum parallel tempering, and it incorporates an update step into the training that allows for the exchange of neighboring ANN configurations. We study instances from two classes of Hamiltonians to demonstrate the utility of our approach using Restricted Boltzmann Machines as our parameterized ANN. The first instance is based on a permutation-invariant Hamiltonian whose landscape stymies the standard training algorithm by drawing it increasingly to a false local minimum. The second instance is four hydrogen atoms arranged in a rectangle, which is an instance of the second quantized electronic structure Hamiltonian discretized using Gaussian basis functions. We study this problem in a minimal basis set, which exhibits false minima that can trap the standard variational algorithm despite the problem’s small size. We show that augmenting the training with quantum parallel tempering becomes useful to finding good approximations to the ground states of these problem instances.

Albash, Tameem↗

Efficient solutions of fermionic systems using artificial neural networks

In this study, we explore the similarities and differences between variational Monte Carlo techniques that employ conventional and artificial neural network representations of the ground-state wave function for fermionic systems. Our primary focus is on shallow neural network architectures, specifically the restricted Boltzmann machine, and we examine unsupervised learning algorithms that are appropriate for modeling complex many-body correlations. We assess the advantages and drawbacks of conventional and neural network wave functions by applying them to a range of circular quantum dot systems. Our findings, which include results for systems containing up to 90 electrons, emphasize the efficient implementation of these methods on both homogeneous and heterogeneous high-performance computing facilities.

97 MATHEMATICS AND COMPUTING↗

D-Wave for High Energy Physics

aWe analyze different problems in high-energy physics where quantum computing, and namely quantum annealers, may be able to provide an edge in coming years: 1) The training of Restricted Boltzmann Machines (RBM) is a natural problem for quantum annealers. We explore RBM trained on a D-Wave machine for classification of galaxy morphologies, including a discussion of how to obtain a Boltzmann distribution from the annealer. 2) For this study, we cast the problem of explaining and predicting the production of particles in high energy collisions (color recombination) as an optimization problem and employ classical and quantum computing techniques to find solutions. In particular, we present a quadratic unconstrained binary optimization (QUBO) formulation for quantum annealers. In parallel to these efforts, we also present the cross-platform XACC package for quantum compilation and execution.

Caldeira, J.↗

Dynamic Topology Reconfiguration of Boltzmann Machines on Quantum Annealers

Boltzmann machines have useful roles in deep learning applications, such as generative data modeling, initializing weights for other types of networks, or extracting efficient representations from high-dimensional data. Most Boltzmann machines use restricted topologies that exclude looping connectivity, as such connectivity creates complex distributions that are difficult to sample. We have used an open-system quantum annealer to sample from complex distributions and implement Boltzmann machines with looping connectivity. Further, we have created policies mapping Boltzmann machine variables to the quantum bits of an annealer. These policies, based on correlation and entropy metrics, dynamically reconfigure the topology of Boltzmann machines during training and improve performance.

Liu, Jeremy (ORCID:0000000172299417)↗

A Quantum Annealing Computer Team Addresses Climate Change Predictability

The near confluence of the successful launch of the Orbiting Carbon Observatory2 on July 2, 2014 and the acceptance on August 20, 2015 by Google, NASA Ames Research Center and USRA of a 1152 qubit D-Wave 2X Quantum Annealing Computer (QAC), offered an exceptional opportunity to explore the potential of this technology to address the scientific prediction of global annual carbon uptake by land surface processes. At UMBC,we have collected and processed 20 months of global Level 2 light CO2 data as well as fluorescence data. In addition we have collected ARM data at 2sites in the US and Ameriflux data at more than 20 stations. J. Dorband has developed and implemented a multi-hidden layer Boltzmann Machine (BM) algorithm on the QAC. Employing the BM, we are calculating CO2 fluxes by training collocated OCO-2 level 2 CO2 data with ARM ground station tower data to infer to infer measured CO2 flux data. We generate CO2 fluxes with a regression analysis using these BM derived weights on the level 2 CO2 data for three Ameriflux sites distinct from the ARM stations. P. Gentine has negotiated for the access of K34 Ameriflux data in the Amazon and is applying a neural net to infer the CO2 fluxes. N. Talik validated the accuracy of the BM performance on the QAC against a restricted BM implementation on the IBM Softlayer Cloud with the Nvidia co-processors utilizing the same data sets. G. Nearing and K. Harrison have extended the GSFC LIS model with the NCAR Noah photosynthetic parameterization and have run a 10 year global prediction of the net ecosystem exchange. C. Pellisier is preparing a BM implementation of the Kalman filter data assimilation of CO2 fluxes. At UMBC, R. Prouty is conducting OSSE experiments with the LISNoah model on the IBM iDataPlex to simulate the impact of CO2 fluxes to improve the prediction of global annual carbon uptake. J. LeMoigne and D. Simpson have developed a neural net image registration system that will be used for MODIS ENVI and will be converted to a BM algorithm implementation on the QAC. The first integer adder has been implemented on the D-Wave 2X by A. Shehab that will perform HAAR wavelets for image compression of MODIS scenes. Finally, based on the next generations of QACs, we are preparing a 5-year performance road map on the scalability of the current QAC algorithms.

Science Data Processing↗