Engineering PapersSearch

SEARCH · Engineering Papers

Results for “RBMs”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

Exploring the energy landscape of RBMs: reciprocal space insights into bosons, hierarchical learning and symmetry breaking

Deep generative models have become ubiquitous due to their ability to learn and sample from complex distributions. Despite the proliferation of various frameworks, the relationships among these models remain largely unexplored, a gap that hinders the development of a unified theory of AI learning. In this work, we address two central challenges: clarifying the connections between different deep generative models and deepening our understanding of their learning mechanisms. We focus on Restricted Boltzmann Machines (RBMs), a class of generative models known for their universal approximation capabilities for discrete distributions. By introducing a reciprocal space formulation for RBMs, we reveal a connection between these models, diffusion processes, and systems of coupled bosons. Our analysis shows that at initialization, the RBM operates at a saddle point, where the local curvature is determined by the singular values of the weight matrix, whose distribution follows the Marc̆enko-Pastur law and exhibits rotational symmetry. During training, this rotational symmetry is broken due to hierarchical learning, where different degrees of freedom progressively capture features at multiple levels of abstraction. This leads to a symmetry breaking in the energy landscape, reminiscent of Landau’s theory. This symmetry breaking in the energy landscape is characterized by the singular values and the weight matrix eigenvector matrix. We derive the corresponding free energy in a mean-field approximation. We show that in the limit of infinite size RBM, the reciprocal variables are Gaussian distributed. Our findings indicate that in this regime, there will be some modes for which the diffusion process will not converge to the Boltzmann distribution. To illustrate our results, we trained replicas of RBMs with different hidden layer sizes using the MNIST dataset. Our findings not only bridge the gap between disparate generative frameworks but also shed light on the fundamental processes underpinning learning in deep generative models.

97 MATHEMATICS AND COMPUTING

Learning Model Structural Uncertainty with Gaussian Processes

The advent of commercially available quantum computers has marked the beginning of quantum computing as a reality. Both quantum gate and annealing computers have been released by major computer hardware companies. In this work, the D-Wave 2XTM quantum annealing computer housed at the NASA Advanced Systems computational facility is investigated to accelerate Machine Learning (ML) for image registration. NASA collects large amounts of images over the globe remotely using space-based monitoring. Images of a fixed areas of the land surface are taken over time. Due to the orbit of the sensors, the viewing angles deviate slightly, and it is necessary to align or register the images precisely to create image time series over the land surface. Unaligned images can lead to substantial analysis errors. These time-series are then used in modeling Earth Systems models such as hydrological, weather, and carbon monitoring models. In this work, we consider the Moderate Resolution Image Spectrometer (MODIS) data collected by the NASA's terra satellite. Artificial Neural Networks (ANNs) is a natural fit for ML modelling of images. Several successes have been reported using machine learning related to image processing. We investigate the use of ML to register MODIS images. ANNs are investigated in combination with a Restricted Boltzmann Machines (RBM) as an auto-encoder. We will present results showing the accuracy and efficiency of this approach.The D-Wave 2XTM quantum annealer samples the ground-state wave-function of a spin-Ising systems with quadratic interactions between qubits and a Chimera connectivity. The system sits in a ~15 mK thermal bath. One can think of the system as being placed in the ground state initially and subject to thermal excitations governed by Boltzmann statistics. If this is assumed true, one can use the statistics from the D-Wave 2XTM to train RBMs. Generating statistics for training Boltzmann machines is an NP-hard problem and constitutes the largest compute cost. We investigate the use of the D-Wave 2XTM to accelerate the training of the RBMs in our ANNs and report on the results.

Kouatchou, Jules

Conditioned quantum-assisted deep generative surrogate for particle-calorimeter interactions

Particle collisions at accelerators like the Large Hadron Collider (LHC), recorded by experiments such as ATLAS and CMS, enable precise standard model measurements and searches for new phenomena. Simulating these collisions significantly influences experiment design and analysis but incurs immense computational costs, projected at millions of CPU-years annually during the high luminosity LHC (HL-LHC) phase. Currently, simulating a single event with Geant4 consumes around 1000 CPU seconds, with calorimeter simulations especially demanding. To address this, we propose a conditioned quantum-assisted generative model, integrating a conditioned variational autoencoder (VAE) and a conditioned restricted Boltzmann machine (RBM). Our RBM architecture is tailored for D-Wave’s Pegasus-structured advantage quantum annealer for sampling, leveraging the flux bias for conditioning. This approach combines classical RBMs as universal approximators for discrete distributions with quantum annealing’s speed and scalability. We also introduce an adaptive method for efficiently estimating effective inverse temperature, and validate our framework on Dataset 2 of CaloChallenge.

97 MATHEMATICS AND COMPUTING

Radial Breathing Modes in Cosmochemistry and Meteoritics

One area of continuing interest in cosmochemistry and meteoritics (C&M) is the identification of the nature of Q-phase, although some researchers in C&M are not reporting relevant portions of Raman spectral data. Q is the unidentified carrier of noble gases in carbonaceous chondrites (CCs). Being carbonaceous, the focus has been on any number of Q-candidates arising from the sp2 hybridization of carbon (C). These all derive from various forms of graphene, a monolayer of C atoms packed into a two-dimensional (2D) hexagonal honeycomb lattice that is the basic building block for graphitic materials of all other dimensions for sp2 allotropes of C. As a basic lattice, 2D graphene can be curled into fullerenes (0D), wrapped into carbon nanotubes or CNTs (1D), and stacked into graphite (3D). These take such additional forms as scroll-like carbon whiskers, carbon fibers, carbon onions, GPCs (graphite polyhedral crystals) [6], and GICs (graphite intercalation compounds). Although all of these have been observed in meteoritics, the issue is which can explain the Q-abundances. In brief, one or more of the 0D-3D sp2 hybridization forms of C is Q. For some Q-candidates, the radial breathing modes (RBMs) are the most important Raman active vibrational modes that exist, and bear a direct relevance to solving this puzzle. Typically in C&M they are ignored when present. Their importance is addressed here as smoking-gun signatures for certain Q-candidates and are very relevant to the ultimate identification of Q.

Wilson, T.L.

Quantum Image Denoising: A Framework via Boltzmann Machines, QUBO, and Quantum Annealing

We investigate a framework for binary image denoising via restricted Boltzmann machines (RBMs) that introduces a denoising objective in quadratic unconstrained binary optimization (QUBO) form and is well-suited for quantum annealing. The denoising objective is attained by balancing the distribution learned by a trained RBM with a penalty term for derivations from the noisy image. We derive the statistically optimal choice of the penalty parameter assuming the target distribution has been well-approximated, and further suggest an empirically supported modification to make the method robust to that idealistic assumption. We also show under additional assumptions that the denoised images attained by our method are, in expectation, strictly closer to the noise-free images than the noisy images are. While we frame the model as an image denoising model, it can be applied to any binary data. As the QUBO formulation is well-suited for implementation on quantum annealers, we test the model on a D-Wave Advantage machine, and also test on data too large for current quantum annealers by approximating QUBO solutions through classical heuristics.

restricted Boltzmann machine

Wildfire Segmentation From Remotely Sensed Data Using Quantum-Compatible Conditional Vector Quantized-Variational Autoencoders

Wildfires represent a critical environmental hazard with multifaceted implications for ecosystems, communities, and public health [1]. The escalating frequency and intensity of wildfires globally have intensified the urgency for robust segmentation methodologies to facilitate effective mitigation, response, and recovery strategies [2]. Accurate wildfire segmentation is pivotal for delineating fire boundaries, assessing progression patterns, and prioritizing resource allocation during emergency scenarios. Furthermore, precise segmentation enables stakeholders, including policymakers, environmental scientists, and emergency responders, to formulate evidence-based strategies, thereby minimizing socio-economic disruptions and ecological degradation. Consequently, advancing wildfire segmentation techniques through innovative technological interventions remains a paramount research imperative. Although foundational in wildfire segmentation, traditional deterministic models exhibit inherent limitations that compromise their efficacy in dynamic and uncertain environments. These models often operate on rigid algorithms prioritizing deterministic classifications, thereby overlooking the inherent complexities and uncertainties associated with wildfire behavior and satellite data variability. Such deterministic frameworks tend to produce oversimplified representations that fail to capture the intricate nuances of evolving fire dynamics, spatial heterogeneity, and environmental interactions [1]. Consequently, the deterministic approach’s propensity for uncertainty collapsing [1, 3] hampers the accuracy, reliability, and applicability of segmentation outcomes in real-world scenarios. Contrastingly, stochastic models offer a more nuanced and adaptable framework for wildfire segmentation. By integrating probabilistic elements into the modeling paradigm, stochastic approaches, particularly probabilistic approaches such as variational auto encoders (VAEs) [4], facilitate comprehensive uncertainty assessment, enabling researchers to quantify and incorporate uncertainties into segmentation outcomes effectively. This probabilistic nature empowers stochastic models to encapsulate variability, account for data inconsistencies, and adapt to evolving environmental conditions, enhancing segmentation accuracy, reliability, and robustness. Embracing stochastic methodologies thus catalyzes advancements in wildfire science by fostering a more holistic, adaptive, and resilient segmentation framework. Despite VAEs demonstrating significant promise in various applications, they come with inherent limitations that have garnered attention within the machine learning community. One of the primary drawbacks lies in their reliance on static priors, which essentially assume a fixed distribution for latent variables, thereby limiting the model’s flexibility to capture complex data structures effectively [5]. This static nature leads to suboptimal representations, especially when dealing with complex and high-dimensional data. Additionally, VAEs often struggle with generating sharp and realistic samples, a phenomenon commonly referred to as mode collapse [5, 7, 6]. Furthermore, the optimization process in VAEs, which involves balancing the reconstruction loss and the regularization term, can sometimes be challenging to fine-tune [7]. In recent efforts to address these shortcomings, alternative approaches like Vector Quantized Variational Auto encoders(VQ-VAEs) [7], address the challenges by incorporating discrete latent variables and leveraging techniques that enhance the quality and diversity of generated samples while maintaining efficient training dynamics. VQ-VAEs propose a dynamic prior distribution generation mechanism that diverges from the static priors commonly associated with traditional VAEs. This dynamic approach allows for more adaptive and context-aware latent variable representations, thereby potentially capturing complex data structures more effectively. Unlike autoregressive prior models such as PixelCNN, which, despite their ability to model dependencies across data dimensions, suffer from significant computational inefficiencies and lack flexibility in handling diverse datasets. In our work, we propose to use a generative quantum-compatible approach to help alleviate the shortcomings of autoregressive prior model in VQ-VAEs. Restricted Boltzmann Machines (RBMs) are a viable alternative prior model that can learn prior distributions in a faster and more flexible manner. In this research endeavor, we meticulously curate a state-of-the-art dataset leveraging satellite MODIS data in conjunction with VIIRS fire masks, derived from Fire Radiative Power (FRP), thereby encapsulating diverse wildfire scenarios and environmental contexts. We developed a conditional VQ-VAE architecture with the RBM prior model that is trained in a supervised manner for segmenting wildfire masks. This innovative approach synergistically harnesses deep learning capabilities, enabling the generation of segmentation maps characterized by heightened precision, granularity, and contextual relevance. Furthermore, replacing the autoregressive prior learning method proposed by the original VQ-VAE with a prior density approximation via quantum-compatible RBM facilitates expedited inference processes, augments flexibility in prior sampling, optimizes computational efficiency and establishes a groundbreaking benchmark in wildfire segmentation methodologies.

quantum machine learning