Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “neural circuits”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Fast decay of classification error in variational quantum circuits

Variational quantum circuits (VQCs) have shown great potential in near-term applications. However, the discriminative power of a VQC, in connection to its circuit architecture and depth, is not understood. To unleash the genuine discriminative power of a VQC, we propose a VQC system with the optimal classical post-processing—maximum-likelihood estimation on measuring all VQC output qubits. Via extensive numerical simulations, we find that the error of VQC quantum data classification typically decays exponentially with the circuit depth, when the VQC architecture is extensive—the number of gates does not shrink with the circuit depth. This fast error suppression ends at the saturation towards the ultimate Helstrom limit of quantum state discrimination. On the other hand, non-extensive VQCs such as quantum convolutional neural networks are sub-optimal and fail to achieve the Helstrom limit, demonstrating a trade-off between ansatz complexity and classification performance in general. To achieve the best performance for a given VQC, the optimal classical post-processing is crucial even for a binary classification problem. To simplify VQCs for near-term implementations, we find that utilizing the symmetry of the input properly can improve the performance, while oversimplification can lead to degradation.

Zhang, Bingzhi↗
DOE OSTI2022

Electronegative metal dopants improve switching variability in Al 2 ⁢O 3 resistive switching devices

Resistive random-access memories are promising for nonvolatile memory and brain-inspired computing applications. High variability and low yield of these devices are key drawbacks hindering reliable training of physical neural networks. In this paper, we show that doping an oxide electrolyte, Al 2 ⁢O 3 , with electronegative metals makes resistive switching significantly more reproducible, surpassing the reproducibility requirements for obtaining reliable hardware neuromorphic circuits. Based on density functional theory calculations, the underlying mechanism is hypothesized to be the ease of creating oxygen vacancies in the vicinity of electronegative dopants due to the capture of the associated electrons by dopant midgap states and the weakening of Al-O bonds. These oxygen vacancies and vacancy clusters also bind significantly to the dopant, thereby serving as preferential sites and building blocks in the formation of conducting paths. Throughout this work, we validate this theory experimentally by implanting different dopants over a range of electronegativities in devices made of multiple alternating layers of Al 2 ⁢O 3 and WN and find superior repeatability and yield with highly electronegative metals, Au, Pt, and Pd. These devices also exhibit a gradual SET transition, enabling multibit switching that is desirable for analog computing.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Quantum reservoir computing implementation on coherently coupled quantum oscillators

Quantum reservoir computing is a promising approach for quantum neural networks, capable of solving hard learning tasks on both classical and quantum input data. However, current approaches with qubits suffer from limited connectivity. We propose an implementation for quantum reservoir that obtains a large number of densely connected neurons by using parametrically coupled quantum oscillators instead of physically coupled qubits. We analyze a specific hardware implementation based on superconducting circuits: with just two coupled quantum oscillators, we create a quantum reservoir comprising up to 81 neurons. We obtain state-of-the-art accuracy of 99% on benchmark tasks that otherwise require at least 24 classical oscillators to be solved. Our results give the coupling and dissipation requirements in the system and show how they affect the performance of the quantum reservoir. Beyond quantum reservoir computing, the use of parametrically coupled bosonic modes holds promise for realizing large quantum neural network architectures, with billions of neurons implemented with only 10 coupled quantum oscillators.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Nonvolatile Electrochemical Random‐Access Memory under Short Circuit

Abstract Electrochemical random‐access memory (ECRAM) is a recently developed and highly promising analog resistive memory element for in‐memory computing. One longstanding challenge of ECRAM is attaining retention time beyond a few hours. This short retention has precluded ECRAM from being considered for inference classification in deep neural networks, which is likely the largest opportunity for in‐memory computing. In this work, an ECRAM cell with orders of magnitude longer retention than previously achieved is developed, and which is anticipated to exceed ten years at 85 °C. This study hypothesizes that the origin of this exceptional retention is phase separation, which enables the formation of multiple effectively equilibrium resistance states. This work highlights the promises and opportunities to use phase separation to yield ECRAM cells with exceptionally long, and potentially permanent, retention times.

Kim, Diana S.↗
DOE OSTI2024

Collective neural network behavior in a dynamically driven disordered system of superconducting loops

Collective properties of complex systems composed of many interacting components such as neurons in our brain can be modeled by artificial networks based on disordered systems. We show that a disordered neural network of superconducting loops with Josephson junctions can exhibit computational properties like categorization and associative memory in the time evolution of its state in response to information from external excitations. Superconducting loops can trap multiples of fluxons in many discrete memory configurations defined by the local free energy minima in the configuration space of all possible states. A memory state can be updated by exciting the Josephson junctions to fire or allow the movement of fluxons through the network as the current through them surpasses their critical current thresholds. Simulations performed with a lumped element circuit model of a 4-loop network show that information written through excitations is translated into stable states of trapped flux and their time evolution. Experimental implementation on a high-Tc superconductor YBCO-based 4-loop network shows dynamically stable flux flow in each pathway characterized by the correlations between junction firing statistics. Neural network behavior is observed as energy barriers separating state categories in simulations in response to multiple excitations, and experimentally as junction responses characterizing different flux flow patterns in the network. The state categories that produce these patterns have different temporal stabilities relative to each other and the excitations. This provides strong evidence for time-dependent (short-to-long-term) memories, that are dependent on the geometrical and junction parameters of the loops, as described with a network model.

Josephson junctions↗
DOE OSTI2021

Efficient Trainability of Linear Optical Modules in Quantum Optical Neural Networks

The existence of “barren plateau landscapes” for generic discrete-variable quantum neural networks, which obstructs efficient gradient-based optimization of cost functions defined by global measurements, would be surprising in the case of generic linear optical modules in quantum optical neural networks due to the tunability of the intensity of continuous variable states and the relevant unitary group having exponentially smaller dimension. We demonstrate that coherent light in m modes can be generically compiled efficiently if the total intensity scales sublinearly with m, and extend this result to cost functions based on homodyne, heterodyne, or photon detection measurement statistics, and to noisy cost functions in the presence of attenuation. Here, we further demonstrate efficient trainability of m mode linear optical quantum circuits for variational mean field energy estimation of positive quadratic Hamiltonians for input states that do not have energy exponentially vanishing with m.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

Probabilistic Neural Computing with Stochastic Devices

Abstract The brain has effectively proven a powerful inspiration for the development of computing architectures in which processing is tightly integrated with memory, communication is event‐driven, and analog computation can be performed at scale. These neuromorphic systems increasingly show an ability to improve the efficiency and speed of scientific computing and artificial intelligence applications. Herein, it is proposed that the brain's ubiquitous stochasticity represents an additional source of inspiration for expanding the reach of neuromorphic computing to probabilistic applications. To date, many efforts exploring probabilistic computing have focused primarily on one scale of the microelectronics stack, such as implementing probabilistic algorithms on deterministic hardware or developing probabilistic devices and circuits with the expectation that they will be leveraged by eventual probabilistic architectures. A co‐design vision is described by which large numbers of devices, such as magnetic tunnel junctions and tunnel diodes, can be operated in a stochastic regime and incorporated into a scalable neuromorphic architecture that can impact a number of probabilistic computing applications, such as Monte Carlo simulations and Bayesian neural networks. Finally, a framework is presented to categorize increasingly advanced hardware‐based probabilistic computing technologies.

Misra, Shashank↗
DOE OSTI2023

Ultrafast radiographic imaging and tracking: An overview of instruments, methods, data, and applications

Ultrafast radiographic imaging and tracking (U-RadIT) use state-of-the-art ionizing particle and light sources to experimentally study sub-nanosecond transients or dynamic processes in physics, chemistry, biology, geology, materials science and other fields. These processes are fundamental to modern technologies and applications, such as nuclear fusion energy, advanced manufacturing, communication, and green transportation, which often involve one mole or more atoms and elementary particles, and thus are challenging to compute by using the first principles of quantum physics or other forward models. One of the central problems in U-RadIT is to optimize information yield through, e.g. high-luminosity X-ray and particle sources, efficient imaging and tracking detectors, novel methods to collect data, and large-bandwidth online and offline data processing, regulated by the underlying physics, statistics, and computing power. We review and highlight recent progress in: (a.) Detectors such as high-speed complementary metal-oxide semiconductor (CMOS) cameras, hybrid pixelated array detectors integrated with Timepix4 and other application-specific integrated circuits (ASICs), and digital photon detectors; (b.) U-RadIT modalities such as dynamic phase contrast imaging, dynamic diffractive imaging, and four-dimensional (4D) particle tracking; (c.) U-RadIT data and algorithms such as neural networks and machine learning, and (d.) Applications in ultrafast dynamic material science using XFELs, synchrotrons and laser-driven sources. Hardware-centric approaches to U-RadIT optimization are constrained by detector material properties, low signal-to-noise ratio, high cost and long development cycles of critical hardware components such as ASICs. Interpretation of experimental data, including comparisons with forward models, is frequently hindered by sparse measurements, model and measurement uncertainties, and noise. Alternatively, U-RadIT make increasing use of data science and machine learning algorithms, including experimental implementations of compressed sensing. Machine learning and artificial intelligence approaches, refined by physics and materials information, may also contribute significantly to data interpretation, uncertainty quantification and U-RadIT optimization.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

QuantumNAT: quantum noise-aware training with noise injection, quantization and normalization

Parameterized Quantum Circuits (PQC) are promising towards quantum advantage on near-term quantum hardware. However, due to the large quantum noises (errors), the performance of PQC models has a severe degradation on real quantum devices. Take Quantum Neural Network (QNN) as an example, the accuracy gap between noise-free simulation and noisy results on IBMQ-Yorktown for MNIST-4 classification is over 60%. Existing noise mitigation methods are general ones without leveraging unique characteristics of PQC; on the other hand, existing PQC work does not consider noise effect. To this end, we present QuantumNAT, a PQC-specific framework to perform noise-aware optimizations in both training and inference stages to improve robustness. We experimentally observe that the effect of quantum noise to PQC measurement outcome is a linear map from noise-free outcome with a scaling and a shift factor. Motivated by that, we propose post-measurement normalization to mitigate the feature distribution differences between noise-free and noisy scenarios. Furthermore, to improve the robustness against noise, we propose noise injection to the training process by inserting quantum error gates to PQC according to realistic noise models of quantum hardware. Lastly, post-measurement quantization is introduced to quantize the measurement outcomes to discrete values, achieving the denoising effect. Extensive experiments on 8 classification tasks using 6 quantum devices demonstrate that QuantumNAT improves accuracy by up to 43%, and achieves over 94% 2-class, 80% 4-class, and 34% 10-class classification accuracy measured on real quantum computers. The code for construction and noise-aware training of PQC is available in the TorchQuantum library.

Wang, Hanrui↗
DOE OSTI2022

Unsupervised quantum circuit learning in high energy physics

Unsupervised training of generative models is a machine learning task that has many applications in scientific computing. Here, in this work, we evaluate the efficacy of using quantum circuit-based generative models to generate synthetic data of high energy physics processes. We use nonadversarial, gradient-based training of quantum circuit Born machines to generate joint distributions over two and three variables.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Learning quantum computers' errors using interpretable neural networks

Learning and reducing the errors and noise in quantum computing systems is necessary for achieving quantum computation’s promise. However, rapid advances in experimental quantum computing are making this task increasingly difficult, because state-of-the-art systems now contain hundreds of qubits and many characterization techniques are hard to apply at this scale. Furthermore, complex kinds of errors in these systems, such as crosstalk and non-Markovian effects, must be understood and decreased, but these errors are challenging to study with most existing methods. In this project, we explored using neural networks for scalable characterization of complex errors in quantum computers. We proposed and demonstrated characterizing a quantum computer’s errors with neural networks that have interpretable parameters corresponding to the rates of different kinds of errors, within a sparse Lindbladian parameterization for errors. To enable scaling to many qubit systems, these networks then predict how these errors combine within quantum circuits and impact their outcomes using an efficient approximations. We demonstrated these networks ability to learn coherent crosstalk errors and context-dependent errors in a simulated 4-qubit system.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Ensemble models for circuit topology estimation, fault detection and classification in distribution systems

This paper presents a methodology for simultaneous fault detection, classification, and topology estimation for adaptive protection of distribution systems. The methodology estimates the probability of the occurrence of each one of these events by using a hybrid structure that combines three sub-systems, a convolutional neural network for topology estimation, a fault detection based on predictive residual analysis, and a standard support vector machine with probabilistic output for fault classification. The input to all these sub-systems is the local voltage and current measurements. A convolutional neural network uses these local measurements in the form of sequential data to extract features and estimate the topology conditions. The fault detector is constructed with a Bayesian stage (a multitask Gaussian process) that computes a predictive distribution (assumed to be Gaussian) of the residuals using the input. Since the distribution is known, these residuals can be transformed into a Standard distribution, whose values are then introduced into a one-class support vector machine. The structure allows using a one-class support vector machine without parameter cross-validation, so the fault detector is fully unsupervised. Finally, a support vector machine uses the input to perform the classification of the fault types. All three sub-systems can work in a parallel setup for both performance and computation efficiency. In conclusion, we test all three sub-systems included in the structure on a modified IEEE123 bus system, and we compare and evaluate the results with standard approaches.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Probabilistic partition of unity networks for high–dimensional regression problems

We explore the probabilistic partition of unity network (PPOU-Net) model in the context of high-dimensional regression problems and propose a general framework focusing on adaptive dimensionality reduction. With the proposed framework, the target function is approximated by a mixture of experts model on a low-dimensional manifold, where each cluster is associated with a fixed-degree polynomial. We present a training strategy that leverages the expectation maximization (EM) algorithm. During the training, we alternate between (i) applying gradient descent to update the DNN coefficients; and (ii) using closed-form formulae derived from the EM algorithm to update the mixture of experts model parameters. Under the probabilistic formulation, step (ii) admits the form of embarrassingly paralleliazable weighted least-squares solves. The PPOU-Nets consistently outperform the baseline fully-connected neural networks of comparable sizes in numerical experiments of various data dimensions. Here, we also explore the proposed model in applications of quantum computing, where the PPOU-Nets act as surrogate models for cost landscapes associated with variational quantum circuits.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Subtleties in the trainability of quantum machine learning models

A new paradigm for data science has emerged, with quantum data, quantum models, and quantum computational devices. This field, called quantum machine learning (QML), aims to achieve a speedup over traditional machine learning for data analysis. However, its success usually hinges on efficiently training the parameters in quantum neural networks, and the field of QML is still lacking theoretical scaling results for their trainability. Some trainability results have been proven for a closely related field called variational quantum algorithms (VQAs). While both fields involve training a parametrized quantum circuit, there are crucial differences that make the results for one setting not readily applicable to the other. In this work, we bridge the two frameworks and show that gradient scaling results for VQAs can also be applied to study the gradient scaling of QML models. Our results indicate that features deemed detrimental for VQA trainability can also lead to issues such as barren plateaus in QML. Consequently, our work has implications for several QML proposals in the literature. In addition, we provide theoretical and numerical evidence that QML models exhibit further trainability issues not present in VQAs, arising from the use of a training dataset. We refer to these as dataset-induced barren plateaus. These results are most relevant when dealing with classical data, as here the choice of embedding scheme (i.e., the map between classical data and quantum states) can greatly affect the gradient scaling.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Smart pixel sensors: towards on-sensor filtering of pixel clusters with deep learning

Highly granular pixel detectors allow for increasingly precise measurements of charged particle tracks. Next-generation detectors require that pixel sizes will be further reduced, leading to unprecedented data rates exceeding those foreseen at the High- Luminosity Large Hadron Collider. Signal processing that handles data incoming at a rate of $\mathcal{O}$(40 MHz) and intelligently reduces the data within the pixelated region of the detector at rate will enhance physics performance at high luminosity and enable physics analyses that are not currently possible. Using the shape of charge clusters deposited in an array of small pixels, the physical properties of the traversing particle can be extracted with locally customized neural networks. In this first demonstration, we present a neural network that can be embedded into the on-sensor readout and filter out hits from low momentum tracks, reducing the detector's data volume by 57.1%–75.7%. The network is designed and simulated as a custom readout integrated circuit with 28 nm CMOS technology and is expected to operate at less than 300 μW with an area of less than 0.2 mm 2 . The temporal development of charge clusters is investigated to demonstrate possible future performance gains, and there is also a discussion of future algorithmic and technological improvements that could enhance efficiency, data reduction, and power per area.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Smart pixel sensors Towards on-sensor filtering of pixel clusters with deep learning

High granularity silicon pixel sensors are at the heart of energy frontier particle physics collider experiments. At an collision rate of 40\,MHz, these detectors create massive amounts of data. Signal processing that handles data incoming at those rate and intelligently reduces the data within the pixelated region of the detector \textit{at rate} will enhance physics performance and enable physics analyses that are not currently possible. Using the shape of charge clusters deposited in an array of small pixels, the physical properties of the traversing particle can be extracted with locally customized neural networks. In this first work, we present a neural network that can be embedded into the on-sensor readout and filter out hits from low momentum tracks, reducing the detector's data volume by 54.4-75.4\%. The network is designed and simulated as a custom readout integrated circuit with 28\,nm CMOS technology and is expected to operate at less than 300\,$\mu W$ with an area of less than 0.2\,mm$^2$.

43 PARTICLE ACCELERATORS↗
DOE OSTI2022

Trainability of Dissipative Perceptron-Based Quantum Neural Networks

Several architectures have been proposed for quantum neural networks (QNNs), with the goal of efficiently performing machine learning tasks on quantum data. Rigorous scaling results are urgently needed for specific QNN constructions to understand which, if any, will be trainable at a large scale. Here, we analyze the gradient scaling (and hence the trainability) for a recently proposed architecture that we call dissipative QNNs (DQNNs), where the input qubits of each layer are discarded at the layer’s output. We find that DQNNs can exhibit barren plateaus, i.e., gradients that vanish exponentially in the number of qubits. Moreover, we provide quantitative bounds on the scaling of the gradient for DQNNs under different conditions, such as different cost functions and circuit depths, and show that trainability is not always guaranteed. Here our work represents the first rigorous analysis of the scalability of a perceptron-based QNN.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Deep learning enables nanoscale X-ray 3D imaging with limited data

Deep neural network can greatly improve tomography reconstruction with limited data. A recent effort of combining ptycho-tomography model with the 3D U-net demonstrated a significant reduction in both the number of projections and computation time, and showed its potential for integrated circuit imaging that requires high-resolution and fast measurement speed.

47 OTHER INSTRUMENTATION↗