VLSI Design of Cellular Neural Networks with Annealing and Optical Input Capabilities
A cellular neural network (CNN) is locally connected, massively paralleled computing system with.
Neural Networks↗
SEARCH · Engineering Papers
Results for “Cellular Neural Networks”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Smart-Pixel Array Processors Based on Optimal Cellular Neural Networks for Space Sensor Applications
A smart-pixel cellular neural network (CNN) with hardware annealing capability, digitally programmable synaptic weights, and multisensor parallel interface has been under development for advanced space sensor applications. The smart-pixel CNN architecture is a programmable multi-dimensional array of optoelectronic neurons which are locally connected with their local neurons and associated active-pixel sensors. Integration of the neuroprocessor in each processor node of a scalable multiprocessor system offers orders-of-magnitude computing performance enhancements for on-board real-time intelligent multisensor processing and control tasks of advanced small satellites. The smart-pixel CNN operation theory, architecture, design and implementation, and system applications are investigated in detail. The VLSI (Very Large Scale Integration) implementation feasibility was illustrated by a prototype smart-pixel 5x5 neuroprocessor array chip of active dimensions 1380 micron x 746 micron in a 2-micron CMOS technology.
Fang, Wai-Chi↗
From neural-based object recognition toward microelectronic eyes
Engineering neural network systems are best known for their abilities to adapt to the changing characteristics of the surrounding environment by adjusting system parameter values during the learning process. Rapid advances in analog current-mode design techniques have made possible the implementation of major neural network functions in custom VLSI chips. An electrically programmable analog synapse cell with large dynamic range can be realized in a compact silicon area. New designs of the synapse cells, neurons, and analog processor are presented. A synapse cell based on Gilbert multiplier structure can perform the linear multiplication for back-propagation networks. A double differential-pair synapse cell can perform the Gaussian function for radial-basis network. The synapse cells can be biased in the strong inversion region for high-speed operation or biased in the subthreshold region for low-power operation. The voltage gain of the sigmoid-function neurons is externally adjustable which greatly facilitates the search of optimal solutions in certain networks. Various building blocks can be intelligently connected to form useful industrial applications. Efficient data communication is a key system-level design issue for large-scale networks. We also present analog neural processors based on perceptron architecture and Hopfield network for communication applications. Biologically inspired neural networks have played an important role towards the creation of powerful intelligent machines. Accuracy, limitations, and prospects of analog current-mode design of the biologically inspired vision processing chips and cellular neural network chips are key design issues.
Sheu, Bing J.↗
On-board Neural Processor Design for an Intelligent Multi-sensor Microspacecraft
A compact VLSI neural processor based on the Optimization Cellular Neural Network (OCNN)has been under development to provide a wide range of support for an intelligent remote sensing microspacecraft which requires both high bandwidth communication and high-performance computing for on-board data analysis, thematic data reduction, synergy of multiple types of sensors, and other smart-sensor functions. The OCNN architecture is a programmable multi-dimensional array of neurons which are locally connected with their local neurons. The OCNN operation theory, architecture, design and implementation, prototype chip, and system applications have been investigated in detail and presented in this paper.
array↗
Compact Receivers and Smart-Pixel Chips for Optical Interconnects and Signal Processing
A continuous-time shift-invariant cellular neural network (CNN) with hardware annealing
Smart↗
DEEP CELLULAR RECURRENT NEURAL ARCHITECTURE FOR EFFICIENT MULTIDIMENSIONAL TIME-SERIES DATA PROCESSING
Efficient processing of time series data is a fundamental yet challenging problem in pattern recognition. Though recent developments in machine learning and deep learning have enabled remarkable improvements in processing large scale datasets in many application domains, most are designed and regulated to handle inputs that are static in time. Many real-world data, such as in biomedical, surveillance and security, financial, manufacturing and engineering applications, are rarely static in time, and demand models able to recognize patterns in both space and time. Current machine learning (ML) and deep learning (DL) models adapted for time series processing tend to grow in complexity and size to accommodate the additional dimensionality of time. Specifically, the biologically inspired learning based models known as artificial neural networks that have shown extraordinary success in pattern recognition, tend to grow prohibitively large and cumbersome in the presence of large scale multi-dimensional time series biomedical data such as EEG. Consequently, this work aims to develop representative ML and DL models for robust and efficient large scale time series processing. First, we design a novel ML pipeline with efficient feature engineering to process a large scale multi-channel scalp EEG dataset for automated detection of epileptic seizures. With the use of a sophisticated yet computationally efficient time-frequency analysis technique known as harmonic wavelet packet transform and an efficient self-similarity computation based on fractal dimension, we achieve state-of-the-art performance for automated seizure detection in EEG data. Subsequently, we investigate the development of a novel efficient deep recurrent learning model for large scale time series processing. For this, we first study the functionality and training of a biologically inspired neural network architecture known as cellular simultaneous recurrent neural network (CSRN). We obtain a generalization of this network for multiple topological image processing tasks and investigate the learning efficacy of the complex cellular architecture using several state-of-the?art training methods. Finally, we develop a novel deep cellular recurrent neural network (CDRNN) architecture based on the biologically inspired distributed processing used in CSRN for processing time series data. The proposed DCRNN leverages the cellular recurrent architecture to promote extensive weight sharing and efficient, individualized, synchronous processing of multi-source time series data. Experiments on a large scale multi-channel scalp EEG, and a machine fault detection dataset show that the proposed DCRNN offers state-of-the-art recognition performance while using substantially fewer trainable recurrent units.
Vidyaratne, Lasitha S.↗
Deep Cellular Recurrent Network for Efficient Analysis of Time-Series Data With Spatial Information
Efficient processing of large-scale time series data is an intricate problem in machine learning. Conventional sensor signal processing pipelines with hand engineered feature extraction often involve huge computational cost with high dimensional data. Deep recurrent neural networks have shown promise in automated feature learning for improved time-series processing. However, generic deep recurrent models grow in scale and depth with increased complexity of the data. This is particularly challenging in presence of high dimensional data with temporal and spatial characteristics. Consequently, this work proposes a novel deep cellular recurrent neural network (DCRNN) architecture to efficiently process complex multi-dimensional time series data with spatial information. Here, the cellular recurrent architecture in the proposed model allows for location-aware synchronous processing of time series data from spatially distributed sensor signal sources. Extensive trainable parameter sharing due to cellularity in the proposed architecture ensures efficiency in the use of recurrent processing units with high-dimensional inputs. This study also investigates the versatility of the proposed DCRNN model for classification of multi-class time series data from different application domains. Consequently, the proposed DCRNN architecture is evaluated using two time-series datasets: a multichannel scalp EEG dataset for seizure detection, and a machine fault detection dataset obtained in-house. The results suggest that the proposed architecture achieves state-of-the-art performance while utilizing substantially less trainable parameters when compared to comparable methods in the literature.
60 APPLIED LIFE SCIENCES↗
Surrogate modeling of Cellular-Potts agent-based models as a segmentation task using the U-Net neural network architecture
The Cellular-Potts model is a powerful and ubiquitous framework for developing computational models for simulating complex multicellular biological systems. Cellular-Potts models (CPMs) are often computationally expensive due to the explicit modeling of interactions among large numbers of individual model agents and diffusive fields described by partial differential equations (PDEs). In this work, we develop a convolutional neural network (CNN) surrogate model using a U-Net architecture that accounts for periodic boundary conditions. We use this model to accelerate the evaluation of a mechanistic CPM previously used to investigate in vitro vasculogenesis. The surrogate model was trained to predict 100 computational steps ahead (Monte-Carlo steps, MCS), accelerating simulation evaluations by a factor of 562 times compared to single-core CPM code execution on CPU. Over short timescales of up to 3 recursive evaluations, or 300 MCS, our model captures the emergent behaviors demonstrated by the original Cellular-Potts model such as vessel sprouting, extension and anastomosis, and contraction of vascular lacunae. This approach demonstrates the potential for deep learning to serve as a step toward efficient surrogate models for CPM simulations, enabling faster evaluation of computationally expensive CPM simulations of biological processes.
97 MATHEMATICS AND COMPUTING↗
Mesoscale Cellular Convection Detection and Classification Using Convolutional Neural Networks: Insights From Long-Term Observations at ARM Eastern North Atlantic Site
Marine boundary layer clouds are crucial in Earth's climate system. They frequently manifest as closed or open cell mesoscale cellular convection (MCC). MCC clouds are challenging to represent accurately in current climate models, highlighting the need for detailed observational data sets and in-depth analyses. This study utilizes over 8 years of observations from the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) User Facility Eastern North Atlantic (ENA) site at Graciosa Island, Azores, to investigate these clouds. We first apply a convolutional neural network with a U-Net architecture to classify open and closed cells, marking the first application of such an approach for automatically detecting MCC patterns from ground-based radar measurements. This method addresses some observational gaps in satellite data related to low temporal resolution, nighttime challenges, and limited vertical structure capture. The analysis of the MCC cases shows clear differences between closed and open MCCs: Closed MCC clouds are characterized by lower cloud tops and bases, shallower cloud geometrical depth, weaker horizontal wind speeds, stronger atmospheric stability, and a more homogeneous liquid water path than open MCCs. Finally, we demonstrate two potential applications of our radar-based MCC classifications: (a) facilitating the investigation of aerosol-cloud interactions and (b) exploring meteorological factors along with MCC's evolution by integrating satellite imagery and back-trajectory analysis. The identified MCC cases offer a valuable resource for the scientific community to study MCC processes further and improve climate model accuracy.
54 ENVIRONMENTAL SCIENCES↗
Hot Chips and Hot Interconnects for High End Computing Systems
I will discuss several processors: 1. The Cray proprietary processor used in the Cray X1; 2. The IBM Power 3 and Power 4 used in an IBM SP 3 and IBM SP 4 systems; 3. The Intel Itanium and Xeon, used in the SGI Altix systems and clusters respectively; 4. IBM System-on-a-Chip used in IBM BlueGene/L; 5. HP Alpha EV68 processor used in DOE ASCI Q cluster; 6. SPARC64 V processor, which is used in the Fujitsu PRIMEPOWER HPC2500; 7. An NEC proprietary processor, which is used in NEC SX-6/7; 8. Power 4+ processor, which is used in Hitachi SR11000; 9. NEC proprietary processor, which is used in Earth Simulator. The IBM POWER5 and Red Storm Computing Systems will also be discussed. The architectures of these processors will first be presented, followed by interconnection networks and a description of high-end computer systems based on these processors and networks. The performance of various hardware/programming model combinations will then be compared, based on latest NAS Parallel Benchmark results (MPI, OpenMP/HPF and hybrid (MPI + OpenMP). The tutorial will conclude with a discussion of general trends in the field of high performance computing, (quantum computing, DNA computing, cellular engineering, and neural networks).
Saini, Subhash↗
VLSI neuroprocessors
Electronic and optoelectronic hardware implementations of highly parallel computing architectures address several ill-defined and/or computation-intensive problems not easily solved by conventional computing techniques. The concurrent processing architectures developed are derived from a variety of advanced computing paradigms including neural network models, fuzzy logic, and cellular automata. Hardware implementation technologies range from state-of-the-art digital/analog custom-VLSI to advanced optoelectronic devices such as computer-generated holograms and e-beam fabricated Dammann gratings. JPL's concurrent processing devices group has developed a broad technology base in hardware implementable parallel algorithms, low-power and high-speed VLSI designs and building block VLSI chips, leading to application-specific high-performance embeddable processors. Application areas include high throughput map-data classification using feedforward neural networks, terrain based tactical movement planner using cellular automata, resource optimization (weapon-target assignment) using a multidimensional feedback network with lateral inhibition, and classification of rocks using an inner-product scheme on thematic mapper data. In addition to addressing specific functional needs of DOD and NASA, the JPL-developed concurrent processing device technology is also being customized for a variety of commercial applications (in collaboration with industrial partners), and is being transferred to U.S. industries. This viewgraph p resentation focuses on two application-specific processors which solve the computation intensive tasks of resource allocation (weapon-target assignment) and terrain based tactical movement planning using two extremely different topologies. Resource allocation is implemented as an asynchronous analog competitive assignment architecture inspired by the Hopfield network. Hardware realization leads to a two to four order of magnitude speed-up over conventional techniques and enables multiple assignments, (many to many), not achievable with standard statistical approaches. Tactical movement planning (finding the best path from A to B) is accomplished with a digital two-dimensional concurrent processor array. By exploiting the natural parallel decomposition of the problem in silicon, a four order of magnitude speed-up over optimized software approaches has been demonstrated.
Kemeny, Sabrina E.↗
All-Electric Nonassociative Learning in Nickel Oxide
Habituation and sensitization represent nonassociative learning mechanisms in both non-neural and neural organisms. They are essential for a range of functions from survival to adaptation in dynamic environments. Design of hardware for neuroinspired computing strives to emulate such features driven by electric bias and can also be incorporated into neural network algorithms. Herein, cellular-like learning in oxygen-deficient NiO x devices is demonstrated. Both habituation learning and sensitization response can be achieved in a single device by simply controlling the magnitude of the electric field. Spontaneous memory relaxations and dynamic redistribution of oxygen vacancies under electric bias enable such learning behavior of NiO x under sequential training. These characteristics in simple device arrays are implemented to learn alphabets as well as demonstrate simulated algorithmic use cases in digit recognition. Transition metal oxides with carefully prepared defect concentrations can be highly sensitive to electronic structure perturbations under moderate electrical stimulus and serve as building blocks for next-generation neuroinspired computing hardware.
36 MATERIALS SCIENCE↗
Convolutional Neural Network for Segmenting Micro-X-ray Computed Tomography Images of Wood Cellular Structures
To further enhance the performance of wood products, improved tools are needed to study in situ cellular scale phenomena like mechanical deformations and moisture swelling. Micro-X-ray computed tomography (μXCT) using brilliant synchrotron light sources now has the spatial and temporal resolution for real-time visualization of phenomena in three-dimensional cellular structures. However, the tradeoff for speed includes the loss of intensity contrast between different types of materials within the imaged structure, such as cell wall and air in wood. This loss of contrast prevents traditional histogram-based segmentation methods from being used effectively. A new convolutional neural network (CNN) approach was therefore developed to segment fast μXCT images of wood into cell wall and air volumes. The fast μXCT and segmentation were demonstrated in the study of moisture swelling in loblolly pine (Pinus taeda) earlywood and latewood cellular structures conditioned at 0%, 33%, 75%, and 95% relative humidity (RH). The CNN segmentation results had a mean intersection over union (IoU) metric accuracy of 96%. Initial analysis of the swelling in the latewood revealed cell walls swelled about 25% when conditioned from 0% to 95% RH. Additionally, the widths of ray cell lumina in the transverse plane of latewood could be observed to increase at higher RH. The segmentation method presented here will facilitate future quantitative analyses in in situ μXCT studies of wood and other similar cellular materials.
Arzola-Villegas, Xavier (ORCID:0000000305369766)↗
Predicting the Functional State of Protein Kinases Using Interpretable Graph Neural Networks
Kinases are a family of proteins that function as molecular switches, regulating several essential cellular activities such as cell proliferation. Dysfunctional kinases are implicated in several types of cancers and hence they are actively pursued as drug targets. Given the vast number of complex kinase structures that are available in the protein data bank (PDB), there is a necessity to develop methodologies that can identify structurally important moieties of the kinases in an automated fashion, for such techniques can be instrumental in identifying novel drug targets. In this work, we develop a graph neural network (GNN) based deep learning framework for classifying the functionally active and inactive states of a large set of eukaryotic protein kinases, making use of their 3D structure from the PDB. We show that GNN based machine learning models can classify protein states with an accuracy greater than 97%. We further use the GNN models to automatically identify regions of the kinases that are important for its function. For this purpose, Gradient-weighted Class Activation Mapping (Grad-CAM) was implemented on the protein graphs. Remarkably, Grad-CAM consistently identifies the highly conserved DFG motif as the most important part of the protein across the entire kinome, without any prior input. Other regions of the hydrophobic core such as the HRD motif were also identified by the interpretable GNN framework, consistent with the literature. We discuss the significance of each of these regions in detail.
Ashwin Ravichandran↗
Spatiotemporal analysis of 3D human iPSC-derived neural networks using a 3D multi-electrode array
While there is a growing appreciation of three-dimensional (3D) neural tissues (i.e., hydrogel-based, organoids, and spheroids), shown to improve cellular health and network activity to mirror brain-like activity in vivo , functional assessment using current electrophysiology techniques (e.g., planar multi-electrode arrays or patch clamp) has been technically challenging and limited to surface measurements at the bottom or top of the 3D tissue. As next-generation MEAs, specifically 3D MEAs, are being developed to increase the spatial precision across all three dimensions (X, Y, Z), development of improved computational analytical tools to discern region-specific changes within the Z dimension of the 3D tissue is needed. In the present study, we introduce a novel computational analytical pipeline to analyze 3D neural network activity recorded from a “bottom-up” 3D MEA integrated with a 3D hydrogel-based tissue containing human iPSC-derived neurons and primary astrocytes. Over a period of ~6.5 weeks, we describe the development and maturation of 3D neural activity (i.e., features of spiking and bursting activity) within cross sections of the 3D tissue, based on the vertical position of the electrode on the 3D MEA probe, in addition to network activity (identified using synchrony analysis) within and between cross sections. Then, using the sequential addition of postsynaptic receptor antagonists, bicuculline (BIC), 2-amino-5-phosphonovaleric acid (AP-5), and 6-cyano-5-nitroquinoxaline-2,3-dione (CNQX), we demonstrate that networks within and between cross sections of the 3D hydrogel-based tissue show a preference for GABA and/or glutamate synaptic transmission, suggesting differences in the network composition throughout the neural tissue. The ability to monitor the functional dynamics of the entire 3D reconstructed neural tissue is a critical bottleneck; here we demonstrate a computational pipeline that can be implemented in studies to better interpret network activity within an engineered 3D neural tissue and have a better understanding of the modeled organ tissue.
3D culture↗
Neural nets on the MPP
The Massively Parallel Processor (MPP) is an ideal machine for computer experiments with simulated neural nets as well as more general cellular automata. Experiments using the MPP with a formal model neural network are described. The results on problem mapping and computational efficiency apply equally well to the neural nets of Hopfield, Hinton et al., and Geman and Geman.
Hastings, Harold M.↗
Spiking Neurons for Analysis of Patterns
Artificial neural networks comprising spiking neurons of a novel type have been conceived as improved pattern-analysis and pattern-recognition computational systems. These neurons are represented by a mathematical model denoted the state-variable model (SVM), which among other things, exploits a computational parallelism inherent in spiking-neuron geometry. Networks of SVM neurons offer advantages of speed and computational efficiency, relative to traditional artificial neural networks. The SVM also overcomes some of the limitations of prior spiking-neuron models. There are numerous potential pattern-recognition, tracking, and data-reduction (data preprocessing) applications for these SVM neural networks on Earth and in exploration of remote planets. Spiking neurons imitate biological neurons more closely than do the neurons of traditional artificial neural networks. A spiking neuron includes a central cell body (soma) surrounded by a tree-like interconnection network (dendrites). Spiking neurons are so named because they generate trains of output pulses (spikes) in response to inputs received from sensors or from other neurons. They gain their speed advantage over traditional neural networks by using the timing of individual spikes for computation, whereas traditional artificial neurons use averages of activity levels over time. Moreover, spiking neurons use the delays inherent in dendritic processing in order to efficiently encode the information content of incoming signals. Because traditional artificial neurons fail to capture this encoding, they have less processing capability, and so it is necessary to use more gates when implementing traditional artificial neurons in electronic circuitry. Such higher-order functions as dynamic tasking are effected by use of pools (collections) of spiking neurons interconnected by spike-transmitting fibers. The SVM includes adaptive thresholds and submodels of transport of ions (in imitation of such transport in biological neurons). These features enable the neurons to adapt their responses to high-rate inputs from sensors, and to adapt their firing thresholds to mitigate noise or effects of potential sensor failure. The mathematical derivation of the SVM starts from a prior model, known in the art as the point soma model, which captures all of the salient properties of neuronal response while keeping the computational cost low. The point-soma latency time is modified to be an exponentially decaying function of the strength of the applied potential. Choosing computational efficiency over biological fidelity, the dendrites surrounding a neuron are represented by simplified compartmental submodels and there are no dendritic spines. Updates to the dendritic potential, calcium-ion concentrations and conductances, and potassium-ion conductances are done by use of equations similar to those of the point soma. Diffusion processes in dendrites are modeled by averaging among nearest-neighbor compartments. Inputs to each of the dendritic compartments come from sensors. Alternatively or in addition, when an affected neuron is part of a pool, inputs can come from other spiking neurons. At present, SVM neural networks are implemented by computational simulation, using algorithms that encode the SVM and its submodels. However, it should be possible to implement these neural networks in hardware: The differential equations for the dendritic and cellular processes in the SVM model of spiking neurons map to equivalent circuits that can be implemented directly in analog very-large-scale integrated (VLSI) circuits.
Huntsberger, Terrance↗
A deep generative model for deciphering cellular dynamics and in silico drug discovery in complex diseases
Human diseases are characterized by intricate cellular dynamics. Single-cell transcriptomics provides critical insights, yet a persistent gap remains in computational tools for detailed disease progression analysis and targeted in silico drug interventions. Here we introduce UNAGI, a deep generative neural network tailored to analyse time-series single-cell transcriptomic data. This tool captures the complex cellular dynamics underlying disease progression, enhancing drug perturbation modelling and screening. When applied to a dataset from patients with idiopathic pulmonary fibrosis, UNAGI learns disease-informed cell embeddings that sharpen our understanding of disease progression, leading to the identification of potential therapeutic drug candidates. Validation using proteomics reveals the accuracy of UNAGI’s cellular dynamics analysis, and the use of the fibrotic cocktail-treated human precision-cut lung slices confirms UNAGI’s predictions that nifedipine, an antihypertensive drug, may have anti-fibrotic effects on human tissues. UNAGI’s versatility extends to other diseases, including COVID, demonstrating adaptability and confirming its broader applicability in decoding complex cellular dynamics beyond idiopathic pulmonary fibrosis, amplifying its use in the quest for therapeutic solutions across diverse pathological landscapes.
Neural Network↗