Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “CAMERAS”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 397 records · Page 22
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

The Sensor Dilemma in Intelligent Transportation Systems

Intelligent Transportation Systems (ITS) are at the forefront in advancing the way we interact and perceive with the transportation network. This revolution is fueled by the significant advancement in sensor perception technologies such as Radar, LiDAR and Video Imaging which are the most popular modalities for ITS. Real-time perception data from these sensors allows intelligent infrastructure side decision making to improve the energy, efficiency and safety at traffic intersections. As traffic departments across the United States are transitioning from traditional loop detectors / emulators and embracing newer technologies, they are often left with a dilemma in choosing a sensor technology for infrastructure-based perception which is reliable, inexpensive, easy to setup and has robust performance in varying weather conditions. However, choosing a sensor which checks all boxes is not straightforward as every sensor type has unique benefits and drawbacks. Radar is excellent at detecting long range vehicles and weather resistance but lacks high resolution. LiDAR is expensive and weather-sensitive, while cameras provide rich visual data at a low cost but are constrained by lighting and visibility. This study examines Radar, LiDAR and camera sensors capabilities to ascertain whether any of these qualifies as the "best" sensor for ITS perception. Through this evaluation, we hope to draw attention to the necessity of National Renewable Energy Laboratory's (NREL) Infrastructure Perception and Control (IPC) framework which presents a multi-sensor track data fusion engine to assimilate multiple data streams in order to provide robust and reliable perception. While no single sensor can meet all the demands of ITS, a hybrid approach combining multiple sensor modalities like Radar, LiDAR and cameras, offers the most robust solution for enhancing the safety and efficiency in intelligent transportation systems.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2020

An effective survey method for studying volant species activity and behavior at tall structures

The effects of anthropogenic modification of air space on wildlife, particularly volant species, is not fully understood. Thus, it is essential to understand wildlife-interactions with tall structures to implement effective mitigation strategies. Yet, we are currently lacking standard protocols for visual surveys of wildlife behavior at such heights. Our study sought to determine an effective, repeatable method using readily available night vision and thermal technology to survey wildlife at tall structures. Using bats as the taxonomic group of interest, we (1) created a key to identify bats and their behavior, (2) compared the effectiveness of 2 different technologies, and (3) assessed optimal equipment placement to visually capture bat activity and behavior in proximity to wind turbine towers. For the latter, we tested thermal cameras at four distances from the base of the tower. The results of our study revealed that thermal cameras captured ∼34% more flying animals than night vision at a 2 m distance. However, due to the heat signature of the turbine towers themselves, it was challenging to identify behaviors and interactions that occurred in close proximity to the towers. In contrast, it was difficult to identify bats approaching the towers using night vision, yet we were able to clearly observe interactions with the towers themselves. With regards to equipment placement, we visually captured more bats with the thermal cameras placed 2 m from the tower base compared to farther distances. From our findings, we recommend that when using either thermal or night vision technology at tall structures, they be placed 2 m from the base to effectively observe interactions along the length of these structures. In addition, we further recommend that consideration be given to the use of these two technology types together to effectively conduct such surveys. If these survey techniques are incorporated into standard protocols, future surveys at a variety of tall structures are likely to become comparable and repeatable, thereby more effectively informing any mitigation strategies that may be required.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Machine Vision-based Robot Manipulators for Nuclear Applications - 20370

Decommissioning and dismantling of nuclear facilities are major challenges facing the nuclear industry. Robot manipulators with capabilities of restoring nuclear structures and significantly prolonging nuclear power generation in addition to their applications in decommissioning and dismantling (D and D) would immensely ease the above-mentioned challenge. Ability to remotely sense/monitor and analyze dangerous and hazardous environments such as nuclear reactors as well as performing reparatory tasks in such environments using robot manipulators require additional information from vision sensors. Structural sample collection and inspection as well as restoration require object and position information. Computer vision is used to obtain precise 3D position information. The current work uses the Sawyer robot with the integrated Cognex camera. The research presents an approach to facilitate and improve operations in a nuclear reactor using robotic vision control. The vision control system is modeled with Augmented Image Space-based visual servoing approach. Results showing accurate robot control. Object recognition is used to recognize the image and calculate it poses. To transform coordinates of the object's pose from the camera frame to the robotic frame transformation matrices are employed. Future work will employ a 3D camera (with depth information) using Denso robot. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2021

Evaluation of a Variable-Aperture Full-Ring SPECT System using Large-Area Pixelated CZT Modules: A Simulation Study for Brain SPECT Applications

Single photon emission computed tomography (SPECT) scanner using cadmium zinc telluride (CZT) offers improved imaging capability over conventional NaI (Tl)-based SPECT scanner. We aim to demonstrate a full-ring SPECT system design with eight large-area CZT detectors that can be used for a broad spectrum of SPECT radiopharmaceuticals and provides higher sensitivity and better spatial resolution than those of conventional NaI (Tl)-based gamma cameras. A newly-designed full-ring SPECT system is composed of 8 large-area CZT cameras (128 mm × 179.2 mm effective area) that can be independently swiveled around their own axes of rotation independently and can have radial motion for varying aperture sizes that can be adapted to different sizes of imaging volume. Extended projection data were generated by conjoining projections of two adjacent detectors to overcome the limited field-of-view (FOV) by each CZT camera. Using Monte Carlo simulations, we evaluated this new system design with digital phantoms including a Derenzo hot-rod phantom and a Zubal brain phantom. Comparison of performance metrics such as spatial resolution, sensitivity, contrast-to-noise ratio (CNR), and contrast-recovery ratio were made between our design and a conventional SPECT scanner. The proposed scanner could result in up to about 3 times faster in acquisition time over conventional scan time at same acquisition time per step. The spatial resolution of our proposed scanner was similar or better to that of the conventional scanner, and there was significant performance improvement over the conventional scanner particularly in sensitivity (approximately 4 times). Overall, we successfully reconstructed the phantom image for both 99 mTc-based perfusion and 123 I-based dopamine transporter (DaT) brain studies simulated for our new design. In particular, the striatal/background contrast-recovery ratio in 3-to-1 reference ratio was over the 0.8 for the 123 I-based DaT study. In conclusion, we demonstrated the potential of our new full-ring CZT SPECT design, showing improved performance metrics such as improved system sensitivity and CNR while maintaining other important imaging parameters such as spatial resolution.

36 MATERIALS SCIENCE↗
DOE OSTI2022

In situ melt pool measurements for laser powder bed fusion using multi sensing and correlation analysis

Laser powder bed fusion is a promising technology for local deposition and microstructure control, but it suffers from defects such as delamination and porosity due to the lack of understanding of melt pool dynamics. To study the fundamental behavior of the melt pool, both geometric and thermal sensing with high spatial and temporal resolutions are necessary. This work applies and integrates three advanced sensing technologies: synchrotron X-ray imaging, high-speed IR camera, and high-spatial-resolution IR camera to characterize the evolution of the melt pool shape, keyhole, vapor plume, and thermal evolution in Ti–6Al–4V and 410 stainless steel spot melt cases. Aside from presenting the sensing capability, this paper develops an effective algorithm for high-speed X-ray imaging data to identify melt pool geometries accurately. Preprocessing methods are also implemented for the IR data to estimate the emissivity value and extrapolate the saturated pixels. Quantifications on boundary velocities, melt pool dimensions, thermal gradients, and cooling rates are performed, enabling future comprehensive melt pool dynamics and microstructure analysis. The study discovers a strong correlation between the thermal and X-ray data, demonstrating the feasibility of using relatively cheap IR cameras to predict features that currently can only be captured using costly synchrotron X-ray imaging. Such correlation can be used for future thermal-based melt pool control and model validation.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Colloidal quantum dot based infrared detectors: extending to the mid-infrared and moving from the lab to the field

Quantum dots (QDs) that absorb in the mid-wave infrared (MWIR) regime (3–5 μm) have recently generated significant interest as possible detector materials for MWIR cameras, with promises to reduce materials and device fabrication costs and potentially increase device operating temperatures. However, these materials have been primarily explored in single pixel devices – not multi-pixel cameras – and their current performance lags behind that of commercially available MWIR cameras based on single crystalline materials. To realize the potential of QD MWIR photodetectors with imaging capabilities, improvements are needed in the quality of the MWIR-absorbing QD materials, ligands dictating inter-QD charge transport, ordering of QD films, device architectures, and scalability of these methods to larger areas. With three families of QDs being researched – lead-, silver-, and mercury-based chalcogenides – as well as a multitude of possible capping ligands and film deposition techniques, the experimental phase space for MWIR QDs is vast. Here, we provide a roadmap, considering the pros and cons of various film deposition and ligand exchange techniques, as well as reintroduce lessons learned over years of research on QD film formation. We also extracted and created a database of reported quantum dot photodetectors’ performance and fabrication methods and have developed an interactive data visualization dashboard for this database, which provides researchers in the field a quick snapshot of the existing state of the art. Finally, we outline figures of merit and information that should be presented in papers moving forward which would help to clarify the reported results, as well as offer ideas for future steps. We provide a database visualization of the QD photodetector literature at https://public.tableau.com/app/profile/tom.nakotte/viz/IRQDphotodetectors_16384709473480/Dashboard1#1.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Preparation of the Multi-Site Data Processing at the Vera C. Rubin Observatory

The Vera C. Rubin Observatory’s Legacy Survey of Space and Time (LSST) Camera is scheduled to start taking data in the summer of 2025. The Data Release Production will run the LSST Science Pipe software at data facilities in the US, France and the UK. The LSST Science Pipeline consists of complex directed acyclic graphs (DAGs) of tasks. Rubin will use the Production and Distributed Analysis (PanDA) workflow and workload management system to orchestrate this complex workflow and the distribution of workloads to the data facilities. When run end-to-end by a team of data production staff, this processing (the Science Pipelines, distributed by the workflow and workload management system) is referred to as a 'campaign'. This paper describes the central services and data facility specific services that support this multi-site data process model, including the service deployment infrastructure, the workload and workflow system, the Campaign Management tools, and connection to Rubin Data Management. This paper will also mention the experience of processing the Rubin Commissioning Camera data. All these are part of the effort to scale up the processing capabilities for the expected very large data volume from the LSST Camera.

Yang, Wei [SLAC]↗
DOE OSTI2020

Experimental system for studying temperature gradient-driven fracture of oxide nuclear fuel out of reactor

Temperature gradients in ceramic light water reactor (LWR) uranium dioxide (UO 2 ) nuclear fuel pellets generate thermal stresses that cause fractures in the fuel beginning early in the life of fresh fuel. The combination of heating due to fission and forced convective cooling on the exterior of LWR fuel rods generates a temperature profile that is difficult to replicate outside the reactor environment. In the present study, a state-of-the-art experimental set-up using electrical heating to replicate fission heating was built and surrogate fuel materials such as ceria (CeO 2 ) were used to validate the system. Cracking experiments were conducted on these surrogates by inducing reactivity-initiated-accident (RIA) like temperature gradients in the pellets via induction and direct resistance heating. Induction heating was done using copper coils and molybdenum susceptors which heated the surrogates to a threshold temperature that is sufficiently high for the fuel material to conduct current. Thereafter, direct resistance heating was used by a D.C. power supply to introduce volumetric heating to replicate LWR operating conditions analogous to fission heating. The pellets were held against nickel electrodes and mounted on a boron nitride test-stand. All the tests were carried out in a stainless-steel vacuum chamber. Simultaneous real-time dual imaging of the surrogate pellet surface has been implemented using an optical and infrared camera system which will be mounted along axial and perpendicular directions to the pellet surface respectively. A beam-splitter was used to split the incoming radiation from the sample into two halves. While one of the beams is transmitted from the splitter through a bandpass filter to obtain optical images, the other beam is reflected from the splitter to the thermal camera to capture full field temperature gradients of the as fabricated pellet surface during crack initiation and propagation. In the current series of tests, a 2-color pyrometer was used for recording and comparing the surface and centerline temperatures of the surrogate pellets in lieu of the thermal camera. A LabVIEW data acquisition system has been set up for collecting useful data during experiments.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2021

High-resolution x-ray radiography with Fresnel zone plates on the University of Rochester’s OMEGA Laser Systems

Experiments performed at the Laboratory for Laser Energetics with a continuous-wave (cw) x-ray source and on the OMEGA and OMEGA EP Laser Systems have utilized a Fresnel zone plate (FZP) to obtain x-ray images with a spatial resolution as small as ~1.5 μm. Such FZP images were obtained with a charge-coupled device or a framing camera at energies ranging from 4.5 keV to 6.7 keV using x-ray line emission from both the cw source and high-intensity, laser-beam–illuminated metal foils. In all cases, the resolution test results are determined from patterns and grids backlit by these sources. Here, the resolutions obtained are shown to be due to a combination of the spectral content of the x-ray sources and detector resolution limited by the magnification of the images (14× to 22×). High-speed framing cameras were used to obtain FZP images with frame times as short as ~30 ps. Double-shell implosions on OMEGA were backlit by laser-irradiated Fe foils, thus obtaining a framing-camera–limited, FZP-image resolution of ~3 μm–4 μm.

47 OTHER INSTRUMENTATION↗
DOE OSTI2022

3D velocity map imaging of electrons with TPX3CAM

In this report we demonstrate three-dimensional velocity map imaging of low energy electrons using a TPX3CAM, where the three-dimensional momentum information [$p_x, p_y, p_z$] is encoded in position and timing [x, y, t] of hits on the camera sensor. We make use of the camera sensor for the [x, y] information and a constant fraction discriminator and fast time to digital converter in the camera for the time information. We illustrate the capabilities of our apparatus by presenting above threshold ionization measurements of xenon, which produces well defined structures in the momentum resolved photoelectron yield.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Upgrade of the Lyman-alpha diagnostic system on DIII-D for main chamber edge neutral studies

The LLAMA (Lyman Alpha Measurement Apparatus) pinhole camera diagnostic had previously been deployed on DIII-D to measure radial profiles of the Lyman-α (Ly-α) deuterium neutral line brightness across the plasma boundary in the lower chamber to infer neutral deuterium density and ionization rate profiles. This system has recently been upgraded with a new diagnostic head, named ALPACA, that also encloses two pinhole cameras and duplicates the LLAMA views in the upper chamber. Similar to LLAMA, ALPACA provides two times 20 lines of sight, viewing the plasma edge on the inboard and outboard sides with a radial resolution of ~2.5 cm (FWHM) and an effective time resolution of ~1 ms that allows for the investigation of inter-ELM dynamics. The extended Ly-α system provides better coverage to study neutrals in experiments with various plasma shapes utilizing both the upper and lower divertors. Furthermore, post-campaign calibration of the LLAMA diagnostic has successfully been demonstrated for the first time. This was facilitated by various upgrades to the calibration set-up and detailed measurements of the emissivity distribution of the Ly-α calibration source using a pinhole collimator. It was found that the sensitivity of the inboard LLAMA pinhole camera was reduced by a factor of 2.0 ± 0.2 over the course of six months of plasma operation in 2021. In conclusion, the upgraded Ly-α system, equipped with improved absolute calibration, will provide key input for neutral fueling and pedestal particle transport studies and for 2D edge transport code validation on the DIII-D tokamak.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Broadband coherent anti-Stokes Raman scattering (BCARS) microscopy for rapid, label-free biological imaging

Broadband coherent anti-Stokes Raman scattering (BCARS) microscopy is a label-free imaging approach that provides detailed chemical information at high spatial resolution in a sample through nonlinear, coherent excitation of molecular vibrations and detection of Raman spectra. While its utility for biological imaging has been demonstrated, many aspects of this technique must mature before it can be widely adopted. One of the areas of required improvement is imaging speed—most BCARS implementations involve sample rastering, which limits imaging speed. Beam scanning can provide faster BCARS imaging but presents some unique challenges. Here, we describe a beam-scanning BCARS microscopy system that improves spatial resolution twofold and imaging speed by fivefold over a previous beam-scanning implementation. These enhancements were enabled by an improvement in supercontinuum power and the use of a sCMOS camera for its high data transfer rate and low read noise. Implementation of the sCMOS camera required correction for the significant pixel-to-pixel background and photon response nonuniformity. Here, we report on the method that we implemented for calibrating and correcting the pixel-to-pixel differences in sCMOS camera noise.

Dixon, Jessica Z. [Georgia Institute of Technology↗
DOE OSTI2021

Pellet triggering of edge localized modes in low collisionality pedestals at DIII-D

Edge localized modes (ELMs) are triggered using deuterium pellets injected into plasmas with ITER-relevant low collisionality pedestals, and the resulting peak ELM energy fluence is reduced by approximately 25%–50% relative to natural ELMs destabilized at similar pedestal pressures. Cryogenically frozen deuterium pellets are injected from the low-field side of the DIII-D tokamak at frequencies lower than the natural ELM frequency, and heat flux is measured by infrared cameras. Ideal MHD pedestal stability calculations show that without pellet injection, these low collisionality pedestals were limited by their current density (peeling-limited) rather than their pressure gradient (ballooning-limited). ELM triggering success correlates strongly with pellet mass, consistent with the theory that a large pressure perturbation is required to trigger an ELM in low collisionality discharges that are far from the ballooning stability boundary. For sufficiently large pellets, both instantaneous and time-integrated ELM energy deposition measured by infrared cameras is reduced with respect to naturally occurring ELMs at the inner strike point, which is the position where it is largest for natural ELMs. Energy fluence at the outer strike point is less effected. Cameras observing both heat flux and D-alpha emission often find significant toroidally asymmetric striations in the outboard far scrape-off layer resulting from ELMs that are triggered by pellets. Toroidal asymmetries at the inner strike point are similar between natural and pellet-triggered ELMs, indicating that the reduction in peak heat flux and total fluence at that location is robust for the conditions reported here.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Digital coded exposure formation of frames from event-based imagery

Abstract Event-driven neuromorphic imagers have a number of attractive properties including low-power consumption, high dynamic range, the ability to detect fast events, low memory consumption and low band-width requirements. One of the biggest challenges with using event-driven imagery is that the field of event data processing is still embryonic. In contrast, decades worth of effort have been invested in the analysis of frame-based imagery. Hybrid approaches for applying established frame-based analysis techniques to event-driven imagery have been studied since event-driven imagers came into existence. However, the process for forming frames from event-driven imagery has not been studied in detail. This work presents a principled digital coded exposure approach for forming frames from event-driven imagery that is inspired by the physics exploited in a conventional camera featuring a shutter. The technique described in this work provides a fundamental tool for understanding the temporal information content that contributes to the formation of a frame from event-driven imagery data. Event-driven imagery allows for the application of arbitrary virtual digital shutter functions to form the final frame on a pixel-by-pixel basis. The proposed approach allows for the careful control of the spatio-temporal information that is captured in the frame. Furthermore, unlike a conventional physical camera, event-driven imagery can be formed into any variety of possible frames in post-processing after the data is captured. Furthermore, unlike a conventional physical camera, coded-exposure virtual shutter functions can assume arbitrary values including positive, negative, real, and complex values. The coded exposure approach also enables the ability to perform applications of industrial interest such as digital stroboscopy without any additional hardware. The ability to form frames from event-driven imagery in a principled manner opens up new possibilities in the ability to use conventional frame-based image processing techniques on event-driven imagery.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Plot-level rapid screening for photosynthetic parameters using proximal hyperspectral imaging

Abstract Photosynthesis is currently measured using time-laborious and/or destructive methods which slows research and breeding efforts to identify crop germplasm with higher photosynthetic capacities. We present a plot-level screening tool for quantification of photosynthetic parameters and pigment contents that utilizes hyperspectral reflectance from sunlit leaf pixels collected from a plot (~2 m×2 m) in <1 min. Using field-grown Nicotiana tabacum with genetically altered photosynthetic pathways over two growing seasons (2017 and 2018), we built predictive models for eight photosynthetic parameters and pigment traits. Using partial least squares regression (PLSR) analysis of plot-level sunlit vegetative reflectance pixels from a single visible near infra-red (VNIR) (400–900 nm) hyperspectral camera, we predict maximum carboxylation rate of Rubisco (Vc,max, R2=0.79) maximum electron transport rate in given conditions (J1800, R2=0.59), maximal light-saturated photosynthesis (Pmax, R2=0.54), chlorophyll content (R2=0.87), the Chl a/b ratio (R2=0.63), carbon content (R2=0.47), and nitrogen content (R2=0.49). Model predictions did not improve when using two cameras spanning 400–1800 nm, suggesting a robust, widely applicable and more ‘cost-effective’ pipeline requiring only a single VNIR camera. The analysis pipeline and methods can be used in any cropping system with modified species-specific PLSR analysis to offer a high-throughput field phenotyping screening for germplasm with improved photosynthetic performance in field trials.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

Methods for In Situ Electroluminescence Imaging of Photovoltaic Modules Under Varying Environmental Conditions

Electroluminescence (EL) imaging is a powerful tool used to identify defects in photovoltaic solar cells. Typically, this type of characterization is performed in the dark using a current injection that equals short-circuit current measured at standard test conditions (STC). Restricting imaging to such a temperature range limits the information obtained about the module and cells. However, it is not trivial to develop a tool that would allow for EL imaging to be performed under a wider range of temperatures. Here we demonstrate an in situ EL imaging capability developed within an environmental chamber that allows for control of sample temperatures between –40 and 90 °C. Additionally, we demonstrate EL imaging of 4-cell mini-modules (MiMo) under front-side mechanical loading. A Raspberry Pi-connected camera with short-pass filter removed is used for EL imaging. The camera is low-cost with a small form-factor, appropriate for use in a harsh, enclosed environment. The camera is installed within a thermally isolating housing mounted within the environmental chamber. Three example cases are given for MiMos that exhibit various forms of degradation including solder fatigue and cell cracking. It is shown that by measuring at conditions above and below STC, different behaviors may be identified. In some cases, restricting characterization to STC may lead to a failure to detect damage existing in the sample.

14 SOLAR ENERGY↗
DOE OSTI2024

Event-to-Video Conversion for Overhead Object Detection

Collecting overhead imagery using an event camera is desirable due to the energy efficiency of the image sensor compared to standard cameras. However, event cameras complicate downstream image processing, especially for complex tasks such as object detection. In this paper, we investigate the viability of event streams for overhead object detection. We demonstrate that across a number of standard modeling approaches, there is a significant gap in performance between dense event representations and corresponding RGB frames. We establish that this gap is, in part, due to a lack of overlap between the event representations and the pre-training data that the object detectors were initially trained on through a number of experiments. Then, apply an off-the-shelf event-to-video conversion tool that converts event streams into gray-scale video to close this gap. We demonstrate that this approach results in a large performance increase, outperforming even event-specific object detection techniques on our overhead target task. These results suggest that better aligning event representations with existing large pre-trained models may result in greater short-term performance gains compared to end-to-end event-specific architectural improvements.

machine learning (ML), computer vision, Neuromorph↗
DOE OSTI2022

Dictionary Learning with Accumulator Neurons

The Locally Competitive Algorithm (LCA) uses local competition between non-spiking leaky integrator neurons to infer sparse representations, allowing for potentially real-time execution on massively parallel neuromorphic architectures such as Intel's Loihi processor. Here, we focus on the problem of inferring sparse representations from streaming video using dictionaries of spatiotemporal features optimized in an unsupervised manner for sparse reconstruction. Non-spiking LCA has previously been used to achieve unsupervised learning of spatiotemporal dictionaries composed of convolutional kernels from raw, unlabeled video. We demonstrate how unsupervised dictionary learning with spiking LCA (\hbox{S-LCA}) can be efficiently implemented using accumulator neurons, which combine a conventional leaky-integrate-and-fire (\hbox{LIF}) spike generator with an additional state variable that is used to minimize the difference between the integrated input and the spiking output. We demonstrate dictionary learning across a wide range of dynamical regimes, from graded to intermittent spiking, for inferring sparse representations of both static images drawn from the CIFAR database as well as video frames captured from a DVS camera. On a classification task that requires identification of the suite from a deck of cards being rapidly flipped through as viewed by a DVS camera, we find essentially no degradation in performance as the LCA model used to infer sparse spatiotemporal representations migrates from graded to spiking. We conclude that accumulator neurons are likely to provide a powerful enabling component of future neuromorphic hardware for implementing online unsupervised learning of spatiotemporal dictionaries optimized for sparse reconstruction of streaming video from event based DVS cameras.

artificial intelligence↗