Event-Based Sensing and Signal Processing in the Visual, Auditory, and Olfactory Domain: A Review

The nervous systems converts the physical quantities sensed by its primary receptors into trains of events that are then processed in the brain. The unmatched efficiency in information processing has long inspired engineers to seek brain-like approaches to sensing and signal processing. The key principle pursued in neuromorphic sensing is to shed the traditional approach of periodic sampling in favor of an event-driven scheme that mimicks sampling as it occurs in the nervous system, where events are preferably emitted upon the change of the sensed stimulus. In this paper we highlight the advantages and challenges of event-based sensing and signal processing in the visual, auditory and olfactory domains. We also provide a survey of the literature covering neuromorphic sensing and signal processing in all three modalities. Our aim is to facilitate research in event-based sensing and signal processing by providing a comprehensive overview of the research performed previously as well as highlighting conceptual advantages, current progress and future challenges in the field.

1. Introduction

Neuromorphic computing has recently garnered much interest. This emerging technology promises lower power and lower latency than established methods for sensing and computing by emulating principles of information processing in the brain. A key concept of neuromorphic computing is event-based sensing, inspired by the property of sensory neurons in the nervous system to preferably respond to changes of the sensed quantity, rather than to continuously report its current level. The latter approach is represented by the established principle of periodic sampling, alongside the signal processing toolbox based on Discrete Fourier Transform (DFT). While the periodic sampling/DFT approach has been extremely successful, it suffers from several inherent weaknesses. First, it is in practice restricted to bandlimited signals due to the aliasing problem. Second, periodic sampling may waste energy when signals change only intermittently. Third, there is an inherent limitation of the minimum achievable latency imposed by the sampling interval; when using DFT for signal processing this gets worse as it requires a window of samples.

Event-based sensing represents a family of sampling schemes where the signal drives the sampling. A sensing element, such as a pixel, a filter bank element, or a gas sensor, emits an event when the signal crosses a threshold. This sampling scheme is often called “send-on-delta.” Other, largely synonymous terms are “event-driven sampling,” “absolute-deadband sampling,” “Lebesgue-sampling,” among others; the specifics of event triggering allow for tuning of the algorithm (Vasyutynskyy and Kabitzsch, 2010). Previous work has analysed the suitability of signal-driven sampling schemes for different kinds of signals (Liu et al., 2019), highlighting that the send-on-delta sampling scheme is particularly suitable for signals that are sparse, intermittent, and not bandlimited.

The event-driven sensing paradigm has been pioneered in vision, but recently also found its way into other sensory modalities. Here, we provide a survey on event-based sensing and signal processing approaches in modalities: vision, sound, and olfaction. The purpose of this survey is designed to put these three modalities in context and provide an overview of the publications in the field.

Event-based vision sensors have evolved during the last decade from an existence almost exclusively limited to a few select research labs to now being widely available as commercial products from several manufacturers worldwide, with several established application domains. Their bio-inspired operation differs from traditional vision systems in that their sampling regime is not frame-based and periodic; instead, sampling at each individual pixel is driven by the signal itself. Just as Ganglion cells in the retina emit a spike when a certain feature (i.e., brightness) in their receptive field changes, event-based vision sensors emit an event when a pixel detects that brightness crosses a threshold (“send-on-delta”). Vision signals are often very broad-band and can require extremely short sensor latencies to be captured accurately, while also being sparse in time. These are favorable conditions for event-based sampling schemes (Liu et al., 2019).

The transition from periodic sampling to event-driven sampling has also been implemented in the auditory domain (Lyon and Mead, 1988). Again, close inspiration from biological auditory systems has guided system development, mimicking the operation of biological cochlea, the mechanical properties of which implement a filter bank that enables spectral decomposition of the audio signal, subsequently transduced by hair cells into spikes. The output of an event-based silicon cochlea is a sparse stream of digital address-events, that represent the address of active channels, with each channel representating a frequency band. In theory, silicon cochleas could increase the sensitivity to rapid changes in the spectral composition of audio signals, since they do not require windowed Fourier transform which inherently introduces a lag in signal processing. However, the bandlimited nature of auditory signals facilitates the choice of a periodic sampling frequency that will allow efficient processing within the expected variation of the signal (Liu et al., 2019). Nevertheless, substantial amounts of research have explored the principles of operation and demonstrated use cases for event-based silicon cochlea, which we summarize below.

As a third example of event-based sensing we cover Olfaction, the sense of smell. There is a long history of seeking bioinspiration for electronic nose systems. One perhaps representative example is the NEUROCHEM project that ran from 2008 to 2011 (Marco et al., 2013). It brought together scientists from different disciplines around bio-inspired olfaction systems. Olfactory stimuli are carried by turbulent dispersal, which imposes wide-band fluctuations on their concentration at a given point in space. The physical properties of turbulent processes indicate that odour dispersion results in an intermittent signal where long periods of “blanks” are interspersed with brief and wide-band “whiffs” (Celani et al., 2014). These turbulence-induced fluctuations can be very rapid, and carry information that could be helpful in locating odour sources—an essential task for foraging or mate-seeking biological agents as well as in robotic gas sensing, e.g., for environmental and factory monitoring, or disaster management (Mylne and Mason, 1991; Schmuker et al., 2016). Given that rapid fluctuations olfactory signals carry useful information, it is not surprising that progress in Olfactory Neuroscience has recently uncovered that animals can decode very short transients in olfactory stimuli (Szyszka et al., 2014; Erskine et al., 2019). In summary, olfaction signals could be very well amenable to event-based sensing since they combine sparseness and intermittent detection with very rapid fluctuations. However, the olfactory domain has also seen the least exploration from the three modalities that we cover in this survey, highlighting the great potential for future research.

1.1. Principles of Event-Based Signal Processing

Once an event is generated by detecting a threshold-crossing in the signal, it is emitted as a data structure typically containing two pieces of information: 1. An address, e.g., the coordinates of the pixel that emitted the event, or the index of the filter bank, or the gas sensor instance, and 2. the time of event creation. In real-time systems, time can represent itself and only the address of the sensor needs to be transmitted. This protocol is commonly called Address-Event Representation (AER).

Algorithms for AER signal processing are largely independent of periodic sampling, and therefore do not suffer from aliasing. Moreover, information acquisition is driven by the spatio-temporal changes in the signal itself, therefore inherently capturing the dynamics of the underlying scene, unlike frame-based systems where these dynamics first have to be reconstructed from the sequence of samples. AER algorithms also have promising properties for parallelization and composition. AER processing modules have the inherent capability to be assembled into hierarchical structures (Serrano-Gotarredona et al., 2009). This is due to the fact that the communication between the AER modules can be made completely asynchronous, i.e., without having to rely on central synchronization. Previous work has developed “glue modules,” such as AER splitters, mergers and mappers that connect these individual processors together (Gomez-Rodriguez et al., 2006).

Despite all these benefits, conventional signal processing algorithms cannot be used for these systems. Unlocking the full potential of AER systems often requires designing new algorithms, starting from first principles in the event-based paradigm. The review of existing event-based algorithms is therefore an integral part of this survey. Notably, there exists a research community around event-based signal processing and control (Miskowicz, 2016), but so far the cross-pollination to brain-inspired event-based sensing has been limited.

1.2. Structure of This Paper

In this paper, we survey the literature published on neuromorphic and event-based sensing in vision, auditory and olfactory sensing systems. Our goal was 2-fold: to identify shared challenges for event-based sensing in these modalities, and to provide a collection of references covering relevant work in these three areas to facilitate research bridging between areas.

Event-based vision is by far the most advanced modality concerning event-based technology and applications, and therefore it takes up most of the space in the survey section of this paper. Auditory event-based sensing has received much less attention, and olfactory even more so. Comprehensive review and survey papers covering these modalities are much harder to find than for vision, and we hope that our contribution will help the inclined reader to identify relevant primary research in these areas.

Finally, the focus on sensory systems indicates that there is a gap in research in the area of more general event-based processing. We discuss this at the end of the paper and point out some domains that may show potential for event-based approaches to data analytics.

The rest of this paper is organized as follows: in section 2 we review the existing surveys on event-based signal processing approaches. Section 3 reviews primary literature on event-based vision systems. Event-based auditory systems are covered in section 4, and olfactory systems in section 5. Section 6 provides a summary of the references covered. Finally, in section 7 we conclude the paper by discussing the main takeaways from this survey and potential future work in this area.

Readers may want to initially focus on sections 1 and 7 sections and treat the survey sections 2 to 5 as reference collections that may enable a “deep-dive” into each field.

2. Previous Surveys and Benchmarks

Among the first papers that review a relevant field is a survey on neuromorphic vision sensors (Etienne-Cummings and Van der Spiegel, 1996). Performed more than two decades ago, when the field was still at its infancy, that survey provides a history of implementing neuromorphic sensors. More recently a short survey of silicon retinas and cochleae has been presented (Delbruck and Liu, 2012), providing a history of recent advances in the area. In Delbruck (2016), a perspective on developments of event-based vision sensors, algorithms and applications over the period of 2002–2016 is presented. The most recent and likely most comprehensive survey on event-based vision contains “everything that has ever been written” on the topic (Delbrück, personal communication; Gallego et al., 2020).

Specialising on the design of VLSI neuromorphic circuits for event-based signal processing, in Indiveri (2008) an overview of selective attention systems based on neuromorphic winner-take-all networks, ranging from single-chip vision sensors to multi-chip systems is presented. In another work, a very short survey of neuromorphic chips is presented in Liu and Wang (2009), which introduces the required hardware and summarizes the applications.

A good discussion on recent developments in neuromorphic optical sensing and artificial vision is presented in Posch et al. (2014). The paper introduces the functionality of biological retinas and then provides an overview of existing neuromorphic vision systems and their implementation. Then the paper expands to a discussion on the design of silicon retinas and neuromorphic vision devices.

A mini-review of current neuromorphic approaches for vision, auditory and olfactory sensors (Vanarse et al., 2016) provides a useful review on some state-of-the-art approaches, but it covers only a small proportion of research in this area.

A literature survey and taxonomy of artificial olfactory systems is presented in Kowadlo and Russell (2008). In Moraud and Chicca (2011), a short review on the field of bio-inspired autonomous navigation and neuromorphic chemical sensing is presented. In Chicca et al. (2013), a brief review of neuromorphic olfactory sensors can be found. A review on neuromorphic approaches for artificial olfactory systems is performed in Vanarse et al. (2017).

To support continuous improvement of algorithms and methods, there is a need for challenging event-based datasets. Benchmark data sets exist that are specifically crafted to assist model design, refinement and testing using event-based signal processing algorithms. In one of the first major efforts, in Tan et al. (2015) some guidelines for the creation of neuromorphic vision benchmarks and challenges is presented. In Gibson et al. (2014a), a data set of 11 moving scenes recorded by a dynamic vision sensor is generated. In Li et al. (2017), using a dynamic vision sensor, 10,000 frame-based images are converted into event streams. The conversion is performed by repeated closed-loop smooth movement of frame-based images. In Serrano-Gotarredona and Linares-Barranco (2015), two sets of data are proposed released for event-based object recognition. One set was obtained by browsing poker card decks and the other was generated by displaying 10,000 moving symbols. In another work (Hu et al., 2016) mostly dynamic vision tasks like action recognition or tracking are targeted for benchmark data collection. In Zhu et al. (2018), a data set with a synchronized stereo pair event-based camera system for 3D perception is presented, which is collected in a variety of illuminations, environments and camera mountings. A data set is presented in Binas et al. (2017) which is a recording of DVS cameras in driving applications. The data are collected under different conditions like daytime, night, dry, wet surface, and different driving speeds. Several event-based algorithms and a remarkable JAVA framework for the DVS can be found at jAER (2021).

3. Event-Based Vision Systems

Machine vision has seen the greatest uptake of event-based sensing and signal processing approaches so far. There are many approaches to develop silicon retinas, examples of which include (Etienne-Cummings et al., 2000; Costas-Santos et al., 2007; Delbruck, 2008; Lichtsteiner et al., 2008; Delbrück et al., 2010; Matolin et al., 2010; Sulzbachner and Kogler, 2010; Camunas-Mesa et al., 2011, 2012; Leñero-Bardallo et al., 2011; Posch et al., 2011; Serrano-Gotarredona et al., 2013; Darwish et al., 2015; Garćıa et al., 2016; Zheng et al., 2016).

Event-based vision has clear advantages over frame-based approaches. First, event-based vision systems report the exact times of relative brightness changes rather than providing a snapshot of absolute brightness at all locations in the visual field. Absolute pixel illumination is not an invariant property of the environment (Lowe, 2004), and it has been hypothesized that this is the reason why many current algorithms fail in uncontrolled conditions (Reinhard et al., 2010).

Second, periodic sampling imposes restrictions on the amount of information that can be extracted from a scene. For example, it has been shown that the human eye can resolve visual dynamics at up to 1 KHz, because this is where natural scenes contain most of the information; Even a sampling rate of 60 Hz can lead to an information loss of around 75% (Akolkar et al., 2015a).

Third, event based sensors can achieve very short latencies that are only constrained by the minimal response time of the sensor, because they only collect data when something is happening, whereas frame-based systems are forced to obey the fixed inter-frame interval. Moreover, periodic sampling suffers from a trade-off between low latency and high inter-frame redundancy, whereas event-driven sampling schemes avoid transmitting temporally redundant information while maintaining the capability of low-latency response to changes. Low latency and avoidance of redundant information acquisition also largely eliminates motion blur.

These properties make event-based vision highly suitable for applications like robotic platforms (Mueggler et al., 2014), where standard cameras with their relatively high latency and computationally expensive sensing and processing pipeline are sub-optimal. It should however be noted though that low-power and low-bandwidth sensing can only be achieved in scenes with sparse activity. Richly textured visual scenes or auditory scenes with high levels of white noise might cause very high event rates, and in consequence power consumption.

3.1. Applications

Since proposed, these devices have found their way in many applications. Here we provide a review on applications of event-based vision systems.

3.1.1. Tracking

Arguably, due to the nature of these sensors, tracking is the most straight forward application of DVS cameras. Tracking with conventional machine vision algorithms is a computationally expensive task. However, as DVS cameras only transmit changes in the images, they are inherently suitable for tracking moving objects. For this reason, among all the applications, the largest number of research is performed in tracking.

3.1.1.1. Object Tracking

A hierarchical neuromorphic system for tracking objects is presented in Gómez-Rodŕıguez et al. (2011), where two processing layers work in cascade for first detecting moving objects and then tracking them with crossing trajectories. In Zong et al. (2018), MLS surface fitting and local plane fitting methods are employed to identify the images collected by a DVS camera for tracking objects. The system is tested on uniform and high speed motion and it is shown that it can filter noise and reach high accuracy and robustness.

Frame-based tracking systems become less accurate as the speed of objects increases. They are also susceptible to changes in illumination. The authors in Saner et al. (2014) approach this problem by combining a DVS and a frame-based camera, such that tracking is performed based on the frame-based system, but the DVS device is used to capture the information about the changes in the scene in the time interval between consecutive frames.

In Delbruck et al. (2015), a human vs. computer slot car racing is devised, where a DVS camera is used to track both cars and control the break and throttle of the racing car. The low latency provided by the DVS camera results in consistent outperformance of human drivers by the computer.

3.1.1.2. Satellite Tracking and Space-Situational Awareness

The high dynamic range of an event-based camera is exploited to track satellites using a ground-based telescope in full daylight (Cohen et al., 2019). A dataset is also provided (Afshar et al., 2020).

3.1.1.3. Multiple Object Tracking

Some works targeted specifically multiple object tracking, for example, Gómez-Rodŕıguez et al. (2010) that presents a cascade architecture for that purpose. In Linares-Barranco et al. (2015), a lattice structured FPGA framework has been presented that allows uncorrelated-event noise removal for tracking multiple objects. The system is capable of adapting itself to fast or slow and large or small objects.

3.1.1.4. Stereo Tracking

Most tracking algorithms use one DVS camera which provides a 2D representation of the environment. Some works have tried to employ two cameras so they can build a 3D map of the environment resulting in a better tracking system, for example, Schraml et al. (2010b), that aimed at tracking people in 3D. The system is capable of detecting and tracking people within a 4 m range with a refresh rate of the depth map of up to 200 Hz. In another work (Müller and Conradt, 2012), two cameras independently track an object. Then a self-adjusted neural network maps the 2D angular coordinates into a Cartesian 3D position of the object.

3.1.1.5. Camera Movement

Tracking can be used to calculate the camera movement. In Kim et al. (2008) a DVS is used to track accurate camera rotation to build a persistent mosaic of a scene. Another work (Reinbacher et al., 2017) proposed panoramic camera tracking. The authors show that the spatial position of the events is enough for simultaneous tracking and mapping, and there is no need for the appearance of the imaged scene point.

3.1.1.6. Camera Pose Estimation

Using a probabilistic generative event model in a Bayesian filtering framework, a camera pose estimation algorithm is designed in Gallego et al. (2015, 2016). This research design the likelihood function used in the filter to process the observed events. Based on the physical characteristics of the DVS, the authors propose the use of the contrast residual as a measure of how well the estimated pose explains the observed events. The authors extend their work (Gallego et al., 2018a) by tackling the problem of accurate, low-latency tracking of a camera from an existing photometric depth map built upon classic dense reconstruction pipelines. Using cubic splines, in Mueggler et al. (2015c) the pose of a DVS camera is estimated by a smooth curve in the space of rigid-body motion, with the trajectory curve being optimized according to the incoming events.

3.1.1.7. Feature Tracking

In some tasks, the camera tries to track some features in the scene. In Lagorce et al. (2015b), a DVS camera is used for tracking of multiple visual features. In the research, various kernels, such as Gaussian, Gabor, combinations of Gabor functions and arbitrary user-defined kernels are used to track features from incoming events. The trackers are capable of handling variations in position, scale and orientation by using multiple pools of trackers. In Ni et al. (2015) a pattern tracking algorithm is proposed, in which the pattern tracking iteratively updates the model location and orientation to match the 2D image plane based on the arrival of events. Since the algorithm tracks patterns, it is capable of tracking objects even if they are subject to transformations. Another example of using DVS cameras for tracking corner-event features is Alzugaray and Chli (2018a).

3.1.1.8. Micro Particle Tracking

In Ni et al. (2012), an asynchronous event-based Hough circle transform is developed to track micro particles. The system allows for a robust multiobject position detection at a frequency of several kHz at a low computational cost. Measurements in turbulent fluid flows often require high-speed imaging techniques. These systems are usually limited by the amount of memory available on-board. In Drazen et al. (2011) a DVS camera is used for particle tracking which enables a 100-fold reduction in bandwidth and data storage. A fast-flow visualization method is presented in for tracking buoyant soap bubbles. The data analysis in this work relies on Kalman filters to associate the events with traces and to reconstruct the path and velocity of particles.

3.1.1.9. Sub-atomic Particle Tracking

An extensive parallel tracking system is designed in Neri et al. (2015), that allows real-time tracking withe a latency of <1 μs. The retina architecture is organized in three main blocks. The first block is a buffer that stores the hit information according to a hold logic. This module gets activated when downstream modules are busy. The second block is a pool of engines that process the hits. And the third block calculates the track parameters. The authors present the testbeam results in Neri et al. (2017).

3.1.1.10. Car Tracking

An embedded vision system for tracking cars has been designed in Litzenberger et al. (2006c) which offers a one millisecond timestamp resolution.

3.1.1.11. Person Tracking

In Pikatkowska et al. (2012), the problem of multiple person tracking in the occurrence of high occlusions is addressed. The authors apply Gaussian Mixture Models for detection, description and tracking individuals.

3.1.1.12. Robotics

In many robotic applications, the agility of robots is limited by their sensing pipeline. A DVS camera is used in Censi et al. (2013) for robot pose tracking to increase robot agility, demonstrating that tracking performance is unaffected by fast motion. An autonomous target tracking approach is proposed in Jiang et al. (2017) for a snake-like robot. Using the Hough transform based on spiking neural networks the target pole is detected as two parallel lines from the visual input. The pose and periodic motion features of the robot are combined to develop an adaptive tracking based on the estimated depth information. In order to design a tracker which is robust to temporal variations due to the relative movement at different velocity of camera and target, a new algorithm is developed in Glover and Bartolozzi (2017). The authors develop a particle filter that follows the target position within the spatio-temporal data, while rejecting the clutter events that occur as a robot moves in an environment. The tracker is used in a robot vision system.

3.1.2. Classification

One main application of event-based cameras is in classification. Among the first studies that used an event-based camera for classification is Schraml et al. (2010a), in which an algorithm for pedestrian classification is proposed that makes use of density and distance metrics for clustering asynchronous events generated by scene dynamics. In Chen et al. (2012) an algorithm is developed for categorization of human postures that employs a combination of event-based hardware and bio-inspired software architecture.

In O'Connor et al. (2013), a method based on the Siegert approximation is proposed for integrate-and-fire neurons to map an offline-trained deep belief network onto an event-driven spiking neural network. They use this system in character recognition in presence of distraction. The event-based classification algorithms' performance lags far behind their frame-based counterparts. The authors in Sironi et al. (2018) cite two reasons, first the lack of low level representations and architectures, and second the lack of real-world event-based data-sets. To tackle these, the authors introduce an event-based feature representation and release a dataset for object classification.

3.1.3. Stereo Matching

A variety of computer vision applications require a 3D structure of the real-world scene. This task is usually performed by a stereo vision, which consists of two cameras observing the same scene from two different angles. Since these two cameras capture slightly different pictures, an algorithm is needed to match corresponding pixels that are projections of the same scene in the images. In frame-based approaches, the frames from two digital cameras are processed pixel by pixel and the patterns that match in both stereo frames are found. When using event-based cameras, it is the events that should be processed to yield such information. This means that whole new set of algorithms are needed to perform this task. In this part of the paper, we review the works that are performed in this area.

In Dominguez-Morales et al. (2011), the existing frame-based stereo matching algorithms are discussed and then an AER stereo matching algorithm is proposed that exploits some of the principles in frame-based stereo matching. In Kogler et al. (2010, 2011a,b); Kogler (2016), the time difference between the received pixels is used as matching criterion. The authors use a global optimization scheme that is designed to deal with sparse data to minimize the matching cost. The work also designs a filter that analyzes the disparities around pixels. In Carneiro et al. (2013), a novel N-ocular 3D reconstruction algorithm is proposed that allows preserving the original dynamics of the scene. This results in a more robust 3D reconstruction.

In a research (Rogister et al., 2012), it is shown that matching on the timing of the events provides information about the 3D objects, when combined with geometric constraints using the distance to the epipolar lines. The approach is capable of filtering out the incorrect matches and can accurately reconstruct the depth model. Because of the geometry of the sensors, estimating the epipolar geometry constraints is difficult. In Benosman et al. (2011), it is shown that these constraints are a consequence of the static frames, and using event-based cameras can, to some extent, overcome this limitation. The authors present a model for asynchronous event-based vision that is used to derive a new concept of epipolar geometry based on the temporal information of the pixels.

3.1.3.1. Cooperative Neural Networks

A modification of the cooperative network is used in Piatkowska et al. (2013, 2014) to store the history of the recent activity in the scene. This is used to serve as spatiotemporal context used in disparity calculation for the events. In this system, the network constantly evolves as events arrive, the network constantly evolves. The work then is further improved (Piatkowska et al., 2017) to reduce the error by over 50%. A dynamic cooperative neural network is used in Firouzi and Conradt (2016) in which the interaction between cooperative cells applies cross-disparity uniqueness-constraints and within-disparity continuity-constraints, to asynchronously extract disparity for each new event. This work is then extended in Dikov et al. (2017), where a spiking neural network is implemented on SpiNNaker.

A different approach is presented in Osswald et al. (2017) that unifies the domains of perceptual neuroscience and machine vision. In this research, a spiking neural network is proposed that is inspired by cooperative network of Marr and Poggio (1976) and is capable of computing stereo correspondence from the visual stream of neuromorphic vision sensors. Because of the dynamic properties of the neuromorphic neural networks, their co-localization of memory and computation and their size, these networks offer possible solution to the Von Neumann bottleneck problem, which is a promising platform for stereo vision systems.

3.1.3.2. Gabor Filter

The use of Gabor filter in extracting information about the orientation of the object edges that produce the events is studied in Camuñas-Mesa et al. (2014a) and Camunas-Mesa et al. (2014). The authors apply the matching algorithm to the events produced by the Gabor filter instead of the events produced by the DVS, therefore increasing the number of constraints applied to the matching algorithm. Their results show that this technique improves the final 3D reconstruction.

3.1.3.3. Using Single Camera

In conventional stereo matching algorithms, a set of camera from different angles are used to find a dense 3D structure of the scene. In Rebecq et al. (2016, 2018), however, it is investigated how one single DVS camera can be used to build a semi-dense 3D structure. DVS cameras have two characteristics that make this possible: they respond to edges, which naturally provide semi-dense geometric information about the scene and they provide continuous measurements of the scene. In another work (Kim et al., 2016), a single DVS 3D reconstruction algorithm is proposed which is based on three decoupled probabilistic filters, each estimating 6-DoF camera motion, scene log intensity gradient and scene inverse depth relative to a keyframe.

3.1.3.4. Similarity Measure

Performing stereo matching requires a type of similarity measure that defines a criteria based on which the corresponding pixels are found. In Schraml et al. (2015) a cost function is proposed which uses a similarity measure based on event distributions. A novel feature descriptor is proposed in Zou et al. (2016) which can describe the local context or distribution of the event data and constructs an effective similarity measure for data matching. Considering the correlation of incoming events, in Eibensteiner et al. (2017), in addition to the spatial information, the time of the occurrence of the events is also used as part of the similarity measure. In Zihao Zhu et al. (2018), the velocity of the camera and a range of disparities are used to synchronize the positions of the events as if they were captured at a single point in time. The authors propose a novel cost over these time synchronized event disparity volumes that rewards similarity between volumes and penalizes blurriness. In Zhou et al. (2018), of the optimization of an energy function is designed to exploit small-baseline spatiotemporal consistency of events triggered across the image planes. To reduce the uncertainty of the estimation, a probabilistic depth-fusion strategy is developed. The method does not require the motion of the camera or prior knowledge about the scene.

3.1.3.5. Verification Approaches

Stereo matching with DVS cameras is a new field and there is an emerging community of scientists that develop algorithms and methods for the problem. The existing ground truth data and evaluation platforms that are proposed for frame-based systems cannot be used for event-based systems. Therefore, there is a need for new metric and verification methods to measure the performance of the proposed algorithms. In Sulzbachner et al. (2010), a tool for synthetic scene generation, ground truth generation and algorithm verification is proposed. In another work (Kogler et al., 2013), a new approach for the evaluation of stereo matching algorithms is presented.

3.1.4. Recognition

Object recognition is one of the main fields in machine vision and as a new technology, event-based cameras have found their way in the field. Camera sensor networks are a network of camera in an environment than collectively capture and process visual information. Due to the number of cameras, these systems require high computational power. In Teixeira et al. (2006), a pattern recognition algorithm is designed for a network of event-based cameras to identify some hand gesture signs. In Ahn et al. (2011), a bare hand gesture recognition algorithm is proposed that recognizes three gestures in rock-paper-scissors game. In Amir et al. (2017), an event-based camera and an event-based processor with one million spiking neurons are used for human gesture recognition. They report that their algorithm recognizes gestures with a latency of 105 ms. A hardware implementation of event-based data processing is presented in Hofstätter et al. (2011), where an event-based camera is used for object recognition.

Solving texture recognition task with an event-based sensor is targeted in Pérez-Carrasco et al. (2010), where the authors show that the recognition rate has not degraded when new sensors are used. In Negri et al. (2018), an event-based camera is used to recognize the shape of poker signs. Combining an event-based sensor with a convolutional neural network, an object recognition and orientation estimation algorithm is proposed in Ghosh et al. (2014), which shows very high accuracy at real-time speed. In Orchard et al. (2015), a spiking hierarchical model is presented for object recognition which show that the temporal information of the events can be used in object recognition in a simpler way than traditional methods.

An event-based camera is used in Reverter Valeiras et al. (2016) to solve the 3D pose estimation problem. While in frame-bases systems the sampling frequency is 30–60 Hz, the authors take advantage of event-based cameras and design a pose estimation algorithm that achieve a temporal resolution of resolution of several hundreds of kHz on a conventional laptop.

3.1.5. Detection

Published reports of event-based cameras being used for detection are still comparably scarce. A face detection algorithm is proposed in Barua et al. (2016), in which a patch-based model for the events is developed. The designed system is capable of reconstructing 2,000 frames per second. In Cannici et al. (2018), two neural network architectures are proposed for object detection, where one network integrates events into surfaces and one that uses convolutional and max pooling layers to exploit the sparsity of camera events. An FPGA implementation of retinal ganglion cell model is designed in Moeys et al. (2016b) which detects moving objects. The authors use this processing in conjunctions with a DVS to extrapolate information about object position. Using a DVS, a car detection algorithm is proposed in Chen (2018) which by employing convolutional neural network handles motion blur and poor illumination conditions problems.

Hand gesture recognition is also studied in Lee et al. (2014), where a neuromorphic post-processing hardware is used. In this work, the motion trajectories of hands are detected, segmented and translated into discrete feature vectors. These feature vectors are then classified via hidden Markov models. In Alzugaray and Chli (2018b), an event-based camera is used for corner detection and tracking. They report promising results at with a speed four times higher than conventional algorithms. Corner detection is also studied in Clady et al. (2015), where a luminance-free method is developed.

Using event-based cameras, a line detection algorithm is proposed in Seifozzakerini et al. (2016, 2017), where Hough Transform is employed in spiking neural networks. In another work (Brändli et al., 2016), a line segment detector is proposed which tries to infer which events are caused by the motion of the same spatial feature by parameterizing the event streams as a set of line segments.

In event-based processing in textured scenes, millions of events are generated per second that require great computational power. To tackle this problem, a research (Mueggler et al., 2017a) proposes a method to reduce the stream of event to a corner event stream. They design a corner detection algorithm that reduces the event rate by a factor of 20. The commonly used Harris corner detector is used in Vasco et al. (2016), where the frames are replaced by a stream events. The research test their method on a DVS camera mounted on a robot.

Sun sensors are navigational tools used in spacecrafts to detect the position of the Sun. In Farian et al. (2015), an event-based sensor is designed that is composed of two lines of pixels that perform in parallel and two optical slits aligned above the chip. The sensor is capable of directly detecting the position of the Sun and so no further processing is required.

3.1.6. Localization and Odometry

Fast localization is crucial in many applications like driving and maneuvering, which traditional cameras can seldom provide. Due to their sampling speed, event-based cameras are very suitable for localization and odometry. Among the first efforts to use event-based cameras for localization is Weikersdorfer and Conradt (2012), which adopts a condensation particle filter tracker and demonstrates robust performance at low computational cost. Another work (Weikersdorfer et al., 2013) proposes a localization and mapping method that offers real-time performance on standard computing hardware. A fast localization algorithm is proposed in Yuan and Ramalingam (2016), in which a fast spatio-temporal binning scheme is developed to detect lines from events. A 3-D model of the world is then constructed which is used to estimate sensor pose. In Milford et al. (2015), an event-based camera is used for simultaneous localization and mapping.

In one of the main first attempts in event-based odometry, a novel event-based tracking approach based on image-to-model alignment is combined with a 3-D reconstruction algorithm in a parallel fashion (Rebecq et al., 2017b). The proposed system runs in real time and supports high dynamic range input with strong illumination changes.

Odometry is to measure the ego-motion of a camera, used, e.g., in robotics. Event-based cameras have great potential for Odometry as they can track fast movement accurately without blurring and quantization. However, new algorithms are required to exploit the sensor's characteristic. The first research that uses event-based cameras in odometry is Kueng et al. (2016) and Mueggler et al. (2017b), in which the features are detected in the grayscale frames and then tracked using stream of events. These features are then fed to an odometry algorithm. In Zhu et al. (2017b), an event-based odometry algorithm is proposed that is asynchronous and provides measurement updates at a rate proportional to the camera velocity.

In Horstschäfer (2016), using an accelerometer and a gyroscope, an a technique is presented for image and event stabilization of an event camera. The camera is then used for visual odometry of a robot. An odometry algorithm is proposed in Rebecq et al. (2017a) which tracks a set of features via overlapping spatio-temporal windows to construct motion event frames. The results presented in the work suggest that their algorithm outperforms state-of-the art conventional approaches with much lower computational expense. In Mueggler et al. (2018) an algorithm is proposed in which the camera trajectory is approximated by a smooth curve in the space of rigid-body motions using cubic splines, which reduces the number of variables in trajectory estimation problems.

3.1.7. Motion Detection

Motion detection has many applications and is an important area in machine vision research. The first research that uses an event-based camera for motion detection is presented in Ruedi (1996), where a simple retina of 23 by 23 pixels is used. A new motion detection algorithm is proposed in Barranco et al. (2009), where by integrating temporal feature results, a new matching algorithm with high stability is obtained. A clustering method is proposed in Schraml and Belbachir (2010) which exploits the sparse spatio-temporal representation of events for detecting moving objects. In Abdul-Kreem and Neumann (2015), the spatio-temporal filtering scheme suggested by Adelson and Bergen (1985) is adopted to make it consistent with the event representation. Finding representative features for motion information is another field of research which is targeted in Sullivan and Lawson (2017), where conventional neural networks are used to extract features.

A unifying framework is presented in Gallego et al. (2018b), in which several computer vision problems are solved: motion, depth and optical flow estimation. By maximizing an objective function, the point trajectories on the image plane are found that are best aligned with the event data.

Bio-inspired systems for motion detection have incorporated mechanisms from the visual system into spiking networks to achieve motion detection (Ridwan and Cheng, 2017; Dalgaty et al., 2018).

Optical flow is the pattern of apparent motion of objects in a scene created by its motion. In Rueckauer and Delbruck (2016), nine optical flow event-based algorithms are compared. To perform the comparison, a dataset of two synthesized and three real samples is created. The authors have made the data sets and the source codes for the algorithms publicly available. Some studies use neuromorphic networks for processing the output of event-based sensors. In Giulioni et al. (2016), an architecture for robust optical flow extraction with an analog neuromorphic multi-chip system is proposed. The algorithm uses a feed-forward network of analog neurons, and the computation is supported by the time of spike emissions. The optical flow is extracted based on time lag in the activation of nearby retinal neurons.

Finding the optical flow using a DVS camera is performed in Benosman et al. (2014), where it is shown that the precise optical flow orientation and amplitude can be estimated with a local differential approach on the surface defined by coactive events. In Bardow et al. (2016) an algorithm is designed that simultaneously finds the optical flow and the brightness of the images. In this work, a cost function is defined and minimized that contains the asynchronous event data and the spatial and temporal regularization within a sliding window with time interval.

An optical flow algorithm called adaptive block-matching is proposed in Liu and Delbrück (2018) which uses time slices of accumulated events, that are adaptively rotated on the input events and optic flow results. The rotation is performed in such a way to ensure the generated slices have sufficient features for matching.

Another example of event-based motion detection include Barranco et al. (2015a), the algorithm in Liu and Delbruck (2017) which mimics motion estimation methods used in MPEG, and the method developed in Gallego and Scaramuzza (2017) for angular velocity estimation.

In Barranco et al. (2014), a comparison between conventional vision algorithms and event-based cameras is performed. The authors show that due to the nature of event-based cameras, motion detection is much easier with these sensors, and they can easily outperform computer vision methods in accuracy and speed.

Event-based cameras have been reported to be evaluated for motion detection applications. For example Litzenberger and Sabo (2012) asks if event-based cameras can be used for optical motion analysis in sports, with a positive result. In Mueggler et al. (2015a), two DVS cameras are used to estimate the trajectory of objects that are thrown at a quadrotor. The object's trajectory is estimated using an Extended Kalman Filter with a mixed state space.

3.1.8. Transportation Systems

Machine vision algorithms are widely used in transportation systems. The requirement for low latency processing plays to the strengths of event-based algorithms. A vision system is described in Litzenberger et al. (2006a) for counting vehicles simultaneously on up to four lanes of a motorway. The authors report fast, low power and robust vehicle counting. In another study (Litzenberger et al., 2006b), a silicon retina is used for vehicle speed estimation that measures the velocity of vehicles on four lanes simultaneously, under variable lighting and atmospheric conditions. A system for real-time classification of vehicles into cars and trucks is described in Gritsch et al. (2008), which achieves an accuracy of over 90%. An application in a pre-crash warning system is proposed in Kogler et al. (2009), where a silicon retina-based stereo vision algorithm achieves a temporal resolution of 1ms, across various lighting conditions.

3.1.9. Healthcare

In recent years, computer vision has found many applications in healthcare, and applications of event-based processing are emerging in this field. Among the first attempts is the work published in Fu et al. (2008a,b), where a vision system is designed to detect accidental falls in elderly home care applications. Compared to frame-based methods, the system reports a fall at ten times higher temporal resolution and shows 84% higher bandwidth efficiency as it transmits fall events. In Belbachir et al. (2012) a stereo matching algorithm is used on two DVS cameras to provide a 3D vision system for fall detection that achieves over 90% positive detections. The authors argue that one advantage of using DVS cameras is privacy as it does not record the true images of the scenes.

In Ghaderi et al. (2015), a wearable mobility device is designed to assist the blind with navigation and object avoidance. In this system, two DVS cameras are used to provide a 3D vision, which is converted via an individualized head-related translate function into a 3D output sound. This device is then improved in Everding et al. (2016).

In order to decrease the transmission delay of visual and non-visual medical records, DVS cameras and edge computing are employed in Chen et al. (2017) reducing the transmission delay by 89.15–86.88%. Optical recording of neural activity requires cameras capable of detecting small temporal contrast with sample rate of 1 kHz. Using CMOS sensors is very challenging as they require high data rates of up to 1 Gb/s. To overcome this, a DVS camera is used for the task in Taverni et al. (2017), that suggests long-term use of the sensor in neural recordings can be very beneficial.

Using a DVS camera, a system is designed in Gaspar et al. (2016) which can be used as a retinal prosthesis or vision augmentation. An algorithm based on integrate and fire neuron model is used in this work to emulate temporal contrast sensitive retinal ganglion cells.

3.1.10. Industry

Many industrial applications require very high sampling rate. For example, monitoring a turbine with thousands of rpm poses a serious challenge to frame-based vision systems. In Perez-Peña et al. (2011), a DVS-based surveillance video system is designed for ultra fast industrial environments, that monitors a machine with a rotating part at 6,000 rpm, with good results.

Flow visualization in wind tunnel testing is of crucial importance for practical applications. In Borer (2014), DVS cameras are used for tracking neutrally buoyant soap bubbles. The authors use three cameras to build a 3D reconstruction, where two cameras provide 3D vision and the third camera increases the reliability of detection in areas with poor lighting, poor background contrast or with reflections.

3.1.11. Segmentation

Segmentation is the process of partitioning an image into multiple sets of pixels and is a common task in computational vision. In the first attempt to design a segmentation algorithm for event-based cameras, a contour detection algorithm is proposed in Barranco et al. (2015b), where structured random forests are used to find the location of contours and their border ownership. These contours are used for the segmentation of the scene. In Surovich et al. (2017) a dynamic segmentation of moving objects is proposed that uses a DVS with a linear polarizing filter. The authors use wavelet transform to analyze the local spatio-temporal content of the images. Segmentation requires high computational power, and in most applications, performing real-time segmentation is very difficult. In Thakur et al. (2017), the random walker algorithm is adapted to a spiking neuromorphic processor to perform real-time segmentation of the scene. The system can perform segmentation at the speed of 1,000 images per second.

Segmentation can benefit from color cues. Yet, the original DVS camera does not transmit color information. In Marcireau et al. (2018), a dichroic beam splitter is thus used to decompose the input light into red, green and blue lights, and then send them to three DVS cameras. The output of these cameras are then processed to perform color segmentation and tracking.

A new event-based protocol is proposed in Darwish et al. (2017), that suppresses spatial redundancies of event-based cameras. The activity of the event-based camera is limited to the effective and relevant information in the scene; therefore, the data flow is drastically reduced. The authors propose a cost-free image segmentation using their method.

3.1.12. Robotics

Many tasks in robotics require reliable and low-latency sensing, hence posing a promising field for applying event-based cameras (Camuñas-Mesa et al., 2014b).

3.1.12.1. Obstacle Avoidance

Among the first studies that used event-based cameras in a real robot is Clady et al. (2014), where these sensors are used to design a fast obstacle avoidance method. The use of event-based cameras in obstacle avoidance problem in robots was then continued in Blum et al. (2017) and Milde et al. (2017), where the authors show how it is possible to achieve functional robot obstacle avoidance control strategies using a mixed signal analog/digital neuromorphic processor with an event-based sensor.

In Milde et al. (2015), an obstacle avoidance system is designed which is based on optic flow. To extract optic flow, the authors use a plane fitting algorithm that estimates the relative velocity in a small spatio-temporal cuboid. The depth structure is then derived from the translational optic flow.

3.1.12.2. Balancing and Control

Conradt's pencil balancing robot was using two event-based vision sensors to sense deviations from the vertical with low latency, and was demode at numerous conferences in the era (Conradt et al., 2009a,b). Event-based sensors are also used in Mueller et al. (2015a,b) for feedback control of mobile robotic systems. The work is continued in Singh et al. (2016) to investigate the problem of quadratically stabilizing a continuous time linear time invariant system using event-based cameras.

3.1.12.3. Flying Robots

Low-latency processing of visual information is crucial for flying robots as thy require fast reactions. A new event-based method to compute optic flow for miniaturized indoor flying robots has been demonstrated in Conradt (2015), that can be embedded on the robot due to its low power requirements and small form-factor. In another work (Orchard et al., 2009), event-based cameras are used for planetary landing tasks. In Hordijk et al. (2017), the “local plane fitting” algorithm is extended to obtain an improved and more computationally efficient optical flow estimation method. The developed algorithms are implemented in a constant divergence landing controller on a quadrotor.

3.1.12.4. Actuators and Manipulation

Many robots require vision for manipulating the environment. In Linares-Barranco et al. (2007), an event-based camera is used for visual sensing, processing and actuating a robot that mimics human behavior. To reproduce human movements, a spike processing strategy is proposed in Perez-Peña et al. (2013) that uses a silicon retina to find the trajectory of human movement. In another work (Jimenez-Fernandez et al., 2009), the actuators of a robot are controlled, based on the input from a camera, to move the robot on a line on the floor.

Precise information about the position of objects and manipulators is crucial in object manipulation tasks where the grippers lack force sensing. To provide a haptic feedback, an artificial retina is used in Bolopion et al. (2012) that provides high update rate of the moving objects and a frame-based camera is devised to provide the position of the object. In Ni et al. (2012), an event-based iterative closet point algorithm is proposed to track a micro-gripper's position. The authors use both a DVS camera and a frame-based camera, where the temporal precision of the asynchronous silicon retina is used to provide a haptic feedback to assist users during manipulation tasks, and the frame-based camera is used to retrieve the position of the object.

When grasping objects, human fingers have very sensitive touch receptors that enable us to apply the precise pressure needed to grasp items. Too low pressure can lead to grasping failure and too much pressure may damage the object. In Rigi et al. (2018), event-based cameras are used to develop algorithms for detecting incipient slip, stress distribution and object vibration. They compare their results with a high speed 1,000 fps camera and show good performance with a very small (44.1 ms) latency.

3.1.12.5. Maneuvering and Navigation

The agility of robots is limited by the latency of their perception. Therefore, event-based cameras can by useful to support high speed robot maneuvers. To achieve a faster vision, the first onboard perception system for 6-DOF localization during high-speed maneuvering of a quadrotor is presented in Mueggler et al. (2014). A DVS is used in Delbruck et al. (2014) to extract motion parallax cues relating to 3D scene structure in the a navigation task, with better performance than frame-based approaches. A guidance system inspired by honeybee vision was proposed in Serres et al. (2016). The simulated bee is equipped with a compound eye comprising 10 sensors, two optic flow regulators that update the control signals, and three event-based controllers.

3.1.12.6. Vision and Attention

In Klein et al. (2015), two DVS cameras are mounted in a robot head to provide vision. The authors designed an image stitching algorithm to represent a scene larger than the field of view of each of the retinas. In another work (Moeys et al., 2016a), a DVS camera is used on a head of a predator robot that follows a prey robot. Robot goalies require very fast reaction time which is hard to achieve with frame based systems. In Delbruck and Lang (2013) and Delbruck and Lichtsteiner (2007) a fast self-calibrating robotic goalie is designed which offers low latency and CPU load. In another work (Becanovic et al., 2002), a neuromorphic analog VLSI sensor is combined with a digital omni-directional vision system. The system is used on a robot for locating a ball and directing the actuators for a goal keeper robot. In order to achieve a fast interaction with the environment, an attention system is developed for a humanoid robot in Rea et al. (2013). The authors report low-latency systems for the attention task.

3.2. Algorithms

There are some studies that propose new ways of processing event based vision signals. In this section we review papers that have come with new algorithms for DVS camera data.

3.2.1. Mapping

Convolutional neural networks (LeCun et al., 1989) inherently operate on frame-based principles. For many large-scale systems, event-based processing modules are impractical. In Pérez-Carrasco et al. (2013), an intermediate solution is presented. First, a database of training frames is generated by binning, i.e., collecting events during fixed time intervals. Second, a frame-driven convolutional neural network is trained to perform object recognition. Third, the learned parameters of the frame-driven convolutional network are mapped to an event-driven convolutional network. Finally, the timing parameters of the event-driven network are fine-tuned to optimize the recognition task.

3.2.2. Filtering

In signal processing, filtering refers to the prepossessing that is applied on the signals for feature detection and extractions. In image processing for example, it is performed to find features like corners, edges, and so on. In Ieng et al. (2014), a filtering methodology is proposed for event-based cameras. The authors propose asynchronous linear and non-linear filtering techniques. In Bidegaray-Fesquet (2015) the effect of noise and uncertainty on levels on the filtering of event data is investigated. The authors analyze the errors in terms of standard deviation of the normal distribution.

3.2.3. Lifetime Estimation

An algorithm is proposed in Mueggler et al. (2015b) that estimates the life-time of events from DVS cameras. The estimation is performed based on its velocity on the image plane. The application of such an algorithm is the construction of sharp gradient images at any time instant.

3.2.4. Classification

Conventional neural networks cannot be directly applied to the classification tasks for event-based data. In Li et al. (2018), it is shown how the deep representation learned with an originally optimized CNN is efficiently transferred to the event-based classification task. In this method, a spike-event coding is used and implemented based on the subthreshold dynamic of the leaky integrate-and-file model.

3.2.5. Compression

By only sending changes in the intensity of pixels, DVS cameras inherently perform high speed video compression. In Brandli et al. (2014), a decompression algorithm is proposed that performs an online optimization of the event decoding in real time. The system exhibits an adaptive compression ratio that depending on the activity in the scene can reach up to 1,800 for stationary scenes.

In order to design a compression algorithm for event-based data, an analysis on the spike firing mechanism and the redundancies of spike data generated from DVS is performed in Bi et al. (2018). The authors then propose a cube-based coding framework comprising three strategies, namely macro-cube partitioning structure, address-prior mode and time-prior mode.

A new compression algorithm for still images is proposed in Doutsi et al. (2015) which uses event-based sampling. In this algorithm, a bio-inspired filter is applied to the image and then the retinal-filtered image is fed to a sampler. To reconstruct the original image, the spike train produced by the sampler is decoded.

3.2.6. Prediction

A spiking neural network with learnable delays is used in Gibson et al. (2014b) to predict temporal sequences of the incoming events from a DVS camera. The system is capable of learning the temporal structure of space-time events, adapt to multiple scales and is able to predict future events in a video sequence. Using a DVS camera, a method is presented in Kaiser et al. (2018) to learn movements from visual predictions. The proposed method consists of two phases. First is learning a visual prediction model for a given movement and second is minimizing the visual prediction error.

3.2.7. High-Speed Frame Capturing

Event cameras only transmit light intensity changes in the scene, so they lack information about all the pixels. A method is proposed in Liu et al. (2017b), to recover a scene, in which the foreground exhibits fast motion and background is static. Frames taken from a conventional camera are first matched to events taken from a DVS camera, then the high-speed events are used to generate the image sequences between consecutive frames. Motion blur in frame-based cameras refers to the apparent streaking of moving objects in a photograph that occurs when part of the image being recorded changes during the exposure. In Pan et al. (2018), the blur generation process is modeled by associating the event data to a latent image. The method is called event-based double integral model that reconstructs a high frame rate, sharp video from a single blurry frame and its event data.

3.2.8. Spiking Neural Networks

Due to specific characteristics of event-driven signals, conventional machine learning techniques cannot be directly used for these signals. Therefore learning systems should be designed that are specifically suitable for these data. A new evolving neural network is developed in Dhoble et al. (2012) that utilizes both rank-order spike coding, also known as time to first spike, and temporal spike coding. The authors implement the system for a classification problem on event-based data from a DVS camera. A novel method for training an event-driven classifier within a spiking neural network system is proposed in Stromatias et al. (2017), which uses the activity provided by an arbitrary topology of prior network layers to build histograms and train the classifier in the frame domain. This way of building histograms captures the dynamics of spikes immediately before the classifier. The system is applied to data from a DVS camera.

3.2.9. Data Transmission

Normally, brain-machine interfaces emphasize faithful transmission of the recorded signals. An alternative approach is taken in Corradi and Indiveri (2015) that proposes a neural recording system is proposed for compressing data. This event-based system applies signal processing and neural computation to extract relevant information from the large amount of collected raw data. It transmits only the low-bandwidth outcome of the processing to remote computation modules.

3.2.10. Fusion

In order to process the output of event-based cameras more accurately, different networks including convolutional and recurrent neural networks are ensembled in Neil and Liu (2016) to jointly solve a recognition task. The authors show that the performance of the algorithm is higher than individual networks.

3.2.11. Hybrid Methods

Event-based vision systems offer fast visual processing with low computational requirements. However, high level visual processing, like, e.g., object recognition, is still a challenge for these devices. Some studies try to accomplish both objectives by combining the advantages of both systems. However, active vision systems need real time and high-level processing at the same time. In Sonnleithner and Indiveri (2011a,b, 2012), dedicated VSLI hardware is designed that implements an event-based network of spiking neurons for real-time processing in combination with a conventional vision system. A low-resolution event-based system responds in real-time to moving objects and produces fast reactive motor outputs. A conventional high-resolution machine vision system performs object recognition task.

In Weikersdorfer et al. (2014), a DVS and a frame-based camera are combined to produce a sparse stream of depth-augmented 3D points. The authors state a smaller amount of generated data and a continuous representation of motions as advantages of this system.

In order to combine the strength of both type of sensors, a frame-based video sensor is used along with an event-based camera in Leow and Nikolic (2015). The system is applied to a variety of applications including video-compression, foveated imaging on the moving objects, object tracking and velocity estimation.

3.2.12. Matching

In Moser (2015), a new approach for matching event sequences is proposed that is based on Hermann Weyl's discrepancy norm.

3.2.13. Feature Extraction

Feature extraction plays an important role in many machine learning applications. The problem is to determine which features from the signal should be extracted for processing, and how. In frame-based computer vision, the features are often defined as a function of the luminance of the pixels within an image. Temporal information of the scene is often not present, e.g., because the source material contains only still frames, or it is of comparably low precision, due to an underlying assumption that 24 frames/s are enough for applications with only moderately fast-changing scenes. Event-based cameras enable extracting different features as they capture temporal information of the scene at high precision. Feature extraction from event-based signals and their application in higher-level computer vision was the subject of many studies, that we review in the following.

3.2.13.1. Vehicle Detection

A spiking neural network is introduced in Bichler et al. (2011, 2012) to extract temporally correlated features from spike-based dynamic vision sensors. A spiking neural network is used in this work, in which the neurons become sensitive to patterns of pixels with correlated activation times. The authors employ a spike-timing-dependent plasticity scheme, where the synapses that do not contribute to spike activation are depressed. The system is developed for detecting cars passing a freeway.

3.2.13.2. Gesture Recognition

In Ahn (2012), local and global feature extraction methods are employed. First the local extraction method uses segmentation to extract smaller number of features from a long sequence of raw gesture events. This is called local because it only considers neighboring events. The global extraction transforms the local features to construct higher level features. The authors use an evolutionary algorithm for feature selection step.

3.2.13.3. Robot Vision

A new time oriented visual feature extraction method is presented in Lagorce et al. (2013), which is base on echo-state networks. The method is unsupervised and is suitable for high dynamic environments.

3.2.13.4. Hardware Implementation

An FPGA design of an analog-to-feature converter is presented in del Campo et al. (2013), which learns a dictionary of features from an event-based signal using matching pursuit and Hebbian learning. The code is sparse and suitable for neuromorphic processors. In Hoseini and Linares-Barranco (2018), using FPGA, a digital circuit is proposed for extracting frequency of rotating objects in real time. This feature can be used, along with other features for recognizing objects with rotating parts. In Yousefzadeh et al. (2015), a 2D convolution event-based processing unit it proposed to extract features from an input event flow. The system is highly parallel and can benefit from FPGA arrays.

3.2.13.5. Optical Flow

In Koeth et al. (2013), it is shown how motion features with spatio-temporal profile can be self-organized using correlations of precise spike intervals. The authors show that their framework forms topologic organization of features in a way similar to human brain.

A luminance-free feature extraction method is proposed in Clady et al. (2017) which performs by mapping the distribution of optical flow along the contours of the moving objects into a matrix. Using speed-tuned temporal kernels, the optical flow is integrated locally or globally in a speed direction coordinate frame-based grid. This ensures that the features equitably represent the distribution of the normal motion with respect to the moving edges.

Most feature tracking methods rely on building a model of events and then computing optical flow by assigning events to corresponding models. This, however, results in a lower quality optical flow and shorter flow tracks. In Zhu et al. (2017a), a soft data association modeled with probabilities is presented which is computed in an expectation maximization scheme. To enable longer tracks, the method also computes the affine deformation with respect to the initial point and use the resulting residual as a measure of persistence. Thus, in this method, varying temporal integration, different for each feature is achieved.

3.2.13.6. Feature Extraction Algorithms

Convolution of Gabor filters over the image is a standard technique in conventional feature extraction. In Tsitiridis et al. (2015), a spiking neural network is used to exploit the temporal nature of the signals. In this method, a biologically inspired Gabor feature approach is presented. The neural network has a hierarchical structure and provides a flexible approach that reduces computation. In Lagorce et al. (2015a), a new computational architecture for learning and encoding spatio-temporal features is presented, based on a set of predictive recurrent reservoir networks, competing via winner-take-all selection. The features in this method are learned in an unsupervised scheme.

In Chandrapala and Shi (2016), a novel architecture called the event-based Generative Adaptive Subspace Self-Organizing Map, for feature extraction is proposed. The system is inspired by cortical models of visual processing and is based on the concepts of sparsity and temporal slowness. In this model, layers of units can be cascaded to learn feature extractors with different levels of complexity.

In Peng et al. (2017), a feature extraction method is proposed which is based on bag of events probability theory. In this approach, each object is represented as a joint probability distribution of events. The authors claim five main advantages: First, the algorithm uses statistical learning methods with good mathematical foundations. Second, it has only one hyper-parameter, therefore reducing the effort spent in parameter tuning. Third, it is an online learning algorithm and does not require data collection. Fourth, it offers competitive results in real time. And finally, the approach requires very simple operations of addition and multiplication.

A new feature is proposed in Negri (2017) that is computed based on an extended local binary pattern (LBP) operator. The feature characterizes the connectivity of the asynchronous events in a two dimensional space. This feature can also be organized on histograms and combined with other features as histograms of oriented events.

A new set of features called time-surfaces is presented in Lagorce et al. (2017), which can be used to create a hierarchical pattern recognition architecture. In this model, the subsequent layers in the hierarchy extract increasingly abstract features using increasingly large spatio-temporal windows. The idea in this work is to use temporal information to create contexts in the form of time-surfaces which represent the temporal activity in a local neighborhood. The first layer in this hierarchy operates on a group of pixels and each subsequent layer feature unit performs operation on the output of previous feature unit.

3.2.13.7. Hybrid Cameras

In order to combine the advantages of event-based cameras with frame-based technology, DAVIS cameras are proposed that consist of a frame-based camera and a DVS camera that fills the information gap between consecutive frames. In Tedaldi et al. (2016), a new feature extraction method is proposed for these type of cameras, in which the frames are first processed and features are detected. These features are then tracked in the blind time between the frames using the events. The system uses an iterative geometric a registration approach for feature tracking.

3.3. Analysis and Modeling

Analyzing and modeling the behavior artificial retinas can help understand, and hence devise ways to improve, their performance In this section we perform an overview on this line of research.

3.3.1. Analysis

In Yousefzadeh et al. (2018) a study is performed on saccades, and it is shown that performing more saccades in different directions can result in more accurate object recognition. Since adding more saccades adds latency and power consumption, the authors propose an intelligent saccadic movement paradigm that reduces the number necessary saccades without sacrificing recognition accuracy. The authors then use a neural-network algorithm that learns to control the saccades, thus further reducing the latency.

The impact of fixational eye movements for a DVS camera is investigated in Löhr and Neumann (2018). The authors use a mirror system to generate the virtual eye movements, and analyze the shape of the Fourier spectrum of random motions of the recordings for stationary and moving features.

A DVS and jAER are integrated in Franco et al. (2013) and an analysis is performed on the system to describe a method to develop new applications in jAER. The paper also describes two applications of the system: tracking objects and constructing images from spikes.

In event-based systems, sampling is induced by the signal, rather than by an external clock. Therefore, mathematical theory of frame-bases systems cannot accurately be applied to these systems. In Grybos (2015), event-based signal processing and the application of irregular sampling theory and frames are studied for event-based signal reconstruction. The method consists of the application of the frame algorithm enhanced with adaptive weight method for signal reconstruction.

3.3.2. Modeling

3.3.2.1. Modeling Retinal Ganglion Cells

An event-based system is developed in Katz et al. (2012a) which models the behavior of retinal ganglion cells. A DVS camera sends the events to a micro-controller processing unit which implements an interrupt driven model of an Approach Sensitive Retinal Ganglion Cells (AS-RGC). Accurate modeling of retinal information processing is studied in Lorach et al. (2012), where the spatial and temporal properties of the ganglion cells in mammalian retina are modeled. A DVS camera is combined in this work with a model pulling non-linear sub-units to reproduce the parallel filtering and temporal coding that occurs in retina.

It is often assumed that neuromorphic technology, i.e., technology that mimicks biological neuronal computing architectures, can potentially help to understand the functionality of nervous system. However, existing neuromorphic systems usually fail to represent true behavior biological sensors and neurons. To overcome this, a neuroid-based ganglion retina cell model is presented in Argüello et al. (2013) that is capable of reproducing the essential features of the photo-receptor response to illumination. A real-time visual system emulator is developed in Kawasetsu et al. (2014) as a combination of hardware retina emulator and SpiNNaker chips, to model neural activity in the retina and visual cortex.

Modeling the early detection of approaching dark objects, which is the functionality of one type of retinal ganglion cells is studied in Liu et al. (2017a). The Java software and FPGA hardware implementation of this type of cells is conducted and it is shown that this model can have applications in developing attention systems.

3.3.2.2. Modeling Event-Based Sensors

In Katz et al. (2012b), a high frame-rate USB camera is used to model the behavior of a DVS camera. The PS3-Eye camera performs at 125 fps, and is integrated into a jAER (2021) software which does real-time event-based sensor processing. A variational model is presented in Munda et al. (2018) that accurately models the behavior of DVS cameras, that is formulated on per-event-basis, where information about the asynchronous nature of events are incorporated via an event manifold induced by the relative time-stamps of events. This model enables the reconstruction of intensity images with arbitrary frame rate in real-time.

3.3.2.3. Modeling of Cortical Mechanisms

In Tschechne et al. (2014), a new approach is proposed for modeling of cortical mechanism of motion detection. The model combines filters with spatio-temporal and direction specificity. The model is then used to record test stimuli, articulated motion and ego-motion.

3.4. Hardware Design

Several efforts to develop hardware systems dedicated to processing event-based vision signals exist. A hardware model of a selective attention mechanism implemented on a VLSI chip is presented in Indiveri (2000), that is used with analog neuromorphic circuits. The device can be used as a transceiver module for multichip neuromorphic vision systems. In Serrano-Gotarredona et al. (2006), a neuromorphic cortical-layer microchip is presented that computes processes 2-D convolutions of event-based vision data. The microchip is able to process 128 × 128 pixels and can be tiled up for higher resolutions. In another work (Vogelstein et al., 2007), a mixed-signal VLSI system is devised for spike-based vision processing. The model exploits arbitrary and re-configurable connectivity between cells in the multichip architecture.

A new vision hardware system called CAVIAR is developed in Serrano-Gotarredona et al. (2009), in order to propose computational neuroscience and machine vision that allows construction of modular, multilayered, hierarchical and salable sensory processing learning and actuating systems. The system is a massively parallel hardware that consists of a retina, programmable kernel, WTA chip, spatio-temporal processing chip, AER mapping and splitting FPGA and a computer-AER interfacing FPGA.

In Bartolozzi et al. (2011) a robotic vision system is proposed that comprises two DVS cameras with a dedicated processor, a General Address Event Processor and a FPGA that connects the sensors to the processors. A software module collects the events for further processing. The system is capable of interaction with real world in real time.

The HMAX model was proposed (Serre et al., 2007) to truly model the visual cortex (Riesenhuber and Poggio, 1999, 2000a,b). An event-based implementation of the model is proposed in Folowosele et al. (2011) to show its ability in classifying basic shapes.

4. Event-Based Auditory Systems

Digital audio recording devices decompose signals using classical digital signal processing techniques. In biological auditory systems, sound signals are decomposed into frequency bands by the mechanical properties of the basilar membrane in the cochlea. Hair cells transduce the band-passed components into neural pulses that are then propagated to higher auditory areas in the brain. Auditory information like speech, music, and environmental noise is temporally structured. The brain is thought to achieve a computational advantage by exploiting the timing of action potentials to code information, compared to mere rate codes. Several studies have thus explored the potential of event-based processing in auditory processing, which we review in this section.

The development of silicon cochleas for signal processing purposes has seen significant effort. Interested readers are referred to Chan et al. (2007), Wen and Boahen (2009), Liu et al. (2010), Koickal et al. (2011), Wang et al. (2015), Yang (2015), and Jiménez-Fernández et al. (2017).

4.1. Applications

4.1.1. Localization

Using two microphones and a pair of silicon cochlea, a neuromorphic sound localization system is proposed in van Schaik et al. (2009) and Yue-Sek Chan et al. (2010). The algorithm proposed in this work is adaptive and supports online learning. A binaural event-based sound localization is presented in Finger and Liu (2011), which implements a spike-based correlation of the spikes and measures the Inter-aural Time Differences (ITD) between the arrival of a sound to the two cochleas. When a spike arrives, the algorithm updates a possible distribution of ITD, which offers a faster solution to the problem compared to conventional cross-correlation methods. A probabilistic model for sound localization using silicon cochlea is presented in Anumula et al. (2018). Instead of using the timing of the spikes to find ITDs, this work uses spikes to support a distribution model of the ITDs.

In order to enhance the perceptual sensation on a hearing aid system, a neuromorphic sound localization circuit is designed in Park et al. (2013). The system is comprised of leaky integrate-and-fire neurons that are optimized to reduce the synaptic circuit noises.

4.1.2. Echolocation

A bat-inspired, event-based localization method is proposed in Abdalla and Horiuchi (2008), which produces qualitatively similar, direction-dependent, spectral features in the same ultrasonic frequency range used by the big brown bat. The input sound signal generated by the an ultrasonic cochlea are sent to spiking neurons which then convert these spikes to spike trains. The authors use pattern recognition algorithms to estimate the azimuth and elevation of the ultrasonic chirps.

4.1.3. Micro-Doppler Sonar

The relative velocity of objects to an observer can be estimated via the frequency shift of the sound produced by the objects. This phenomenon is used by some animals like bats and dolphins to navigate and locate objects. A system for micro-Doppler sonar is presented in Figliolia et al. (2015), which uses a silicon cochlea with acoustic fovea and AER.

4.1.4. Speech Recognition

A speech perception algorithm is proposed in Näger et al. (2002) which uses a model of human cochlea with spiking neural network. The network employees synaptic plasticity to learn patterns by establishing characteristic delay structures.

The authors of Jansen and Niyogi (2009) design point process models to operate on sparse detector-based representation of speech signals and apply them to speech recognition tasks. They show that this system can operate at a comparable level to a basic hidden Markov model.

A speaker-independent isolated digit recognition system is designed in Abdollahi and Liu (2011), that works based on cochlear image maps based on the spikes from a silicon cochlea. The cochlear maps were found by means of time-binned spike counts, low-pass filtered spike trains and the Radon spike count method. These maps are then fed to a support vector machine for classification.

A spiking neural network composed of three types of integrate-and-fire neurons is proposed in Miró-Amarante et al. (2017) that is capable of recognizing vowel phonemes. The neural network is described in VHDL for detecting Spanish words.

4.1.5. Speaker Identification

A method for speaker identification employing a silicon cochlea and limit-cycle statistics is proposed in Chakrabartty and Liu (2010). The authors employ a Gini-support vector machine classifier and use spike rates, inter-speak-interval distributions and inter-spike-velocity features. In Li et al. (2012), auditory features representing fading histograms of inter-spike intervals and channel activity distributions are extracted from the output of a silicon cochlea. Then a linear support vector machine is used to classify the feature vectors.

4.1.6. Sound Recognition

In Jäckel et al. (2010), a sound recognition system is designed that uses a silicon cochlea and classical hidden Markov model. The system is trained to recognize two different sound of a clap or a bass drum in presence of noise. A neuromorphic auditory system for feature extraction and an artificial neural network are used in Cerezuela-Escudero et al. (2016) to recognize 12 musical notes in presence of white noise.

4.1.7. Sensor Fusion

In Chan et al. (2012), a pair of silicon cochlea and a silicon retina are combined on a robotic platform to allow the robot to learn sound localization through visual feedback and a sound localization algorithm. This work is an extension on the work presented in van Schaik et al. (2009) and Yue-Sek Chan et al. (2010), where only silicon cochlea was used for localization. The authors report that the combination with a silicon retina improves sound localization.

The combination of visual and auditory information can help resolve ambiguities in sensing. In Akolkar et al. (2015b), event-based vision and auditory systems are combined to design a collision detection algorithm for application in robotics. Collisions are distinguishable from mere occlusions by on the sound the collision produces. Salient sensory events must therefore be detected by vision and auditory systems at the same time. This requires very high temporal resolution and is challenging for frame-based systems.

In Rios-Navarro et al. (2015), an event-based camera and auditory systems are used together to measure the rotation frequency of a motor. The system uses a FPGA and performs in real-time.

4.1.8. Feature Extraction

In Anumula et al. (2018), the effectiveness of frame-based features generated using spike counts and constant event binning is investigated. The authors propose a pre-processing method which applies an exponential kernel on the events to better preserve timing information. In Acharya et al. (2018) the authors extend their feature extraction approach to fixed number of bins and fixed bin size methods.

5. Event-Based Olfactory Systems

Olfaction plays an essential role in many activities, e.g., food foraging, trail following, mating, bonding, navigation, and detection of threats. Artificial olfaction has great potential in many areas, including hazard detection, food safety, industrial and environmental monitoring, disaster management, crop monitoring, medical diagnosis, among others. However, gas sensor technology still lags far behind what is available the visual and auditory domains. Designing artificial olfactory systems faces a number of challenges, including coping with slow sensor recovery, sensor drift compensation, concentration-invariant recognition, orthogonalization of odor patterns, mixture separation, and odor identification against complex backgrounds and interferents, among others (see Pearce et al., 2003 for a review). Event-based approaches could help mitigate some of these issues.

The front end of the olfactory system in vertebrates consists of a large number of olfactory receptor neurons that fire action potentials upon encountering volatile chemicals in inhaled air. Humans have around 5 million olfactory receptor cells, each of which expresses a only one of about 350 possible olfactory receptors. Different olfactory receptors differ in their molecular receptive ranges. One type of receptor may respond to a range of odorants, and one odorant typically elicits responses in a range of receptor. A combinatorial code emerges that encodes the identity of an odorant by the pattern of olfactory receptors that is activated (Bieri et al., 2004).

Artificial olfactory systems follow a similar principle, where arrays of sensors with partly overlapping response characteristics are combined in a frame-based manner and subsequently processed to extract activation patterns that can be assigned to particular odorants. In practice, gas sensor signals are often collected over long periods of time, like tens of seconds to several minutes, and subsequently averaged to remove turbulence-induced signal variations and other noise. This approach invariably introduces latency and removes any information-containing turbulence-induced information from the signal. In contrast, event-based olfactory systems try to mimic the key principles of biological olfaction, by transmitting events only when the gas concentration changes.

Here, we review gas sensing approaches that use direct inspiration from biology, dedicated hardware systems for neuromorphic gas sensing, and algorithms that have been suggested to improve gas sensing using neuromorphic principles.

5.1. Bio-Inspired Olfaction Systems

5.1.1. Vertebrate Olfactory System

An artificial chemosensing system is presented in White et al. (1998) and White and Kauer (1999) which is based on neural circuits of the vertebrate olfactory system. An array of chemosensors which are designed to produce similar response to olfactory sensory neurons is used as input that produces spatiotemporal patterns. These patterns are recognized by a delay line neural network. The system is devised to encode the vapor identity by the spatial patterning of activity in the neural network and vapor intensity is encoded by response latency. The identity and intensity information are then separated into two distinct codes. This serves as a discriminator among organic vapors.

Inspired by olfactory structure of mammals, an artificial olfactory bulb is presented in Jing et al. (2016), which consists of olfactory receptor neurons and mitral, granule, periglomerular and short axon cells. The model transforms the input of gas sensors into neuron spikes that simplifies the feature generation step. The system is used in liquor classification.

5.1.2. Insect Olfactory System

Since the output of odor sensors is usually real-time, continuous, noisy, lacks a precise onset signal and accurate classification often requires temporal information, many neuronal network models fail to operate properly in practice. To investigate the potentials and suitability of biomimetic classifiers for real-world sensor data, a research is performed in Diamond et al. (2016). In this work, inspired by insect antennal lobe, a generic classifier is designed to identify 20 individual chemical odors.

In Pearce et al. (2013, 2014), a biologically-constrained neuromorphic spiking model of the insect antennal lobe is presented that detects the concentration of chemical components of a material. The system is dynamic and uses winner-takes-all or winnerless competition depending on the inhibition and symmetry of its connections. The authors employ spike timing-dependent plasticity in their model and show that this is able to organize weights into a stable configuration.

5.1.3. Honeybee Olfactory System

The honeybee's olfactory pathway is decomposed into its local circuits and processing stages in Hausler et al. (2011) and Schmuker et al. (2011). The authors demonstrate functional role of these organs and build a model a spiking neuronal network models of them by designing a probabilistic classifier. In another work (Kasap and Schmuker, 2013), also inspired by honeybee antennal lobe, unsupervised learning of the lateral inhibition structure is presented. The authors use inhibitory spike-timing dependent plasticity in a computational model for multivariate data processing. In this system, the inhibitory connectivity self-organizes to reflect the correlation between input channels. It is shown in this paper that local learning produces an inhibitory connectivity that reduces channel correlation and is suitable for a neuromorphic architecture. This line of work further gave rise to the first published implementation of a spiking network for multivariate pattern recognition on neuromorphic hardware (Schmuker et al., 2014).

5.1.4. Stereo Olfaction

In Rochel et al. (2002), inspired by animal olfactory systems in tracking odors, a stereo sniffing system is designed that tracks specific odors. In this system, first the gas-concentration gradient is estimated, and then the gas is recognized. The authors use spiking neural networks to implement this biologically inspired system.

5.2. Hardware Systems

In this section we review the research that have developed hardware systems specifically designed for odor recognition.

5.2.1. Hardware Design

A hardware architecture for chemical classifiers is presented in Abdel-Aty-Zohdy et al. (2010) which takes advantage of Sampling Spiking Neural Networks (SSNN). The chip records learning statistics and can be used in parallel with other SSNN co-processors to build very large systems.

5.2.2. VLSI

Among the first attempts to develop a VLSI spiking neuromorphic is Koickalb et al. (2004) and Pearce et al. (2005). In this work, an olfactory bulb model, a reduced 70-element chemosensor array and the silicon implementation are presented. An adaptive neuromorphic VLSI olfaction device with on-chip chemosensor array is designed in Koickal et al. (2006, 2007). The system processes temporal spiking signals and classifies the odors. In Hsieh and Tang (2012) a VLSI neuromorphic spiking neural network olfactory system is designed that uses sub-threshold oscillation and onset-latency representation, in order to reduce power consumption. The authors use the synaptic weights between the mitral and cortical cells according to an spike-timing-dependent plasticity learning rule.

5.2.3. Gas Recognition

In another work (Ng et al., 2011), a CMOS gas recognition chip is presented which encodes sensor outputs into spikes with the firing delay mapping the strength of the simulation. The circuit processes the spikes and looks for match within a library of spatio-temporal signatures. Exploiting fundamental characteristics of the olfactory pathway, a simple spike based gas recognition technique is presented in Al Yamani et al. (2012a,b). The system is designed for detecting ethanol, methane and carbon monoxide. Gas recognition is performed in this system by looking for a match within a library of spatio-temporal spike patterns. In Hassan et al. (2015), instead of the logarithmic time encoding model, spike codes are formed from transient features (similar to Muezzinoglu et al., 2009), thus eliminating the need for regression.

5.3. Modeling and Algorithms

Algorithms for modeling olfactory systems and for improving recognition performance have been proposed by a range of studies.

5.3.1. Accelerated Event-Based Gas Sensing

In Drix and Schmuker (2021) a Kalman-filter based algorithms is described that can decode fast transients (in the order of one second) from metal-oxide sensors. It uses an event-based signal representation to detect gas onset with high temporal precision. An application in gas source direction detection is demonstrated.

5.3.2. Event-Based Source-Distance Estimation

In Schmuker et al. (2016) and event-based (albeit non-spiking) algorithm is proposed that exploits turbulence-induced signal fluctuations to estimate the distance of a gas source in a wind tunnel.

5.3.3. Neural Networks

A spiking olfactory bulb model is implemented in programmable logging and combined with a Hebbian learning rule in Guerrero-Rivera and Pearce (2007). The system is able to store attractors which correspond to odor patterns, and can classify learnt odors.

In Beyeler et al. (2010), the topology of biological networks is studied, and it is analysed how network activity depends on various parameters of the theoretical models. The authors' aim is to shed light on how network structure relates to filtering and enhancement of recognition performance.

5.3.4. Neuromorphic Design

A network model of the glomerular layer of the mammalian olfactory bulb is implemented in neuromorphic hardware in Imam et al. (2012). In Martinelli et al. (2009), an artificial olfactory system is proposed and implemented on FPGA. The model is based on a direct spike conversion of the input signal and digital glomerular signal processing for spikes.

5.3.5. Computational Modeling

Inspired by the biological principle of distributed coding, and olfactory receptor neurons converging in a chemotopic fashion onto glomerular units in the olfactory pathway, in Raman et al. (2006a), a computational model of chemical sensor arrays is presented. The work presents a monotonic concentration-response model that maps the sensor inputs into a distributed activation pattern across receptor models. Then a self-organizing model of chemotopic convergence is used to simulate the projection onto glomerular units in the olfactory bulb.

5.3.6. Contrast Enhancement

In order to enhance the discrimination of multivariate patterns from gas sensor arrays, a signal processing model is presented in Raman et al. (2006b), which improves the separability between input odor patterns. The model captures chemotopic convergence of sensory neurons onto the olfactory bulb and center on-off surround lateral interactions. The features are projected onto a two dimensional lattice which results in odor-specific spatial patterning. These patterns are then fed to a network of mitral cells to enhance the contrast among odors and decouples odor identity from intensity.

5.3.7. Spike Latency

To study the hypothesis that neurons transmit the most meaningful information via the first spikes, and that spike latency acts as a descriptor of the information content, an artificial sensory system is designed with a single layer of spiking neurons in Di Natale (2011). The authors assessed the system's capability to discriminate between distinct chemicals and mixtures, and studying the effect of lateral inhibition. The authors considered both the spikes latency and the average firing rate as the output of the network. Experiments with the system show that the average firing rate offers the best separation among stimuli, while latency offers discrimination in shorter time. These results aligned with observations in biological olfaction.

A latency-based e-nose system is designed in Chen et al. (2011) to achieve power-efficient, compact and robust gas identification, using rank order and spike distance classification algorithms.

6. Summary of Reviewed Research

In this paper we reviewed research on event-based signal processing, focusing on visual, auditory and olfactory systems. We did not attempt to cover event-based control systems, since the field is an independent area with large amount of research that would require a separate review. There is also a wide range of research on neuromorphic engineering which we did not cover in this paper due to constraints in scope and size; readers so inclined are referred to other excellent survey papers (Cauwenberghs, 1998; Liu and Wang, 2009; Liu et al., 2009; Nawrocki et al., 2016; Vanarse et al., 2016; James et al., 2017; Schuman et al., 2017).

There is an interesting trend in the volume of published research in event-based visual, auditory and olfactory systems during the last two decades (Figure 1). Few papers have been published prior to 2006, when a significant increase in interest saw a surge in publications. After 2006, there is steady increase in the research on the subject, indicating that the potential of the technology has been discovered.

FIGURE 1

Figure 1. The number of reviewed papers on event-based visual, auditory and olfactory systems in each year since 1996.

Tables 1, 2 summarize the structure of this paper and the papers reviewed.

TABLE 1

Table 1. The paper structure.

TABLE 2

Table 2. The paper structure.

7. Conclusion and Discussion

Event-based sensing and signal processing has been applied to many applications, with promising results and several conceptual advantages. First, event-based systems only collect meaningful information; therefore, the redundant data are not transferred and processed, enabling a more efficient encoding scheme. These systems can operate in an asynchronous fashion, not limited by the constraints induced by a global clock. Second, information is reported instantaneously, in contrast to periodic-sampling systems that quantize based on their sampling rate. Also, temporal information on very short timescales can be captured, without the constraints imposed by the Nyquist Theorem. Third, event-based sensors provide the potential for lower power operation when sampling sparse signals, which can be an advantage in power-constrained scenarios like mobile devices or medical implants. Fourth, many event-based systems are implemented in a modular fashion, allowing to increase computational power by composition and parallelism.

Nevertheless, event-based systems are still in their infancy, especially in the auditory and olfactory domain. This technology provides promising potential, but also comes with a set of unique challenges, outlined below.

The greatest bottleneck for growth in neuromorphic sensing is probably that a comprehensive theoretical framework to formally describe and analyse event-based sensing and signal processing algorithms has yet to emerge. This hampers the development of event-based algorithms and applications. Cross-pollination from engineering-focused event-based research communities may provide a way forward.

The lack of a theoretical framework also complicates the translation of traditional machine learning algorithms from the frame-based into the event-based domain. While attempts of such translations have been successful, they invariably resulted in a performance penalty compared to the frame-based implementation (although this penalty can sometimes be very small). The opposite direction is also a challenge: interfacing asynchronous, event-based systems with frame-based, clocked digital systems. A naive approach is to simply create frames from the event-based signal representation, but clearly this is not optimal since it risks to render the advantages of event-based sensing and signal processing void. Future research is therefore expected to focus on developing machine learning techniques that are specifically designed for event-based systems. A promising line of research could exploit the inherent compatibility of event-based sensing with spiking networks, potentially in combination with operation on accelerated neuromorphic hardware systems.

In addition to translation from frame-based algorithms into the event-based domain, Traditional machine vision and signal processing approaches already offer a great number of tools and techniques that help solve many common tasks, e.g., image enhancement, image restoration, depth identification, etc., which are readily available in widely-used and well-documented software packages. However, researchers building event-based approaches often must start from scratch. The barrier for adopting event-based technology for tasks outside basic research is therefore rather high, and sometimes the cost may be perceived too high to outweigh the gains. A standard toolbox for event-based signal processing could be a game-changing asset to boost the accessibility of this promising technology.

Most applications described for visual event-based signal processing are simple tasks of detection and tracking. Conventional machine vision algorithms are now developed for much more sophisticated tasks. Interesting and relevant tasks like face recognition, human behavior analysis, medical diagnosis, product inspection, etc. are currently far beyond what could reasonably be achieved with event-based vision. This is likely not due to an inherent limitation of the event-based approach; after all, the human brain can perform all these tasks and it operates in an event-driven fashion. Rather, improvements in neuromorphic hardware and, perhaps more importantly, event-driven algorithms will be needed to compete with state-of-the-art machine vision, speech recognition and gas sensing solutions.

The vast majority of event-based systems until now have been designed for vision, with applications in the auditory domain emerging, and prototypes having been demonstrated in olfaction. Other areas of data processing that do not explicitly deal with sensing a physical quantity are less well-explored in the event-based signal processing community, in spite their inherent suitability for the processing scheme; For example, areas like internet security, traffic data analysis, human behavior analysis, finance, physical experiments, healthcare data, social media, video surveillance, etc., generate data that can be interpreted as events. For example, the data in traffic systems comprise some events, like accidents, breaks, turns, etc., that could be processed in event-based scheme. In finance, events like an large cargo ship getting stuck and blocking a major trade route can affect the price of oil. In healthcare, changes in blood pressure, blood sugar, heart beat rate can indicate specific problems. Research in these areas has yet to be permeated by event-based data processing strategies.

Event-based sensing and signal processing also provides a few interesting avenues for research that may be crucial for the future development of the field; for example, exploration of event-based noise and noise-tolerance, “anti-patterns” for event-based sensing, signal compression, and cryptography.

While noise reduction in event-based vision has been addressed previously (Padala et al., 2018; Xie et al., 2018), there remains a need for a theoretical treatment for the problem. Also, the effect of the noise on existing algorithms should be studied. Future algorithms should be developed that are robust to such noise.

“Anti-patterns” for event-based signal processing refer to specific types of data or environment where event-based approaches struggle or fail, while traditional approaches cope well or have been engineered to overcome issues. For example, textured visual scenes can cause extremely high event-counts that could overload the event transmission fabric. This problem effectively limits the pixel count of event-based vision sensors as a function of bus capacity. Similar anti-patterns could exist in event-based olfaction and audio processing, but have yet to be identified, studied, and have solutions provided for. Often, a look at neuroscience could suggest promising solutions. For example, the mammalian retina already provides a fascinating wealth of signal processing before any spikes are generated (Baden et al., 2016).

Signal compression in the event-based domain is an interesting topic. Event-driven sampling of sparse signals inherently implements a compression of the input information compared to periodic and thus temporally redundant sampling. Still there is a need for more research on data compression for these sensors, especially in the visual domain where event counts even for sparse signals can grow very quickly when using high-resolution sensors. We reviewed some approaches in this direction, but more theoretical analysis and practical algorithms could help develop more efficient compression mechanisms.

Cryptography plays an important role when dealing with sensitive data. Many algorithms exist for efficient encryption of sensitive audio and video signals. This area of research is mostly absent from current trends in the event-based research community. Yet, event-based signals might be sensitive and thus require encryption before transmission. Specifically designed encryption algorithms for event-based data is an important domain which has not been targeted yet.

Finally, specific to olfaction, a large challenge is the availability of powerful sensors. Current gas sensing technology lags behind olfactory capabilities of animals, and even insects. A particular problem is temporal resolution, at least if portability and low power consumption are desired. Event-based approaches exist to narrow the gap between technology and biology (e.g., Drix and Schmuker, 2021). Improvements in gas sensing technology will without doubt catalyze progress in event-based olfaction as well.

Author Contributions

Both authors listed have made a substantial, direct and intellectual contribution to the work, and approved it for publication.

Funding

MS was funded from EU H2020 Grant 945539 (Human Brain Project SGA3) and MRC grant MR/T046759/1 NeuroNex: From Odor to Action within the NSF/CIHR/DFG/FRQ/UKRI-MRC Next Generation Networks for Neuroscience Program (NSF Award no. 2014217).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Abdalla, H., and Horiuchi, T. K. (2008). “Spike-based acoustic signal processing chips for detection and localization,” in 2008 IEEE Biomedical Circuits and Systems Conference, 225–228. doi: 10.1109/BIOCAS.2008.4696915

CrossRef Full Text | Google Scholar

Abdel-Aty-Zohdy, H. S., Allen, J. N., and Ewing, R. L. (2010). “Spiking neural network e-nose classifier chip,” in Proceedings of the IEEE 2010 National Aerospace Electronics Conference, 374–378. doi: 10.1109/NAECON.2010.5712980

CrossRef Full Text | Google Scholar

Abdollahi, M., and Liu, S. (2011). “Speaker-independent isolated digit recognition using an aer silicon cochlea,” in 2011 IEEE Biomedical Circuits and Systems Conference (BioCAS), 269–272. doi: 10.1109/BioCAS.2011.6107779

CrossRef Full Text | Google Scholar

Abdul-Kreem, L. I., and Neumann, H. (2015). “Estimating visual motion using an event-based artificial retina,” in International Joint Conference on Computer Vision, Imaging and Computer Graphics (Springer), 396–415. doi: 10.1007/978-3-319-29971-6_21

CrossRef Full Text | Google Scholar

Acharya, J., Patil, A., Li, X., Chen, Y., Liu, S. C., and Basu, A. (2018). A comparison of low-complexity real-time feature extraction for neuromorphic speech recognition. Front. Neurosci. 12:160. doi: 10.3389/fnins.2018.00160

PubMed Abstract | CrossRef Full Text | Google Scholar

Adelson, E. H., and Bergen, J. R. (1985). Spatiotemporal energy models for the perception of motion. Josa A 2, 284–299. doi: 10.1364/JOSAA.2.000284

PubMed Abstract | CrossRef Full Text | Google Scholar

Afshar, S., Nicholson, A. P., Schaik, A. v., and Cohen, G. (2020). Event-based object detection and tracking for space situational awareness. IEEE Sens. J. 20, 15117–15132. doi: 10.1109/JSEN.2020.3009687

CrossRef Full Text | Google Scholar

Ahn, E. Y., Lee, J. H., Mullen, T., and Yen, J. (2011). “Dynamic vision sensor camera based bare hand gesture recognition,” in 2011 IEEE Symposium On Computational Intelligence For Multimedia, Signal And Vision Processing (IEEE), 52–59. doi: 10.1109/CIMSIVP.2011.5949251

CrossRef Full Text | Google Scholar

Ahn, E. Y. Y. (2012). Evolutionary-Based Feature Extraction for Gesture Recognition Using a Motion Camera.

Google Scholar

Akolkar, H., Meyer, C., Clady, X., Marre, O., Bartolozzi, C., Panzeri, S., et al. (2015a). What can neuromorphic event-driven precise timing add to spike-based pattern recognition? Neural Comput. 27, 561–593. doi: 10.1162/NECO_a_00703

PubMed Abstract | CrossRef Full Text | Google Scholar

Akolkar, H., Valeiras, D. R., Benosman, R., and Bartolozzi, C. (2015b). “Visual-auditory saliency detection using event-driven visual sensors,” in 2015 International Conference on Event-Based Control, Communication, and Signal Processing (EBCCSP), 1–6. doi: 10.1109/EBCCSP.2015.7300674

CrossRef Full Text | Google Scholar

Al Yamani, J. H. J., Boussaid, F., Bermak, A., and Martinez, D. (2012a). Glomerular latency coding in artificial olfaction. Front. Neuroeng. 4:18. doi: 10.3389/fneng.2011.00018

PubMed Abstract | CrossRef Full Text | Google Scholar

Al Yamani, J. H. J., Boussaid, F., Bermak, A., and Martinez, D. (2012b). “Bio-inspired gas recognition based on the organization of the olfactory pathway,” in 2012 IEEE International Symposium on Circuits and Systems, 1391–1394. doi: 10.1109/ISCAS.2012.6271503

CrossRef Full Text | Google Scholar

Alzugaray, I., and Chli, M. (2018a). “ACE: an efficient asynchronous corner tracker for event cameras,” in 2018 International Conference on 3D Vision (3DV) (IEEE), 653–661. doi: 10.1109/3DV.2018.00080

CrossRef Full Text | Google Scholar

Alzugaray, I., and Chli, M. (2018b). Asynchronous corner detection and tracking for event cameras in real time. IEEE Robot. Autom. Lett. 3, 3177–3184. doi: 10.1109/LRA.2018.2849882

CrossRef Full Text | Google Scholar

Amir, A., Taba, B., Berg, D., Melano, T., McKinstry, J., Di Nolfo, C., et al. (2017). “A low power, fully event-based gesture recognition system,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 7243–7252. doi: 10.1109/CVPR.2017.781

CrossRef Full Text | Google Scholar

Anumula, J., Ceolini, E., He, Z., Huber, A., and Liu, S. (2018). “An event-driven probabilistic model of sound source localization using cochlea spikes,” in 2018 IEEE International Symposium on Circuits and Systems (ISCAS), 1–5. doi: 10.1109/ISCAS.2018.8351856

CrossRef Full Text | Google Scholar

Anumula, J., Neil, D., Delbruck, T., and Liu, S. C. (2018). Feature representations for neuromorphic audio spike streams. Front. Neurosci. 12:23. doi: 10.3389/fnins.2018.00023

PubMed Abstract | CrossRef Full Text | Google Scholar

Argüello, E., Silva, R., Huerta, M., and Castillo, C. (2013). “New trends in computational modeling: a neuroid-based retina model,” in 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 4561–4564. doi: 10.1109/EMBC.2013.6610562

PubMed Abstract | CrossRef Full Text | Google Scholar

Baden, T., Berens, P., Franke, K., Rosón, M. R., Bethge, M., and Euler, T. (2016). The functional diversity of retinal ganglion cells in the mouse. Nature 529, 345–350. doi: 10.1038/nature16468

PubMed Abstract | CrossRef Full Text | Google Scholar

Bardow, P., Davison, A. J., and Leutenegger, S. (2016). “Simultaneous optical flow and intensity estimation from an event camera,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). doi: 10.1109/CVPR.2016.102

CrossRef Full Text | Google Scholar

Barranco, F., Díaz, J., Ros, E., and Del Pino, B. (2009). Visual system based on artificial retina for motion detection. IEEE Trans. Syst. Man Cybernet. B Cybernet. 39, 752–762. doi: 10.1109/TSMCB.2008.2009067

PubMed Abstract | CrossRef Full Text | Google Scholar

Barranco, F., Fermüller, C., and Aloimonos, Y. (2014). Contour motion estimation for asynchronous event-driven cameras. Proc. IEEE 102, 1537–1556. doi: 10.1109/JPROC.2014.2347207

CrossRef Full Text | Google Scholar

Barranco, F., Fermuller, C., and Aloimonos, Y. (2015a). “Bio-inspired motion estimation with event-driven sensors,” in International Work-Conference on Artificial Neural Networks (Springer), 309–321. doi: 10.1007/978-3-319-19258-1_27

CrossRef Full Text | Google Scholar

Barranco, F., Teo, C. L., Fermuller, C., and Aloimonos, Y. (2015b). “Contour detection and characterization for asynchronous event sensors,” in The IEEE International Conference on Computer Vision (ICCV). doi: 10.1109/ICCV.2015.63

CrossRef Full Text | Google Scholar

Bartolozzi, C., Clercq, C., Mandloi, N., Rea, F., Indiveri, G., Fasnacht, D., et al. (2011). EMORPH: towards neuromorphic robotic vision. Proc. Comput. Sci. 7, 163–165. doi: 10.1016/j.procs.2011.09.027

CrossRef Full Text | Google Scholar

Barua, S., Miyatani, Y., and Veeraraghavan, A. (2016). “Direct face detection and video reconstruction from event cameras,” in 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), 1–9. doi: 10.1109/WACV.2016.7477561

CrossRef Full Text | Google Scholar

Becanovic, V., Indiveri, G., Kobialka, H. U., and Plöger, P. G. (2002). Silicon Retina Sensing Guided by Omni-Directional Vision.

Google Scholar

Belbachir, A. N., Litzenberger, M., Schraml, S., Hofstätter, M., Bauer, D., Schön, P., et al. (2012). “CARE: a dynamic stereo vision sensor system for fall detection,” in 2012 IEEE International Symposium on Circuits and Systems, 731–734. doi: 10.1109/ISCAS.2012.6272141

CrossRef Full Text | Google Scholar

Benosman, R., Clercq, C., Lagorce, X., Ieng, S., and Bartolozzi, C. (2014). Event-based visual flow. IEEE Trans. Neural Netw. Learn. Syst. 25, 407–417. doi: 10.1109/TNNLS.2013.2273537

CrossRef Full Text | Google Scholar

Benosman, R., Ieng, S. H., Rogister, P., and Posch, C. (2011). Asynchronous event-based Hebbian epipolar geometry. IEEE Trans. Neural Netw. 22, 1723–1734. doi: 10.1109/TNN.2011.2167239

PubMed Abstract | CrossRef Full Text | Google Scholar

Beyeler, M., Stefanini, F., Proske, H., Galizia, G., and Chicca, E. (2010). “Exploring olfactory sensory networks: simulations and hardware emulation,” in 2010 Biomedical Circuits and Systems Conference (BioCAS), 270–273. doi: 10.1109/BIOCAS.2010.5709623

CrossRef Full Text | Google Scholar

Bi, Z., Dong, S., Tian, Y., and Huang, T. (2018). “Spike coding for dynamic vision sensors,” in 2018 Data Compression Conference, 117–126. doi: 10.1109/DCC.2018.00020

CrossRef Full Text | Google Scholar

Bichler, O., Querlioz, D., Thorpe, S. J., Bourgoin, J., and Gamrat, C. (2011). “Unsupervised features extraction from asynchronous silicon retina through spike-timing-dependent plasticity,” in The 2011 International Joint Conference on Neural Networks, 859–866. doi: 10.1109/IJCNN.2011.6033311

CrossRef Full Text | Google Scholar

Bichler, O., Querlioz, D., Thorpe, S. J., Bourgoin, J. P., and Gamrat, C. (2012). Extraction of temporally correlated features from dynamic vision sensors with spike-timing-dependent plasticity. Neural Netw. 32, 339–348. doi: 10.1016/j.neunet.2012.02.022

PubMed Abstract | CrossRef Full Text | Google Scholar

Bidegaray-Fesquet, B. (2015). “Noise and offset in the iir filtering of event-based sampled data,” in 2015 International Conference on Event-based Control, Communication, and Signal Processing (EBCCSP), 1–4. doi: 10.1109/EBCCSP.2015.7300694

CrossRef Full Text | Google Scholar

Bieri, S., Monastyrskaia, K., and Schilling, B. (2004). Olfactory receptor neuron profiling using sandalwood odorants. Chem. Senses 29, 483–487. doi: 10.1093/chemse/bjh050

PubMed Abstract | CrossRef Full Text | Google Scholar

Binas, J., Neil, D., Liu, S. C., and Delbruck, T. (2017). Ddd17: end-to-end davis driving dataset. arXiv[Preprint].arXiv:1711.01458.

Google Scholar

Blum, H., Dietmüller, A., Milde, M. B., Conradt, J., Indiveri, G., and Sandamirskaya, Y. (2017). “A neuromorphic controller for a robotic vehicle equipped with a dynamic vision sensor,” in Robotics: Science and Systems. doi: 10.15607/RSS.2017.XIII.035

CrossRef Full Text | Google Scholar

Bolopion, A., Ni, Z., Agnus, J., Benosman, R., and Régnier, S. (2012). “Stable haptic feedback based on a dynamic vision sensor for microrobotics,” in 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE), 3203–3208. doi: 10.1109/IROS.2012.6385557

CrossRef Full Text | Google Scholar

Borer, D., Delbruck, T., and Rösgen, T. (2017). Three-dimensional particle tracking velocimetry using dynamic vision sensors. Exp. Fluids 58:165. doi: 10.1007/s00348-017-2452-5

CrossRef Full Text | Google Scholar

Borer, D. J. (2014). 4D flow visualization with dynamic vision sensors (Ph.D. thesis), ETH Zurich, Zurich, Switzerland.

Google Scholar

Brandli, C., Muller, L., and Delbruck, T. (2014). “Real-time, high-speed video decompression using a frame- and event-based davis sensor,” in 2014 IEEE International Symposium on Circuits and Systems (ISCAS), 686–689. doi: 10.1109/ISCAS.2014.6865228

CrossRef Full Text | Google Scholar

Brändli, C., Strubel, J., Keller, S., Scaramuzza, D., and Delbruck, T. (2016). “Elised—an event-based line segment detector,” in 2016 Second International Conference on Event-based Control, Communication, and Signal Processing (EBCCSP), 1–7. doi: 10.1109/EBCCSP.2016.7605244

CrossRef Full Text | Google Scholar

Brette, R., Rudolph, M., Carnevale, T., Hines, M., Beeman, D., Bower, J. M., et al. (2007). Simulation of networks of spiking neurons: a review of tools and strategies. J. Comput. Neurosci. 23, 349–398. doi: 10.1007/s10827-007-0038-6

PubMed Abstract | CrossRef Full Text | Google Scholar

Camunas-Mesa, L., Acosta-Jimenez, A., Zamarreño-Ramos, C., Serrano-Gotarredona, T., and Linares-Barranco, B. (2011). A 32 × 32 pixel convolution processor chip for address event vision sensors with 155 ns event latency and 20 meps throughput. IEEE Trans. Circuits Syst. I Reg. Pap. 58, 777–790. doi: 10.1109/TCSI.2010.2078851

CrossRef Full Text | Google Scholar

Camunas-Mesa, L., Zamarreno-Ramos, C., Linares-Barranco, A., Acosta-Jimenez, A. J., Serrano-Gotarredona, T., and Linares-Barranco, B. (2012). An event-driven multi-kernel convolution processor module for event-driven vision sensors. IEEE J. Solid State Circuits 47, 504–517. doi: 10.1109/JSSC.2011.2167409

CrossRef Full Text | Google Scholar

Camunas-Mesa, L. A., Serrano-Gotarredona, T., Ieng, S. H., Benosman, R., and Linares-Barranco, B. (2014). On the use of orientation filters for 3D reconstruction in event-driven stereo vision. Front. Neurosci. 8:48. doi: 10.3389/fnins.2014.00048

PubMed Abstract | CrossRef Full Text | Google Scholar

Camuñas-Mesa, L. A., Serrano-Gotarredona, T., and Linares-Barranco, B. (2014b). “Event-driven sensing and processing for high-speed robotic vision,” in 2014 IEEE Biomedical Circuits and Systems Conference (BioCAS) Proceedings (IEEE), 516–519. doi: 10.1109/BioCAS.2014.6981776

CrossRef Full Text | Google Scholar

Camuñas-Mesa, L. A., Serrano-Gotarredona, T., Linares-Barranco, B., Ieng, S., and Benosman, R. (2014a). “Event-driven stereo vision with orientation filters,” in 2014 IEEE International Symposium on Circuits and Systems (ISCAS), 257–260. doi: 10.1109/ISCAS.2014.6865114

PubMed Abstract | CrossRef Full Text | Google Scholar

Cannici, M., Ciccone, M., Romanoni, A., and Matteucci, M. (2018). Event-based convolutional networks for object detection in neuromorphic cameras. CoRR abs/1805.07931. doi: 10.1109/CVPRW.2019.00209

PubMed Abstract | CrossRef Full Text | Google Scholar

Carneiro, J., Ieng, S. H., Posch, C., and Benosman, R. (2013). Event-based 3D reconstruction from neuromorphic retinas. Neural Netw. 45, 27–38. doi: 10.1016/j.neunet.2013.03.006

PubMed Abstract | CrossRef Full Text | Google Scholar

Cauwenberghs, G. (1998). “Neuromorphic learning VLSI systems: a survey,” in Neuromorphic Systems Engineering (Springer), 381–408. doi: 10.1007/978-0-585-28001-1_17

CrossRef Full Text | Google Scholar

Celani, A., Villermaux, E., and Vergassola, M. (2014). Odor landscapes in turbulent environments. Phys. Rev. X 4:041015. doi: 10.1103/PhysRevX.4.041015

CrossRef Full Text | Google Scholar

Censi, A., Strubel, J., Brandli, C., Delbruck, T., and Scaramuzza, D. (2013). “Low-latency localization by active led markers tracking using a dynamic vision sensor,” in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IEEE), 891–898. doi: 10.1109/IROS.2013.6696456

CrossRef Full Text | Google Scholar

Cerezuela-Escudero, E., Jimenez-Fernandez, A., Paz-Vicente, R., Dominguez-Morales, J. P., Dominguez-Morales, M. J., and Linares-Barranco, A. (2016). “Sound recognition system using spiking and MLP neural networks,” in Artificial Neural Networks and Machine Learning-ICANN 2016, eds A. E. Villa, P. Masulli, and A. J. Pons Rivero (Cham: Springer International Publishing), 363–371. doi: 10.1007/978-3-319-44781-0_43

CrossRef Full Text | Google Scholar

Chakrabartty, S., and Liu, S. (2010). “Exploiting spike-based dynamics in a silicon cochlea for speaker identification,” in Proceedings of 2010 IEEE International Symposium on Circuits and Systems, 513–516. doi: 10.1109/ISCAS.2010.5537578

CrossRef Full Text | Google Scholar

Chan, V., Jin, C., and van Schaik, A. (2012). Neuromorphic audio-visual sensor fusion on a sound-localising robot. Front. Neurosci. 6:21. doi: 10.3389/fnins.2012.00021

CrossRef Full Text | Google Scholar

Chan, V., Liu, S., and van Schaik, A. (2007). AER ear: a matched silicon cochlea pair with address event representation interface. IEEE Trans. Circuits Syst. I Reg. Pap. 54, 48–59. doi: 10.1109/TCSI.2006.887979

CrossRef Full Text | Google Scholar

Chandrapala, T. N., and Shi, B. E. (2016). “Invariant feature extraction from event based stimuli,” in 2016 6th IEEE International Conference on Biomedical Robotics and Biomechatronics (BioRob), 1–6. doi: 10.1109/BIOROB.2016.7523449

CrossRef Full Text | Google Scholar

Chen, H. T., Ng, K. T., Bermak, A., Law, M. K., and Martinez, D. (2011). Spike latency coding in biologically inspired microelectronic nose. IEEE Trans. Biomed. Circuits Syst. 5, 160–168. doi: 10.1109/TBCAS.2010.2075928

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, N. F. (2018). “Pseudo-labels for supervised learning on dynamic vision sensor data, applied to object detection under ego-motion,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, 644–653. doi: 10.1109/CVPRW.2018.00107

CrossRef Full Text | Google Scholar

Chen, S., Akselrod, P., Zhao, B., Carrasco, J. A. P., Linares-Barranco, B., and Culurciello, E. (2012). Efficient feedforward categorization of objects and human postures with address-event image sensors. IEEE Trans. Pattern Anal. Mach. Intell. 34, 302–314. doi: 10.1109/TPAMI.2011.120

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, Z., Shikh-Bahaei, T., Luff, P., and Shikh-Bahaei, M. (2017). “Edge caching and dynamic vision sensing for low delay access to visual medical information,” in 2017 39th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 1238–1241. doi: 10.1109/EMBC.2017.8037055

PubMed Abstract | CrossRef Full Text | Google Scholar

Chicca, E., Schmuker, M., and Nawrot, M. (2013). Neuromorphic Sensors, Olfaction. New York, NY: Springer New York.

Google Scholar

Clady, X., Clercq, C., Ieng, S. H., Houseini, F., Randazzo, M., Natale, L., et al. (2014). Asynchronous visual event-based time-to-contact. Front. Neurosci. 8:9. doi: 10.3389/fnins.2014.00009

CrossRef Full Text | Google Scholar

Clady, X., Ieng, S. H., and Benosman, R. (2015). Asynchronous event-based corner detection and matching. Neural Netw. 66, 91–106. doi: 10.1016/j.neunet.2015.02.013

PubMed Abstract | CrossRef Full Text | Google Scholar

Clady, X., Maro, J. M., Barré, S., and Benosman, R. B. (2017). A motion-based feature for event-based pattern recognition. Front. Neurosci. 10:594. doi: 10.3389/fnins.2016.00594

PubMed Abstract | CrossRef Full Text | Google Scholar

Cohen, G., Afshar, S., Morreale, B., Bessell, T., Wabnitz, A., Rutten, M., et al. (2019). Event-based sensing for space situational awareness. J. Astronaut. Sci. 66, 125–141. doi: 10.1007/s40295-018-00140-5

CrossRef Full Text | Google Scholar

Conradt, J. (2015). “On-board real-time optic-flow for miniature event-based vision sensors,” in 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO) (IEEE), 1858–1863. doi: 10.1109/ROBIO.2015.7419043

CrossRef Full Text | Google Scholar

Conradt, J., Berner, R., Cook, M., and Delbruck, T. (2009a). “An embedded aer dynamic vision sensor for low-latency pole balancing,” in 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops (IEEE), 780–785. doi: 10.1109/ICCVW.2009.5457625

CrossRef Full Text | Google Scholar

Conradt, J., Cook, M., Berner, R., Lichtsteiner, P., Douglas, R. J., and Delbruck, T. (2009b). “A pencil balancing robot using a pair of AER dynamic vision sensors,” in 2009 IEEE International Symposium on Circuits and Systems (IEEE), 781–784. doi: 10.1109/ISCAS.2009.5117868

CrossRef Full Text | Google Scholar

Corradi, F., and Indiveri, G. (2015). A neuromorphic event-based neural recording system for smart brain-machine-interfaces. IEEE Trans. Biomed. Circuits Syst. 9, 699–709. doi: 10.1109/TBCAS.2015.2479256

PubMed Abstract | CrossRef Full Text | Google Scholar

Costas-Santos, J., Serrano-Gotarredona, T., Serrano-Gotarredona, R., and Linares-Barranco, B. (2007). A spatial contrast retina with on-chip calibration for neuromorphic spike-based AER vision systems. IEEE Trans. Circuits Syst. I Reg. Pap. 54, 1444–1458. doi: 10.1109/TCSI.2007.900179

CrossRef Full Text | Google Scholar

Dalgaty, T., Vianello, E., Ly, D., Indiveri, G., De Salvo, B., Nowak, E., et al. (2018). “Insect-inspired elementary motion detection embracing resistive memory and spiking neural networks,” in Conference on Biomimetic and Biohybrid Systems (Springer), 115–128. doi: 10.1007/978-3-319-95972-6_13

CrossRef Full Text | Google Scholar

Darwish, A., Abbass, H., Fesquet, L., and Sicard, G. (2017). “Event-driven image sensor application: event-driven image segmentation,” in 2017 3rd International Conference on Event-Based Control, Communication and Signal Processing (EBCCSP), 1–6. doi: 10.1109/EBCCSP.2017.8022820

CrossRef Full Text | Google Scholar

Darwish, A., Fesquet, L., and Sicard, G. (2015). “RTL simulation of an asynchronous reading architecture for an event-driven image sensor,” in 2015 International Conference on Event-Based Control, Communication, and Signal Processing (EBCCSP), 1–4. doi: 10.1109/EBCCSP.2015.7300666

CrossRef Full Text | Google Scholar

del Campo, S. M., Albertsson, K., Nilsson, J., Eliasson, J., and Sandin, F. (2013). “FPGA prototype of machine learning analog-to-feature converter for event-based succinct representation of signals,” in 2013 IEEE International Workshop on Machine Learning for Signal Processing (MLSP), 1–6. doi: 10.1109/MLSP.2013.6661996

CrossRef Full Text | Google Scholar

Delbruck, T. (2008). “Frame-free dynamic digital vision,” in Proceedings of International Symposium on Secure-Life Electronics, Advanced Electronics for Quality Life and Society, 21–26.

Google Scholar

Delbruck, T. (2016). “Neuromorophic vision sensing and processing,” in 2016 46th European Solid-State Device Research Conference (ESSDERC) (IEEE), 7–14. doi: 10.1109/ESSDERC.2016.7599576

CrossRef Full Text | Google Scholar

Delbruck, T., and Lang, M. (2013). Robotic goalie with 3 ms reaction time at 4% cpu load using event-based dynamic vision sensor. Front. Neurosci. 7:223. doi: 10.3389/fnins.2013.00223

PubMed Abstract | CrossRef Full Text | Google Scholar

Delbruck, T., and Lichtsteiner, P. (2007). “Fast sensory motor control based on event-based hybrid neuromorphic-procedural system,” in 2007 IEEE International Symposium on Circuits and Systems (IEEE), 845–848. doi: 10.1109/ISCAS.2007.378038

CrossRef Full Text | Google Scholar

Delbrück, T., Linares-Barranco, B., Culurciello, E., and Posch, C. (2010). “Activity-driven, event-based vision sensors,” in Proceedings of 2010 IEEE International Symposium on Circuits and Systems (ISCAS) (IEEE), 2426–2429. doi: 10.1109/ISCAS.2010.5537149

CrossRef Full Text | Google Scholar

Delbruck, T., and Liu, S. C. (2012). Event-Based Silicon Retinas and Cochleas. Vienna: Springer Vienna.

Google Scholar

Delbruck, T., Pfeiffer, M., Juston, R., Orchard, G., Müggler, E., Linares-Barranco, A., et al. (2015). “Human vs. computer slot car racing using an event and frame-based Davis vision sensor,” in 2015 IEEE International Symposium on Circuits and Systems (ISCAS), 2409–2412. IEEE. doi: 10.1109/ISCAS.2015.7169170

CrossRef Full Text | Google Scholar

Delbruck, T., Villanueva, V., and Longinotti, L. (2014). “Integration of dynamic vision sensor with inertial measurement unit for electronically stabilized event-based vision,” in 2014 IEEE International Symposium on Circuits and Systems (ISCAS) (IEEE), 2636–2639. doi: 10.1109/ISCAS.2014.6865714

CrossRef Full Text | Google Scholar

Dhoble, K., Nuntalid, N., Indiveri, G., and Kasabov, N. (2012). “Online spatio-temporal pattern recognition with evolving spiking neural networks utilising address event representation, rank order, and temporal spike learning,” in The 2012 International Joint Conference on Neural Networks (IJCNN), 1–7. doi: 10.1109/IJCNN.2012.6252439

CrossRef Full Text | Google Scholar

Di Natale, C. (2011). An investigation on the role of spike latency in an artificial olfactory system. Front. Neuroeng. 4:16. doi: 10.3389/fneng.2011.00016

PubMed Abstract | CrossRef Full Text | Google Scholar

Diamond, A., Schmuker, M., Berna, A., Trowell, S., and Nowotny, T. (2016). Classifying continuous, real-time e-nose sensor data using a bio-inspired spiking network modelled on the insect olfactory system. Bioinspir. Biomimet. 11:026002. doi: 10.1088/1748-3190/11/2/026002

PubMed Abstract | CrossRef Full Text | Google Scholar

Dikov, G., Firouzi, M., Röhrbein, F., Conradt, J., and Richter, C. (2017). “Spiking cooperative stereo-matching at 2 ms latency with neuromorphic hardware,” in Biomimetic and Biohybrid Systems, eds M. Mangan, M. Cutkosky, A. Mura, P. F. Verschure, T. Prescott, and N. Lepora (Cham: Springer International Publishing), 119–137. doi: 10.1007/978-3-319-63537-8_11

CrossRef Full Text | Google Scholar

Dominguez-Morales, M., Cerezuela-Escudero, E., Jimenez-Fernandez, A., Paz-Vicente, R., Font-Calvo, J., Inigo-Blasco, P., et al. (2011). “Image matching algorithms in stereo vision using address-event-representation: a theoretical study and evaluation of the different algorithms,” in Proceedings of the International Conference on Signal Processing and Multimedia Applications (IEEE), 1–6.

Google Scholar

Doutsi, E., Fillatre, L., Antonini, M., and Gaulmin, J. (2015). “Event-based coding of still images using a bio-inspired frame,” in 2015 International Conference on Event-Based Control, Communication, and Signal Processing (EBCCSP), 1–8. doi: 10.1109/EBCCSP.2015.7300678

CrossRef Full Text | Google Scholar

Drazen, D., Lichtsteiner, P., Häfliger, P., Delbrück, T., and Jensen, A. (2011). Toward real-time particle tracking using an event-based dynamic vision sensor. Exp. Fluids 51:1465. doi: 10.1007/s00348-011-1207-y

CrossRef Full Text | Google Scholar

Drix, D., and Schmuker, M. (2021). Resolving fast gas transients with metal oxide sensors. ACS Sensors 6, 688–692. doi: 10.1021/acssensors.0c02006

PubMed Abstract | CrossRef Full Text | Google Scholar

Eibensteiner, F., Brachtendorf, H. G., and Scharinger, J. (2017). “Event-driven stereo vision algorithm based on silicon retina sensors,” in 2017 27th International Conference Radioelektronika (RADIOELEKTRONIKA), 1–6. doi: 10.1109/RADIOELEK.2017.7937602

CrossRef Full Text | Google Scholar

Erskine, A., Ackels, T., Dasgupta, D., Fukunaga, I., and Schaefer, A. T. (2019). Mammalian olfaction is a high temporal bandwidth sense. bioRxiv 570689. doi: 10.1101/570689

CrossRef Full Text | Google Scholar

Etienne-Cummings, R., and Van der Spiegel, J. (1996). Neuromorphic vision sensors. Sens Actuators A Phys. 56, 19–29. doi: 10.1016/0924-4247(96)01277-0

CrossRef Full Text | Google Scholar

Etienne-Cummings, R., Van der Spiegel, J., Mueller, P., and Zhang, M. Z. (2000). A foveated silicon retina for two-dimensional tracking. IEEE Trans. Circuits Syst. II Analog Digit. Signal Process. 47, 504–517. doi: 10.1109/82.847066

CrossRef Full Text | Google Scholar

Everding, L., Walger, L., Ghaderi, V. S., and Conradt, J. (2016). “A mobility device for the blind with improved vertical resolution using dynamic vision sensors,” in 2016 IEEE 18th International Conference on e-Health Networking, Applications and Services (Healthcom), 1–5. doi: 10.1109/HealthCom.2016.7749459

CrossRef Full Text | Google Scholar

Farian, Ł., Häfliger, P., and Leñero-Bardallo, J. A. (2015). “Miniaturized sun sensor with in-pixel processing for attitude determination of micro space probes,” in 2015 International Conference on Event-based Control, Communication, and Signal Processing (EBCCSP), 1–6. doi: 10.1109/EBCCSP.2015.7300688

CrossRef Full Text | Google Scholar

Figliolia, T., Murray, T. S., and Andreou, A. G. (2015). Acoustic micro-doppler signal processing with foveated electronic cochlea. Electron. Lett. 51, 132–134. doi: 10.1049/el.2014.3711

CrossRef Full Text | Google Scholar

Finger, H., and Liu, S. (2011). “Estimating the location of a sound source with a spike-timing localization algorithm,” in 2011 IEEE International Symposium of Circuits and Systems (ISCAS), 2461–2464. doi: 10.1109/ISCAS.2011.5938102

CrossRef Full Text | Google Scholar

Firouzi, M., and Conradt, J. (2016). Asynchronous event-based cooperative stereo matching using neuromorphic silicon retinas. Neural Process. Lett. 43, 311–326. doi: 10.1007/s11063-015-9434-5

CrossRef Full Text | Google Scholar

Folowosele, F., Vogelstein, R. J., and Etienne-Cummings, R. (2011). Towards a cortical prosthesis: implementing a spike-based hmax model of visual object recognition in silico. IEEE J. Emerg. Select. Top. Circuits Syst. 1, 516–525. doi: 10.1109/JETCAS.2012.2183409

CrossRef Full Text | Google Scholar

Franco, J. A. G., del Valle Padilla, J. L., and Ortega Cisneros, S. (2013). “Event-based image processing using a neuromorphic vision sensor,” in 2013 IEEE International Autumn Meeting on Power Electronics and Computing (ROPEC), 1–6. doi: 10.1109/ROPEC.2013.6702715

CrossRef Full Text | Google Scholar

Fu, Z., Culurciello, E., Lichtsteiner, P., and Delbruck, T. (2008a). “Fall detection using an address-event temporal contrast vision sensor,” in 2008 IEEE International Symposium on Circuits and Systems, 424–427. doi: 10.1109/ISCAS.2008.4541445

CrossRef Full Text | Google Scholar

Fu, Z., Delbruck, T., Lichtsteiner, P., and Culurciello, E. (2008b). An address-event fall detector for assisted livingapplications. IEEE Trans. Biomed. Circuits Syst. 2, 88–96. doi: 10.1109/TBCAS.2008.924448

CrossRef Full Text | Google Scholar

Gallego, G., Delbruck, T., Orchard, G. M., Bartolozzi, C., Taba, B., Censi, A., et al. (2020). Event-based vision: a survey. IEEE Trans. Pattern Anal. Mach. Intel. 1. doi: 10.1109/TPAMI.2020.3008413

CrossRef Full Text | Google Scholar

Gallego, G., Forster, C., Mueggler, E., and Scaramuzza, D. (2015). Event-based camera pose tracking using a generative event model. arXiv[Preprint].arXiv:1510.01972.

Google Scholar

Gallego, G., Lund, J., Mueggler, E., Rebecq, H., Delbruck, T., and Scaramuzza, D. (2016). Event-based 6-DoF camera tracking for high-speed applications. arXiv[Preprint].arXiv:1607.034682.

Google Scholar

Gallego, G., Lund, J. E., Mueggler, E., Rebecq, H., Delbruck, T., and Scaramuzza, D. (2018a). Event-based 6-DoF camera tracking from photometric depth maps. IEEE Trans. Pattern Anal. Mach. Intell. 40, 2402–2412. doi: 10.1109/TPAMI.2017.2769655

PubMed Abstract | CrossRef Full Text | Google Scholar

Gallego, G., Rebecq, H., and Scaramuzza, D. (2018b). “A unifying contrast maximization framework for event cameras, with applications to motion, depth, and optical flow estimation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 3867–3876. doi: 10.1109/CVPR.2018.00407

CrossRef Full Text | Google Scholar

Gallego, G., and Scaramuzza, D. (2017). Accurate angular velocity estimation with an event camera. IEEE Robot. Autom. Lett. 2, 632–639. doi: 10.1109/LRA.2016.2647639

CrossRef Full Text | Google Scholar

García, G. P., Camilleri, P., Liu, Q., and Furber, S. (2016). “PYDVS: an extensible, real-time dynamic vision sensor emulator using off-the-shelf hardware,” in 2016 IEEE Symposium Series on Computational Intelligence (SSCI) (IEEE), 1–7.

Google Scholar

Gaspar, N., Sondhi, A., Evans, B., and Nikolic, K. (2016). “A low-power neuromorphic system for retinal implants and sensory substitution,” in 2016 IEEE Biomedical Circuits and Systems Conference (BioCAS), 78–81. doi: 10.1109/BioCAS.2016.7833729

CrossRef Full Text | Google Scholar

Ghaderi, V. S., Mulas, M., Pereira, V. F. S., Everding, L., Weikersdorfer, D., and Conradt, J. (2015). “A wearable mobility device for the blind using retina-inspired dynamic vision sensors,” in 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 3371–3374. doi: 10.1109/EMBC.2015.7319115

PubMed Abstract | CrossRef Full Text | Google Scholar

Ghosh, R., Mishra, A., Orchard, G., and Thakor, N. V. (2014). “Real-time object recognition and orientation estimation using an event-based camera and CNN,” in 2014 IEEE Biomedical Circuits and Systems Conference (BioCAS) Proceedings (IEEE), 544–547. doi: 10.1109/BioCAS.2014.6981783

CrossRef Full Text | Google Scholar

Gibson, T. A., Heath, S., Quinn, R. P., Lee, A. H., Arnold, J. T., Sonti, T. S., et al. (2014a). “Event-based visual data sets for prediction tasks in spiking neural networks,” in International Conference on Artificial Neural Networks (Springer), 635–642. doi: 10.1007/978-3-319-11179-7_80

CrossRef Full Text | Google Scholar

Gibson, T. A., Henderson, J. A., and Wiles, J. (2014b). “Predicting temporal sequences using an event-based spiking neural network incorporating learnable delays,” in 2014 International Joint Conference on Neural Networks (IJCNN), 3213–3220. doi: 10.1109/IJCNN.2014.6889850

CrossRef Full Text | Google Scholar

Giulioni, M., Lagorce, X., Galluppi, F., and Benosman, R. B. (2016). Event-based computation of motion flow on a neuromorphic analog neural platform. Front. Neurosci. 10:35. doi: 10.3389/fnins.2016.00035

PubMed Abstract | CrossRef Full Text | Google Scholar

Glover, A., and Bartolozzi, C. (2017). “Robust visual tracking with a freely-moving event camera,” in 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE), 3769–3776. doi: 10.1109/IROS.2017.8206226

CrossRef Full Text | Google Scholar

Gómez-Rodríguez, F., Miró-Amarante, L., Diaz-del Rio, F., Linares-Barranco, A., and Robotics, G. J. (2010). “Real time multiple objects tracking based on a bio-inspired processing cascade architecture,” in Proceedings of 2010 IEEE International Symposium on Circuits and Systems (IEEE), 1399–1402. doi: 10.1109/ISCAS.2010.5537277

CrossRef Full Text | Google Scholar

Gómez-Rodríguez, F., Miró-Amarante, L., Rivas, M., Jimenez, G., and Diaz-del Rio, F. (2011). “Neuromorphic real-time objects tracking using address event representation and silicon retina,” in International Work-Conference on Artificial Neural Networks (Springer), 133–140. doi: 10.1007/978-3-642-21501-8_17

CrossRef Full Text | Google Scholar

Gomez-Rodriguez, F., Paz, R., Linares-Barranco, A., Rivas, M., Miro, L., Vicente, S., et al. (2006). “AER tools for communications and debugging,” in Proceedings of 2006 IEEE International Symposium on Circuits and Systems 2006, ISCAS 2006 (IEEE), 4. doi: 10.1109/ISCAS.2006.1693319

CrossRef Full Text | Google Scholar

Gritsch, G., Litzenberger, M., Donath, N., and Kohn, B. (2008). “Real-time vehicle classification using a smart embedded device with a ‘silicon retina' optical sensor,” in 2008 11th International IEEE Conference on Intelligent Transportation Systems (IEEE), 534–538. doi: 10.1109/ITSC.2008.4732575

CrossRef Full Text | Google Scholar

Grybos, A. (2015). “Reconstruction of event-based sampled signal using adaptive weights method,” in 2015 International Conference on Event-Based Control, Communication, and Signal Processing (EBCCSP), 1–4. doi: 10.1109/EBCCSP.2015.7300695

CrossRef Full Text | Google Scholar

Guerrero-Rivera, R., and Pearce, T. C. (2007). “Attractor-based pattern classification in a spiking FPGA implementation of the olfactory bulb,” in 2007 3rd International IEEE/EMBS Conference on Neural Engineering, 593–599. doi: 10.1109/CNE.2007.369742

CrossRef Full Text | Google Scholar

Hassan, M., Bermak, A., Ali, A. A. S., and Amira, A. (2015). “Gas identification with spike codes in wireless electronic nose: a potential application for smart green buildings,” in 2015 SAI Intelligent Systems Conference (IntelliSys), 457–462. doi: 10.1109/IntelliSys.2015.7361180

CrossRef Full Text | Google Scholar

Hausler, C., Nawrot, M. P., and Schmuker, M. (2011). “A spiking neuron classifier network with a deep architecture inspired by the olfactory system of the honeybee,” in 2011 5th International IEEE/EMBS Conference on Neural Engineering, 198–202. doi: 10.1109/NER.2011.5910522

CrossRef Full Text | Google Scholar

Hofstätter, M., Litzenberger, M., Matolin, D., and Posch, C. (2011). “Hardware-accelerated address-event processing for high-speed visual object recognition,” in 2011 18th IEEE International Conference on Electronics, Circuits, and Systems (IEEE), 89–92. doi: 10.1109/ICECS.2011.6122221

CrossRef Full Text | Google Scholar

Hordijk, B. J. P., Scheper, K. Y. W., and de Croon, G. C. H. E. (2017). Vertical landing for micro air vehicles using event-based optical flow. CoRR abs/1702.00061.

Google Scholar

Horstschäfer, T. (2016). Parallel Tracking, Depth Estimation, and Image Reconstruction With an Event Camera.

Google Scholar

Hoseini, S., and Linares-Barranco, B. (2018). “Real-time temporal frequency detection in FPGA using event-based vision sensor,” in 2018 IEEE 14th International Conference on Intelligent Computer Communication and Processing (ICCP), 271–278. doi: 10.1109/ICCP.2018.8516629

CrossRef Full Text | Google Scholar

Hsieh, H., and Tang, K. (2012). Vlsi implementation of a bio-inspired olfactory spiking neural network. IEEE Trans. Neural Netw. Learn. Syst. 23, 1065–1073. doi: 10.1109/TNNLS.2012.2195329

PubMed Abstract | CrossRef Full Text | Google Scholar

Hu, Y., Liu, H., Pfeiffer, M., and Delbruck, T. (2016). DVS benchmark datasets for object tracking, action recognition, and object recognition. Front. Neurosci. 10:405. doi: 10.3389/fnins.2016.00405

PubMed Abstract | CrossRef Full Text | Google Scholar

Ieng, S., Posch, C., and Benosman, R. (2014). Asynchronous neuromorphic event-driven image filtering. Proc. IEEE 102, 1485–1499. doi: 10.1109/JPROC.2014.2347355

CrossRef Full Text | Google Scholar

Imam, N., Cleland, T., Manohar, R., Merolla, P., Arthur, J., Akopyan, F., et al. (2012). Implementation of olfactory bulb glomerular-layer computations in a digital neurosynaptic core. Front. Neurosci. 6:83. doi: 10.3389/fnins.2012.00083

PubMed Abstract | CrossRef Full Text | Google Scholar

Indiveri, G. (2000). Modeling selective attention using a neuromorphic analog VLSI device. Neural Comput. 12, 2857–2880. doi: 10.1162/089976600300014755

PubMed Abstract | CrossRef Full Text | Google Scholar

Indiveri, G. (2008). Neuromorphic VLSI models of selective attention: from single chip vision sensors to multi-chip systems. Sensors 8, 5352–5375. doi: 10.3390/s8095352

PubMed Abstract | CrossRef Full Text | Google Scholar

Jäckel, D., Moeckel, R., and Liu, S. (2010). “Sound recognition with spiking silicon cochlea and hidden Markov models,” in 6th Conference on Ph.D. Research in Microelectronics Electronics, 1–4.

Google Scholar

jAER (2021). Open Source jAER Software Project. jAER.

James, C. D., Aimone, J. B., Miner, N. E., Vineyard, C. M., Rothganger, F. H., Carlson, K. D., et al. (2017). A historical survey of algorithms and hardware architectures for neural-inspired and neuromorphic computing applications. Biol. Inspir. Cogn. Archit. 19, 49–64. doi: 10.1016/j.bica.2016.11.002

CrossRef Full Text | Google Scholar

Jansen, A., and Niyogi, P. (2009). Point process models for event-based speech recognition. Speech Commun. 51, 1155–1168. doi: 10.1016/j.specom.2009.05.008

CrossRef Full Text | Google Scholar

Jiang, Z., Bing, Z., Huang, K., Chen, G., Cheng, L., and Knoll, A. (2017). “Event-based target tracking control for a snake robot using a dynamic vision sensor,” in International Conference on Neural Information Processing (Springer), 111–121. doi: 10.1007/978-3-319-70136-3_12

CrossRef Full Text | Google Scholar

Jiménez-Fernández, A., Cerezuela-Escudero, E., Miró-Amarante, L., Domínguez-Morales, M. J., de Asís Gómez-Rodríguez, F., Linares-Barranco, A., et al. (2017). A binaural neuromorphic auditory sensor for FPGA: a spike signal processing approach. IEEE Trans. Neural Netw. Learn. Syst. 28, 804–818. doi: 10.1109/TNNLS.2016.2583223

PubMed Abstract | CrossRef Full Text | Google Scholar

Jimenez-Fernandez, A., Lujan-Martinez, C., Paz-Vicente, R., Linares-Barranco, A., Jimenez, G., and Civit, A. (2009). “From vision sensor to actuators, spike based robot control through address-event-representation,” in International Work-Conference on Artificial Neural Networks (Springer), 797–804. doi: 10.1007/978-3-642-02478-8_100

CrossRef Full Text | Google Scholar

Jing, Y. Q., Meng, Q. H., Qi, P. F., Zeng, M., and Liu, Y. J. (2016). Signal processing inspired from the olfactory bulb for electronic noses. Meas. Sci. Technol. 28:015105. doi: 10.1088/1361-6501/28/1/015105

CrossRef Full Text | Google Scholar

Kaiser, J., Melbaum, S., Tieck, J. C. V., Roennau, A., Butz, M. V., and Dillmann, R. (2018). “Learning to reproduce visually similar movements by minimizing event-based prediction error,” in 2018 7th IEEE International Conference on Biomedical Robotics and Biomechatronics (Biorob), 260–267. doi: 10.1109/BIOROB.2018.8487959

CrossRef Full Text | Google Scholar

Kasap, B., and Schmuker, M. (2013). “Improving odor classification through self-organized lateral inhibition in a spiking olfaction-inspired network,” in 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), 219–222. doi: 10.1109/NER.2013.6695911

CrossRef Full Text | Google Scholar

Katz, M. L., Lutterbeck, C., and Nikolic, K. (2012a). “An implementation of magnocellular pathways in event-based retinomorphic systems,” in 2012 IEEE Biomedical Circuits and Systems Conference (BioCAS), 17–20. doi: 10.1109/BioCAS.2012.6418480

CrossRef Full Text | Google Scholar

Katz, M. L., Nikolic, K., and Delbruck, T. (2012b). “Live demonstration: behavioural emulation of event-based vision sensors,” in 2012 IEEE International Symposium on Circuits and Systems, 736–740. doi: 10.1109/ISCAS.2012.6272143

CrossRef Full Text | Google Scholar

Kawasetsu, T., Ishida, R., Sanada, T., and Okuno, H. (2014). “A hardware system for emulating the early vision utilizing a silicon retina and spinnaker chips,” in 2014 IEEE Biomedical Circuits and Systems Conference (BioCAS) Proceedings, 552–555. doi: 10.1109/BioCAS.2014.6981785

CrossRef Full Text | Google Scholar

Kim, H., Handa, A., Benosman, R., Ieng, S. H., and Davison, A. J. (2008). Simultaneous mosaicing and tracking with an event camera. J. Solid State Circ. 43, 566–576. doi: 10.5244/C.28.26

CrossRef Full Text | Google Scholar

Kim, H., Leutenegger, S., and Davison, A. J. (2016). “Real-time 3D reconstruction and 6-DoF tracking with an event camera,” in Computer Vision-ECCV 2016, eds B. Leibe, J. Matas, N. Sebe, and M. Welling (Cham: Springer International Publishing), 349–364. doi: 10.1007/978-3-319-46466-4_21

CrossRef Full Text | Google Scholar

Klein, P., Conradt, J., and Liu, S. C. (2015). “Scene stitching with event-driven sensors on a robot head platform,” in 2015 IEEE International Symposium on Circuits and Systems (ISCAS) (IEEE), 2421–2424. doi: 10.1109/ISCAS.2015.7169173

CrossRef Full Text | Google Scholar

Koeth, F., Marques, H., and Delbruck, T. (2013). Self-organisation of motion features with a temporal asynchronous dynamic vision sensor. Biol. Inspir. Cogn. Archit. 6, 8–11. doi: 10.1016/j.bica.2013.05.010

CrossRef Full Text | Google Scholar

Kogler, J. (2016). Design and evaluation of stereo matching techniques for silicon retina cameras (Ph.D. thesis), Vienna University of Technology, Vienna, Austria.

Google Scholar

Kogler, J., Eibensteiner, F., Humenberger, M., Gelautz, M., and Scharinger, J. (2013). “Ground truth evaluation for event-based silicon retina stereo data,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. doi: 10.1109/CVPRW.2013.98

CrossRef Full Text | Google Scholar

Kogler, J., Humenberger, M., and Sulzbachner, C. (2011a). “Event-based stereo matching approaches for frameless address event stereo data,” in International Symposium on Visual Computing (Springer), 674–685. doi: 10.1007/978-3-642-24028-7_62

CrossRef Full Text | Google Scholar

Kogler, J., Sulzbachner, C., Eibensteiner, F., and Humenberger, M. (2010). “Address-event matching for a silicon retina based stereo vision system,” in 4th International Conference from Scientific Computing to Computational Engineering, 17–24.

Kogler, J., Sulzbachner, C., Humenberger, M., and Eibensteiner, F. (2011b). “Address-event based stereo vision with bio-inspired silicon retina imagers,” in Advances in Theory and Applications of Stereo Vision (IntechOpen). doi: 10.5772/12941

CrossRef Full Text | Google Scholar

Kogler, J., Sulzbachner, C., and Kubinger, W. (2009). “Bio-inspired stereo vision system with silicon retina imagers,” in International Conference on Computer Vision Systems (Springer), 174–183. doi: 10.1007/978-3-642-04667-4_18

CrossRef Full Text | Google Scholar

Koickal, T. J., Fulvi-Maria, C., Covingtonc, J. A., Tanc, F. S., Gardnerc, J. W., and Hamiltonb, A. (2004). Silicon-Based Neuromorphic Olfactory Pathway Implementation.

Google Scholar

Koickal, T. J., Hamilton, A., Pearce, T. C., Tan, S. L., Covington, J. A., and Gardner, J. W. (2006). “Analog VLSI design of an adaptive neuromorphic chip for olfactory systems,” in 2006 IEEE International Symposium on Circuits and Systems, 4550. doi: 10.1109/ISCAS.2006.1693641

CrossRef Full Text | Google Scholar

Koickal, T. J., Hamilton, A., Tan, S. L., Covington, J. A., Gardner, J. W., and Pearce, T. C. (2007). Analog VLSI circuit implementation of an adaptive neuromorphic olfaction chip. IEEE Trans. Circuits Syst. I Reg. Pap. 54, 60–73. doi: 10.1109/TCSI.2006.888677

CrossRef Full Text | Google Scholar

Koickal, T. J., Latif, R., Gouveia, L., Mastropaolo, E., Wang, S., Hamilton, A., et al. (2011). “Design of a spike event coded RGT microphone for neuromorphic auditory systems,” in 2011 IEEE International Symposium of Circuits and Systems (ISCAS), 2465–2468. doi: 10.1109/ISCAS.2011.5938103

CrossRef Full Text | Google Scholar

Kowadlo, G., and Russell, R. A. (2008). Robot odor localization: a taxonomy and survey. Int. J. Robot. Res. 27, 869–894. doi: 10.1177/0278364908095118

CrossRef Full Text | Google Scholar

Kueng, B., Mueggler, E., Gallego, G., and Scaramuzza, D. (2016). “Low-latency visual odometry using event-based feature tracks,” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE), 16–23. doi: 10.1109/IROS.2016.7758089

CrossRef Full Text | Google Scholar

Lagorce, X., Ieng, S., and Benosman, R. (2013). “Event-based features for robotic vision,” in 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems, 4214–4219. doi: 10.1109/IROS.2013.6696960

CrossRef Full Text | Google Scholar

Lagorce, X., Ieng, S. H., Clady, X., Pfeiffer, M., and Benosman, R. B. (2015a). Spatiotemporal features for asynchronous event-based data. Front. Neurosci. 9:46. doi: 10.3389/fnins.2015.00046

PubMed Abstract | CrossRef Full Text | Google Scholar

Lagorce, X., Meyer, C., Ieng, S. H., Filliat, D., and Benosman, R. (2015b). Asynchronous event-based multikernel algorithm for high-speed visual features tracking. IEEE Trans. Neural Netw. Learn. Syst. 26, 1710–1720. doi: 10.1109/TNNLS.2014.2352401

PubMed Abstract | CrossRef Full Text | Google Scholar

Lagorce, X., Orchard, G., Galluppi, F., Shi, B. E., and Benosman, R. B. (2017). HOTS: a hierarchy of event-based time-surfaces for pattern recognition. IEEE Trans. Pattern Anal. Mach. Intell. 39, 1346–1359. doi: 10.1109/TPAMI.2016.2574707

PubMed Abstract | CrossRef Full Text | Google Scholar

LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., et al. (1989). Backpropagation applied to handwritten zip code recognition. Neural Comput. 1, 541–551. doi: 10.1162/neco.1989.1.4.541

CrossRef Full Text | Google Scholar

Lee, J. H., Delbruck, T., Pfeiffer, M., Park, P. K., Shin, C. W., Ryu, H., et al. (2014). Real-time gesture interface based on event-driven processing from stereo silicon retinas. IEEE Trans. Neural Netw. Learn. Syst. 25, 2250–2263. doi: 10.1109/TNNLS.2014.2308551

PubMed Abstract | CrossRef Full Text | Google Scholar

Leñero-Bardallo, J. A., Serrano-Gotarredona, T., and Linares-Barranco, B. (2011). A 3.6μs latency asynchronous frame-free event-driven dynamic-vision-sensor. IEEE J. Solid State Circuits 46, 1443–1455. doi: 10.1109/JSSC.2011.2118490

CrossRef Full Text | Google Scholar

Leow, H. S., and Nikolic, K. (2015). “Machine vision using combined frame-based and event-based vision sensor,” in 2015 IEEE International Symposium on Circuits and Systems (ISCAS), 706–709. doi: 10.1109/ISCAS.2015.7168731

CrossRef Full Text | Google Scholar

Li, C., Delbruck, T., and Liu, S. (2012). “Real-time speaker identification using the AEREAR2 event-based silicon cochlea,” in 2012 IEEE International Symposium on Circuits and Systems, 1159–1162. doi: 10.1109/ISCAS.2012.6271438

CrossRef Full Text | Google Scholar

Li, H., Li, G., Ji, X., and Shi, L. (2018). Deep representation via convolutional neural network for classification of spatiotemporal event streams. Neurocomputing 299, 1–9. doi: 10.1016/j.neucom.2018.02.019

CrossRef Full Text | Google Scholar

Li, H., Liu, H., Ji, X., Li, G., and Shi, L. (2017). CIFAR10-DVS: an event-stream dataset for object classification. Front. Neurosci. 11:309. doi: 10.3389/fnins.2017.00309

PubMed Abstract | CrossRef Full Text | Google Scholar

Lichtsteiner, P., Posch, C., and Delbruck, T. (2008). A 128 × 128 120 dB 15 μs latency asynchronous temporal contrast vision sensor. IEEE J. Solid State Circuits 43, 566–576. doi: 10.1109/JSSC.2007.914337

CrossRef Full Text | Google Scholar

Linares-Barranco, A., Gomez-Rodriguez, F., Jimenez-Fernandez, A., Delbruck, T., and Lichtensteiner, P. (2007). “Using FPGA for visuo-motor control with a silicon retina and a humanoid robot,” in 2007 IEEE International Symposium on Circuits and Systems (IEEE), 1192–1195. doi: 10.1109/ISCAS.2007.378265

CrossRef Full Text | Google Scholar

Linares-Barranco, A., Gómez-Rodríguez, F., Villanueva, V., Longinotti, L., and Delbrück, T. (2015). “A USB3. 0 FPGA event-based filtering and tracking framework for dynamic vision sensors,” in 2015 IEEE International Symposium on Circuits and Systems (ISCAS) (IEEE), 2417–2420. doi: 10.1109/ISCAS.2015.7169172

CrossRef Full Text | Google Scholar

Litzenberger, M., Bauer, D., Donath, N., Garn, H., Kohn, B., Posch, C., et al. (2006a). “Embedded vehicle counting system with ‘silicon retina' optical sensor,” in AIP Conference Proceedings, Vol. 860 (AIP), 360–367. doi: 10.1063/1.2361239

CrossRef Full Text | Google Scholar

Litzenberger, M., Kohn, B., Belbachir, A., Donath, N., Gritsch, G., Garn, H., et al. (2006b). “Estimation of vehicle speed based on asynchronous data from a silicon retina optical sensor,” in 2006 IEEE Intelligent Transportation Systems Conference (IEEE), 653–658. doi: 10.1109/ITSC.2006.1706816

CrossRef Full Text | Google Scholar

Litzenberger, M., Posch, C., Bauer, D., Belbachir, A., Schon, P., Kohn, B., et al. (2006c). “Embedded vision system for real-time object tracking using an asynchronous transient vision sensor,” in 2006 IEEE 12th Digital Signal Processing Workshop & 4th IEEE Signal Processing Education Workshop (IEEE), 173–178. doi: 10.1109/DSPWS.2006.265448

CrossRef Full Text | Google Scholar

Litzenberger, S., and Sabo, A. (2012). Can silicon retina sensors be used for optical motion analysis in sports? Proc. Eng. 34, 748–753. doi: 10.1016/j.proeng.2012.04.128

CrossRef Full Text | Google Scholar

Liu, H., Rios-Navarro, A., Moeys, D. P., Delbruck, T., and Linares-Barranco, A. (2017a). “Neuromorphic approach sensitivity cell modeling and FPGA implementation,” in Artificial Neural Networks and Machine Learning-ICANN 2017, eds A. Lintas, S. Rovetta, P. F. Verschure, and A. E. Villa (Cham: Springer International Publishing), 179–187. doi: 10.1007/978-3-319-68600-4_22

CrossRef Full Text | Google Scholar

Liu, H. C., Zhang, F. L., Marshall, D., Shi, L., and Hu, S. M. (2017b). High-speed video generation with an event camera. Visual Comput. 33, 749–759. doi: 10.1007/s00371-017-1372-y

CrossRef Full Text | Google Scholar

Liu, J., and Wang, C. (2009). “A survey of neuromorphic engineering-biological nervous systems realized on silicon,” in IEEE Circuits and Systems International Conference on Testing and Diagnosis 2009, ICTD 2009 (IEEE), 1–4. doi: 10.1109/CAS-ICTD.2009.4960772

CrossRef Full Text | Google Scholar

Liu, J. H., Wang, C. Y., and An, Y. Y. (2009). “A survey of neuromorphic vision system: biological nervous systems realized on silicon,” in 2009 International Conference on Industrial Mechatronics and Automation, 154–157. doi: 10.1109/ICIMA.2009.5156583

CrossRef Full Text | Google Scholar

Liu, M., and Delbruck, T. (2017). “Block-matching optical flow for dynamic vision sensors: algorithm and FPGA implementation,” in 2017 IEEE International Symposium on Circuits and Systems (ISCAS) (IEEE), 1–4. doi: 10.1109/ISCAS.2017.8050295

CrossRef Full Text | Google Scholar

Liu, M., and Delbrück, T. (2018). ABMOF: a novel optical flow algorithm for dynamic vision sensors. arXiv[Prepint].arXiv:1805.03988.

Google Scholar

Liu, S., van Schaik, A., Mincti, B. A., and Delbruck, T. (2010). “Event-based 64-channel binaural silicon cochlea with Q enhancement mechanisms,” in Proceedings of 2010 IEEE International Symposium on Circuits and Systems, 2027–2030. doi: 10.1109/ISCAS.2010.5537164

CrossRef Full Text | Google Scholar

Liu, S. C., Rueckauer, B., Ceolini, E., Huber, A., and Delbruck, T. (2019). Event-driven sensing for efficient perception: vision and audition algorithms. IEEE Signal Process. Mag. 36, 29–37. doi: 10.1109/MSP.2019.2928127

CrossRef Full Text | Google Scholar

Löhr, M. P. R., and Neumann, H. (2018). “Contrast detection in event-streams from dynamic vision sensors with fixational eye movements,” in 2018 IEEE International Symposium on Circuits and Systems (ISCAS), 1–5. doi: 10.1109/ISCAS.2018.8351084

CrossRef Full Text | Google Scholar

Lorach, H., Benosman, R., Marre, O., Ieng, S. H., Sahel, J. A., and Picaud, S. (2012). Artificial retina: the multichannel processing of the mammalian retina achieved with a neuromorphic asynchronous light acquisition device. J. Neural Eng. 9:066004. doi: 10.1088/1741-2560/9/6/066004

PubMed Abstract | CrossRef Full Text | Google Scholar

Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110. doi: 10.1023/B:VISI.0000029664.99615.94

CrossRef Full Text | Google Scholar

Lyon, R. F., and Mead, C. (1988). An analog electronic cochlea. IEEE Trans. Acoust. Speech Signal Process. 36, 1119–1134. doi: 10.1109/29.1639

CrossRef Full Text | Google Scholar

Marcireau, A., Ieng, S. H., Simon-Chane, C., and Benosman, R. B. (2018). Event-based color segmentation with a high dynamic range sensor. Front. Neurosci. 12:135. doi: 10.3389/fnins.2018.00135

PubMed Abstract | CrossRef Full Text | Google Scholar

Marco, S., Gutiérrez-Gálvez, A., Lansner, A., Martinez, D., Rospars, J. P., Beccherelli, R., et al. (2013). A biomimetic approach to machine olfaction, featuring a very large-scale chemical sensor array and embedded neuro-bio-inspired computation. Microsyst. Technol. 20, 729–742. doi: 10.1007/s00542-013-2020-8

CrossRef Full Text | Google Scholar

Marr, D., and Poggio, T. (1976). Cooperative computation of stereo disparity. Science 194, 283–287. doi: 10.1126/science.968482

CrossRef Full Text | Google Scholar

Martinelli, E., Dini, F., Pennazza, G., Canosa, M., D'Amico, A., and Di Natale, C. (2009). A Novel Bio-inspired Digital Signal Processing Method for Chemical Sensor Arrays. Berlin; Heidelberg: Springer Berlin Heidelberg.

Google Scholar

Matolin, D., Wohlgenannt, R., Litzenberger, M., and Posch, C. (2010). “A load-balancing readout method for large event-based PWM imaging arrays,” in Proceedings of 2010 IEEE International Symposium on Circuits and Systems, 361–364. doi: 10.1109/ISCAS.2010.5537778

CrossRef Full Text | Google Scholar

Milde, M. B., Bertrand, O. J. N., Benosmanz, R., Egelhaaf, M., and Chicca, E. (2015). “Bioinspired event-driven collision avoidance algorithm based on optic flow,” in 2015 International Conference on Event-based Control, Communication, and Signal Processing (EBCCSP), 1–7. doi: 10.1109/EBCCSP.2015.7300673

CrossRef Full Text | Google Scholar

Milde, M. B., Dietmüller, A., Blum, H., Indiveri, G., and Sandamirskaya, Y. (2017). “Obstacle avoidance and target acquisition in mobile robots equipped with neuromorphic sensory-processing systems,” in 2017 IEEE International Symposium on Circuits and Systems (ISCAS) (IEEE), 1–4. doi: 10.1109/ISCAS.2017.8050984

CrossRef Full Text | Google Scholar

Milford, M., Kim, H., Leutenegger, S., and Davison, A. (2015). “Towards visual slam with event-based cameras,” in The Problem of Mobile Sensors Workshop in Conjunction With RSS.

Miró-Amarante, L., Gómez-Rodríguez, F., Jiménez-Fernández, A., and Jiménez-Moreno, G. (2017). A spiking neural network for real-time spanish vowel phonemes recognition. Neurocomputing 226, 249–261. doi: 10.1016/j.neucom.2016.12.005

CrossRef Full Text | Google Scholar

Miskowicz, M. (Ed.). (2016). Event-Based Control and Signal Processing. Embedded Systems. Boca Raton, FL: CRC Press.

Google Scholar

Moeys, D. P., Corradi, F., Kerr, E., Vance, P., Das, G., Neil, D., et al. (2016a). “Steering a predator robot using a mixed frame/event-driven convolutional neural network,” in 2016 Second International Conference on Event-based Control, Communication, and Signal Processing (EBCCSP) (IEEE), 1–8. doi: 10.1109/EBCCSP.2016.7605233

CrossRef Full Text | Google Scholar

Moeys, D. P., Delbrück, T., Rios-Navarro, A., and Linares-Barranco, A. (2016b). “Retinal ganglion cell software and FPGA model implementation for object detection and tracking,” in 2016 IEEE International Symposium on Circuits and Systems (ISCAS), 1434–1437. IEEE. doi: 10.1109/ISCAS.2016.7527520

CrossRef Full Text | Google Scholar

Moraud, E. M., and Chicca, E. (2011). Toward neuromorphic odor tracking: perspectives for space exploration. Acta Futura 4, 9–19. doi: 10.5167/uzh-98513

CrossRef Full Text | Google Scholar

Moser, B. A. (2015). Matching event sequences approach based on weyl's discrepancy norm. In 2015 International Conference on Event-based Control, Communication, and Signal Processing (EBCCSP), 1–5. doi: 10.1109/EBCCSP.2015.7300676

CrossRef Full Text | Google Scholar

Mueggler, E., Bartolozzi, C., and Scaramuzza, D. (2017a). “Fast event-based corner detection,” in British Machine Vision Conference (BMVC), Vol. 1. doi: 10.5244/C.31.33

CrossRef Full Text | Google Scholar

Mueggler, E., Baumli, N., Fontana, F., and Scaramuzza, D. (2015a). “Towards evasive maneuvers with quadrotors using dynamic vision sensors,” in 2015 European Conference on Mobile Robots (ECMR) (IEEE), 1–8. doi: 10.1109/ECMR.2015.7324048

CrossRef Full Text | Google Scholar

Mueggler, E., Forster, C., Baumli, N., Gallego, G., and Scaramuzza, D. (2015b). “Lifetime estimation of events dynamic vision sensors,” in 2015 IEEE International Conference on Robotics and Automation (ICRA), 4874–4881. doi: 10.1109/ICRA.2015.7139876

CrossRef Full Text | Google Scholar

Mueggler, E., Gallego, G., Rebecq, H., and Scaramuzza, D. (2018). Continuous-time visual-inertial odometry for event cameras. IEEE Trans. Robot. 34, 1425–1440. doi: 10.1109/TRO.2018.2858287

CrossRef Full Text | Google Scholar

Mueggler, E., Gallego, G., and Scaramuzza, D. (2015c). “Continuous-time trajectory estimation for event-based vision sensors,” in Robotics: Science and Systems, Vol. 2. doi: 10.15607/RSS.2015.XI.036

CrossRef Full Text | Google Scholar

Mueggler, E., Huber, B., and Scaramuzza, D. (2014). “Event-based 6-DoF pose tracking for high-speed maneuvers,” in 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2014) (IEEE), 2761–2768. doi: 10.1109/IROS.2014.6942940

CrossRef Full Text | Google Scholar

Mueggler, E., Rebecq, H., Gallego, G., Delbruck, T., and Scaramuzza, D. (2017b). The event-camera dataset and simulator: event-based data for pose estimation, visual odometry, and slam. Int. J. Robot. Res. 36, 142–149. doi: 10.1177/0278364917691115

CrossRef Full Text | Google Scholar

Mueller, E., Censi, A., and Frazzoli, E. (2015a). “Efficient high speed signal estimation with neuromorphic vision sensors,” in 2015 International Conference on Event-Based Control, Communication, and Signal Processing (EBCCSP), 1–8. doi: 10.1109/EBCCSP.2015.7300672

CrossRef Full Text | Google Scholar

Mueller, E., Censi, A., and Frazzoli, E. (2015b). “Low-latency heading feedback control with neuromorphic vision sensors using efficient approximated incremental inference,” in 2015 54th IEEE Conference on Decision and Control (CDC) (IEEE), 992–999. doi: 10.1109/CDC.2015.7402002

CrossRef Full Text | Google Scholar

Muezzinoglu, M. K., Vergara, A., Huerta, R., Rulkov, N., Rabinovich, M. I., Selverston, A., et al. (2009). Acceleration of chemo-sensory information processing using transient features. Sens. Actuators B Chem. 137, 507–512. doi: 10.1016/j.snb.2008.10.065

CrossRef Full Text | Google Scholar

Müller, G. R., and Conradt, J. (2012). “Self-calibrating marker tracking in 3D with event-based vision sensors,” in International Conference on Artificial Neural Networks (Springer), 313–321. doi: 10.1007/978-3-642-33269-2_40

CrossRef Full Text | Google Scholar

Munda, G., Reinbacher, C., and Pock, T. (2018). Real-time intensity-image reconstruction for event cameras using manifold regularisation. Int. J. Comput. Vision 126, 1381–1393. doi: 10.1007/s11263-018-1106-2

CrossRef Full Text | Google Scholar

Mylne, K. R., and Mason, P. J. (1991). Concentration fluctuation measurements in a dispersing plume at a range of up to 1000 m. Q. J. R. Meteorol. Soc. 117, 177–206. doi: 10.1256/smsqj.49708

CrossRef Full Text | Google Scholar

Näger, C., Storck, J., and Deco, G. (2002). Speech recognition with spiking neurons and dynamic synapses: a model motivated by the human auditory pathway. Neurocomputing 44–46, 937–942. doi: 10.1016/S0925-2312(02)00494-0

CrossRef Full Text | Google Scholar

Narusuye, K., Kawai, F., and ichi Miyachi, E. (2003). Spike encoding of olfactory receptor cells. Neurosci. Res. 46, 407–413. doi: 10.1016/S0168-0102(03)00131-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Nawrocki, R. A., Voyles, R. M., and Shaheen, S. E. (2016). A mini review of neuromorphic architectures and implementations. IEEE Trans. Electron Dev. 63, 3819–3829. doi: 10.1109/TED.2016.2598413

CrossRef Full Text | Google Scholar

Negri, P. (2017). “Extended LBP operator to characterize event-address representation connectivity,” in Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, eds C. Beltrán-Castañón, I. Nyström, and F. Famili (Cham: Springer International Publishing), 241–248. doi: 10.1007/978-3-319-52277-7_30

CrossRef Full Text | Google Scholar

Negri, P., Serrano-Gotarredona, T., and Linares-Barranco, B. (2018). “Spiking hough for shape recognition,” in Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications, eds M. Mendoza and S. Velastín (Cham: Springer International Publishing), 425–432. doi: 10.1007/978-3-319-75193-1_51

CrossRef Full Text | Google Scholar

Neil, D., and Liu, S. (2016). “Effective sensor fusion with event-based sensors and deep network architectures,” in 2016 IEEE International Symposium on Circuits and Systems (ISCAS), 2282–2285. doi: 10.1109/ISCAS.2016.7539039

CrossRef Full Text | Google Scholar

Neri, N., Abba, A., Caponio, F., Citterio, M., Coelli, S., Fu, J., et al. (2015). “First results of a detector embedded real-time tracking system with artificial retina,” in 2015 IEEE Nuclear Science Symposium and Medical Imaging Conference (NSS/MIC) (IEEE), 1–4. doi: 10.1109/NSSMIC.2015.7581772

CrossRef Full Text | Google Scholar

Neri, N., Abba, A., Caponio, F., Citterio, M., Coelli, S., Fu, J., et al. (2017). Testbeam results of the first real-time embedded tracking system with artificial retina. Nucl. Instr. Methods Phys. Res. Sect. A Acceler. Spectrom. Detect. Assoc. Equip. 845, 607–611. doi: 10.1016/j.nima.2016.05.129

CrossRef Full Text | Google Scholar

Ng, K. T., Boussaid, F., and Bermak, A. (2011). A CMOS single-chip gas recognition circuit for metal oxide gas sensor arrays. IEEE Trans. Circuits Syst. I Reg. Pap. 58, 1569–1580. doi: 10.1109/TCSI.2011.2143090

CrossRef Full Text | Google Scholar

Ni, Z., Bolopion, A., Agnus, J., Benosman, R., and Regnier, S. (2012). Asynchronous event-based visual shape tracking for stable haptic feedback in microrobotics. IEEE Trans. Robot. 28, 1081–1089. doi: 10.1109/TRO.2012.2198930

CrossRef Full Text | Google Scholar

Ni, Z., Ieng, S. H., Posch, C., Régnier, S., and Benosman, R. (2015). Visual tracking using neuromorphic asynchronous event-based cameras. Neural Comput. 27, 925–953. doi: 10.1162/NECO_a_00720

PubMed Abstract | CrossRef Full Text | Google Scholar

Ni, Z., Pacoret, C., Benosman, R., Ieng, S., and Régnier, S. (2012). Asynchronous event-based high speed vision for microparticle tracking. J. Microsc. 245, 236–244. doi: 10.1111/j.1365-2818.2011.03565.x

CrossRef Full Text | Google Scholar

O'Connor, P., Neil, D., Liu, S. C., Delbruck, T., and Pfeiffer, M. (2013). Real-time classification and sensor fusion with a spiking deep belief network. Front. Neurosci. 7:178. doi: 10.3389/fnins.2013.00178

PubMed Abstract | CrossRef Full Text | Google Scholar

Orchard, G., Bartolozzi, C., and Indiveri, G. (2009). “Applying neuromorphic vision sensors to planetary landing tasks,” in 2009 IEEE Biomedical Circuits and Systems Conference, 201–204. doi: 10.1109/BIOCAS.2009.5372048

CrossRef Full Text | Google Scholar

Orchard, G., Meyer, C., Etienne-Cummings, R., Posch, C., Thakor, N., and Benosman, R. (2015). Hfirst: a temporal approach to object recognition. IEEE Trans. Pattern Anal. Mach. Intell. 37, 2028–2040. doi: 10.1109/TPAMI.2015.2392947

PubMed Abstract | CrossRef Full Text | Google Scholar

Osswald, M., Benosman, S. H., Benosman, R., and Indiveri, G. (2017). A spiking neural network model of 3D perception for event-based neuromorphic stereo vision systems. Sci. Rep. 7:40703. doi: 10.1038/srep40703

PubMed Abstract | CrossRef Full Text | Google Scholar

Padala, V., Basu, A., and Orchard, G. (2018). A noise filtering algorithm for event-based asynchronous change detection image sensors on truenorth and its implementation on truenorth. Front. Neurosci. 12:118. doi: 10.3389/fnins.2018.00118

PubMed Abstract | CrossRef Full Text | Google Scholar

Pan, L., Scheerlinck, C., Yu, X., Hartley, R., Liu, M., and Dai, Y. (2018). Bringing a blurry frame alive at high frame-rate with an event camera. CoRR abs/1811.10180. doi: 10.1109/CVPR.2019.00698

CrossRef Full Text | Google Scholar

Park, P. K. J., Ryu, H., Lee, J. H., Shin, C., Lee, K. B., Woo, J., et al. (2013). “Fast neuromorphic localization for binaural hearing aids,” in 2013 35th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), 5275–5278. doi: 10.1109/EMBC.2013.6610739

PubMed Abstract | CrossRef Full Text | Google Scholar

Pearce, T., Karout, S., Capurro, A., Rácz, Z., Gardner, J., and Cole, M. (2013). Rapid processing of chemosensor transients in a neuromorphic implementation of the insect macroglomerular complex. Front. Neurosci. 7:119. doi: 10.3389/fnins.2013.00119

PubMed Abstract | CrossRef Full Text | Google Scholar

Pearce, T., Schiffman, S., Nagle, T., and Gardner, J. W. (Eds.). (2003). Handbook of Machine Olfaction. Weinheim: Wiley-VCH. doi: 10.1002/3527601597

CrossRef Full Text | Google Scholar

Pearce, T. C., Fulvi-Mari, C., Covington, J. A., Tan, F. S., Gardner, J. W., Koickal, T. J., et al. (2005). “Silicon-based neuromorphic implementation of the olfactory pathway,” in Conference Proceedings, 2nd International IEEE EMBS Conference on Neural Engineering 2005, 307–312. doi: 10.1109/CNE.2005.1419619

CrossRef Full Text | Google Scholar

Pearce, T. C., Karout, S., Rácz, Z., Capurro, A., Gardner, J. W., and Cole, M. (2014). Rapid processing of chemosensor transients in a neuromorphic implementation of the insect macroglomerular complex. Flavour 3:P14. doi: 10.1186/2044-7248-3-S1-P14

PubMed Abstract | CrossRef Full Text | Google Scholar

Peng, X., Zhao, B., Yan, R., Tang, H., and Yi, Z. (2017). Bag of events: an efficient probability-based feature extraction method for AER image sensors. IEEE Trans. Neural Netw. Learn. Syst. 28, 791–803. doi: 10.1109/TNNLS.2016.2536741

PubMed Abstract | CrossRef Full Text | Google Scholar

Pérez-Carrasco, J. A., Acha, B., Serrano, C., Camuñas-Mesa, L., Serrano-Gotarredona, T., and Linares-Barranco, B. (2010). Fast vision through frameless event-based sensing and convolutional processing: Application to texture recognition. IEEE Trans. Neural Netw. 21, 609–620. doi: 10.1109/TNN.2009.2039943

PubMed Abstract | CrossRef Full Text | Google Scholar

Pérez-Carrasco, J. A., Zhao, B., Serrano, C., Acha, B., Serrano-Gotarredona, T., Chen, S., et al. (2013). Mapping from frame-driven to frame-free event-driven vision systems by low-rate rate coding and coincidence processing-application to feedforward convnets. IEEE Trans. Pattern Anal. Mach. Intell. 35, 2706–2719. doi: 10.1109/TPAMI.2013.71

PubMed Abstract | CrossRef Full Text | Google Scholar

Perez-Peña, F., Estevez, A. M., Montero-Gonzalez, R. J., Linares-Barranco, A., and Jiménez-Moreno, G. (2011). “Video surveillance at an industrial environment using an address event vision sensor: Comparative between two different video sensor based on a bioinspired retina,” in Proceedings of the International Conference on Signal Processing and Multimedia Applications, 1–4.

Google Scholar

Perez-Peña, F., Morgado-Estevez, A., Linares-Barranco, A., Jimenez-Fernandez, A., Gomez-Rodriguez, F., Jimenez-Moreno, G., et al. (2013). Neuro-inspired spike-based motion: from dynamic vision sensor to robot motor open-loop control through spike-vite. Sensors 13, 15805–15832. doi: 10.3390/s131115805

PubMed Abstract | CrossRef Full Text | Google Scholar

Piatkowska, E., Belbachir, A. N., and Gelautz, M. (2014). Cooperative and asynchronous stereo vision for dynamic vision sensors. Meas. Sci. Technol. 25:055108. doi: 10.1088/0957-0233/25/5/055108

CrossRef Full Text | Google Scholar

Piatkowska, E., Kogler, J., Belbachir, N., and Gelautz, M. (2017). “Improved cooperative stereo matching for dynamic vision sensors with ground truth evaluation,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. doi: 10.1109/CVPRW.2017.51

CrossRef Full Text | Google Scholar

Piatkowska, E., Nabil Belbachir, A., and Gelautz, M. (2013). “Asynchronous stereo vision for event-driven dynamic stereo sensor using an adaptive cooperative approach,” in The IEEE International Conference on Computer Vision (ICCV) Workshops. doi: 10.1109/ICCVW.2013.13

CrossRef Full Text | Google Scholar

Pikatkowska, E., Belbachir, A. N., Schraml, S., and Gelautz, M. (2012). “Spatiotemporal multiple persons tracking using dynamic vision sensor,” in 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (IEEE), 35–40. doi: 10.1109/CVPRW.2012.6238892

CrossRef Full Text | Google Scholar

Posch, C., Matolin, D., and Wohlgenannt, R. (2011). A QVGA 143 dB dynamic range frame-free pwm image sensor with lossless pixel-level video compression and time-domain CDS. IEEE J. Solid State Circuits 46, 259–275. doi: 10.1109/JSSC.2010.2085952

CrossRef Full Text | Google Scholar

Posch, C., Serrano-Gotarredona, T., Linares-Barranco, B., and Delbruck, T. (2014). Retinomorphic event-based vision sensors: bioinspired cameras with spiking output. Proc. IEEE 102, 1470–1484. doi: 10.1109/JPROC.2014.2346153

CrossRef Full Text | Google Scholar

Raman, B., Sun, P. A., Gutierrez-Galvez, A., and Gutierrez-Osuna, R. (2006a). Processing of chemical sensor arrays with a biologically inspired model of olfactory coding. IEEE Trans. Neural Netw. 17, 1015–1024. doi: 10.1109/TNN.2006.875975

PubMed Abstract | CrossRef Full Text | Google Scholar

Raman, B., Yamanaka, T., and Gutierrez-Osuna, R. (2006b). Contrast enhancement of gas sensor array patterns with a neurodynamics model of the olfactory bulb. Sens. Actuators B Chem. 119, 547–555. doi: 10.1016/j.snb.2006.01.035

CrossRef Full Text | Google Scholar

Rea, F., Metta, G., and Bartolozzi, C. (2013). Event-driven visual attention for the humanoid robot icub. Front. Neurosci. 7:234. doi: 10.3389/fnins.2013.00234

PubMed Abstract | CrossRef Full Text | Google Scholar

Rebecq, H., Gallego, G., Mueggler, E., and Scaramuzza, D. (2018). EMVS: event-based multi-view stereo–3D reconstruction with an event camera in real-time. Int. J. Comput. Vision 126, 1394–1414. doi: 10.1007/s11263-017-1050-6

CrossRef Full Text | Google Scholar

Rebecq, H., Gallego, G., and Scaramuzza, D. (2016). “EMVS: event-based multi-view stereo,” in British Machine Vision Conference (BMVC) (York), 1–111.

Google Scholar

Rebecq, H., Horstschaefer, T., and Scaramuzza, D. (2017a). “Real-time visual-inertial odometry for event cameras using keyframe-based nonlinear optimization,” in British Machine Vision Conference (BMVC), Vol. 3. doi: 10.5244/C.31.16

CrossRef Full Text | Google Scholar

Rebecq, H., Horstschäfer, T., Gallego, G., and Scaramuzza, D. (2017b). EVO: a geometric approach to event-based 6-DoF parallel tracking and mapping in real time. IEEE Robot. Autom. Lett. 2, 593–600. doi: 10.1109/LRA.2016.2645143

CrossRef Full Text | Google Scholar

Reinbacher, C., Munda, G., and Pock, T. (2017). “Real-time panoramic tracking for event cameras,” in 2017 IEEE International Conference on Computational Photography (ICCP) (IEEE), 1–9. doi: 10.1109/ICCPHOT.2017.7951488

CrossRef Full Text | Google Scholar

Reinhard, E., Heidrich, W., Debevec, P., Pattanaik, S., Ward, G., and Myszkowski, K. (2010). High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting. Morgan Kaufmann.

Google Scholar

Reverter Valeiras, D., Orchard, G., Ieng, S. H., and Benosman, R. B. (2016). Neuromorphic event-based 3D pose estimation. Front. Neurosci. 9:522. doi: 10.3389/fnins.2015.00522

PubMed Abstract | CrossRef Full Text | Google Scholar

Ridwan, I., and Cheng, H. (2017). “An event-based optical flow algorithm for dynamic vision sensors,” in International Conference Image Analysis and Recognition (Springer), 182–189. doi: 10.1007/978-3-319-59876-5_21

CrossRef Full Text | Google Scholar

Riesenhuber, M., and Poggio, T. (1999). Hierarchical models of object recognition in cortex. Nat. Neurosci. 2:1019. doi: 10.1038/14819

PubMed Abstract | CrossRef Full Text | Google Scholar

Riesenhuber, M., and Poggio, T. (2000a). Computational Models of Object Recognition in Cortex: A Review. Technical report, Massachusetts Institute of Technology, Cambridge, Artificial Intelligence Lab.

Google Scholar

Riesenhuber, M., and Poggio, T. (2000b). Models of object recognition. Nat. Neurosci. 3:1199. doi: 10.21236/ADA458109

CrossRef Full Text | Google Scholar

Rigi, A., Baghaei Naeini, F., Makris, D., and Zweiri, Y. (2018). A novel event-based incipient slip detection using dynamic active-pixel vision sensor (DAVIS). Sensors 18, 333. doi: 10.3390/s18020333

PubMed Abstract | CrossRef Full Text | Google Scholar

Rios-Navarro, A., Cerezuela-Escudero, E., Dominguez-Morales, M., Jimenez-Fernandez, A., Jimenez-Moreno, G., and Linares-Barranco, A. (2015). “Real-time motor rotation frequency detection with event-based visual and spike-based auditory aer sensory integration for FPGA,” in 2015 International Conference on Event-based Control, Communication, and Signal Processing (EBCCSP), 1–6. doi: 10.1109/EBCCSP.2015.7300696

CrossRef Full Text | Google Scholar

Rochel, O., Martinez, D., Hugues, E., and Sarry, F. (2002). “Stereo-olfaction with a sniffing neuromorphic robot using spiking neurons,” in 16th European Conference on Solid-State Transducers–EUROSENSORS (Prague: Colloque Avec Actes et Comité de Lecture Internationale), 4.

Google Scholar

Rogister, P., Benosman, R., Ieng, S. H., Lichtsteiner, P., and Delbruck, T. (2012). Asynchronous event-based binocular stereo matching. IEEE Trans. Neural Netw. Learn. Syst. 23, 347–353. doi: 10.1109/TNNLS.2011.2180025

PubMed Abstract | CrossRef Full Text | Google Scholar

Rueckauer, B., and Delbruck, T. (2016). Evaluation of event-based algorithms for optical flow with ground-truth from inertial measurement sensor. Front. Neurosci. 10:176. doi: 10.3389/fnins.2016.00176

PubMed Abstract | CrossRef Full Text | Google Scholar

Ruedi, P. F. (1996). “Motion detection silicon retina based on event correlations,” in Proceedings of Fifth International Conference on Microelectronics for Neural Networks (IEEE), 23–29. doi: 10.1109/MNNFS.1996.493768

CrossRef Full Text | Google Scholar

Saner, D., Wang, O., Heinzle, S., Pritch, Y., Smolic, A., Sorkine-Hornung, A., et al. (2014). “High-speed object tracking using an asynchronous temporal contrast sensor,” in VMV, 87–94.

Google Scholar

Schmuker, M., Bahr, V., and Huerta, R. (2016). Exploiting plume structure to decode gas source distance using metal-oxide gas sensors. Sens. Actuators B Chem. 235, 636–646. doi: 10.1016/j.snb.2016.05.098

CrossRef Full Text | Google Scholar

Schmuker, M., Häusler, C., Brüderle, D., and Nawrot, M. P. (2011). Benchmarking the impact of information processing in the insect olfactory system with a spiking neuromorphic classifier. BMC Neurosci. 12:P233. doi: 10.1186/1471-2202-12-S1-P233

CrossRef Full Text | Google Scholar

Schmuker, M., Pfeil, T., and Nawrot, M. P. (2014). A neuromorphic network for generic multivariate data classification. Proc. Natl. Acad. Sci. U.S.A. 111, 2081–2086. doi: 10.1073/pnas.1303053111

PubMed Abstract | CrossRef Full Text | Google Scholar

Schraml, S., and Belbachir, A. N. (2010). “A spatio-temporal clustering method using real-time motion analysis on event-based 3D vision,” in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops (IEEE), 57–63. doi: 10.1109/CVPRW.2010.5543810

CrossRef Full Text | Google Scholar

Schraml, S., Belbachir, A. N., and Brändle, N. (2010a). “A real-time pedestrian classification method for event-based dynamic stereo vision,” in 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops (IEEE), 93–99. doi: 10.1109/CVPRW.2010.5543775

CrossRef Full Text | Google Scholar

Schraml, S., Belbachir, A. N., Milosevic, N., and Schön, P. (2010b). “Dynamic stereo vision system for real-time tracking,” in Proceedings of 2010 IEEE International Symposium on Circuits and Systems (IEEE), 1409–1412. doi: 10.1109/ISCAS.2010.5537289

CrossRef Full Text | Google Scholar

Schraml, S., Nabil Belbachir, A., and Bischof, H. (2015). “Event-driven stereo matching for real-time 3D panoramic vision,” in The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). doi: 10.1109/CVPR.2015.7298644

CrossRef Full Text | Google Scholar

Schuman, C. D., Potok, T. E., Patton, R. M., Birdwell, J. D., Dean, M. E., Rose, G. S., et al. (2017). A survey of neuromorphic computing and neural networks in hardware. CoRR abs/1705.06963.

Google Scholar

Seifozzakerini, S., Yau, W. Y., and Mao, K. (2017). “Effect of inhibitory window on event-based hough transform for multiple lines detection,” in Proceedings of the International Conference on Advances in Image Processing (ACM), 39–44. doi: 10.1145/3133264.3133286

CrossRef Full Text | Google Scholar

Seifozzakerini, S., Yau, W. Y., Zhao, B., and Mao, K. (2016). “Event-based hough transform in a spiking neural network for multiple line detection and tracking using a dynamic vision sensor,” in BMVC. doi: 10.5244/C.30.94

CrossRef Full Text | Google Scholar

Serrano-Gotarredona, R., Oster, M., Lichtsteiner, P., Linares-Barranco, A., Paz-Vicente, R., Gómez-Rodríguez, F., et al. (2009). Caviar: A 45K neuron 5M synapse 12G connects/s AER hardware sensory-processing-learning-actuating system for high-speed visual object recognition and tracking. IEEE Trans. Neural Netw. 20, 1417–1438. doi: 10.1109/TNN.2009.2023653

PubMed Abstract | CrossRef Full Text | Google Scholar

Serrano-Gotarredona, R., Serrano-Gotarredona, T., Acosta-Jimenez, A., and Linares-Barranco, B. (2006). A neuromorphic cortical-layer microchip for spike-based event processing vision systems. IEEE Trans. Circuits Syst. I Reg. Pap. 53, 2548–2566. doi: 10.1109/TCSI.2006.883843

CrossRef Full Text | Google Scholar

Serrano-Gotarredona, T., and Linares-Barranco, B. (2015). POKER-DVS and MNIST-DVS. Their history, how they were made, and other details. Front. Neurosci. 9:481. doi: 10.3389/fnins.2015.00481

PubMed Abstract | CrossRef Full Text | Google Scholar

Serrano-Gotarredona, T., Park, J., Linares-Barranco, A., Jiménez, A., Benosman, R., and Linares-Barranco, B. (2013). “Improved contrast sensitivity dvs and its application to event-driven stereo vision,” in 2013 IEEE International Symposium on Circuits and Systems (ISCAS2013), 2420–2423. doi: 10.1109/ISCAS.2013.6572367

CrossRef Full Text | Google Scholar

Serre, T., Wolf, L., Bileschi, S., Riesenhuber, M., and Poggio, T. (2007). Robust object recognition with cortex-like mechanisms. IEEE Trans. Pattern Anal. Mach. Intel. 29, 411–426. doi: 10.1109/TPAMI.2007.56

CrossRef Full Text | Google Scholar

Serres, J., Raharijaona, T., Vanhoutte, E., and Ruffier, F. (2016). “Event-based visual guidance inspired by honeybees in a 3D tapered tunnel,” in 2016 Second International Conference on Event-Based Control, Communication, and Signal Processing (EBCCSP), 1–4. doi: 10.1109/EBCCSP.2016.7605273

CrossRef Full Text | Google Scholar

Singh, P., Yong, S. Z., Gregoire, J., Censi, A., and Frazzoli, E. (2016). “Stabilization of linear continuous-time systems using neuromorphic vision sensors,” in 2016 IEEE 55th Conference on Decision and Control (CDC) (IEEE), 3030–3036. doi: 10.1109/CDC.2016.7798722

CrossRef Full Text | Google Scholar

Sironi, A., Brambilla, M., Bourdis, N., Lagorce, X., and Benosman, R. (2018). “HATS: histograms of averaged time surfaces for robust event-based object classification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 1731–1740. doi: 10.1109/CVPR.2018.00186

CrossRef Full Text | Google Scholar

Sonnleithner, D., and Indiveri, G. (2011a). “Active vision driven by a neuromorphic selective attention system,” in Proceedings of International Symposium on Autonomous Minirobots for Research and Edutainment, AMiRE, Vol. 2011 (Citeseer), 1–10.

Google Scholar

Sonnleithner, D., and Indiveri, G. (2011b). “A neuromorphic saliency-map based active vision system,” in 2011 45th Annual Conference on Information Sciences and Systems, 1–6. doi: 10.1109/CISS.2011.5766145

CrossRef Full Text | Google Scholar

Sonnleithner, D., and Indiveri, G. (2012). “A real-time event-based selective attention system for active vision,” in Advances in Autonomous Mini Robots, eds U. Rückert, S. Joaquin, and W. Felix (Berlin; Heidelberg: Springer Berlin Heidelberg), 205–219. doi: 10.1007/978-3-642-27482-4_21

CrossRef Full Text | Google Scholar

Stromatias, E., Soto, M., Serrano-Gotarredona, T., and Linares-Barranco, B. (2017). An event-driven classifier for spiking neural networks fed with synthetic or dynamic vision sensor data. Front. Neurosci. 11:350. doi: 10.3389/fnins.2017.00350

PubMed Abstract | CrossRef Full Text | Google Scholar

Sullivan, K., and Lawson, W. (2017). “Representing motion information from event-based cameras,” in 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN) (IEEE), 1465–1470. doi: 10.1109/ROMAN.2017.8172497

CrossRef Full Text | Google Scholar

Sulzbachner, C., and Kogler, J. (2010). “A load balancing approach for silicon retina based asynchronous temporal data processing,” in 2010 36th EUROMICRO Conference on Software Engineering and Advanced Applications, 431–435. doi: 10.1109/SEAA.2010.12

CrossRef Full Text | Google Scholar

Sulzbachner, C., Kogler, J., and Eibensteiner, F. (2010). “A novel verification approach for silicon retina stereo matching algorithms,” in Proceedings ELMAR-2010, 467–470.

Google Scholar

Surovich, M., Shrestha, S., Douard, N., Giakos, Z., and Giakos, G. (2017). “Dynamic segmentation using a novel neuromorphic polarimetric imaging system,” in 2017 IEEE International Conference on Imaging Systems and Techniques (IST), 1–6. doi: 10.1109/IST.2017.8261439

CrossRef Full Text | Google Scholar

Szyszka, P., Gerkin, R. C., Galizia, C. G., and Smith, B. H. (2014). High-speed odor transduction and pulse tracking by insect olfactory receptor neurons. Proc. Natl. Acad. Sci. U.S.A. 111, 16925–16930. doi: 10.1073/pnas.1412051111

PubMed Abstract | CrossRef Full Text | Google Scholar

Tan, C., Lallee, S., and Orchard, G. (2015). Benchmarking neuromorphic vision: lessons learnt from computer vision. Front. Neurosci. 9:374. doi: 10.3389/fnins.2015.00374

PubMed Abstract | CrossRef Full Text | Google Scholar

Taverni, G., Moeys, D. P., Voigt, F. F., Li, C., Cavaco, C., Motsnyi, V., et al. (2017). “In-vivo imaging of neural activity with dynamic vision sensors,” in 2017 IEEE Biomedical Circuits and Systems Conference (BioCAS), 1–4. doi: 10.1109/BIOCAS.2017.8325076

CrossRef Full Text | Google Scholar

Tedaldi, D., Gallego, G., Mueggler, E., and Scaramuzza, D. (2016). “Feature detection and tracking with the dynamic and active-pixel vision sensor (DAVIS),” in 2016 Second International Conference on Event-Based Control, Communication, and Signal Processing (EBCCSP), 1–7. doi: 10.1109/EBCCSP.2016.7605086

CrossRef Full Text | Google Scholar

Teixeira, T., Culurciello, E., Park, J. H., Lymberopoulos, D., Barton-Sweeney, A., and Savvides, A. (2006). “Address-event imagers for sensor networks: evaluation and modeling,” in 2006 5th International Conference on Information Processing in Sensor Networks, 458–466. doi: 10.1145/1127777.1127847

CrossRef Full Text | Google Scholar

Thakur, C. S., Molin, J., and Etienne-Cummings, R. (2017). “Real-time image segmentation using a spiking neuromorphic processor,” in 2017 51st Annual Conference on Information Sciences and Systems (CISS), 1–6. doi: 10.1109/CISS.2017.7926171

CrossRef Full Text | Google Scholar

Tschechne, S., Sailer, R., and Neumann, H. (2014). “Bio-inspired optic flow from event-based neuromorphic sensor input,” in Artificial Neural Networks in Pattern Recognition, eds N. El Gayar, F. Schwenker, and C. Suen (Cham: Springer International Publishing), 171–182. doi: 10.1007/978-3-319-11656-3_16

CrossRef Full Text | Google Scholar

Tsitiridis, A., Conde, C., de Diego, I. M., del Rio Saez, J. S., Gomez, J. R., and Cabello, E. (2015). “Gabor feature processing in spiking neural networks from retina-inspired data,” in 2015 International Joint Conference on Neural Networks (IJCNN), 1–8. doi: 10.1109/IJCNN.2015.7280352

CrossRef Full Text | Google Scholar

van Schaik, A., Chan, V., and Jin, C. (2009). “Sound localisation with a silicon cochlea pair,” in 2009 IEEE International Conference on Acoustics, Speech and Signal Processing, 2197–2200. doi: 10.1109/ICASSP.2009.4960054

CrossRef Full Text | Google Scholar

Vanarse, A., Osseiran, A., and Rassau, A. (2016). A review of current neuromorphic approaches for vision, auditory, and olfactory sensors. Front. Neurosci. 10:115. doi: 10.3389/fnins.2016.00115

PubMed Abstract | CrossRef Full Text | Google Scholar

Vanarse, A., Osseiran, A., and Rassau, A. (2017). An investigation into spike-based neuromorphic approaches for artificial olfactory systems. Sensors 17:2591. doi: 10.3390/s17112591

PubMed Abstract | CrossRef Full Text | Google Scholar

Vasco, V., Glover, A., and Bartolozzi, C. (2016). “Fast event-based harris corner detection exploiting the advantages of event-driven cameras,” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE), 4144–4149. doi: 10.1109/IROS.2016.7759610

CrossRef Full Text | Google Scholar

Vasyutynskyy, V., and Kabitzsch, K. (2010). “Event-based control: overview and generic model,” in 2010 IEEE International Workshop on Factory Communication Systems Proceedings, 271–279. doi: 10.1109/WFCS.2010.5548623

CrossRef Full Text | Google Scholar

Vogelstein, R. J., Mallik, U., Culurciello, E., Cauwenberghs, G., and Etienne-Cummings, R. (2007). A multichip neuromorphic system for spike-based visual information processing. Neural Comput. 19, 2281–2300. doi: 10.1162/neco.2007.19.9.2281

PubMed Abstract | CrossRef Full Text | Google Scholar

Wang, S., Koickal, T. J., Enemali, G., Gouveia, L., and Hamilton, A. (2015). “Design of a silicon cochlea system with biologically faithful response,” in 2015 International Joint Conference on Neural Networks (IJCNN), 1–7. doi: 10.1109/IJCNN.2015.7280828

CrossRef Full Text | Google Scholar

Weikersdorfer, D., Adrian, D. B., Cremers, D., and Conradt, J. (2014). “Event-based 3D slam with a depth-augmented dynamic vision sensor,” in 2014 IEEE International Conference on Robotics and Automation (ICRA), 359–364. doi: 10.1109/ICRA.2014.6906882

CrossRef Full Text | Google Scholar

Weikersdorfer, D., and Conradt, J. (2012). “Event-based particle filtering for robot self-localization,” in 2012 IEEE International Conference on Robotics and Biomimetics (ROBIO) (IEEE), 866–870. doi: 10.1109/ROBIO.2012.6491077

CrossRef Full Text | Google Scholar

Weikersdorfer, D., Hoffmann, R., and Conradt, J. (2013). “Simultaneous localization and mapping for event-based vision systems,” in International Conference on Computer Vision Systems (Springer), 133–142. doi: 10.1007/978-3-642-39402-7_14

CrossRef Full Text | Google Scholar

Wen, B., and Boahen, K. (2009). A silicon cochlea with active coupling. IEEE Trans. Biomed. Circuits Syst. 3, 444–455. doi: 10.1109/TBCAS.2009.2027127

CrossRef Full Text | Google Scholar

White, J., Dickinson, T. A., Walt, D. R., and Kauer, J. S. (1998). An olfactory neuronal network for vapor recognition in an artificial nose. Biol. Cybernet. 78, 245–251. doi: 10.1007/s004220050430

PubMed Abstract | CrossRef Full Text | Google Scholar

White, J., and Kauer, J. S. (1999). Odor recognition in an artificial nose by spatio-temporal processing using an olfactory neuronal network. Neurocomputing 26–27, 919–924. doi: 10.1016/S0925-2312(98)00137-4

CrossRef Full Text | Google Scholar

Xie, X., Du, J., Shi, G., Yang, J., Liu, W., and Li, W. (2018). “DVS image noise removal using k-SVD method,” in Ninth International Conference on Graphic and Image Processing, Vol. 10615. doi: 10.1117/12.2305260

CrossRef Full Text | Google Scholar

Yang, M. (2015). Silicon retina and cochlea with asynchronous delta modulator for spike encoding (Ph.D. thesis), ETH Zurich, Zurich, Switzerland.

Google Scholar

Yousefzadeh, A., Orchard, G., Serrano-Gotarredona, T., and Linares-Barranco, B. (2018). Active perception with dynamic vision sensors. Minimum saccades with optimum recognition. IEEE Trans. Biomed. Circuits Syst. 12, 927–939. doi: 10.1109/TBCAS.2018.2834428

PubMed Abstract | CrossRef Full Text | Google Scholar

Yousefzadeh, A., Serrano-Gotarredona, T., and Linares-Barranco, B. (2015). “Fast pipeline 128 × 128 pixel spiking convolution core for event-driven vision processing in FPGAS,” in 2015 International Conference on Event-Based Control, Communication, and Signal Processing (EBCCSP), 1–8. doi: 10.1109/EBCCSP.2015.7300698

CrossRef Full Text | Google Scholar

Yuan, W., and Ramalingam, S. (2016). “Fast localization and tracking using event sensors,” in 2016 IEEE International Conference on Robotics and Automation (ICRA) (IEEE), 4564–4571. doi: 10.1109/ICRA.2016.7487657

CrossRef Full Text | Google Scholar

Yue-Sek Chan, V., T. Jin, C., and van Schaik, A. (2010). Adaptive sound localization with a silicon cochlea pair. Front. Neurosci. 4:196. doi: 10.3389/fnins.2010.00196

PubMed Abstract | CrossRef Full Text | Google Scholar

Zheng, Y., Cao, Y., and Chang, C. H. (2016). “A new event-driven dynamic vision sensor based physical unclonable function for camera authentication in reactive monitoring system,” in Hardware-Oriented Security and Trust (AsianHOST), IEEE Asian (IEEE), 1–6. doi: 10.1109/AsianHOST.2016.7835551

CrossRef Full Text | Google Scholar

Zhou, Y., Gallego, G., Rebecq, H., Kneip, L., Li, H., and Scaramuzza, D. (2018). “Semi-dense 3D reconstruction with a stereo event camera,” in The European Conference on Computer Vision (ECCV). doi: 10.1007/978-3-030-01246-5_15

CrossRef Full Text | Google Scholar

Zhu, A. Z., Atanasov, N., and Daniilidis, K. (2017a). “Event-based feature tracking with probabilistic data association,” in 2017 IEEE International Conference on Robotics and Automation (ICRA), 4465–4470. doi: 10.1109/ICRA.2017.7989517

CrossRef Full Text | Google Scholar

Zhu, A. Z., Atanasov, N., and Daniilidis, K. (2017b). “Event-based visual inertial odometry,” in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (IEEE), 5816–5824. doi: 10.1109/CVPR.2017.616

CrossRef Full Text | Google Scholar

Zhu, A. Z., Thakur, D., Ozaslan, T., Pfrommer, B., Kumar, V., and Daniilidis, K. (2018). The multi vehicle stereo event camera dataset: an event camera dataset for 3D perception. IEEE Robot. Autom. Lett. 3, 2032–2039. doi: 10.1109/LRA.2018.2800793

CrossRef Full Text | Google Scholar

Zihao Zhu, A., Chen, Y., and Daniilidis, K. (2018). “Realtime time synchronized event-based stereo,” in The European Conference on Computer Vision (ECCV). doi: 10.1007/978-3-030-01231-1_27

CrossRef Full Text | Google Scholar

Zong, X., Xiao, P., and Wen, S. (2018). “An event camera tracking based on MLS surface fitting algorithm,” in 2018 Chinese Control and Decision Conference (CCDC) (IEEE), 5001–5005. doi: 10.1109/CCDC.2018.8407998

CrossRef Full Text | Google Scholar

Zou, D., Guo, P., Wang, Q., Wang, X., Shao, G., Shi, F., et al. (2016). “Context-aware event-driven stereo matching,” in 2016 IEEE International Conference on Image Processing (ICIP), 1076–1080. doi: 10.1109/ICIP.2016.7532523

CrossRef Full Text | Google Scholar