A memristive spiking neuron with firing rate coding

Perception, decisions, and sensations are all encoded into trains of action potentials in the brain. The relation between stimulus strength and all-or-nothing spiking of neurons is widely believed to be the basis of this coding. This initiated the development of spiking neuron models; one of today's most powerful conceptual tool for the analysis and emulation of neural dynamics. The success of electronic circuit models and their physical realization within silicon field-effect transistor circuits lead to elegant technical approaches. Recently, the spectrum of electronic devices for neural computing has been extended by memristive devices, mainly used to emulate static synaptic functionality. Their capabilities for emulations of neural activity were recently demonstrated using a memristive neuristor circuit, while a memristive neuron circuit has so far been elusive. Here, a spiking neuron model is experimentally realized in a compact circuit comprising memristive and memcapacitive devices based on the strongly correlated electron material vanadium dioxide (VO₂) and on the chemical electromigration cell Ag/TiO_2−x/Al. The circuit can emulate dynamical spiking patterns in response to an external stimulus including adaptation, which is at the heart of firing rate coding as first observed by E.D. Adrian in 1926.

Introduction

In a brain the most prominent processing units are neurons. An archetypical neuron consists of dendrites (the input), soma (the processing unit), and axon (the output) as schematically shown in Figure 1A. Information between neurons is encoded into sequences of identical spikes or action potentials, which appear in spatial and irregular temporal patterns. It is widely believed that in neurons input stimuli are transacted into a firing rate of action potentials at the output. This so-called firing rate hypothesis was first recognized in 1926 by E.D. Adrian (Adrian, 1926, 1928) from the investigation of sensory neurons. Nowadays the firing rate hypothesis is generalized and considered as the basic encoding scheme of neurons in the primary visual cortex, somatosensory cortex, auditory cortex, place cells in the hippocampus and many other brain regions (Barlow, 1961; Laughlin, 1989; Maass and Bishop, 2001; Gerstner and Kistler, 2002; Natelson, 2013). An additional important (and already observed by E. D. Adrian in 1926) aspect of neural signal processing is that neurons only transiently sustain a (high) firing rate, even when the stimulus is permanently applied (Adrian, 1926, 1928). In other words, the transiently decaying signal is encoded via a variable action potential firing rate, as sketched in Figure 1B. Adrian interpreted his findings as a general concept of adaptation in all living species. The decreasing firing rate, while applying a timely constant stimulus, results in reduced sensation, or in other words the species adapts to the outer world (Mausfeld, 2013). In this way sensory adaptation enables the subtraction of spatial and temporal steady signals parts, which leads to a signal renormalization and facilitates the consecutive coding. We would like to emphasize that the firing rate code hypothesis is nowadays extended even to neurons far from the receptor neuron and goes beyond the sensory transduction concept. The concept of spiking neurons is applied to neurons in mammalian brains, since those neurons are directly involved in information processing (Bear et al., 2006; Shepherd and Grillner, 2010).

FIGURE 1

Figure 1. Response to a stimulation principle: (A) Schematic of a single neuron, which can be divided into three functional parts: Dendrites, collect signals from other neurons; cell body (soma), the central processing unit of a neuron; axon, neuronal output stage. (B) Relationship between firing rate of a neuron and the strength of input stimulation reflecting the response to a stimulation principle as proposed by E. D. Adrian in 1926 (Adrian, 1926, 1928; Maass and Bishop, 2001).

In more detail, spikes or action potentials are result from voltage controlled ionic currents in cell membranes and are short electrical pulses in the millisecond range with a peak-to-peak amplitude of about 100 mV. Hodgkin and Huxley described the spike generation for the first time, by using an electronic equivalent circuit model (Hodgkin and Huxley, 1952), which is essentially based on sodium and potassium ion channels and enables the description of electrophysiological properties relevant for the spike's line shape. However, information in neural networks is encoded by the number and timing of those spikes rather than by the spike shape. Hence, simple phenomenological spiking neuron models are of interest rather than detailed conductance-based neuron models (Gerstner and Kistler, 2002), because these kind of models already enable studying neuronal coding, memory, and network dynamics in a simple circuit model (Fitzhugh, 1955; Hindmarsh and Rose, 1984; Mead, 1989; Izhikevich, 2003).

Archetypical neurons may formally be divided into three functional parts, as sketched in Figure 1A: While dendrites serve as input stage of the neuron, collecting signals from other neurons, the cell body (soma) is the central processing unit of a neuron, which generates a spike whenever the membrane potential exceeds a certain threshold V_θ defined at the axon-hillock. Finally, the axon serves as an active transmission line for the generated spikes toward post-synaptic neurons (Bear et al., 2006). A straightforward description of a neuron is given by the integrate-and-fire model (Gerstner and Kistler, 2002), where a resistor R is connected in parallel with a capacitor C and they are driven by a current I(t), which can be expressed as

$\begin{array}{ll}{\tau }_m\frac{du\left(t\right)}{dt}=-u\left(t\right)+R(I,t)I\left(t\right),& \left(1\right)\hfill \end{array}$

where τ_m = RC is the time constant of the circuit and u(t) refers to the membrane potential. Moreover, a threshold electronic circuit is used, so that the neuron is firing whenever u(t) reaches a defined threshold voltage V_θ, whereafter the potential u(t) is reset to a reset potential u_r < V_θ (Gerstner and Kistler, 2002).

To realize compact, real-time, and energy efficient electronic neuron circuits, the analog complementary metal-oxide-semiconductor (CMOS) technology has been successfully employed (Indiveri et al., 2011). This approach goes back to the 80's of the last century, initiated by Carver Mead (Mead, 1989), where the integration of those circuits into very large-scale integration (VLSI) technology offers the possibility to build up real-time autonomous (cognitive) systems (Chicca et al., 2014). However, to further improve such bio-inspired circuits, non-volatile electronic, and/or ionic devices are required which improve the circuit design flexibility combined with a reduced circuit complexity. Memristive devices may fulfill these requirements. Although, theoretically predicted by Chua in 1971, researchers have just realized the useful functionalities of those devices for neural computing in the last couple of years. The ongoing research on memristive devices for neural systems mainly concentrates on the emulation of biological synapses and important synaptic functionalities (Hasegawa et al., 2010; Jo et al., 2010; Ohno et al., 2011; Zamarreño-Ramos et al., 2011; Jeong et al., 2013; Ziegler et al., 2015). In contrast, the entire neural functionality has been studied less (Pickett et al., 2013; Lim et al., 2015).

Mathematically, Equation (1) belongs to the class of van der Pol oscillators (van der Pol, 1926) if R is replaced by voltage-controlled or current-controlled devices with negative differential resistance (NDR). Recently, Pickett et al. (2013) showed that metal to insulator (MIT) phase transition materials are highly attractive for such circuits, since they allow to design inductor free circuits with the advantage of a reduced power consumption, scalability to the nanoscale, and integrability in complex neural network circuits (Lim et al., 2015). In particular, their spiking circuit is an experimental implementation of the neuristor (Crane, 1960) proposed by Hewitt Crane in 1960 and it allows emulation of some important neuronal functions, as for example the all-or-nothing spiking (Pickett et al., 2013; Lim et al., 2015). However, a neuristor cannot be considered as an electronic substitute of a neuron because a neuristor can only enable a subset of neuronal functions. In particular a neuronal coding scheme is missing, which represents the link between stimulus and response (Adrian, 1926; Chapleau, 2007), as sketched in Figure 1A.

In this work we combine the opportunities of memristive devices with phenomenological neuron circuit models to implement an analog memristive spiking neuron circuit. The circuit consists of: (1) a non-linear resistor exhibiting a NDR, (2) a memcapacitance sub-circuit derived from parallel/serial connected capacitors and a memristive device, as well as (3) a passive diode based output stage. We show that this circuit allows to realize a spiking neuronal coding scheme including firing frequency adaptation, where the amount and frequency of generated spikes are depend on the intensity and duration of an external current pulse, as well as on the number of generated spikes. Therefore, the circuit can be considered as a neuromorphic engineered version of the biological activity pattern (significance of a response) to an external stimulus principle, as first observed by A.D. Adrian in 1926 (Adrian, 1926, 1928). For the experimental realization of the neuron circuit the strongly correlated electron material VO₂ patterned in a lateral device structure is used. We show that this device exhibits a S-type shape NDR in a section of his I-V curve. Moreover, a memristive behavior of the neuron circuit is obtained using a memcapacitance, which has been experimentally realized using a capacitive divider with a memristive device in parallel to the second capacitor. Here, we used Ag-doped TiO_2−x as memristive device, which inherent stochastic nature additionally introduces stochastic noise to the neuron model.

The paper is organized as follows: In the Materials and Methods Section particular features of the applied materials are summarized and the film patterning techniques for device fabrication are explained. In the subsequent Section entitled Circuit Layout and Device Characterization the engagement of the individual circuit element to function as firing rate adapter is presented. The most prominent features of our circuit, i.e., fire rate coding, adaptation, and refractoriness, are summarized in the Section Results and Discussion followed by a Conclusion.

Materials and Methods

Device Fabrication

Memristive devices used for the memcapacitive circuit were fabricated from Ag/TiO_2−x /Al planar capacitor structures (a sketch of the material stack is shown in the inset of Figure 5A). The 45 nm Ag bottom electrode was deposited by thermal evaporation on thermally oxidized Si substrates. Standard optical lithography was used to define 50 × 50 μm windows. Afterwards, an 17.6 nm thick TiO_2−x layer was deposited by reactive sputtering, followed by the deposition of a 140 nm Al top electrode and a subsequent lift-off in acetone.

Vanadium dioxide (VO₂) devices were fabricated in a lateral Au/VO₂/Au geometry, as shown in Figure 3B. Therefore, VO₂ films were grown directly on single crystal TiO₂ substrates by Pulsed Laser Deposition (PLD) using a KrF excimer laser of 248 nm in wavelength (Kim and Kwoka, 1994; Petraru et al., 2014). A commercially available sintered ceramic V₂O₅ target was used. During deposition, the temperature of the substrate was kept at 380°C with an oxygen pressure of 1 × 10⁻² mbar in the PLD chamber. The energy density of the laser at the target was about 2.5 J/cm². After VO₂ thin film deposition, electrodes with a separation of 2–4 μm are defined by optical lithography, followed by the deposition of a 40 nm thick Au film and a subsequent lift-off in acetone.

Electrical Measurements

The neuron circuit was realized on a breadboard using commercially available metal-film resistors and polyester film capacitors with a tolerance of, respectively, 1 and 10%. The custom-made electronic devices are externally connected to the circuit board. Therefore, a Süss wafer prober was employed, where the individual memristive cells are electrically contacted through tungsten probe tips. The transient behavior of the neuron circuit was measured using a Tektronix TDS 7104 oscilloscope. As an input signal of the neuron circuit constant current pulses were applied. Therefore, an Agilent E5263A source measurement unit (SMU) was employed. The oscilloscope was used to record the voltage response of the circuit.

Current–voltage measurements (I–V curves) on single devices were obtained using an Agilent E5263A SMU by sweeping the applied current (voltage) and measuring the voltage (current) simultaneously. Additionally, a current (voltage) compliance was set in order to avoid a breakdown of the investigated device by electrical stress.

Circuit Layouts and Device Characterizations

Memristive Spiking Neuron Circuit

The here proposed memristive spiking neuron circuit is shown in Figure 2. The main parts of this circuit are an integrator circuit (in accordance to Equation 1) with a negative differential resistor (blue box in Figure 2) and a spike output branch (red box in Figure 2) used to generate the output spike v_out(t) from the oscillating voltage u(t). In particular, the negative differential resistor causes a breakdown of u(t), when its negative differential regime is reached. This results in an oscillation of u(t) during constant current input i(t). Furthermore, a memcapacitance C_M is used for the integrator circuit, which allows varying the frequency of oscillation in dependence of the charge flow history, i.e., the number of generated voltage spikes. Thus, C_M defines the memristive behavior of the proposed neuron circuit model. At the output stage of the neuron circuit the serial connection of D with R₁ and R₂ allow to emulate a desired line shape of the generated output voltage spikes v_out(t) across R₂ (cf. Figure 2), including a refractory period (labeled by t_ref in the inset of Figure 2). A constant voltage source V_B is added in series to the negative differential resistor which allows resetting the capacitance of C_M after the external current stimulus disappears. The inset of Figure 2 shows typical obtained voltage characteristics for v_out(t) and u(t) for a constant current input i(t) = i₀. In detail, the voltage source V_B causes a shift of u(t) to a negative base voltage, while the diode D builds with R₁ and R₂ a voltage divider which helds v_out(t) constant (defining t_ref) as long as u(t) is smaller than the build-in voltage of D (V_D = 0.7V). If u(t), however, exceeds V_D a spike v_out(t) is initiated with a spike width of ~2 ms, which is in accordance to biological spike times.

FIGURE 2

Figure 2. Circuit scheme used to emulate neuronal functionalities: The circuit consists of an integrator circuit (blue) based on a negative differential resistor and a memcapacitor C_M, as well as on an output branch with a diode D in series with two ohmic resistors R₁ and R₂ (red) which deliver the spikes v_out(t) from the oscillating voltage u(t). Inset: Typical oscillation for a constant input current of 0.25 mA. While the constant voltage source V_B shifts the base voltage to a negative value, the voltage divider with the diode D cause a constant output voltage v_out(t) in an interval (labeld as t_ref) in which u(t) is smaller than the build-in voltage of the diode. The used device parameters of the circuit were R₁ = 47 kΩ, R₂ = 10 kΩ, C_M = C₀ = 0.068 μF, V_B = −3.5 V.

The key devices of this circuit are the strongly correlated electron material vanadium dioxide (VO₂), which provides a NDR and the memcapacitor C_M, which has been realized by a sub-circuit containing a capacitive divider and an Ag/TiO_2−x/Al electrochemical metallization cell (cf. Figure 4A). Both devices are discussed in the following first, before going deeper in the analysis of the neuron circuit. In the following both devices are discussed individually, with a subsequent deeper analysis of the complete neuron circuit in the Results and Discussion section.

VO₂-based Negative Differential Resistor

As resistor R for the presented spiking neuron circuit (cf. Figure 2) a current-controlled VO₂ device was used, which exhibits a NDR. In particular, VO₂ belongs to the class of strongly correlated electron materials and exhibits a MIT phase transition (Mott—Peierls transition) at approximately 60°C (Morin, 1959; Nakano et al., 2012; Natelson, 2013) accompanied by a structural phase transition from the high-temperature tetragonal phase to the low-temperature monoclinic phase. A typical resistance vs. temperature curve of the here fabricated VO₂ film is shown in Figure 3A. A constant voltage of 1 V was applied to the film, while the current was recorded simultaneously. The temperature was ramped from 30 to 95°C and back to 30°C. In particular, we found the Mott transition temperature at 58°C during heating (red curve in Figure 3A) together with a hysteresis of ~10°C during cooling (blue curve in Figure 3A). Moreover, a resistance change of more than four orders in magnitude was observed which reflects a good quality of the VO₂ film. The quality of the VO₂ could be further verified by using X-ray diffraction spectroscopy (see inset of Figure 3A), which exhibits peaks corresponding to the (001) crystal orientation of the TiO₂ substrate and to the VO₂ (40-2)_M1 monoclinic phase M1, as expected at room temperature (Andersson, 1956).

Instead of increasing the substrate temperature to introduce the Mott transition also electrical stress can be used, where Joule heating of local filamentary grain structures has been identified as the origin of the resistance switch (Driscoll et al., 2012; Guénon et al., 2013). Recently, such two terminal devices, in which the Mott transition can be introduced by electrical stress, were identified as memristive devices (Chua, 1971) with a transient memory (Pickett et al., 2013). A typical current-voltage characteristic of our device is depicted in Figure 3B together with a sketch of the device structure. Therefore, the applied current was ramped between 0 and 1.5 mA, while simultaneously the voltage was measured. Starting from the initial high resistance of the device (${\text{R}}_{\text{VO}2}^{\text{H}}$ = 55 kΩ), the device resistance decreases gradually to a lower resistance of ${\text{R}}_{\text{VO}2}^{\text{L}}$ = 2–14 kΩ if the applied current-voltage threshold Θ_thr is exceeded (cf. Figure 3B).

FIGURE 3

Figure 3. Electrical characteristics of the VO₂ device: (A) Resistance versus temperature characteristic. Inset of (A): X-ray 2θ-ω scan of the VO₂ film on a TiO₂ (001) substrate measured with Cu K_α radiation. The observed peaks correspond to TiO₂ (002) and to VO₂ (40-2)_M1 (insulating phase, space group P2₁/c, a_M1 = 5.743 Å, b_M1 = 4.517 Å, c_M1 = 5.375 Å, and β_M1 = 122.618°) (B) Current voltage characteristic of a lateral VO₂ device together with a schematic sketch of the device structure. In all sub-figures indicating the red arrows the heating cycle and the blue arrows showing the cooling direction.

Ag-doped TiO2_-x -based Memcapacitive Device Circuit

In analogy to memristive devices (Chua, 1971), memcapacitors are defined by a capacitance which depends on the charge flow history, i.e., C_M = C_M(q,t). The memcapacitor was theoretically proposed by Di Ventra et al. (2009) in 2009 and shows promising functionalities for adaptive circuits (Traversa et al., 2013). To experimentally realize a memcapacitance, we used the sub-circuit shown in Figure 4A. This circuit consists of two individual capacitances C₁ and C₂ arranged in a capacitive divider with a single memristive device R_M connected in parallel to C₂, (labeled as $C_2^{eff}$ in Figure 4). In this configuration the resistance of the memristive device can be varied in accordance to the charge flow history, i.e., R_M = R_M(q,t). The therefore obtained memcapacitive behavior (depending on the resistances of R_M) can be seen by regarding the total capacitance C_M of the sub-circuit, which can be expressed as

$\begin{array}{l}{C}_M=\frac{C_1{C}_2^{eff}}{C_1+C_2^{eff}}\text{ with }{C}_2^{eff}\left({u_m,R}_M,t\right)\\ =\frac{1}{u_m}\left(q\left(t\right)-q_m\left({u_m,R}_M,t\right)\right).& \left(2\right)\hfill \end{array}$

Here u_m defines the voltage drop across $C_2^{eff}$, while q and q_m are the total charge of the capacitive branch and the charge stored in between C₁ and $C_2^{eff}$, respectively. Hence, $C_2^{eff}$ depends on R_M(q,t) which provides a memcapactive behavior as proposed theoretically by Di Ventra et al. (2009).

FIGURE 4

Figure 4. Experimental realization of a memcapacitor: (A) Schematic drawing of the memcapactitive circuit including a memristive device. (B) Calculated impedance phase φ_RM||C2 between the memristive device and the capacitor C₂ as function of the resistance R_M.

In order to gain some more insight into the functionality of the memcapacitive circuit and to give advices for the design of memristive devices it is worth to look at the total impedance of $C_2^{eff}$, which reads.

$\begin{array}{ll}{Z}_{R_M||C_2}(R_M)=\frac{1}{\frac{1}{R_M}+j\omega C_2}.& \left(3\right)\hfill \end{array}$

In particular, Equation (3) implies that $C_2^{eff}$ (and therewith C_M) is affected whenever the impedance phase φ_RM||C2 between C₂ and R_M is less than 90°. However, to fulfill this condition R_M must be varied in accordance to C₂. In this investigation C₁ and C₂ have been selected in respect to biological time scales, which is for a single spike in the range of a few ms. By further taking the resistance of the VO₂ negative differential resistor (cf. Figure 3) into account, C₁ and C₂ have been chosen to 0.165 and 0.068 μF, respectively. Hence, to ensure that Δφ_RM||C2 is less than 90°, R_M must be variable in between 1 and 100 kΩ, as depicted in Figure 4B. In particular, the impedance phase of $C_2^{eff}$ will be most sensitive to R_M changes when R_M and C₂ contribute to the overall impedance magnitude roughly equally. As we will show as next, a Ag-doped TiO_2−x based memristive device fulfills this requirement.

In Figure 5A typical current-voltage characteristic (I-V curve) obtained on a single Ag/TiO_2−x/Al memristive device is shown together with a sketch of the device structure. By sweeping the bias voltage between 1.4 and –0.5 V the device resistance changes at a positive set voltage of V_set = 0.95 V from the initial high resistance state of 1 MΩ to the low resistance state of 1 kΩ and vice versa at negative voltage at a reset voltage of V_Reset = −0.2 V. In order to avoid a device breakdown a current compliance of 0.1 mA was set. To analyze the resistance switching of the memristive device in some more detail single voltage pulses of 10 V in height and 2 ms in width are applied to an individual Ag/TiO_2−x/Al cell. In particular, that voltage pulse corresponds to the maximal possible voltage which can be dropped across the memristive cell when the cell is operating in the neuron circuit of Figure 2. The obtained change in resistance is shown in Figure 5B. We found that the device resistance is decreased from initially 1–0.8 kΩ under such voltage pulses. Therefore, the recorded resistances are within the memcapacitive interval estimated from Equation (3) and marked by two red dashed lines in Figures 4B, 5B.

FIGURE 5

Figure 5. Electrical characteristics of the memristive device: (A) Measured I-V curve of an Ag-doped TiO_2-x -based memristive device together with a sketch of the layer sequence of this cell. A current compliance of 0.1 mA has been set. (B) Measured resistance variation of the device by applying 2 ms voltage pulses with an amplitude of 10 V. The dashed red lines show the desired memcapacitive range obtained from Figure 4B. (C) Distribution of the set voltage obtained from 620 identically voltage sweeps using a current compliance of 0.1 mA. The red curve is a Gaussian data fit.

We would like to mention that the resistive switching process in the used Ag/TiO_2−x/Al memristive cells has an inherent stochastic nature (Gaba et al., 2013), where the needed number of voltage pulses to set the device resistance depends on local Ag migration processes. A more detailed analysis of the stochastic behavior of the used Ag/TiO_2−x/Al memristive device is shown in Figure 5C. Therein, the set voltages V_set of 620 consecutive current-voltage cycles are shown, which has been obtained from I-V measurements in which the voltage was ramped from 0 to 2 V and a current compliance of 0.1 mA was set. By fitting the experimental data with a Gaussian function, an average set voltage of 0.64 V was obtained with a full width at half maximum of 0.25 V. In particular, the device stochastic influences the transient dynamics of the neuron circuit, as we will discuss it below.

Results and Discussion

In the following we discuss the memristive spiking neuron model (depicted in Figure 2) in detail. For a clearer presentation of this circuit we split the analysis in two parts. First, we discuss the emulation of fire-rate coding, i.e., how the circuit emulates dynamical spiking patterns in response to an external stimulus. Thereafter, the emulation of adaptation and refractoriness will be addressed. We therefore like in particular to focus on the adaptive/memristive behavior of our neuron circuit.

Firing Rate Coding

The relevant mechanisms for the emulation of spike rate coding of our proposed neuron circuit scheme (Figure 2), can be dissected in that the memcapacitance (C_M) is fixed. By replacing the TiO₂ based sub-circuit with a constant capacitor C₀ this can be achieved and we obtain a circuit as shown in Figure 6A. The therewith recorded voltage characteristics u(t) and v_out(t) for different current inputs i(t) are shown in Figure 6B, while the used parameters of the circuit devices read as R₁ = 47 kΩ, R₂ = 10 kΩ, C_M = C₀ = 0.068 μF, V_B = −3.5 V. As a result we found that a current strength up to 0.15 mA will affect no spike generation within the investigated time interval, while current strengths of 0.25 and 0.4 mA triggers the circuit to spike with different numbers of spikes. In order to study this point in some more detail, single constant current pulses of 20 ms and amplitudes ranging from 0 to 0.95 mA were applied to the neuron circuit. The circuit produced fire frequencies (number of spikes per second) as function of the applied currents are depicted in Figure 6C. While for i(t) smaller then the VO₂ threshold current I_Θ (cf. Figure 3B) no oscillations are evoked, input currents above I_Θ generate a spiking of the circuit. Moreover, the generated frequency of spikes increased linearly with increasing current which allows to directly relat the spike frequency to the intensity of the stimulus. This represents the biological firing rate mechanism, as sketched in Figure 1B (Adrian, 1926; Chapleau, 2007). In technical terms, the linear increase in the fire frequency is related to the slope of the negative differential branch of the VO₂ device (cf. Figure 3B). Consequently, the total interval for the frequency coding is defined by the length of the negative differential branch which is for the here realized VO₂ cells for current strengths varying in between 0.4 and 0.95 mA. Because the firing rate is constant for a constant given stimulus (i.e., the firing rate exhibits no transient decay), the onset f₀(I) and steady f_∞(I) curve are identical (Benda and Herz, 2003).

FIGURE 6

Figure 6. Emulation of fire frequency coding: (A) Circuit layout to emulate firing frequency coding. In contrast to Figure 2 C_M was replaced by a constant capacitance C₀. (B) Recorded spike pattern for different current inputs. (C) Measured oscillation frequencies as function of the input current i(t). I_Θ denotes the threshold value for the spike initiation. Parameters used for the circuit: R₁ = 47 kΩ, R₂ = 10 kΩ, C₀ = 0.068 μF, V_B = −3.5 V.

Adaptation and Refractoriness

As furthermore recognized in 1926 by Adrian (Adrian, 1926, 1928), the firing rate of neurons transiently decreases rather than staying constant (cf. Figure 1B), if both the receptor and organisms habituate to a persistent stimulus. In particular, Adrian expected that the post transient decrease in the firing rate might reflect the degree of habituation of the stimulus, i.e., adaptation of an external stimulus (Adrian, 1928). Nowadays, adaptation is believed to be the essential process of a signaling system to be better suited to environmental changes and it can be observed at nearly any level of biological systems (Maass and Bishop, 2001). In the spike trains of regularly firing neurons, the adaptation of their firing frequency during sustained current input is believed to be fundamental in forward masking, selective attention, and in the synchronization of neuronal assemblies (Maass and Bishop, 2001; Fuhrmann et al., 2002).

In order to emulate fire frequency adaptation the capacitance C₀, shown in Figure 6A, has been replaced by a memcapacitance C_M = C_M(t) (cf. Figure 4) which leads to the circuit presented in Figure 2 and Figure 7B. In Figure 7A the therewith obtained voltage characteristics for u(t) and v_out(t) for a constant current input of 0.5 mA are presented. The used device parameters of the circuit were R₁ = 1 MΩ, R₂ = 47 kΩ, C₁ = 0.165 μF, C₂ = 0.068 μF, V_B = −5.5 V. As a main result we found that the frequency of spike initiation is clearly decreased after the first eight spikes, while the amplitude of the individual spikes is nearly unaffected. This finding can also be observed in Figure 7C, where a close-up view of two of the spikes of Figure 7A is shown. While the black curve in Figure 7C corresponds to one of the first spikes of the voltage course of v_out (compare also Figure 7A), the second spike (red curve) is cut from the last part of Figure 7A. In particular, their amplitudes of roughly 0.36 V and their resting potentials of u_r = −15 mV are varying slightly, while their particular spike widths and refractory period widths vary significantly. This differences in the spike width can be directly related to Equation (1), where the memconductance (cf. Equation 2) introduces a memristive time constant τ_m = R_VO2(i,t) C_M(R_M,u_m,t) for the current integration. The effect of such a memristive time constant can be further analyzed from experimental data (depicted in Figure 7C) by regarding the corresponding phase plots, as shown in Figure 7D. From this plot it can be seen that the rate of the voltage change of v_out varies little. However, the initial phases vary significantly when C_M changes, as it can be seen from the enlarged parts of the phase plots, depicted in the insets of Figure 7D (see yellow frame). In particular, we observed a much stronger rise in the rate of the voltage change of v_out for the initial spike (black curve) compared to the final spike (red curve). In this context it is worth mentioning that these features are also observed in cortical neurons and it is believed that the dynamics of spike initiation is a unique feature, which can qualitatively change the nature of neuronal encoding (Naundorf et al., 2006).

FIGURE 7

Figure 7. Emulation of adaptation: (A) Measured spike pattern for a constant current input based on a memcapacitance C_M(t). (B) Layout of the investigated circuit which corresponds to the one depicted in Figure 2. (C) Characteristics of two individual spikes and corresponding phase diagram (D), where the insets show the initial phase of the spikes. Parameters used for the circuit: R₁ = 1MΩ, R₂ = 47kΩ, C₁ = 0.165μF, C₂ = 0.068μF, V_B = −5.5 V.

We would like to remark that in contrast to biological findings an abrupt switch rather than a gradually, continuous decrease of the fire frequency was obtained. This, in fact, belongs to the switching mechanism of the memristive device and might change if the Ag/TiO_2−x/Al cell is replaced by a memristive device which shows a more gradual change in device resistance. However, the principles of the proposed adaptation emulation can be directly transferred to other types of memristive devices. A further important aspect of the used memristive cell is the inherent stochastic nature of the resistance switching process (cf. Figure 5C). In particular, this device behavior introduces stochastic noise to both the pulse width of a single spike and the number of spikes needed before a pulse frequency adaptation occurs (cf. Figure 7A). For example, for the spike course shown in Figure 7A, the width of the second and third spike is larger than those of spike number four, while a “real” adaptation of the firing frequency was obtained after eight voltage spikes. However, such a stochastic behavior might be of interest for a variety of network applications and are in agreement with biological neurons (McDonnell and Ward, 2011).

In general, adaptation of the neurons firing rate has to be balanced with refractoriness. Refractoriness is a general characteristic of neurons to ensure that consecutive spikes are not overlapping and defines the (forward) direction of spikes in nerve cells. In order to incorporate a refractory period, a diode was connected in series with R₁ and R₂ in the output branch of the neuron circuit of Figure 2 (red column). Further of importance for the emulation of a refractory period was a negative base voltage which has been generated by the constant voltage source V_B (cf. Figure 2). In particular, V_B causes a negative offset of the circuit induced oscillation of u(t), while the diode D affects that all voltages of u(t) smaller than the built-in voltage of the diode (V = 0.7 V) leading to a constant output voltage v_out(t) (see for example inset of Figure 2). Therefore, the resistances R₁ and R₂ have been chosen much smaller than resistance value of the diode below the built-in voltage, so that most of the voltage u(t) is dropping across D below 0.7 V. A refractory period can then be defined by the time interval for which v_out(t) is stabilized by the diode (labeled as t_ref1,2 in Figure 7C), i.e., the time at which u(t) is smaller than 0.7 V. Hence, in the framework of an I-F neuron model (according to Equation 1) the built-in voltage of the diode defines the threshold voltage for the spike initiation.

Conclusion

In conclusion, a memristive spiking neuron circuit has been experimentally realized by using a VO₂-based negative differential resistor and a memcapacitor based on an Ag/TiO_2−x/Al memristive cell. The circuit allows emulation of basic neuronal functionalities, including spike coding, firing frequency adaptation in real time and shows a refractory period. Moreover, the obtained spike times are consistent with the spike duration in biological systems. Further, we have shown that the combination of a memristive device and a capacitive divider allows to experimentally realizing a memcapacitance. Therewith, we were able to show that the use of a memcapacitance in an NDR oscillator allows to mimic dynamic neuronal components in which the circuit induced oscillation is changing in dependence of the charge flow history, i.e., on the number of spikes generated before. Thus, the use of a memcapacitance introduces a memristive behavior of an I-F neuron. Since such spiking neuron models are important conceptual tools for the analysis and emulation of neuronal dynamics, a memristive neuron might open important new opportunities for the realization of neuronal networks.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

Funding

This work was supported by the Deutsche Forschungsgemeinschaft (DFG) through FOR2093.

References

Adrian, E. D. (1926). The impulses produced by sensory nerve endings. J. Physiol. 61, 49–72.

PubMed Abstract | Google Scholar

Adrian, E. D. (1928). The Basis of Sensation. New York, NY: W. W. Norton.

Google Scholar

Andersson, G. (1956). Studies on vanadium oxides. II. The crystal structure of vanadium dioxide. Acta Chem. Scand. 10, 623–628. doi: 10.3891/acta.chem.scand.10-0623

CrossRef Full Text | Google Scholar

Barlow, H. (1961). “Possible principles underlying the transformation of sensory messages,” in Sensory Communication, ed W. Rosenblith (Cambridge, MA: MIT Press), 217–234.

Bear, M. F., Connors, B. W., and Paradiso, M. A. (2006). Neuroscience: Exploring the Brain. Hagerstown, MD: Lippincott Williams and Wilkins.

Benda, J., and Herz, A. V. (2003). A universal model for spike-frequency adaptation. Neural Comput. 15, 2523–2564. doi: 10.1162/089976603322385063

PubMed Abstract | CrossRef Full Text | Google Scholar

Chapleau, M. W. (2007). Exciting times in sensory transduction from A(drian) to Z. J. Physiol. 582, 13–14. doi: 10.1113/jphysiol.2007.135137

PubMed Abstract | CrossRef Full Text | Google Scholar

Chicca, E., Stefanini, F., Bartolozzi, C., and Indiveri, G. (2014). Neuromorphic electronic circuits for building autonomous cognitive systems. Proc. IEEE 102, 1367–1388. doi: 10.1109/JPROC.2014.2313954

CrossRef Full Text | Google Scholar

Chua, L. O. (1971). Memristor-The missing circuit element. IEEE Trans. Circuit Theory 18, 507–519. doi: 10.1109/TCT.1971.1083337

CrossRef Full Text | Google Scholar

Crane, H. D. (1960). The neuristor. IRE Trans. Electron. Comp. EC-9, 370–371. doi: 10.1109/TEC.1960.5219861

CrossRef Full Text | Google Scholar

Di Ventra, M., Pershin, Y. V., and Chua, L. O. (2009). Circuit elements with memory: memristors, memcapacitors, and meminductors. Proc. IEEE 97, 1717–1724. doi: 10.1109/JPROC.2009.2021077

CrossRef Full Text | Google Scholar

Driscoll, T., Quinn, J., DiVentra, M., Basov, D. N., Seo, G., Lee, Y. W., et al. (2012). Current oscillations in vanadium dioxide: evidence for electrically triggered percolation avalanches. Phys. Rev. B 86:094203. doi: 10.1103/physrevb.86.094203

CrossRef Full Text | Google Scholar

Fitzhugh, R. (1955). Mathematical models of threshold phenomena in the nerve membrane. Bull. Math. Biophys. 17, 257–278. doi: 10.1007/BF02477753

CrossRef Full Text | Google Scholar

Fuhrmann, G., Markram, H., and Tsodyks, T. (2002). Spike frequency adaptation and neocortical rhythms. J. Neurophysiol. 88, 761–770. doi: 10.1152/jn.00848.2001

PubMed Abstract | CrossRef Full Text | Google Scholar

Gaba, S., Sheridan, R., Zhou, J., Choi, S., and Lu, W. (2013). Stochastic memristive devices for computing and neuromorphic applications. Nanoscale 5, 5872–5878. doi: 10.1039/c3nr01176c

PubMed Abstract | CrossRef Full Text | Google Scholar

Gerstner, W., and Kistler, W. M. (2002). Spiking Neuron Models. Cambridge: Cambridge University Press.

Google Scholar

Guénon, S., Scharinger, S., Wang, S., Ramírez, J. G., Koelle, D., Kleiner, R., et al. (2013). Electrical breakdown in a V₂O₃ device at the insulator to metal transition. Europhys. Lett. 101:57003. doi: 10.1209/0295-5075/101/57003

CrossRef Full Text | Google Scholar

Hasegawa, T., Ohno, T., Terabe, K., Tsuruoka, T., Nakayama, T., Gimzewski, J. K., et al. (2010). Learning abilities achieved by a single solid−state atomic switch. Adv. Mater. 22, 1831–1834. doi: 10.1002/adma.200903680

PubMed Abstract | CrossRef Full Text | Google Scholar

Hindmarsh, J. L., and Rose, R. M. (1984). A model of neuronal bursting using three first order differential equations. Proc. R. Soc. Lond. 221, 87–102. doi: 10.1098/rspb.1984.0024

PubMed Abstract | CrossRef Full Text | Google Scholar

Hodgkin, A. L., and Huxley, A. F. (1952). A quantitative description of membrane current and its application to conduction and excitation in nerve. J. Physiol. 117, 500–544. doi: 10.1113/jphysiol.1952.sp004764

PubMed Abstract | CrossRef Full Text | Google Scholar

Indiveri, G., Linares-Barranco, B., Hamilton, T. J., van Schaik, A., Etienne-Cummings, R., Delbruck, T., et al. (2011). Neuromorphic silicon neuron circuits. Front. Neurosci. 5:73. doi: 10.3389/fnins.2011.00073

PubMed Abstract | CrossRef Full Text | Google Scholar

Izhikevich, E. M. (2003). Simple model of spiking neurons. IEEE Trans. Neural Netw. 14, 1569–1572. doi: 10.1109/TNN.2003.820440

PubMed Abstract | CrossRef Full Text | Google Scholar

Jeong, D. S., Kim, I., Ziegler, M., and Kohlstedt, H. (2013). Towards artificial neurons and synapses: a materials point of view. RSC Adv. 3, 3169–3183. doi: 10.1039/c2ra22507g

CrossRef Full Text | Google Scholar

Jo, S. H., Chang, T., Ebong, I., Bhadviya, B. B., Mazumder, P., and Lu, W. (2010). Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10, 1297–1301. doi: 10.1021/nl904092h

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, D. H., and Kwoka, H. S. (1994). Pulsed laser deposition of VO₂ thin films. Appl. Phys. Lett. 65, 3188.

Google Scholar

Laughlin, S. B. (1989). The role of sensory adaptation in the retina. J. Exp. Biol. 146, 39–62.

PubMed Abstract | Google Scholar

Lim, H., Kornijcuk, V., Seok, J. Y., Kim, S., Kim, I., Hwang, C. S., et al. (2015). Reliability of neural information conveyed by unreliable neuristor-based integrate-and-fire neurons: a model study. Sci. Rep. 5, 1–15. doi: 10.1038/srep09776

CrossRef Full Text | Google Scholar

Maass, W., and Bishop, C. M. (2001). Pulsed Neural Networks. Cambridge: The MIT Press.

Google Scholar

Mausfeld, R. (2013). “The biological function of sensory systems,” in Neurosciences, eds C. G. Galizia, P. M. Lledo, and R. Mausfeld (Heidelberg: Springer-Verlag), 239–252.

McDonnell, M. D., and Ward, L. M. (2011). The benefits of noise in neural systems: bridging theory and experiment. Nat. Rev. Neurosci. 12, 415–426. doi: 10.1038/nrn3061

PubMed Abstract | CrossRef Full Text | Google Scholar

Mead, C. (1989). Analog VLSI and Neural Systems. Boston, MA: Addison-Wesley. doi: 10.1007/978-1-4613-1639-8

CrossRef Full Text

Morin, F. J. (1959). Oxides which show a metal-to-insulator transition at the neel temperature. Phys. Rev. Lett. 3, 34–36. doi: 10.1103/PhysRevLett.3.34

CrossRef Full Text | Google Scholar

Nakano, M., Shibuya, K., Okuyama, D., Hatano, T., Ono, S., Kawasaki, Y., et al. (2012). Collective bulk carrier delocalization driven by electrostatic surface charge accumulation. Nature 487, 459–462. doi: 10.1038/nature11296

PubMed Abstract | CrossRef Full Text | Google Scholar

Natelson, D. (2013). Condensed matter physics: a solid triple point. Nature 500, 408–409. doi: 10.1038/500408a

PubMed Abstract | CrossRef Full Text | Google Scholar

Naundorf, B., Wolf, F., and Volgushev, M. (2006). Unique features of action potential initiation in cortical neurons. Nature 440, 1060–1063. doi: 10.1038/nature04610

PubMed Abstract | CrossRef Full Text | Google Scholar

Ohno, T., Hasegawa, T., Tsuruoka, T., Terabe, K., Gimzewski, J. K., and Aono, M. (2011). Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. Nat. Mater. 10, 591–595. doi: 10.1038/nmat3054

PubMed Abstract | CrossRef Full Text | Google Scholar

Petraru, A., Soni, R., and Kohlstedt, H. (2014). Voltage controlled biaxial strain in VO₂ films grown on 0.72Pb(Mg_1∕3Nb_2∕3)-0.28PbTiO₃ crystals and its effect on the transition temperature. Appl. Phys. Lett. 105, 092902. doi: 10.1063/1.4894536

CrossRef Full Text

Pickett, M. D., Medeiros-Ribeiro, G., and Williams, R. S. (2013). A scalable neuristor built with Mott memristors. Nat. Mater. 12, 114–117. doi: 10.1038/nmat3510

PubMed Abstract | CrossRef Full Text | Google Scholar

Shepherd, G., and Grillner, S. (2010). Handbook of Brain Microcircuits. Oxford: Oxford University Press.

Google Scholar

Traversa, F. L., Pershin, Y. V., and Di Ventra, M. (2013). Memory models of adaptive behavior. IEEE Trans. Neural Netw. Learn. Syst. 24, 1437–1448. doi: 10.1109/tnnls.2013.2261545

PubMed Abstract | CrossRef Full Text | Google Scholar

van der Pol, B. (1926). On relaxation-oscillations. Lond. Edinb. Dublin Phil. Mag. J. Sci. 7, 978–992. doi: 10.1080/14786442608564127

CrossRef Full Text | Google Scholar

Zamarreño-Ramos, C., Camuñas-Mesa, L. A., Pérez-Carrasco, J. A., Masquelier, T., Serrano-Gotarredona, T., and Linares-Barranco, B. (2011). On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex. Front. Neurosci. 5:26. doi: 10.3389/fnins.2011.00026

PubMed Abstract | CrossRef Full Text

Ziegler, M., Riggert, C., Hansen, M., Bartsch, T., and Kohlstedt, H. (2015). Memristive Hebbian plasticity model: device requirements for the emulation of Hebbian plasticity based on memristive devices. IEEE Trans. Biomed. Circuits Syst. 9, 197–206. doi: 10.1109/TBCAS.2015.2410811

PubMed Abstract | CrossRef Full Text | Google Scholar