A comparison of two sleep spindle detection methods based on all night averages: individually adjusted vs. fixed frequencies

Sleep spindles are frequently studied for their relationship with state and trait cognitive variables, and they are thought to play an important role in sleep-related memory consolidation. Due to their frequent occurrence in NREM sleep, the detection of sleep spindles is only feasible using automatic algorithms, of which a large number is available. We compared subject averages of the spindle parameters computed by a fixed frequency (FixF) (11–13 Hz for slow spindles, 13–15 Hz for fast spindles) automatic detection algorithm and the individual adjustment method (IAM), which uses individual frequency bands for sleep spindle detection. Fast spindle duration and amplitude are strongly correlated in the two algorithms, but there is little overlap in fast spindle density and slow spindle parameters in general. The agreement between fixed and manually determined sleep spindle frequencies is limited, especially in case of slow spindles. This is the most likely reason for the poor agreement between the two detection methods in case of slow spindle parameters. Our results suggest that while various algorithms may reliably detect fast spindles, a more sophisticated algorithm primed to individual spindle frequencies is necessary for the detection of slow spindles as well as individual variations in the number of spindles in general.

Introduction

Sleep spindles are oscillations emerging from interacting thalamocortical, corticothalamic, and reticular networks in NREM sleep (Steriade and Deschenes, 1984; Amzica and Steriade, 2000; Steriade, 2000; Fogel and Smith, 2011), which are thought to play an important role in sleep-related brain plasticity (Genzel et al., 2014). Due to their trait-like nature and relationship to plasticity, sleep spindles are frequently studied as candidate indexes of individual variations in cognitive performance. Sleep spindles are remarkably individual features: sleep spindle parameters are characterized by high intra-individual stability and inter-individual variability (De Gennaro et al., 2005), a strong genetic background (De Gennaro et al., 2008), and a correlation with anatomical properties of the brain (Piantoni et al., 2013; Saletin et al., 2013).

Due to their high prevalence and specific signal properties automatic detection methods have proven to be viable and preferable alternatives to visual detection. Some of the earliest studies (Broughton et al., 1978; Campbell et al., 1980) used phase-locked loop devices for automatic sleep spindle detection and already reported an adequate agreement with visual detection. An early combined software-hardware system (Ferri et al., 1989) also reliably replicated visual spindle detection results. Software solutions for automatic spindle detection were introduced somewhat later (Schimicek et al., 1994) and reported relatively high (approx. 70%) specificity for 90% sensitivity, while an improved method (Devuyst et al., 2006) could increase this to almost 76% in a clinical sample. More recently, sophisticated automatic sleep spindle detection methods using artificial neural networks (Acır and Güzeliş, 2004; Ventouras et al., 2005) and decision trees (Duman et al., 2009) reached even higher performance, with correct classification frequently exceeding 90%.

Automatic sleep spindle recognition was further refined by adapting algorithms that take into account the inter-individual differences in sleep spindle activity, which vastly exceed intra-individual variation (De Gennaro et al., 2005) and emerge—among others—as a function of age and sex (Driver et al., 1996; Carrier et al., 2001; Huupponen et al., 2002; Genzel et al., 2012). Sleep spindle detection methods have been developed to operate with individually adjusted amplitude limits (Huupponen et al., 2000, 2007; Ray et al., 2010). A novel algorithm (Bódizs et al., 2009; Ujma et al., 2014) based on the electrophysiological fingerprint theory of human sleep (De Gennaro et al., 2005, 2008) is the Individual Adjustment Method (IAM), which takes into account inter-individual variations not only in the amplitude, but also in the frequency of sleep spindles. In the IAM, sleep spindles are therefore not only detected based on individual amplitude thresholds, but also within the exact frequency bands where they are present in a given individual. A similarly adaptive detection method (based on a probabilistic model) is reported in Nonclercq et al. (2013).

A comparison of four different spindle detection methods (Huupponen et al., 2007) reported acceptable, but not overwhelming concordance. A recent study (Warby et al., 2014) investigated the agreement in spindle detection between expert human raters, non-experts recruited in an internet crowdsourcing effort, and automatic detection algorithms. Concordance was strongest among human experts, followed by non-experts operation in a crowdsourcing scheme, and weakest among automatic algorithms.

While the progress in automatic sleep spindle detection methods is impressive, there are numerous concerns which must be addressed in this field. A practical criticism may arise from the fact that automatic sleep spindle detections are frequently validated against visual detections: however, agreement in the visual scoring of spindles is not perfect (Campbell et al., 1980; Warby et al., 2014), the visual detection of spindles is often considered as a consensus from several raters which may bias results (Ray et al., 2010), and—despite stronger agreement among human raters than algorithms (Warby et al., 2014)—the use of human expert opinion as an absolute gold standard is philosophically questionable in itself (Bódizs et al., 2009).

Further criticism must be given to the fact of the use of standard signal detection terminology (such as sensitivity and specificity) in case of sleep spindle detection algorithms. Sleep spindles are frequent phenomena, but even so the vast majority of a sleep EEG recording does not consist of sleep spindles. Therefore, correct negative classifications are by far the most common result produced by any sleep spindle detector, which might drastically inflate specificity. The ratio of correct hits and false detections—including misses and false positives—would be a much more conservative, but also more informative measure of detection performance.

Sleep spindles are not only biological signals, but important markers of individual traits (De Gennaro et al., 2005, 2008) as well as powerful correlates of human cognition (among others: Bódizs et al., 2005; Schabus et al., 2006; Fogel et al., 2007; Ujma et al., 2014). Therefore, an alternative option in order to assess detection algorithms would be to investigate how much they can reproduce trait-like individual averages (instead of comparing individual spindle detections).

To our knowledge, it has never been investigated how strongly spindle measures of different detection methods are correlated if not individual spindle detections, but subject averages are considered. This can evidently not predicted from the signal detection characteristics of the comparison of individual spindle detections of various methods—albeit the literature usually reports moderate agreement between the individual spindle detections of different algorithms, it is unknown whether the different spindle samples obtained by different methods approximate the same individual averages. Therefore, the aim of our study was to reveal the correlation between individual sleep spindle parameters calculated with two different detection methods.

Materials and Methods

Subjects

We examined polysomnographic data of 161 healthy volunteers (88 males, 73 females, age between 17 years and 69 years, mean age 29.4 years, StD 10.7 years) recorded on the second night spent in a sleep laboratory. All procedures were approved by the responsible institution's ethical board and subjects gave informed consent. A semi-structured interview excluded any history of neurologic or psychiatric disease, but six subjects suffered from frequent nightmares. Subjects were free of drugs and prescription medication (except for contraceptives, all data self-reported). Alcohol and excessive caffeine consumption (over two cups of coffee before noon) was not allowed. Eight subjects were smokers, while the rest were non-smokers (self-reported). This dataset used for analysis was the same as in Ujma et al. (2014), except for the inclusion of one female subject who was excluded from the previous study due to her unavailable IQ score.

Sleep Recordings

All subjects spent two nights in a sleep laboratory and polysomnographic data from the 2nd night was used for analysis. Since the study was performed in cooperation between multiple sleep laboratories, recordings were performed in four slightly different designs.

For 31 subjects, recordings were performed with 18 EEG electrodes using a Flat Style SLEEP La Mont Headbox device with a HBX32-SLP preamplifier (La Mont Medical Inc. USA), with a sampling rate of 249 Hz, hardware prefiltering 0.5–70 Hz and a precision of 12 bit.

For 16 subjects signals were collected, prefiltered (0.33–1500 Hz, 40 dB/decade anti-aliasing hardware input filter), amplified and digitized with 4096 Hz/channel sampling rate (synchronous) and 12 bit resolution by using the 32 channel EEG/polysystem (Brain-Quick BQ 132S, Micromed, Italy). A further 40 dB/decade anti-aliasing digital filter was applied by digital signal processing which low-pass filtered the data at 450 Hz. Finally, the digitized and filtered EEG was undersampled at 1024 Hz.

For 114 subjects, recordings were performed with a Comlab 32 Digital Sleep Lab device (Schwarzer, Germany) with a sampling rate of 250 Hz, hardware prefiltering 0.53–70 Hz and a precision of 8 bit. In 94 of these subjects, 22 EEG electrode sites were used, while in the others 20 subjects 10 EEG electrodes were used. Common recording sites in all subjects which were used in the analysis were Fp1, Fp2, F3, F4, Fz, F7, F8, C3, C4, Cz, P3, P4, T3, T4, T5, T6, O1, and O2, all referred to the mathematically linked mastoids. For the 20 subjects with only 10 electrodes, data from Fp1, Fp2, F3, F4, C3, C4, P3, P4, O1 and O2 was available and for the other electrodes these subjects were treated as missing data.

In order to correct for potentially different baseline amplitudes depending on the recording device (Vasko et al., 1997), the analog-digital conversion and filtering characteristics of all recording devices were measured and sleep spindle amplitudes were corrected for the measured differences as follows (Ujma et al., 2014). We determined the amplitude reduction rate of each recording system by calculating the proportion between digital (measured) and analog (generated) amplitudes of sinusoid signals at typical sleep spindle frequencies (10, 11, 12, 13, 14, and 15 Hz) for both inducing (40 and 355 μV amplitude) signals. Machine-specific amplitude reduction rates were given as the mean amplitude rate between digital and analog values at the two amplitudes and six measured frequencies. Sleep spindle amplitudes were corrected by dividing their calculated values by the amplitude reduction rate of the recording system. Given the individual- and derivation-specific adjustment inherent to both the Fixed frequency method (FixF) and the IAM, sleep spindle densities and durations are amplitude-insensitive measures. Thus, there is no need for the compensation of the different recording systems in these values.

Sleep recordings of the second nights were scored according to standard criteria (Iber et al., 2007) on a 20 s basis and artifacts were removed by visual inspection on a 4 s basis. Sleep spindle analysis was performed on artifact-free segments of NREM sleep.

Analyses

Fixed frequency method of sleep spindle analysis

For the FixF method we determined the 11–13 Hz range as a slow spindle frequency band and the 13–15 Hz window as a fast spindle frequency band. These frequencies were selected to ensure consistency with previous studies (Schabus et al., 2006, 2007, 2008; Chatburn et al., 2013), which used a similar approach for the separation of slow and fast spindles.

Sleep spindles were automatically detected within artifact-free NREM sleep periods on every EEG derivation. For slow spindle detection, data were bandpass-filtered between 11 Hz and 13 Hz. The root mean squares of the filtered signals were determined for 0.25 s length time windows. Next a threshold was calculated at the 95th percentile of the root mean square values for every EEG derivation. A spindle was identified when at least two consecutive root mean square time points exceeded the threshold, and the duration criterion (≥0.5 s) was met. Four spindle characteristics were calculated; these were density (number of spindles/min); amplitude (peak-to-peak difference in voltage, expressed in μV); duration (s), and frequency (number of cycles/s, in Hz). The same procedure was followed for detecting fast spindles, using a band pass filter of 13–15 Hz (Schabus et al., 2007; Gruber et al., 2013).

Sleep spindle analysis according to the IAM

The second sleep spindle detection algorithm was the IAM (Bódizs et al., 2009). This sleep spindle detection method takes into account both inter-individual variations and intra-individual consistency in sleep spindle frequency (De Gennaro et al., 2005, 2008), analyzing sleep spindles at the individual peak frequency for all subjects.

The IAM procedure (Bódizs et al., 2009) consisted of several steps as described below (illustrated on Figure 1).

i. Average amplitude spectra. Non-overlapping 4 s artifact-free NREM sleep EEG segments are Hanning-tapered (50%), then zero-padded to 16 s. Average amplitude spectra of all-night NREM sleep EEG derivations is computed between 9 Hz and 16 Hz by using an FFT routine (frequency resolution: 0.0625 Hz).

ii. Individually adjusted frequency limits of slow and fast sleep spindles. Determination of the individual slow and fast sleep spindle frequencies is based on second order derivatives of the 9–16 Hz amplitude spectra. In order to avoid small fluctuations in convex and concave segments average amplitude spectra of 0.0625 Hz resolution (i) is downsampled (decimated) by a factor of 4 (resulting in a resolution of 0.25 Hz) before calculating the derivation-specific second-order derivatives in this frequency range. Derivation-specific second order derivatives of the amplitude spectra are then averaged over all EEG derivations resulting in a whole-scalp second order derivative for each subject. Individual-specific frequency limits of sleep spindles are defined as pairs of zero crossing points encompassing a negative peak in the whole-scalp second order derivatives. These zero-crossing points are rounded to the closest bins within the high-resolution (0.0625 Hz) amplitude spectra obtained in step i. Two pairs of individual-specific frequency limits and corresponding ranges are defined (one for slow and one for fast spindles). In cases of uncertainty (lack of zero crossing points indicating slow spindles or partial overlap between slow and fast sleep spindles in some cases), frequencies with predominance of power in averaged frontal (Fp1, Fp2, F3, F4, Fz, F7, F8) over averaged centro-parietal (C3, C4, Cz, P3, P4) amplitude spectra were considered as slow spindle frequencies (N = 18). There was no case of uncertainty related to the individual-specific frequency boundaries of fast sleep spindles.

iii. Individual-specific spindle middle frequencies. Slow spindle middle frequency of a given subject was quantified as the arithmetic mean of the individual-specific lower and upper limits for slow spindling as obtained above (ii). In case of fast sleep spindling the arithmetic mean of the lower and the upper frequency limits of fast sleep spindles were considered.

iv. Individual- and derivation-specific amplitude criteria for sleep spindles. Spindles are defined as those EEG segments contributing to the peak region of the average amplitude spectrum. Hence we intended to obtain an amplitude criterion corresponding to the line determined by the y-values (μV) pertaining to the individually adjusted pairs of frequency limits (ii) in the average amplitude spectra (i).

a. The number of high resolution (0.0625 Hz) frequency bins (i) falling in the individual-specific slow- and fast sleep spindle frequency ranges (ii) is determined.

b. The amplitude spectral values (i) at the individually adjusted frequency limits for slow and fast sleep spindles (ii) are determined. This is performed in a derivation-specific manner.

c. Number of bins for slow and fast sleep spindling (iv/a) are multiplied by the arithmetic mean of the pairs of derivation-specific amplitude spectral values for slow and fast sleep spindle frequency limits (iv/b), respectively. Outcomes are individual- and derivation specific amplitude criteria for slow and fast sleep spindle detections.

v. Envelopes of sleep spindling. EEG data is band-pass filtered for the slow and fast spindle frequency ranges by using an FFT-based Gaussian filter with 16 s windows: f(x) = e^(−(x− x_m)/(w/2)), where x varies between zero and the Nyquist frequency according to the spectral resolution, x_m is the middle frequency of the spindle range (iii), and w is the width of the spindle range (ii) (ii and iii). Filtered signal is rectified and smoothed by a moving average weighted with a Hanning window of 0.1 s length and multiplied with π /2 (the latter is the inverse of the mean of a rectified sine wave).

vi. Detection and characterization of sleep spindles. If envelopes of this band-pass filtered and rectified data (v) exceed the individual and derivation-specific threshold as defined above (iv) for at least 0.5 s, a sleep spindle is detected. Sleep spindles detected this way are analyzed and average sleep spindle density (number of spindles per minute), sleep spindle duration (s), as well as median and maximum amplitude (expressed as all-night means of intra-spindle envelopes in μV at the middle of the detected spindles and at the maxima of the spindles, respectively) is calculated for the subject.

FIGURE 1

Figure 1. The Individual Adjustment Method (IAM) of sleep spindle analysis. (A) Four-second EEG epoch Hanning-tapered and zero padded to 16 s. (B) Fast Fourier Transformation (FFT) is used to calculate 9–16 Hz average amplitude spectra of all night NREM sleep EEG from Hanning-tapered and zero-padded segments (derivations: Fp1, Fp2, F3, F4, Fz, F7, F8, T3, T4, T5, T6, C3, C4, Cz, P3, P4, O1, O2 referred to the mathematically-linked mastoids). (C) Amplitude spectra are decimated (down-sampled) by a factor of 4. (D) Second order derivatives of the decimated amplitude spectra. (E) Calculating the whole-scalp second order derivatives by averaging all series. The resulting average series is overplotted with the averaged frontal (Fp1, Fp2, F3, F4, Fz, F7, F8) and centro-parietal (C3, C4, Cz, P3, P4) amplitude spectra (the left-side Y axis is for average second-order derivatives, while the second Y axis on the right is for average amplitude spectra). Appropriate zero-crossing points encompassing individual-specific slow and fast sleep spindle bands are selected on the 9–16 Hz frequency scale. (F) Derivation-specific amplitude criteria are calculated. (G) Thresholding of the envelopes of the slow and fast-spindle filtered signal.

Statistics

FixF and IAM spindle parameters were compared using paired-sample t-tests (α = 0.05). The Benjamini-Hochberg method of false detection rate correction was performed in order to correct for multiple comparisons.

We computed Pearson's point-moment correlation coefficients between comparable sleep spindle measures (that is, sleep spindle parameters computed from the same electrode) produced by IAM, and the FixF method.

Results

IAM Frequency Bands

For the IAM method, individual slow spindle lower frequency limits ranged from 8.98 Hz to 12.95 Hz (mean: 10.96 Hz), while higher frequency limits ranged from 10.14 Hz to 13.7 Hz (mean: 11.9 Hz). Slow spindle middle frequencies ranged from 9.59 to 13.28 Hz (mean: 11.43 Hz). Fast spindle lower frequency limits ranged from 11.82 Hz to 14.77 Hz (mean: 13.06 Hz), while higher frequency limits ranged from 13.04 Hz to 16.03 Hz (mean: 14.36 Hz). Fast spindle middle frequencies ranged from 12.49 Hz to 15.38 Hz (mean: 13.71 Hz).

Individual slow spindle frequency bands were on average 0.94 Hz wide (range: 0.34–2.2 Hz). Individual fast spindle frequency bands were on average 1.3 Hz wide (range: 0.84–1.89 Hz).

Figure 2 shows the distribution of individual sleep spindle frequencies.

FIGURE 2

Figure 2. Distribution of individual sleep spindle frequency bands across the 9–16 Hz frequency domain based on visual inspection of the sleep EEG spectrum and the zero crossings of the second-order derivatives thereof. Blue markers indicate slow spindle middle frequencies, while red markers show fast spindle middle frequencies. Negative and positive error bars illustrate lower and higher individual frequency limits, respectively. Thick lines highlight the 11 Hz, 13 Hz, and 15 Hz thresholds used in the FixF method. Subjects have been ordered by slow spindle middle frequency to ensure better visibility.

FixF vs. IAM Spindle Parameters

IAM provides an approximately twice higher sleep spindle density than the FixF method in case of both slow and fast spindles as well as 1.5–2 times longer sleep spindle durations. Standard deviations of the individual averages of the FixF parameters are much smaller than in case of IAM parameters, even proportionally to the lower mean values.

Sleep spindle parameters are shown in Table 1. It must be noted that while FixF and IAM amplitude measures are displayed and compared, they are not expected to be on the same scale due to the narrower frequency band of IAM and the fact that in the FixF method amplitude was expressed as the mean maximum peak-to-peak voltage difference within a spindle, while in IAM amplitude was defined as the mean maximum of intra-spindle envelopes of the individually band-passed EEG.

TABLE 1

Table 1. Sleep spindle parameters calculated by IAM and the fixed frequency method (FixF).

The difference between comparable FixF and IAM spindle parameters is significant in all cases at p < 0.0001, and all comparisons remain significant after correction for multiple comparisons.

Correlations Between FixF and IAM Spindle Parameters

Despite the differences in the results, individual spindle parameters obtained with the FixF and IAM methods are strongly correlated in case of the amplitude and duration of fast spindles. These correlations are always over 0.5 for amplitude and over 0.4 for duration and they are highest (>0.8 for amplitude, >0.7 for duration) in derivations where fast spindles are most prominent (central and parietal electrodes) as well as in occipital derivations. There is, surprisingly, a negative correlation between fast spindle density calculated by the IAM and the FixF method.

There is only a week concordance between FixF and IAM slow spindle parameters. There is no significant FixF-IAM correlation in case of slow spindle density and duration, and only a modest correlation in case of slow spindle amplitude (r < 0.5 except for F3).

Table 2 presents the Pearson correlation coefficients depicting the linear relationship between corresponding IAM and FixF spindle parameters on all electrodes.

TABLE 2

Table 2. Correlation coefficients and p-values between compatible sleep spindle parameters calculated by IAM and the fixed frequency method.

Given that 1) our sample consisted of several datasets recorded on various EEG devices and 2) the FixF ranges we analyzed—while based on previous literature—did not correspond well to the frequency ranges computed by IAM, we re-analyzed our sample divided in subsamples as well as with different FixF ranges set with slow spindles between 10 Hz and 12.5 Hz and fast spindles between 12.5 Hz and 15 Hz. In both re-analyses, we attempted to replicate our most prominent results, and investigated fast spindle parameters on P4 and slow spindle parameters on F3. F3 was selected over Fz because of the higher availability of this electrode in the sample.

Results are similar across subsamples: that is, fast spindle density is negatively correlated; slow spindle density and duration are not correlated, slow spindle amplitude is moderately and positively correlated while fast spindle duration and amplitude are strongly and positively correlated. FixF-IAM correlations for slow spindles on F3 are as follows for density (r_Budapest1 = 0.427, p = 0.016; r_Budapest2 = −0.032, p = 0.908; r_Munich = 0.129, p = 0.086), duration (r_Budapest1 = 0.086, p = 0.647; r_Budapest2 = −0.143, p = 0.597; r_Munich = −0.072, p = 0.448) and amplitude (r_Budapest1 = 0.353, p = 0.052; r_Budapest2 = 0.498, p = 0.049; r_Munich = 0.519, p < 0.001). FixF-IAM correlations for fast spindles on P4 are as follows for density (r_Budapest1 = −0.28, p = 0.127; r_Budapest2 = −0.282, p = 0.291; r_Munich = −0.359, p < 0.001), duration (r_Budapest1 = 0.844, p < 0.001; r_Budapest2 = 0.661, p = 0.005; r_Munich = 0.805, p < 0.001) and amplitude (r_Budapest1 = 0.75, p < 0.001; r_Budapest2 = 0.798, p < 0.001; r_Munich = 0.861, p < 0.001).

Application of the new frequency bands also did not change the pattern of consistency of our methods significantly. With the 10–12.5 Hz FixF windows, FixF-IAM correlations for slow spindles on F3 are the following: r_density = 0.083, p = 0.292; r_duration = −0.069, p = 0.39; r_amplitude = 0.419, p < 0.001. With the 12.5–15 Hz FixF windows, FixF-IAM correlations for fast spindles on P4 are the following: r_density = −0.149, p = 0.06; r_duration = 0.802, p < 0.001; r_amplitude = 0.66, p < 0.001.

Discussion

While previous studies compared sleep spindle detections between various manual and automatic methods (Huupponen et al., 2007; Warby et al., 2014), to our knowledge no previous study compared individual averages of sleep spindle parameters calculated by various methods. Moreover, comparisons of individual detections were usually performed on many spindles from a small number of subjects. We investigated the convergent validity of two well-known algorithms by correlating all-night averages of individual sleep spindle parameters in a large database of subjects. In this approach, the agreement between individual detections is admittedly less important than agreement between individual averages. Overall, our results highlight both similarities and differences in the two sleep spindle detection methods we compared, and they do not provide overwhelming evidence for the convergence of the two methods.

IAM is tuned to individual spindle frequencies as well as individual and derivation-specific amplitude limits, making it inherently more sensitive as evidenced by higher spindle density and longer duration. FixF, on the other hands, focuses on the upper 5% of the amplitude distribution of filtered EEG signals. While FixF appears to detect “the tips of the icebergs” with this approach, the fast spindles detected by FixF are able to realistically approximate the same fast spindle durations and amplitudes as the IAM. Concordance is much weaker, however, in case of slow spindle amplitude, completely absent in case of slow spindle density and duration, while a very surprising negative correlation between fast spindle densities were found. To explain these findings, some empirical tendencies must be considered.

First, while the 13–15 Hz FixF window for fast spindles was similar to the empirically determined individual frequencies of IAM fast spindles, this was not the case for the 11–13 Hz slow spindle window. Fast spindle middle frequencies were below 13 Hz in only 11.24% of all cases and over 15 Hz in 1.87% of cases, while slow spindle middle frequencies were below 11 Hz in 27.5% of all cases and over 13 Hz in 1.25% of all cases. This poor demarcation of slow spindles in the FixF method might explain why FixF slow spindle parameters correlate more strongly with IAM fast spindle parameters than IAM slow spindle parameters (FixF slow vs. IAM fast correlations on Cz: r_density = −0.092 p = 0.275; r_duration = 0.547 p < 0.001; r_amplitude = 0.603 p < 0.001, with similar tendencies on all electrodes, see Table 2 for correlations with IAM slow spindle parameters). This finding, together with poor agreement on density measures suggests that some FixF slow spindles may actually be classified as fast spindles by the IAM procedure and vice versa, explaining the confusion in both density measures and slow spindle parameters in general. This phenomenon is exemplified by some dissimilar findings in the field. That is, both slow and fast sleep spindle measures correlate with cognitive abilities in cases when the FixF method is used (Schabus et al., 2006, 2008), while in case of IAM fast spindles are much more stable correlates of cognitive performance (Bódizs et al., 2005, 2008; Ujma et al., 2014). It must be noted that sleep spindles are not stationary sinusoidal processes: they are known to shift frequencies (chirp). Negative spindle chirps (decreasing frequencies) have been reported in humans (Andrillon et al., 2011; Schonwald et al., 2011), while increasing spindle frequencies were reported in rats (Sitnikova et al., 2014). These frequency shifts are not large enough to eclipse the difference between slow and fast spindles (Andrillon et al., 2011) but spectral chirps arising in spindles close to the 13 Hz boundary might be large enough to make them “jump” it and be detected in the opposite category.

Second, the average width of the individual fast spindle frequency band was 1.3 Hz, while in case of slow spindles it was only 0.94 Hz. That is, our results show that individual fast spindle frequency bands rarely fell outside the 13–15 Hz range and they were generally closer to the 2 Hz window of the FixF method than slow spindle frequency bands. The fact the re-analysis with FixF bands resembling the empirically determined individual frequency bands of IAM (10–12.5/12.5–15 Hz, compare with IAM frequency bands on Figure 2) did not significantly improve concordance between the two methods suggests that the differences in individual spindle bandwidth may be even more important for the lack of concordance between the two methods than the mere whereabouts of the frequency limits. This is in line with previous results from an adaptive, probabilistic model (Nonclercq et al., 2013) which reported a similar robustness to the input frequency range.

Based on the above findings we hypothesize that the lack of consistency between FixF and IAM slow spindle parameters is caused by the above factors: IAM slow spindles are determined at a lower and narrower frequency, with a larger distance from fast spindle frequencies in the same subject. The same phenomenon might be speculated to explain the negative correlation between IAM and FixF fast spindle density: in subjects with higher numbers of fast spindles (by IAM definitions) around the 13 Hz cutoff point cross-contamination with slow spindles may have been elevated in the FixF method.

There is little consistency in the sleep spindle detection methods used in previous research literature concerning the relationship between spindles and human cognition. Not all studies about the relationship between sleep spindle parameters and individual differences in psychometric variables separated slow and fast spindles: many analyzed sleep spindles in general or spectral power from a broader frequency band (Clemens et al., 2005; Fogel and Smith, 2006; Fogel et al., 2007; Tucker and Fishbein, 2009; Lustenberger et al., 2012; Gruber et al., 2013). Most studies which specifically analyzed slow and fast spindles and their correlation with psychometric variables used a post-hoc classification of spindles based on their central frequency, usually with 13 Hz as the split point (Schabus et al., 2006, 2008; Chatburn et al., 2013). Other studies used a slightly different ad-hoc division of sigma power into slow (11.5–12.5 Hz) and fast (13.5–14.5 Hz) sigma bands (Bang et al., 2014). Only a handful of studies relied on individually determined spindle frequencies, either by using the IAM method (Bódizs et al., 2005, 2008; Ujma et al., 2014) by computing individual relative sigma power defined as power ± 2 Hz around a single maximal spectral peak relative to the background EEG (Gottselig et al., 2002; Geiger et al., 2011) or by using an adaptive, probabilistic method (Nonclercq et al., 2013).

In sum, our results show that in case of fast spindles, duration and amplitude can be estimated reliably with both fixed and individual frequency methods. Much less consistency can be reached in case of slow spindles, and fixed cutoff frequencies may also lead to a poor separation of slow and fast spindles. Our results suggest that the cutoff frequencies and bandwidths for slow and fast spindles must be selected carefully and individually determined frequency bands should be considered.

It is notable that the concordance between the two methods is generally highest on typical spindle locations (frontal electrodes for slow spindles and centro-parietal electrodes for fast spindles). Concordance is usually lowest on temporal leads, but remains relatively high in occipital leads, in line with the relatively high spindle amplitude on these electrodes reported in the same dataset (Ujma et al., 2014). Lead-specific findings suggest that the lack of concordance between different spindle detection algorithms is especially problematic when non-prominent (e.g., temporal) leads are investigated.

There are limitations of our study that must be mentioned. First, the technical standards of the American Academy of Sleep Medicine (2007) are not met in several subsamples of our study. That is, the analog-to-digital conversion rate is low (8 bits) in the largest subsample (N = 114), while the sampling rates are close to the minimally required values (249 and 250 Hz) in two subsamples. Second, while the study compared methodically well-established methods with previous practical applications in science, it must be acknowledged that IAM and the FixF algorithm operate with different philosophical underpinnings, they are designed to detect different features: the FixF method considers the background-relative amplitude of the filtered signal as the key feature of a spindle event, while IAM looks for an amplitude threshold based on the inflection points of the individual EEG spectrum (IAM and FixF detections, together with visual detections are illustrated on Figure 3). Therefore, a perfect agreement between their results cannot be expected, and in the absence of a “gold standard” the inherent superiority of any method cannot be ascertained.

FIGURE 3

Figure 3. Visual and automatic sleep spindle detections in a 20 s N2 sleep segment recorded from a healthy adult male. Unfiltered EEG with the visual detections of an expert rater are shown in the top data line. Appropriately filtered EEG with automatic (IAM and FixF) slow/fast spindle detections are shown in the bottom 4 data lines.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

Acır, N., and Güzeliş, C. (2004). Automatic recognition of sleep spindles in EEG by using artificial neural networks. Expert Syst. Appl. 27, 451–458. doi: 10.1016/j.eswa.2004.05.007

CrossRef Full Text | Google Scholar

Amzica, F., and Steriade, M. (2000). Integration of low-frequency sleep oscillations in corticothalamic networks. Acta Neurobiol. Exp. (Wars) 60, 229–245.

Pubmed Abstract | Pubmed Full Text | Google Scholar

Andrillon, T., Nir, Y., Staba, R. J., Ferrarelli, F., Cirelli, C., Tononi, G., et al. (2011). Sleep spindles in humans: insights from intracranial EEG and unit recordings. J. Neurosci. 31, 17821–17834. doi: 10.1523/JNEUROSCI.2604-11.2011

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Bang, J. W., Khalilzadeh, O., Hämäläinen, M., Watanabe, T., and Sasaki, Y. (2014). Location specific sleep spindle activity in the early visual areas and perceptual learning. Vision Res. 99, 162–171. doi: 10.1016/j.visres.2013.12.014

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Bódizs, R., Kis, T., Lazar, A. S., Havran, L., Rigo, P., Clemens, Z., et al. (2005). Prediction of general mental ability based on neural oscillation measures of sleep. J. Sleep Res. 14, 285–292. doi: 10.1111/j.1365-2869.2005.00472.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Bódizs, R., Körmendi, J., Rigó, P., and Lázár, A. S. (2009). The individual adjustment method of sleep spindle analysis: methodological improvements and roots in the fingerprint paradigm. J. Neurosci. Methods 178, 205–213. doi: 10.1016/j.jneumeth.2008.11.006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Bódizs, R., Lázár, A., and Rigo, P. (2008). Correlation of visuospatial memory ability with right parietal EEG spindling during sleep. Acta Physiol. Hung. 95, 297–306. doi: 10.1556/APhysiol.95.2008.3.5

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Broughton, R., Healey, T., Maru, J., Green, D., and Pagurek, B. (1978). A phase locked loop device for automatic detection of sleep spindles and stage 2. Electroencephalogr. Clin. Neurophysiol. 44, 677–680. doi: 10.1016/0013-4694(78)90134-7

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Campbell, K., Kumar, A., and Hofman, W. (1980). Human and automatic validation of a phase-locked loop spindle detection system. Electroencephalogr. Clin. Neurophysiol. 48, 602–605. doi: 10.1016/0013-4694(80)90296-5

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Carrier, J., Land, S., Buysse, D. J., Kupfer, D. J., and Monk, T. H. (2001). The effects of age and gender on sleep EEG power spectral density in the middle years of life (ages 20–60 years old). Psychophysiology 38, 232–242. doi: 10.1111/1469-8986.3820232

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Chatburn, A., Coussens, S., Lushington, K., Kennedy, D., Baumert, M., and Kohler, M. (2013). Sleep spindle activity and cognitive performance in healthy children. Sleep 36, 237–243. doi: 10.5665/sleep.2380

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Clemens, Z., Fabo, D., and Halasz, P. (2005). Overnight verbal memory retention correlates with the number of sleep spindles. Neuroscience 132, 529–535. doi: 10.1016/j.neuroscience.2005.01.011

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

De Gennaro, L., Ferrara, M., Vecchio, F., Curcio, G., and Bertini, M. (2005). An electroencephalographic fingerprint of human sleep. Neuroimage 26, 114–122. doi: 10.1016/j.neuroimage.2005.01.020

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

De Gennaro, L., Marzano, C., Fratello, F., Moroni, F., Pellicciari, M. C., Ferlazzo, F., et al. (2008). The electroencephalographic fingerprint of sleep is genetically determined: a twin study. Ann. Neurol. 64, 455–460. doi: 10.1002/ana.21434

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Devuyst, S., Dutoit, T., Didier, J. F., Meers, F., Stanus, E., Stenuit, P., et al. (2006). Automatic sleep spindle detection in patients with sleep disorders. Conf. Proc. IEEE Eng. Med. Biol. Soc. 1, 3883–3886. doi: 10.1109/IEMBS.2006.259298

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Driver, H. S., Dijk, D. J., Werth, E., Biedermann, K., and Borbely, A. A. (1996). Sleep and the sleep electroencephalogram across the menstrual cycle in young healthy women. J. Clin. Endocrinol. Metab. 81, 728–735.

Pubmed Abstract | Pubmed Full Text | Google Scholar

Duman, F., Erdamar, A., Erogul, O., Telatar, Z., and Yetkin, S. (2009). Efficient sleep spindle detection algorithm with decision tree. Expert Syst. Appl. 36, 9980–9985. doi: 10.1016/j.eswa.2009.01.061

CrossRef Full Text | Google Scholar

Ferri, R., Dequae, P., Colognola, R. M., Musumeci, S. A., and Bergonzi, P. (1989). A simple electronic and computer system for automatic spindle detection. Neurophysiol. Clin. 19, 171–177. doi: 10.1016/S0987-7053(89)80057-7

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Fogel, S. M., Nader, R., Cote, K. A., and Smith, C. T. (2007). Sleep spindles and learning potential. Behav. Neurosci. 121, 1–10. doi: 10.1037/0735-7044.121.1.1

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Fogel, S. M., and Smith, C. T. (2006). Learning-dependent changes in sleep spindles and Stage 2 sleep. J. Sleep Res. 15, 250–255. doi: 10.1111/j.1365-2869.2006.00522.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Fogel, S. M., and Smith, C. T. (2011). The function of the sleep spindle: a physiological index of intelligence and a mechanism for sleep-dependent memory consolidation. Neurosci. Biobehav. Rev. 35, 1154–1165 doi: 10.1016/j.neubiorev.2010.12.003

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Geiger, A., Huber, R., Kurth, S., Ringli, M., Jenni, O. G., and Achermann, P. (2011). The sleep EEG as a marker of intellectual ability in school age children. Sleep 34, 181–189.

Pubmed Abstract | Pubmed Full Text | Google Scholar

Genzel, L., Kiefer, T., Renner, L., Wehrle, R., Kluge, M., Grozinger, M., et al. (2012). Sex and modulatory menstrual cycle effects on sleep related memory consolidation. Psychoneuroendocrinology 37, 987–998. doi: 10.1016/j.psyneuen.2011.11.006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Genzel, L., Kroes, M. C., Dresler, M., and Battaglia, F. P. (2014). Light sleep versus slow wave sleep in memory consolidation: a question of global versus local processes? Trends Neurosci. 37, 10–19. doi: 10.1016/j.tins.2013.10.002

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Gottselig, J. M., Bassetti, C. L., and Achermann, P. (2002). Power and coherence of sleep spindle frequency activity following hemispheric stroke. Brain 125, 373–383. doi: 10.1093/brain/awf021

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Gruber, R., Wise, M. S., Frenette, S., Knäauper, B., Boom, A., Fontil, L., et al. (2013). The association between sleep spindles and IQ in healthy school-age children. Int. J. Psychophysiol. 89, 229–240. doi: 10.1016/j.ijpsycho.2013.03.018

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Huupponen, E., Gómez-Herrero, G., Saastamoinen, A., Värri, A., Hasan, J., and Himanen, S.-L. (2007). Development and comparison of four sleep spindle detection methods. Artif. Intell. Med. 40, 157–170. doi: 10.1016/j.artmed.2007.04.003

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Huupponen, E., Himanen, S. L., Varri, A., Hasan, J., Lehtokangas, M., and Saarinen, J. (2002). A study on gender and age differences in sleep spindles. Neuropsychobiology 45, 99–105. doi: 10.1159/000048684

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Huupponen, E., Varri, A., Himanen, S. L., Hasan, J., Lehtokangas, M., and Saarinen. (2000). Optimization of sigma amplitude threshold in sleep spindle detection. J. Sleep Res. 9, 327–334. doi: 10.1046/j.1365-2869.2000.00220.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text

Iber, C., Ancoli-Israel, S., Chesson, A., and Quan, S. (2007). The AASM Manual for the Scoring of Sleep and Associated Events: Rules, Terminology and Technical Specification. Westchester, IL: American Academy of Sleep Medicine.

Lustenberger, C., Maric, A., Durr, R., Achermann, P., and Huber, R. (2012). Triangular relationship between sleep spindle activity, general cognitive ability and the efficiency of declarative learning. PLoS ONE 7:e49561. doi: 10.1371/journal.pone.0049561

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Nonclercq, A., Urbain, C., Verheulpen, D., Decaestecker, C., Van Bogaert, P., and Peigneux, P. (2013). Sleep spindle detection through amplitude-frequency normal modelling. J. Neurosci. Methods 214, 192–203. doi: 10.1016/j.jneumeth.2013.01.015

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Piantoni, G., Poil, S. S., Linkenkaer-Hansen, K., Verweij, I. M., Ramautar, J. R., Van Someren, E. J., et al. (2013). Individual differences in white matter diffusion affect sleep oscillations. J. Neurosci. 33, 227–233. doi: 10.1523/JNEUROSCI.2030-12.2013

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Ray, L. B., Fogel, S. M., Smith, C. T., and Peters, K. R. (2010). Validating an automated sleep spindle detection algorithm using an individualized approach. J. Sleep Res. 19, 374–378. doi: 10.1111/j.1365-2869.2009.00802.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Saletin, J. M., Van Der Helm, E., and Walker, M. P. (2013). Structural brain correlates of human sleep oscillations. Neuroimage 83, 658–668. doi: 10.1016/j.neuroimage.2013.06.021

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Schabus, M., Dang-Vu, T. T., Albouy, G., Balteau, E., Boly, M., Carrier, J., et al. (2007). Hemodynamic cerebral correlates of sleep spindles during human non-rapid eye movement sleep. Proc. Natl. Acad. Sci. 104, 13164–13169. doi: 10.1073/pnas.0703084104

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Schabus, M., Hodlmoser, K., Gruber, G., Sauter, C., Anderer, P., Klosch, G., et al. (2006). Sleep spindle-related activity in the human EEG and its relation to general cognitive and learning abilities. Eur. J. Neurosci. 23, 1738–1746. doi: 10.1111/j.1460-9568.2006.04694.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Schabus, M., Hoedlmoser, K., Pecherstorfer, T., Anderer, P., Gruber, G., Parapatics, S., et al. (2008). Interindividual sleep spindle differences and their relation to learning-related enhancements. Brain Res. 1191, 127–135. doi: 10.1016/j.brainres.2007.10.106

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Schimicek, P., Zeitlhofer, J., Anderer, P., and Saletu, B. (1994). Automatic sleep-spindle detection procedure: aspects of reliability and validity. Clin. Electroencephalogr. 25, 26–29. doi: 10.1177/155005949402500108

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Schonwald, S. V., Carvalho, D. Z., Dellagustin, G., De Santa-Helena, E. L., and Gerhardt, G. J. (2011). Quantifying chirp in sleep spindles. J. Neurosci. Methods 197, 158–164. doi: 10.1016/j.jneumeth.2011.01.025

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Sitnikova, E., Hramov, A. E., Grubov, V., and Koronovsky, A. A. (2014). Time-frequency characteristics and dynamics of sleep spindles in WAG/Rij rats with absence epilepsy. Brain Res. 16, 290–299. doi: 10.1016/j.brainres.2013.11.001

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Steriade, M. (2000). Corticothalamic resonance, states of vigilance and mentation. Neuroscience 101, 243–276. doi: 10.1016/S0306-4522(00)00353-5

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Steriade, M., and Deschenes, M. (1984). The thalamus as a neuronal oscillator. Brain Res. 320, 1–63. doi: 10.1016/0165-0173(84)90017-1

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Tucker, M. A., and Fishbein, W. (2009). The impact of sleep duration and subject intelligence on declarative and motor memory performance: how much is enough? J. Sleep Res. 18, 304–312. doi: 10.1111/j.1365-2869.2009.00740.x

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Ujma, P. P., Konrad, B. N., Genzel, L., Bleifuss, A., Simor, P., Pótari, A., et al. (2014). Sleep spindles and intelligence: evidence for a sexual dimorphism. J. Neurosci. 34, 16358–16368. doi: 10.1523/JNEUROSCI.1857-14.2014

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Vasko, R. C., Brunner, D. P., Monahan, J. P., Doman, J., Boston, J. R., El-Jaroudi, A., et al. (1997). Power spectral analysis of EEG in a multiple-bedroom, multiple-polygraph sleep laboratory. Int. J. Med. Inform. 46, 175–184. doi: 10.1016/S1386-5056(97)00064-6

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Ventouras, E. M., Monoyiou, E. A., Ktonas, P. Y., Paparrigopoulos, T., Dikeos, D. G., Uzunoglu, N. K., et al. (2005). Sleep spindle detection using artificial neural networks trained with filtered time-domain EEG: a feasibility study. Comput. Methods Programs Biomed. 78, 191–207. doi: 10.1016/j.cmpb.2005.02.006

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar

Warby, S. C., Wendt, S. L., Welinder, P., Munk, E. G. S., Carrillo, O., Sorensen, H. B. D., et al. (2014). Sleep-spindle detection: crowdsourcing and evaluating performance of experts, non-experts and automated methods. Nat Methods 11, 385–392. doi: 10.1038/nmeth.2855

Pubmed Abstract | Pubmed Full Text | CrossRef Full Text | Google Scholar