Cell density quantification of high resolution Nissl images of the juvenile rat brain

Nissl histology underpins our understanding of brain anatomy and architecture. Despite its importance, no high-resolution datasets are currently available in the literature for 14-day-old rats. To remedy this issue and demonstrate the utility of such a dataset, we have acquired over 2000 high-resolution images (0.346 μm per pixel) from eight juvenile rat brains stained with cresyl violet. To analyze this dataset, we developed a semi-automated pipeline using open-source software to perform cell density quantification in the primary somatosensory hindlimb (S1HL) cortical column. In addition, we performed cortical layer annotations both manually and using a machine learning model to expand the number of annotated samples. After training the model, we applied it to 262 images of the S1HL, retroactively assigning segmented cells to specific cortical layers, enabling cell density quantification per layer rather than just for entire brain regions. The pipeline improved the efficiency and reliability of cell density quantification while accurately assigning cortical layer boundaries. Furthermore, the method is adaptable to different brain regions and cell morphologies. The full dataset, annotations, and analysis tools are made publicly available for further research and applications.

1 Introduction

Cell density and brain cytoarchitecture are important for many reasons. The spatial arrangement and density of specific cell types are essential for the function of neuronal microcircuits (Neurohr and Amon, 2020). Proper characterization of cell densities facilitates investigation of pathologies such as neurodegenerative diseases that result in the loss of selective cell populations. Cell density and layer information are also useful for construction of detailed cortical microcircuit models (Markram et al., 2015), though quantitative literature remains sparse (Keller et al., 2018). In this study, we used postnatal age 14 days (P14) animals to compare the data with previous electrophysiological studies. Electrophysiology recordings on P14 animals are easier to perform as juvenile cells are less myelinated.

The Nissl stain is widely used in neuroscience to count the total number of cells, yet data for the juvenile rat Wistar Han at age 14 days is lacking. Our review of available rat atlas revealed no existing resources for this age group. Additionally, no adult-age atlases provide sufficient image resolution or magnification to enable accurate cell counting and delineation of cortical layer boundaries in the specific region of interest (S1HL). While several adult rat atlases exist for different strains, including the Wistar Han (Paxinos and Watson, 2014; Blixhavn et al., 2023; Johnson et al., 2012), the Sprague Dawley (Papp et al., 2014) and the Fischer 344 (Goerzen et al., 2020), none offer adequate histological data for the Wistar Han strain.

For adult Wistar Han rats, histological data is limited to the reference atlas by Paxinos and Watson (2014) and the Nissl-Thionine dataset by Blixhavn et al. (2023). However, the image resolution in Paxinos and Watson (2014) is unspecified, the dataset is not publicly available, and the images are presented in PDF format, which is not suitable for cell counting. Although the Blixhavn et al. (2023) atlas includes higher-resolution images from a single adult animal, the publicly available files appear to be down-sampled, further limiting their utility.

In contrast, the mouse atlases from the Allen Institute for Brain Science (2004) cover both juvenile and adult (P56) ages. However, these atlases also present limitations: the image resolution is three times lower than our study and thus insufficient for reliable cell counting and the images are not consecutive, with a 200 μm gap between samples. Moreover, while these atlases define brain regions, they do not provide cell counts, and most datasets are derived from a single animal.

The cerebral cortex is known for its distinct cytoarchitecture, with variations in the size and subdivisions of its cortical layers across different brain areas and species (Palomero-Gallagher and Zilles, 2019). The primary somatosensory hind limb area (S1HL) exhibits a characteristic six-layer structure, including a well-developed layer IV, comparable to that of primates (García-Cabezas et al., 2022; Briggs, 2010). Accurate quantification of cell densities in these cortical layers is essential for understanding cortical organization.

Manual stereology remains the most rigorous method for cell density quantification, but it is both time-consuming and susceptible to observer bias, as subjective differences in cell counting can lead to inconsistencies (West, 1999). From a standpoint of consistency and efficiency, automatic or semi-automatic methods represent a significant improvement over manual approaches.

In this study, we compiled a robust dataset of eight rodent brains stained with cresyl violet, a conventional Nissl stain commonly used for visualizing cell bodies via bright-field microscopy, which provides an accurate estimate of total cell numbers (García-Cabezas et al., 2016; Gurr, 1960; Nestor, 2008; Warr et al., 1981). To streamline the analysis, we enhanced an open-source cell detector for improved segmentation accuracy and developed a semi-automated pipeline for cell segmentation, stereological exclusion, and cell density analysis. These advancements significantly reduced the time required for traditionally laborious tasks.

Machine learning (ML) methods have been previously employed for layer discrimination in the isocortex (Li et al., 2019; Štajduhar et al., 2023; Wagstyl et al., 2020). Building on these approaches, we developed a pipeline that uses similar ML methods to provide a detailed laminar description of the somatosensory cortex in juvenile rats. This method allows for the retroactive assignment of segmented cells to specific layers, enabling the quantification of cell densities within individual layers, not just across broader brain regions.

By utilizing the same strain and age groupe as previously published studies (Keller et al., 2019), our dataset can be directly compared and incorporated into larger studies. Although the pipeline was primarily developed for the analysis of the somatosensory cortex, we tested it on another cortical region.

To our knowledge, this is the first time such a large dataset has been made publicly available, even though it does not qualify as an atlas due to the slicing angle used. Additionally, this is the first instance in which a complete pipeline—from histology processing to cortical layer identification—has been made available to the public. Finally, this works represents the first application of ML techniques to define cortical layer boundaries in the somatosensory cortex of the rat.

2 Materials and methods

2.1 Animals

All animal procedures were approved by the Veterinary Authorities and the Cantonal Commission for Animal Experimentation of the Canton of Vaud, according to the Swiss animal protection laws, under authorization number VD3516. The animals originated from the same provider and from three different litters. Each litter was processed individually on different experimental days.

2.2 Sample preparation

On postnatal day fourteen, rats were transferred to the experimental room in the morning to acclimate. The described procedure was conducted within a consistent 3-h window of the day (09:00–12:00). Initially, the rats were deeply anesthetized using pentobarbital (intraperitoneal dose of 150 mg/kg; concentration of 150 mg/mL). This was succeeded by transcardial perfusion with ice cold 0.1 M phosphate buffer (PB; pH 7.4), followed by cold 4% paraformaldehyde (PFA) in 0.1 M PB. Subsequently, the brain was carefully removed from the skull, post fixed at 4°C in 4% PFA overnight, and then rinsed in 0.1 M PB. The brains underwent a sequential storage process: first in a 15% sucrose solution (in 0.1 M PB) at 4°C for ~24 h, followed by a 30% sucrose solution at 4°C for an additional 24 h. The hemispheres were carefully divided along the midline, after which both right and left hemispheres were precisely sliced sagittally using a cryostat (Leica, VT-1200S) at 50 μm employing an approximate angle rotation of 4 ± 1 degrees along the anterior-posterior axis to optimize alignment with apical dendrites. These brain slices were stored in a cryoprotectant solution (30% v/v ethylene glycol; 30% m/v sucrose in 0.1 M PB) at –20°C, preserving them until cytochemistry assays were executed (within a maximum of two weeks from extraction to cytochemistry).

Brain slices were stained using cresyl violet, a stain specifically targeting cell bodies, including the endoplasmic reticulum, also known as Nissl substance or Nissl bodies. Free-floating sections of 50 μm thickness were transferred from cryoprotectant into 0.1 M PB to thaw and eliminate any cryoprotectant remnants. Subsequently, they were transferred into 0.01 M PB to minimize salt residues before being meticulously mounted onto SuperFrost©glass slides (Thermo Fisher Scientific Inc., Gerhard Menzel B.V. and Co. KG, GE). This mounting was carried out while considering the brain's orientation relative to the midline, from its external to internal regions. Slide-mounted sections were processed using an automated slide stainer Tissue-Tek^® Prisma Plus (Sakura Finetek-Europe, NL). These sections were incubated for 6 min at room temperature (RT = 20°C) in a 0.5% cresyl violet solution in water (with pH adjusted to 2.85 using acetic acid), followed by a brief wash in tap water. The sections underwent dehydration through a series of ethanol concentrations (70, 70, 96, 100, 100%) with each step lasting one minute at RT. Subsequently cleared with two steps of xylene for one minute each at RT, and the sections were mounted using Pertex (Sakura Finetek-Europe, NL) before being cover-slipped using the automated glass coverslipper Tissue-Tek^® Glas™ g2 (Sakura Finetek-Europe, NL). A meticulous assessment of the coloration was conducted and if the staining appeared faint, a repeat staining procedure was carried out.

Stained slides were scanned using an automated slide scanner (Olympus, VS120-L100, GER) equipped with a UPLSAPO 20x/0.75 air objective (Olympus, GER) and a Pike F505 Color camera leading to a pixel size of 0.346 μm/pixel. This is three times higher than commonly used atlases (Allen Institute for Brain Science, 2004; Paxinos and Watson, 2014) (see also Table 1). Each brain slice was entirely scanned. Subsequently, the digital images obtained were meticulously organized and subjected to analysis using the open-source software QuPath v0.3.2 (Bankhead et al., 2017). The tasks of cell segmentation training, cell density analysis, and ML were analyzed employing tools from the Python programming language. We used an imaging pixel size of 0.346 × 0.346 μm on the whole dataset.

Table 1

Table 1. Datasets comparison.

Detailed information regarding the distinct experimental procedures, duration days within the cryoprotectant, incubation times for staining, the count of slices containing S1HL, the methodology of cell detection have been documented and can be found in Supplementary material.

2.3 Cell segmentation

We organized the dataset by structuring the QuPath projects on a per-hemisphere basis (i.e., one project dedicated to each hemisphere, as indicated in Supplementary Table 1). To determine the slice position, the brain distance to midline axis was evaluated by visually matching the closest sagittal section from the Paxinos and Watson (2014) atlas to each sample. This positional data was then integrated into each image as a metadata field called “distance to midline.” Due to the age difference of the specimens between our study and the Paxinos and Watson atlas (respectively, juvenile 40 grams vs. adult 270 grams), the resolution of the atlas plates (50 μm between slices vs. 250 μm between slices for the atlas) and the inherent angle in our slicing procedure (4 ± 1 degrees angle along the anterior-posterior axis vs. 0° for the atlas), a direct comparison of our images with the atlas is not entirely feasible. Because of this slicing angle, the part of the slice spanning from the pia to the hippocampus closely aligns with the published atlas, whereas a linear shift becomes apparent as we move from the hippocampus to the cerebral peduncle. To define the somatosensory cortex anterior border, we used the fact that Layer IV is absent in the motor cortex, whereas it is present in the somatosensory cortex. This helps to determine the start of the S1 region and to exclude slices without a visible Layer IV. To confirm the accuracy of the assigned “distance to midline” values for the acquired images, an expert assessment was conducted.

Each image falling within the S1HL range (spanning from lateral coordinates 1.90–3.18 mm) underwent delineation within the QuPath software. This delineation encompassed defining both the slice contour (SliceContour) and the S1HL brain region (S1HL). Initially, the SliceContour was roughly delineated using a pixel-based thresholder (Resolution: 11 μm/px, RGB average intensity, Gaussian Blur sigma: 4 px, threshold: 211.5) (“Pixel classification—QuPath 0.3.0 documentation,” n.d.). Manual refinement was conducted, with particular focus directed toward obtaining an accurate cortex segmentation. To ensure the integrity of cell density calculations and avoid introducing bias, the cellular aggregates near the pia were excluded from the SliceContour annotation. This exclusion was necessary due to the non-uniform representation of these aggregates in every slice, attributable to cutting artifacts.

We undertook a manual delineation process to define the S1HL brain region and closely replicate the corresponding Paxinos and Watson atlas plate as accurately as possible. The upper boundary of the S1HL region precisely followed the contours outlined in the SliceContour annotation, while the lower boundary was established by the initiation point of the cingulum brain region. In addition, we incorporated some pertinent metadata to each slice, such as a comment section (the “distance to midline” information, and a classification indicating whether the image should be included in the semi-automated analysis: Analyze/True or Analyze/False). This systematic approach was consistently applied across all datasets to ensure the possibility of batch processing.

We opted for using Cellpose, a general deep learning-based segmentation method described to be the most precise in the IoU comparison of different models (Cellpose, Mask R-CNN, Stardist, U-Net3 and U-Net2) (Stringer et al., 2021). Although Cellpose does struggle with handling overlapping cells, this limitation does not impact the primary objective of classifying brain regions based on cell shape and localization features. With Cellpose integrated into QuPath, we were able to generate ground truth cell data directly within the software, which facilitated the training of a custom model. Initially, we selected four images that represent the S1HL region, each sourced from distinct animals, litters, and points along the distance-to-midline axis. On those four images, we randomly positioned three validation and 24 training squares, with each square being (177,16 × 177,16 μm). After applying the pre-trained Cellpose model cyto2 (Stringer et al., 2021) to each square, we engaged in manual corrections through visual inspection and curation of the identified cells. In cases where two cells were lightly stained and overlapping, they were counted as one cell. Similarly, if a darker and smaller cell overlapped with a fainter one, we accurately accounted for both entities. It is important to note that the analysis specifically excluded capillaries, endothelial cells, and the cellular aggregates near the pia. Moreover, the ground truth dataset includes not only neuronal cells but a diverse range of cell types including astrocytes, oligodendrocytes, neurons and glial cells.

Following this refinement, we proceeded to train a custom Cellpose model, denoted as v1. In the interest of enhancing precision, we imposed constraints on the v1 detection phase. This involved considering stacked cells as a single entity instead of two (as in v1), and ensuring that the shape was accurate at the edge of the regions. This led to a refined series of ground truth annotations, in order to train a new custom model v2. The cell segmentation quality was assessed using three recommended metrics developed by an international consortium (Maier-Hein et al., 2022). Unpaired Welch's t-tests were applied to compare results from cyto2 and v2 mean predictions (see Section 2.7). Then, in order to obtain a reference for the performance of the automated counts, we compared these with cell counts from another human annotator. We selected one of the images used for the segmentation training steps and assigned a second human to correct the segmentation provided by cyto2. We then compared this second human annotation (Human 2) and the segmenter final counts v2 to the original human ground truth (Human 1). The accuracy between Human 1 and Human 2 or v2 were also compared using the Welch's t-test.

Once cell segmentation was achieved, reference points (S1HL region of interest (ROI), Outside pia; S1HL contour, S1HL top-right point, S1HL top-left point, S1HL bottom-right point, S1HL bottom-left point) and custom cell measurements (image; name; centroid -x in μm; centroid -y in μm; cell maximum diameter in μm; cell minimum diameter in μm; the distance to annotation with the outside pia in μm) were extracted from QuPath to proceed with stereological analysis.

2.4 Stereology and cell density

The conventional stereological approach to prevent cell overcounting during consecutive slice density estimation involves excluding cells on half of the boundaries of the sampled areas (West, 1999; West and Gundersen, 1990). In our approach, we counted all cells included in the S1HL region without applying such a correction in the -x and -y directions because all cells were fully counted and not sampled. However, in the -z direction, corresponding to the slice thickness and because our datasets contained consecutive slices, such a correction remained necessary. To derive a reasonable -z correction factor, a method involving a random assignment of a -z coordinate and an appropriate diameter to each cell was employed. The cell diameter was estimated from the six nearest neighboring cells, by computing a mean diameter between these cells. With this information, it is possible to identify cells touching the upper surface of the slice. We compared the -z coordinate of the cell added to the half of its diameter to the -z coordinate of the upper surface of the slice. The procedure was iterated a hundred times to facilitate bootstrapping, a statistical procedure. We randomly resampled the cell -z position to create many simulated samples. Once the stereology correction factor was obtained, cell densities as a function of the percentage of depth inside the somatosensory cortex can be calculated.

Initially, the cell centroid coordinates for the -x and -y axis as well as the S1HL polygon coordinates were exported from QuPath and the S1HL polygon cartesian point coordinates were converted into a shapely polygon (a two-dimensional feature with a non-zero area enclosed by a linear ring). Cells outside of the polygon were excluded.

A nonlinear grid consisting of 10 columns and 20 rows was generated within the S1HL polygon. Straight vertical lines were calculated using the endpoints derived from the quadrilateral's top and bottom lines, ensuring an even division. The horizontal lines were formed by multiple segments that mimic the shape of the quadrilateral's top and bottom lines, effectively depicting the brain's depth. The resulting split polygons followed the S1HL's top and bottom lines. This nonlinear grid method was constrained to a convex polygon shape. When the method was applied to a concave shape, we realized that split polygons were not constrained anymore in the horizontal axis, resulting in cells from different somatosensory depth being wrongly considered in the same polygon. Slices with a concave S1HL shape were then excluded from further cell density analysis (slices spanning from lateral coordinates of 3.10–3.25 mm). The resulting S1HL size is consequently reduced within lateral coordinates of 1.90–3.05 mm.

For each split polygon, we counted the number of cells and computed its volume by using the theoretical slice thickness of 50 μm (volume in mm³). The cell densities were then computed as a function of the percentage of the somatosensory depth. For each of the density computed per depth percentage, the mean values and the standard deviation were calculated (cells/mm³).

2.5 Cortical layer boundaries

Defining layer boundaries in an image is challenging for humans. To establish precise laminar delineation within our datasets, we used (Palomero-Gallagher and Zilles, 2019) layering properties. The cortical column extends from Layer I to Layer VI and each layer has a different and representative cell population. Layer I contains small soma sizes with round shape and is sparse in cell density; Layer II contains medium soma sizes with triangular soma shapes and is denser than Layer I; Layer III contains a broader range of soma sizes, with a majority of ovoid shape and has a slightly sparser density than Layer II; Layer IV contains a greater number of small soma sizes with similar ovoid shape and has a similar density as Layer II; Layer V contains small, medium and large soma sizes with a lower density compared with Layer IV (Layer Va) and in the transition to Layer VI, the soma sizes increase drastically and become more distinctly pyramidal in shape (large pyramidal cell population; Layer Vb); Layer VIa has an average to dense cell density, with small and medium soma sizes and lacks large pyramidal cells; Layer VIb is sparse, with a majority of small soma oriented horizontally. The white matter clearly defines the bottom of Layer VI.

We manually curated a subset of thirty-eight distributed images, with each hemisphere represented by two to three images covering the entire S1HL volume. The SliceContour and S1HL annotation, as well as the S1HL cell segmentation (v2) output were imported from the cell segmentation phase. The manually annotated images were curated to form a dataset that can be used to train and evaluate a model (“layer boundaries expert-classified dataset”). In line with the classification established by Brodmann (1909), we-human identified and categorized the following cortical layers: Layer I, Layer II, Layer III, Layer II/III, Layer IV, Layer V, Layer VIa, Layer VIb. In instances where conditions allowed for a clear visual differentiation between Layer II and Layer III, we ensured their distinct separation (Layer II and Layer III). Otherwise, in cases where visual separation was not feasible, we opted to merge these layers (Layer II/III). For measurement purposes, the annotation “Outside Pia” was added, the cell positions registered and the corresponding result file exported.

Once this baseline expert-classified dataset was defined, a supervised ML approach for cell classification in cortical layers was developed. The goals were to decrease the bias that can occur between human annotators and to improve the annotation throughput. Careful feature selection using the permutation importance algorithm (Breiman, 2001) was carried out to reduce the dimensionality of the data and improve the model's performance. Only features with a positive impact on the predictions were kept. This ML task can be defined as a multiclass classification problem at the cell level (i.e., each cell within an image has to be classified as belonging to one of the seven cortical layers classes: Layer I, Layer II, Layer III, Layer IV, Layer V, Layer VIa or Layer VIb). To train the models, the dataset was split into images for training and others used for testing. Models of varying degrees of complexity (such as image segmentation models) could be employed to solve the task. However, due to the large number of pixels in the sample set [one full image contains roughly 46,000 (width) × 31,000 (height) pixels], employing complex convolutional neural network architectures like U-Net (Ronneberger et al., 2015) would result in significant computational overhead and increase processing times without necessarily yielding improved results. Furthermore, we would lose the rich features gained from QuPath, such as area, length, circularity, solidity of the cells. Therefore, we did not explore the feasibility of using segmentation models, but structured the problem as a cell level classification task instead.

The classification displays an intrinsic spatial relationship as each layer sequentially appears in the slices. We therefore experimented with simpler models such as a K-Nearest Neighbor (KNN) (Cover and Hart, 1967) and a Random Forest (RF) (Breiman, 2001). These models were evaluated according to four metrics: accuracy, precision, recall and F1-score (Fawcett, 2006), using a 10-fold cross-validation (Hastie et al., 2001) method. These metrics provide good insights into the model performance, and facilitate comparison across models. Notably, they are computed by averaging the per-class metrics for each model, ensuring that classes with varying sample sizes contribute equally to the final metric values. We tested the models on two paradigms: one in which Layer II was separated from Layer III (LII and LIII distinguished), and another in which Layer II was merged with Layer III (LII/LIII merged). Once the model was trained, we applied it to naive data and assigned a predicted layer to each cell. For each image, the predicted layer to each cell was registered.

2.6 Cell densities and size predictions

With the cells segmented and the predicted layer assigned to each cell, the cell densities and the cell soma sizes as a function of the somatosensory predicted layer can be calculated. We first created the smallest polygon possible containing all the cells of a given predicted layer using the Alpha-Concave hull method (Asaeedi et al., 2017). In this method, the alpha value (α) can be adjusted to tighten or loosen the polygon, thereby integrating points inside or outside the concave hull.

To generalize the method and account for the slight variations in cell density between images, we aimed to find a single α value that could be applied universally. For each α tested, we measured the bounding area surrounding each layer and counted the rejected cells. We selected the value that created a bounding area as close as possible to the actual layer shape without rejecting any cells. This empirical testing approach provided a single α value applicable to all layers, except for Layer I. Due to the much lower cell density in Layer I, the method yielded a smaller α value, corresponding to one-tenth of the value used for the other layers.

The volume of each alpha shape polygon, corresponding to each layer, was computed using the theoretical slice thickness of 50 μm. The number of cells within each alpha shape polygon was counted, and the cell density was subsequently calculated. The cell soma size was determined by utilizing the area (μm²) feature of each cell, assuming a perfect circular morphology. From this, a mean diameter was calculated for analysis.

We applied statistical tests to determine if a bimodal Gaussian for to the cell diameter distribution produces statistically different means. We also tested if there are statistically different cell density differences across layers (see statistical methods).

2.7 Statistical methods

The cell segmentation quality was assessed using several metrics. The Dice Similarity Coefficient (DSC) is the most widely used counting metric in medical image analysis. It measures the overlap between the prediction and the reference and yields a value between 0, for no overlap, and 1, in the case of full overlap. The Intersection over Union (IoU) metric quantifies the degree of overlap between two objects. As IoU approaches 1, the annotation converges to ground truth. The accuracy, commonly used in classification tasks to assess the overall detection performance (with respect to the ground truth), was calculated as follows:

$\begin{array}{ll}\text{Accuracy}=\frac{\text{True Positives}}{\text{True Positives + False Positives + False Negatives}}& (1)\end{array}$

The advantages and disadvantages of each metric are listed in Stringer et al. (2021)).

Unpaired Welch's t-tests were applied to compare results from cyto2 and v2 mean predictions. Results are plotted and p-values are reported with asterisk convention (e.g., *corresponds to p-value ≤ 0.1, ***corresponds to p-value ≤ 0.001). The unpaired Welch's t-test assumes that the means being compared are normally distributed, but does not assume that the samples come from populations with equal variance. The segmentation accuracy between Human 1 and Human 2 or v2 was also compared using the Welch's t-test.

The distribution of cell diameters followed a bimodal Gaussian distribution. Using scikit-learn's GaussianMixture model, we estimated the parameters of a Gaussian mixture distribution for the cell diameters on an image-by-image basis (228 images). This approach yielded two mean diameters for each layer in each image: one representing the smaller cell population and one for the larger cell population. Next, we calculated the average smaller and larger mean diameters across all images for each layer. To assess whether these two mean diameters significantly differ within each layer, we conducted two-sample t-tests, applying a p-value threshold of 0.05 to evaluate the significance.

To determine the statistical significance of cell density differences across layers, first, cell density for each cortical layer and in each image was calculated. Second, these cell density values were aggregated into seven datasets, each representing a distinct cortical layer. Finally, the Kruskal-Wallis H-test using the SciPy kruskal function was performed on the seven datasets, and p-value was calculated to determine the statistical significance of cell density differences across layers. Finally, We used two-sample t-tests corrected for multiple comparisons with the Bonferroni method adjusting the alpha value for the six comparisons (α = 0.05/6 = 0.0083) to assess the difference in cell density between adjacent layers.

We observed a significant improvement in the segmenter accuracy (cyto2: 0.2457 ± 0.0596; v2: 0.5554 ± 0.0643) and in the overlap between the prediction and the reference (Dice Similarity Coefficient metric, DSC) for v2 compared with cyto2 (cyto2: 0.3908 ± 0.0768; v2: 0.7119 ± 0.0542). The intersection over the union (IoU) improvement was below a threshold of p = 0.05 and then cannot be considered statistically significant (cyto2: 0.6383 ± 0.0677; v2: 0.7260 ± 0.0626). A possible explanation for the generalized increase in all metrics can be an improvement in the segmentation task, as v2 seemed to be less biased in favor of annotating larger cells than cyto2. Although statistical analysis with (N_Image = 4) was at the low end limit of applicability due to samples size, the cell segmentation improvement allowed us to calculate and refine the rat somatosensory cortical cell density numbers previously published (Keller et al., 2019; Markram et al., 2015).

3 Results

3.1 Sample preparation

The 8 animals behaved normally and no brain abnormality was noticed at the organ collection. Three had their eyes closed, two had their eyes starting to open, and three had their eyes opened. The animal weights were comprised between 26.8 and 45.6 grams with a mean of 35.48 ± 7.90 grams. Sixteen hemispheres were completely sliced at a thickness of 50 μm and the Nissl staining protocol was applied on 2,077 slices.

During processing, one batch of staining required corrections. Specifically, one dataset needed re-staining due to an acetic acid step that caused a reduction in staining intensity. To rectify this issue, these slides underwent a de-coverslipping process by immersing them in a xylene bath. Subsequently, the cresyl violet staining procedure was repeated. The goal was to restore the precise color pattern and to match the quality and characteristics of the previous successful stainings.

In another dataset, an improper slicing orientation led to the exclusion of a left hemisphere while a distinct right hemisphere was omitted due to the evident hydrocephalic brain anatomy appearance. It is worth noting that the corresponding left hemisphere from the same subject exhibited normal anatomical features.

The slice images (Figure 1A) were mapped into particular regions using the corresponding atlas (Figure 1B). The high resolution of 0.346 × 0.346 μm (Figures 1C, D) allows for semi-automatic and automatic cell segmentation for better cell quantification.

Figure 1

Figure 1. A Nissl-stained sagittal section (A) showing the slice contour and the S1HL boundaries (black full lines) (B) The corresponding Paxinos and Watson atlas plate (Figure 172—Lateral 2.62 mm) (C) A zoom-in of the S1HL brain region (D) with close-up images of cells along the cortical column (1) Layer I and Layer II border (2) Layer III (3) Layer IV and Layer V border (4) Layer V (5) Layer V border and Layer VIa and (6) Layer VIb. Imaging pixel size: 0,346 μm/pixel. Scale bars: 1,000 μm (A, B); 250 μm (C); 50 μm (D).

Each hemisphere dataset was then prepared in QuPath. Using the Paxinos and Watson atlas as a reference, we began by identifying the most similar slice containing the S1HL and matching the overall slice anatomy. The atlas-described distance to the midline was then assigned to this selected slice. From this reference point the appropriate increments or decrements of 50 μm were applied for all slices of the hemisphere. Among the remaining samples, comprising the S1HL brain region, further selection was performed using specific criteria: slices were disregarded if they were damaged, exhibited a curled configuration, lacked proper visualization of the somatosensory cortex, displayed hydrocephalic brain characteristics, or presented severe CPu atrophy. At the end of this process, we drew the Slice Contour, the S1HL ROI, added S1HL reference points and the Outside Pia contour. N_Slices = 199 remained available for further cell densities analysis.

3.2 Cell segmentation

To automatically collect cell positional data from each slice image, we tested a cytoplasm model (“Cellpose—cytoplasm 2.0 model—cyto2”). This model was utilized on high-resolution images to effectively detect and segment nuclei within the S1HL region. However, it became evident that this model had limitations in accurately constraining the segmentation or distinguishing the smallest cells. Additionally, it tended to inaccurately identify two adjacent cells as a single larger cell, thereby impacting the accuracy of the segmentation process. Because the data used in this study differed significantly from the training set used for the Cellpose cyto2 model (most Cellpose training data consists of homogeneous objects, despite the capability of Cellpose to work with non-homogeneous populations), we refined the cyto2 cell segmentation process.

We enhanced the accuracy of cell soma contours, disentangled double-cell instances, and removed artifacts on four images (three squares for validation and twenty-four squares for training, each square size is equal to 177,16 × 177,16 μm). This task was challenging for several reasons: the large number of cells and the need to check and update the contours at each training attempt of the cell detection algorithm. Throughout the process, we adopted a standard software zooming factor of 5.4% to mitigate inconsistencies in drawing precision. This labor-intensive endeavor resulted in the creation of what we refer to as a ground truth cell segmentation dataset (“cell segmentation GT dataset”).

We then compared this augmented segmentation version, referred to as v2, with cyto2 and compared two human annotators (Figure 2A). The accuracy between Human 1 and Human 2 demonstrated a high degree of overlap of 72.5 ± 3.5%. The accuracy between v2 and Human 1 was even higher than that of Human 2 74.2 ± 2.5%. These results give us confidence in the reliability of the submitted segmenter as shown in Figure 2B and in Supplementary Figure 1.

Figure 2

Figure 2. Comparison of the cell segmentation in Layer III (A) from left to right: the human annotation is superposed on the raw image, on cyto2 and on the final v2. The human annotation is represented in cyan, the alternative classifiers in olive and the overlap in purple. (B) Quality control measures of the cell segmentation model comparing cyto2 (blue) with v2 (orange), human 2 (gray), standard deviation (black), Statistical analysis by Welch T-test ****p < 0.0001.

3.3 Stereology and cell density

As our datasets contained consecutive slices, we adapted the classical optical dissector method by adding a virtual -z coordinate to cells to realistically estimate the cell density within our defined volume of interest (see Supplementary Figure 2). The bootstrapping technique excluded only part of the cells (Figure 3A) touching the upper surface of the slice (Figures 3B–E).

Figure 3

Figure 3. Stereological cell exclusion. (A) An example showing the exclusion distribution on one S1HL brain image. Scale bar: 250 μm. (B) Zoom-in of the superposition of the raw image with the 2D visualization of the excluded cells in the -x and -y axis. (C) 2D visualization of the excluded cells in the -x and -z axis. (D) 2D visualization of the excluded cells in the -y and -z axis. (E) 3D visualization of the excluded cells in the three axis. (C–E) panels shows the exclusion of cells being restricted on the slice upper limit.

A total of 9.79% of the cells were excluded from the analysis N_Image = 199, N_{meanDetectedCell} = 13690, N_{meanExcludedCell} = 1340, N_Animal = 8, N_Hemisphere = 14 (7 RH, 7 LH). The proportion of included and excluded cells remains constant throughout the cortical column and the cell soma diameters (Figures 4A–C). This exclusion enhanced the reliability and confidence of our cell density results.

Figure 4

Figure 4. Stereology exclusion result and cell density of the juvenile rat Wistar Han (P14) as a function of percentage of cortical depth of the S1HL brain region. The upper panel shows the stereology exclusion results of the whole dataset. (A) Superposition of excluded and included cells per distance to the pia (red for excluded cells and blue for included cells). (B) Superposition of the diameters of excluded and included cells (red for excluded cells and blue for included cells). (C) The percentage of excluded cells per image. The bottom panel shows cell density as a function of cortical depth with (D) the inter-hemisphere variability (right hemispheres mean in dark red, left hemispheres mean in dark blue, standard deviation in light red and light blue), (E) the cell density of the whole dataset (mean in dark blue, standard deviation in light blue), and (F) the variability between animals (one bar per animal, mean in black, each individual brain slice as a red dot).

Our data reports no significant differences of the juvenile male rat somatosensory cortex cell densities between the right (N_{RightHemisphere} = 7) and the left hemisphere (N_{LeftHemisphere} = 7) (Figure 4D). The sexual dimorphism could not be analyzed as only male datasets were processed.

The cell density curve along the somatosensory cortex is consistent across animals, litters, and experimental days (Figures 4E, F and Supplementary Figure 3). The cell density results for individual brains are shown in Supplementary Figures 4, 5. The total mean density of cells positive for the Nissl substance in the S1HL brain region is 83,100 ± 2,393 per cubic mm (cells/mm³). This result is lower than expected (Morin and Beaulieu, 1994; Markram et al., 2015); however, comparing methods, animal age, brain regions, staining molecules or sample size is challenging. Nevertheless, a study from the late 90s reports a barrel cortex cell density of 79,130 ± 26,440 per cubic mm (same strain, older animal, same fixation procedure, similar methodologies, same cresyl violet staining) (Keller and Carlson, 1999).

3.4 Cortical layer boundaries

The boundary between Layer II and Layer III could be distinguished in 30 out of the 38 images (79% of the cases), while we were not able to separate them in the remaining eight images (21% of the cases) (Figure 5A). We therefore decided to label cells belonging to these indistinguishable layers as being part of a merged Layer II/III (Figure 5B).

Figure 5

Figure 5. Example of the cortical layering annotated by a human expert. (A, B) are the unannotated images of (C, D), respectively. In (C) Layer II is distinguished from Layer III (Lateral 2.90 mm). Whereas, in (D) Layer II and Layer III are merged (Lateral 2.40 mm). Scale bar: 250 μm. Color code: red (Layer I), pink (Layer II), purple (Layer III), brown (Layer II/III), blue (Layer IV), lighter blue (Layer V), cyan (Layer VIa), green (Layer VIb), black (S1HL).

Anatomically, it is more interesting to train a model on separated Layer II and Layer III data. Therefore, we present here results based on two ML models, RF and KNN, trained on separated Layer II and Layer III. The results on the merged LII/LIII are provided in Supplementary Figures 6, 7. We evaluated other models, such as Gradient Boosting and Feed Forward Neural Networks, but they either showed worse performance or had higher computational cost for similar performance.

Out of the 72 available cell features, 19 were selected to enhance computational efficiency and predictive accuracy. A detailed list of these features with their importance is provided in Supplementary Tables 2, 3. The most critical features include the distance to the annotation with the Outside pia (μm) and its smoothed value (Smoothed 50 μm: Distance to annotation with Outside Pia μm), highlighting the intrinsic spatial dependency of the layers. Those two features individually resulted in a decrease in accuracy of 0.2288 ± 0.018 % and 0.1902 ± 0.0015 % when permuted.

To avoid drastic dataset shrinking, we included cells from every image and excluded only those from the undistinguishable Layer II and Layer III. This dataset was then split into 32 images for training, representing 395,926 cells, while the remaining six images containing 76,515 cells were used for testing.

We evaluated KNN and RF performance using four metrics: accuracy, precision, recall and F1-score. The evaluation shows that Layer II, IV and VIb are the most challenging to classify for both models (panel A for KNN and panel C for RF of Figure 6).

Figure 6

Figure 6. Classification metrics for KNN and RF models on Layer II and III separated. (A) KNN generated classification metrics per layer, (B) confusion matrix of the KNN model (C), RF generated classification metrics per layer (D), confusion matrix of the RF model, (E) overall per-model metrics.

A confusion matrix, where each entry represents the number of cells classified into each combination of ground truth and predicted labels is shown in panel B for KNN and panel D for RF of Figure 6. The result shows that almost every cell is either classified in the correct layer, or in the adjacent one. This underlines that the models captured the sequential nature of the layers and that no neighbor cells in space should pertain to distant classes. The results clearly demonstrate the superior performance of the RF model over the KNN, across all metrics and all layers.

We averaged the per-class metrics for each model, ensuring that classes with varying sample sizes contribute equally to the final metric values. The overall KNN prediction accuracy reaches 74%, whereas the RF prediction accuracy is higher and seems to be capped at 87% for separated Layer II and Layer III (Figure 6E). This could be partially influenced by the approximate nature of the ground truth annotation by a human, especially around the layer boundaries. Supplementary Figure 8 compares the results of (Figure 6E) with classes with more samples having a proportionally higher weight in the metrics and shows a minimal difference due to an imbalance of samples.

To further assess the predictions of cortical layers, we visually evaluated them on test images for their biological relevance (Figure 7A; Supplementary Figure 9). A close-up of the RF predictions superposed onto the human annotation is shown in Figure 7B.

Figure 7

Figure 7. A superposition of the RF prediction onto the RAW image with the cell segmented and the human-annotation superposed. (A) RF prediction and human-annotation (black lines) superposed onto the RAW image with the cell segmented (blue). (B) close-up of each layer borders predictions [(1): LI to LII border; (2) LII to LIII border; (3) LIII to LIV border; (4) LIV to LV border; (2) LV to LVIa border; (2) LVIa to LVIb border]. Scale bar in (A) 250 μm and in (B) 50 μm.

As RF outperformed KNN, we applied it to the remaining unannotated N_Images = 262 to predict the S1HL layer boundaries of each image and we visually verified the predicted layers. This drastically increased the layer annotation throughput a human-annotator would reach.

3.5 Cell densities and size prediction

Because the cell density only varies slightly between images, we aimed to find a single α value, for which the bounding area enveloping each layer was as close as possible to the layer shape and that would not reject any cells (Figure 8A). The value of 0.05 is applied on all layers, with the exception of Layer I characterized by a significantly lower cell density, for which we opted for a value of 0.005.

Figure 8

Figure 8. Example of cortical layer boundaries generated with the alpha shape toolbox. (A) Region of interest with cell bodies colored according to layer. Scale bar: 250 μm (B) Cell density per predicted layer [cells/mm³], averaged across all samples, with each orange dot representing one animal. (C) Predicted layer height [μm], averaged across all samples, with each orange dot representing one animal. (D) Mean S1HL cell diameter [μm] per layer under the assumption of a perfect circular morphology. Black dotted lines the Gaussian and red dotted line the center peak of the fit Gaussian. (E) Predicted S1HL cell density per animal (each bar plot represents an animal) [cells/mm³].

We then counted the number of cells located in each alpha shape polygon corresponding to each layer and computed the volume of each polygon by using the theoretical slice thickness of 50 μm. Consequently we found the following mean densities: Layer I contains 32,568 cells/mm³, Layer II contains 90,226 cells/mm³, Layer III contains 85,094 cells/mm³, Layer IV contains 103,016 cells/mm³, Layer V contains 72,005 cells/mm³, Layer VIa contains 89,198 cells/mm³, Layer VIb contains 71,153 cells/mm³ (Figure 8B; Table 2). The cell density per predicted layer and per hemisphere is shown in Supplementary Table 4.

Table 2

Table 2. The cell density per predicted layer (cells/mm³).

Using the layer prediction generated for each cell by the machine learning model, we determined the height of each layer by fitting the cell densities predictions. The height of each layer is the distance between the top and the bottom of a layer. The projected height of the S1HL cortical column is 1,915 μm with a standard deviation of 50.57 μm, detailed height per layer is reported in Figure 8C and in Table 2. These projections are consistent with previously published data (Beaulieu, 1993; DeFelipe et al., 2002; Markram et al., 2015; Meyer et al., 2010).

Once a predicted layer was assigned to each cell, the distribution of cell diameters was calculated under the assumption of a perfect circular morphology of the same area (Figure 8D). The somatosensory cortical column contains two distinct cell populations, with a cell diameter distribution ranging from 0.39 to 27.57 μm; each of the populations consists of multiple subtypes. The first population of small cells spans from 4.80 to 6.09 μm and is present throughout the entire cortical column height. The second population of larger cells peaks from 8.14 to 10.74 μm and is present in all layers except Layer I. Layers I, V, and VIb contained a larger proportion of smaller cells, whereas Layers II, III, IV, VIa contained a larger proportion of larger cells. Supplementary Figure 10 shows the cell diameter distribution per layer, which is well-described by two Gaussian distributions. Supplementary Table 5 displays the mean values for the two Gaussian distributions of cell diameters calculated per image and per layer. To determine the statistical significance of cell density differences across the layers, a Kruskal-Wallis H-test was performed on data obtained from 228 images. This confirmed that cell densities are different across layers and that they are not drawn from the same distribution. The last column of Supplementary Table 5 also shows that cell densities are significantly different across each pair of adjacent layers.

We also confirmed that other cortical regions could be processed similarly with our approach by applying it to the medial parietal association cortex - MPtA (see Supplementary Figure 11).

4 Discussion

We have produced the most detailed publicly available dataset of Nissl-positive cells in a rodent species to date, accompanied by an open-source, semi-automatic cell detection pipeline. This pipeline accurately segments and counts the total number of cells within the somatosensory hindlimb cortical column of eight juvenile rats, stained using a traditional Nissl method. By incorporating machine learning models, the pipeline efficiently assigns layer boundaries and categorizes segmented cells accordingly. The Cellpose algorithm was particularly well-suited to the challenge of dense cell segmentation. We expect that these pipelines could be adapted for use with other staining methods, as long as the sample preparation and acquisition parameters support precise cell annotation for generating reliable ground truth data.

As demonstrated in the original Cellpose paper, the IoU comparison of different models (Cellpose, Mask R-CNN, Stardist, U-Net3 and U-Net2) shows Cellpose's superior precision (doi: 10.1038/s41592-020-01018-x; Figure 4). While Cellpose may struggle with cell overlap, this issue is mitigated using thin (50 μm) slices and has little effect on the classification of brain regions based on cell shape and localization features.

Although superposition of cells may influence the overall cell density results, it does not compromise the layer border properties. We were aware of this limitation during the training phase. Cytoarchitectonic distortions, such as concave/convex cell shapes, do not pose a significant challenge to our analysis. Even when mask convexity decreases, leading to a potential increase in missed cells, Cellpose still performs better than other contemporary models (Stringer et al., 2021).

We adapted the stereological exclusion method to estimate the total number of cells in the S1HL without skipping slices. The exclusion factor (roughly 10%) may be on the higher end, and alternative exclusion analysis methods should be explored in the future. Nevertheless, our results show that the overall cell density is lower than initially expected but remains remarkably consistent across individual animals, litters, and experimental days. A study using similar methods also reported comparable cell density in the barrel cortex.

Ever since Broca reported the left hemisphere's dominance for language (Broca, 1861), brain lateralization has been studied both behaviorally (Güntürkün et al., 2020; Rogers et al., 2013) and anatomically (Elkind et al., 2023; Tobet et al., 1993). Previous data from mice report some variation within the cortical layers (LII/III tendency for left hemisphere being denser than the right hemisphere; LIV and LVI with a tendency for right to be denser than the left; see Figure 3C of Elkind et al. (2023). Our data reports no significant differences in the juvenile male rat somatosensory cortex cell densities between the right and the left hemisphere.

However, comparing datasets in this field is inherently complex due to species and age differences. Additionally, differences in brain regions, fixation protocols, staining techniques, sample sizes, and potential human biases complicate direct comparisons between studies.

Our dataset offers an opportunity for further analysis, beyond the metrics explored in this study, and can be adapted for use in other species. We have high confidence in the dataset's quality, due to its exceptional imaging resolution, the standardized methods employed during data collection, and the remarkable reproducibility of the results. While this investigation focused on the somatosensory cortex, the versatility of the dataset allows for potential applications to other brain regions.

We employed machine learning algorithms to categorize cells into layers, minimizing human bias and achieving an accuracy of 87%. Unlike studies that rely on tissue-level differentiation (Wagstyl et al., 2020), our method provides a cell-level classification, offering finer granularity. This accuracy is on par with recent studies (Štajduhar et al., 2023), despite differences in species, staining techniques and acquisition parameters. Direct experimental validation of layer boundaries using a combination of cell-specific stains remains challenging. Staining data, such as receptor profile curves in the isocortex, are often blurred and the localization of stained layers rarely aligns precisely to cytoarchitectonic boundaries (Palomero-Gallagher and Zilles, 2019).

In our classification approach, spatial location was the most critical factor for layer assignment, which is unsurprising given the columnar organization of the cortex. Since all cells must be assigned to a layer, spatial information played a dominant role, particularly for cells within a given layer. For cells at the layer boundaries, morphological features were more influential. Future studies could explore models trained specifically on these boundary cells, where morphology would likely have a greater impact on classification decisions. While our study aimed to reduce human bias in layer assignment, it is important to note that different brain regions would require re-training of machine learning models to account for their unique properties. Classifying cells situated at layer boundaries remains a significant challenge compared to those located inside layers. Once a layer is defined by the model, cells within that layer can be classified based on their distance from the pia. Future studies should prioritize the development of metrics to evaluate boundary prediction accuracy and explore methods that do not rely solely on spatial information for boundary cells, such as training distinct models for this specific population.

The accuracy of automated annotation may fluctuate in regions characterized by unique neuronal properties or when utilizing different staining techniques. As our approach relies on supervised machine learning, the precision of cell prediction is contingent upon the expertise of human annotators. Nonetheless, our method lays the groundwork for more efficient and standardized cortical layer annotation. Notably, the predicted layer thicknesses in our study are consistent with those reported in previous studies on the barrel cortex (Meyer et al., 2013; Narayanan et al., 2017).

Our study also provides a more detailed characterization of the somatosensory cortex by analyzing the distribution of cell soma diameters within each predicted layer. We identified two primary cell populations: a smaller population with a cell diameter ranging from 4.8 to 6.1 μm and a larger population with a cell diameter ranging from 8.1 to 10.7 μm. The exact locations of the peaks differed slightly according to layer. The smaller population, present throughout the cortical column, is likely composed of inhibitory cells, especially since Layer I predominantly contains these small cell types (Muralidhar et al., 2014). The population with larger diameters likely includes excitatory cells, as it is absent in Layer I but present in all other layers. One should remember that our dataset includes not only neuronal cells but a diverse range of cell types including astrocytes, oligodendrocytes, neurons and glial cells. Accordingly, the cell size difference alone may not be sufficient to definitely categorize them as inhibitory or excitatory neurons.

By automating cell classification into cortical layers, our methodology accelerates a traditionally labor-intensive task. Accurate quantification of cell densities in brain tissues, enabled by our approach, holds significant potential for advancing our understanding of neuropathologies, such as neurodegenerative diseases that affect specific cell populations. Additionally, this work supports the construction of detailed cortical microcircuit models, contributing to a more comprehensive understanding of brain structure. Ultimately, our study lays the groundwork for fully automated, high-throughput investigations leveraging the wealth of histological data being produced worldwide, enabling more efficient exploration of the complexities of cortical microstructure.

Data availability statement

The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: Nissl_1: https://doi.org/10.5281/zenodo.12204988; Nissl_2: https://doi.org/10.5281/zenodo.12514990; Nissl_3: https://doi.org/10.5281/zenodo.12516971; Nissl_4: https://doi.org/10.5281/zenodo.12516982; Nissl_5: https://doi.org/10.5281/zenodo.12517030; Nissl_6: https://doi.org/10.5281/zenodo.12517073; QuPath_LayerBoundaries_GroundTruth_20220927: https://doi.org/10.5281/zenodo.12656468. The source code for the analysis is available at: https://github.com/BlueBrain/layer-recognition.

Ethics statement

The animal study was approved by Cantonal Commission for Animal Experimentation of the Canton of Vaud. The study was conducted in accordance with the local legislation and institutional requirements.

Author contributions

JM: Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Validation, Writing – original draft, Writing – review & editing. JJ: Conceptualization, Data curation, Formal analysis, Methodology, Project administration, Software, Validation, Writing – original draft, Writing – review & editing. OB: Software, Writing – original draft. CZ: Formal analysis, Software, Validation, Writing – original draft. NF: Formal analysis, Software, Validation, Writing – original draft. JP: Data curation, Software, Writing – original draft. YS: Data curation, Writing – review & editing. RP: Conceptualization, Investigation, Formal analysis, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing. DK: Conceptualization, Formal analysis, Methodology, Project administration, Supervision, Writing – original draft, Writing – review & editing. HM: Conceptualization, Funding acquisition, Methodology, Resources, Supervision, Writing – review & editing.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This study was supported by funding from the EPFL to the Laboratory of Neural Microcircuitry (LNMC), and by funding to the Blue Brain Project, a research center of the École Polytechnique Fédérale de Lausanne (EPFL), from the Swiss Government's ETH Board of the Swiss Federal Institutes of Technology.

Acknowledgments

We would like to thank Rodrigo Perin for guidance and the whole LNMC for helpful discussions and feedback, Clémentine Lévy-Fidel, Ying Shi, Maurizio Pezzoli, Koami Gafan, Sara Zantou for dataset preparation, the EPFL-SV Research Core Facilities: the BioImaging and Optics Core Facility for the training and access to the microscopes and in particular José Artacho for his assistance in imaging; the Histology Core Facility for the access to the cryostat and in particular Agnès Hautier for the slices histology, and Karin Holmes for editing assistance.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

The reviewer CC declared a past co-authorship with the authors DK and HM to the handling editor.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fnana.2024.1463632/full#supplementary-material

References

Allen Institute for Brain Science (2004). Allen Mouse Brain Atlas. Available at: http://mouse.brain-map.org (accessed March 19, 2024).

Google Scholar

Asaeedi, S., Didehvar, F., and Mohades, A. (2017). α-Concave hull, a generalization of convex hull. Theor. Comput. Sci. 702, 48–59. doi: 10.1016/j.tcs.2017.08.014

Crossref Full Text | Google Scholar

Bankhead, P., Loughrey, M. B., Fernández, J. A., Dombrowski, Y., McArt, D. G., Dunne, P. D., et al. (2017). QuPath: open source software for digital pathology image analysis. Sci. Rep. 7:16878. doi: 10.1038/s41598-017-17204-5

PubMed Abstract | Crossref Full Text | Google Scholar

Beaulieu, C. (1993). Numerical data on neocortical neurons in adult rat, with special reference to the GABA population. Brain Res. 609, 284–292. doi: 10.1016/0006-8993(93)90884-P

PubMed Abstract | Crossref Full Text | Google Scholar

Blixhavn, C. H., Haug, F.-M., Kleven, H., Puchades, M. A., Bjaalie, J. G., and Leergaard, T. B. (2023). A Timm-Nissl multiplane microscopic atlas of rat brain zincergic terminal fields and metal-containing glia. Sci Data 10:150. doi: 10.1038/s41597-023-02012-6

PubMed Abstract | Crossref Full Text | Google Scholar

Breiman, L. (2001). Random forests. Mach. Learn. 45, 5–32. doi: 10.1023/A:1010933404324

Crossref Full Text | Google Scholar

Briggs, F. (2010). Organizing principles of cortical layer 6. Front. Neural Circuits 4:3. doi: 10.3389/neuro.04.003.2010

PubMed Abstract | Crossref Full Text | Google Scholar

Broca, M. P. (1861). Remarques sur le siège de la facultédu language articulé, suivies d'une observation d'aphémie (perte de la parole). Bull. Soc. Anatom. 6, 330–357.

Google Scholar

Brodmann, K. (1909). Vergleichende Lokalisationslehre der Grosshirnrinde in ihren Prinzipien dargestellt auf Grund des Zellenbaues. Barth: Johann Ambrosius Barth Verlag. Available at: https://de.wikipedia.org/wiki/Johann_Ambrosius_Barth_Verlag

Google Scholar

Cover, T., and Hart, P. (1967). Nearest neighbor pattern classification. IEEE Transact. Inf. Theory 13, 21–27. doi: 10.1109/TIT.1967.1053964

Crossref Full Text | Google Scholar

DeFelipe, J., Alonso-Nanclares, L., and Arellano, J. I. (2002). Microstructure of the neocortex: comparative aspects. J. Neurocytol. 31, 299–316. doi: 10.1023/A:1024130211265

PubMed Abstract | Crossref Full Text | Google Scholar

Elkind, D., Hochgerner, H., Aloni, E., Shental, N., and Zeisel, A. (2023). Sex, strain, and lateral differences in brain cytoarchitecture across a large mouse population. eLife 12:e82376. doi: 10.7554/eLife.82376

PubMed Abstract | Crossref Full Text | Google Scholar

Fawcett, T. (2006). An introduction to ROC analysis. Pattern Recognit. Lett. 27, 861–874. doi: 10.1016/j.patrec.2005.10.010

Crossref Full Text | Google Scholar

Garcéa-Cabezas, M., John, Y. J., Barbas, H., and Zikopoulos, B. (2016). Distinction of neurons, glia and endothelial cells in the cerebral cortex: an algorithm based on cytological features. Front. Neuroanat. 10:107. doi: 10.3389/fnana.2016.00107

PubMed Abstract | Crossref Full Text | Google Scholar

Garcéa-Cabezas, M. n, Hacker, J. L., and Zikopoulos, B. (2022). Homology of neocortical areas in rats and primates based on cortical type analysis: an update of the hypothesis on the dual origin of the neocortex. Brain Struct. Funct. 228, 1069–1093.doi: 10.1007/s00429-022-02548-0

PubMed Abstract | Crossref Full Text | Google Scholar

Goerzen, D., Fowler, C., Devenyi, G. A., Germann, J., Madularu, D., Chakravarty, M. M., et al. (2020). An MRI-derived neuroanatomical atlas of the Fischer 344 rat brain. Sci. Rep. 10:6952. doi: 10.1038/s41598-020-63965-x

PubMed Abstract | Crossref Full Text | Google Scholar

Güntürkün, O., Ströckens, F., and Ocklenburg, S. (2020). Brain lateralization: a comparative perspective. Physiol. Rev. 100, 1019–1063. doi: 10.1152/physrev.00006.2019

PubMed Abstract | Crossref Full Text | Google Scholar

Gurr, E. (1960). Encyclopaedia of Microscopic Stains. London: Leonard Hill Limited.

Google Scholar

Hastie, T., Friedman, J., and Tibshirani, R. (2001). The Elements of Statistical Learning. Springer Series in Statistics. New York, NY: Springer.

Google Scholar

Johnson, G. A., Calabrese, E., Badea, A., Paxinos, G., and Watson, C. (2012). A multidimensional magnetic resonance histology atlas of the wistar rat brain. Neuroimage 62, 1848–1856. doi: 10.1016/j.neuroimage.2012.05.041

PubMed Abstract | Crossref Full Text | Google Scholar

Keller, A., and Carlson, G. C. (1999). Neonatal whisker clipping alters intracortical, but not thalamocortical projections, in rat barrel cortex. J. Comp. Neurol. 412, 83–94. doi: 10.1002/(SICI)1096-9861(19990913)412:1<83::AID-CNE6>3.0.CO;2-7

PubMed Abstract | Crossref Full Text | Google Scholar

Keller, D., Erö, C., and Markram, H. (2018). Cell densities in the mouse brain: a systematic review. Front. Neuroanat. 12:83. doi: 10.3389/fnana.2018.00083

PubMed Abstract | Crossref Full Text | Google Scholar

Keller, D., Meystre, J., Veettil, R. V., Burri, O., Guiet, R., Schürmann, F., et al. (2019). A derived positional mapping of inhibitory subtypes in the somatosensory cortex. Front. Neuroanat. 13:78. doi: 10.3389/fnana.2019.00078

PubMed Abstract | Crossref Full Text | Google Scholar

Li, D., Zavaglia, M., Wang, G., Xie, H., Hu, Y., Werner, R., et al. (2019). Discrimination of the hierarchical structure of cortical layers in 2-photon microscopy data by combined unsupervised and supervised machine learning. Sci. Rep. 9:7424. doi: 10.1038/s41598-019-43432-y

PubMed Abstract | Crossref Full Text | Google Scholar

Maier-Hein, L., Reinke, A., Godau, P., Tizabi, M. D., Christodoulou, E., Glocker, B., et al. (2022). Metrics reloaded: pitfalls and recommendations for image analysis validation. arXiv [preprint]. doi: 10.48550/arXiv.2206.01653

PubMed Abstract | Crossref Full Text | Google Scholar

Markram, H., Muller, E., Ramaswamy, S., Reimann, M., Abdellah, M., Sanchez, C., et al. (2015). Reconstruction and simulation of neocortical microcircuitry. Cell 163, 456–492. doi: 10.1016/j.cell.2015.09.029

PubMed Abstract | Crossref Full Text | Google Scholar

Meyer, H. S., Egger, R., Guest, J. M., Foerster, R., Reissl, S., and Oberlaender, M. (2013). Cellular organization of cortical barrel columns is whisker-specific. Proc. Natl. Acad. Sci. U. S. A. 110, 19113–19118. doi: 10.1073/pnas.1312691110

PubMed Abstract | Crossref Full Text | Google Scholar

Meyer, H. S., Wimmer, V. C., Oberlaender, M., de Kock, C. P., Sakmann, B., and Helmstaedter, M. (2010). Number and laminar distribution of neurons in a thalamocortical projection column of rat vibrissal cortex. Cereb. Cortex 20, 2277–2286. doi: 10.1093/cercor/bhq067

PubMed Abstract | Crossref Full Text | Google Scholar

Morin, F., and Beaulieu, C. (1994). Equivalent cell density in three areas of neonatal rat cerebral cortex. Neurosci. Lett. 176, 85–88. doi: 10.1016/0304-3940(94)90877-X

PubMed Abstract | Crossref Full Text | Google Scholar

Muralidhar, S., Wang, Y., and Markram, H. (2014). Synaptic and cellular organization of layer 1 of the developing rat somatosensory cortex. Front. Neuroanat. 7:52. doi: 10.3389/fnana.2013.00052

PubMed Abstract | Crossref Full Text | Google Scholar

Narayanan, R. T., Udvary, D., and Oberlaender, M. (2017). Cell type-specific structural organization of the six layers in rat barrel cortex. Front. Neuroanat. 11:91. doi: 10.3389/fnana.2017.00091

PubMed Abstract | Crossref Full Text | Google Scholar

Nestor, S. L. (2008). “19 - techniques in neuropathology,” in Theory and Practice of Histological Techniques, 6th Edn, eds. J. D. Bancroft, and M. Gamble (Edinburgh: Churchill Livingstone), 365–403.

Google Scholar

Neurohr, G. E., and Amon, A. (2020). Relevance and regulation of cell density. Trends Cell Biol. 30, 213–225. doi: 10.1016/j.tcb.2019.12.006

PubMed Abstract | Crossref Full Text | Google Scholar

Palomero-Gallagher, N., and Zilles, K. (2019). Cortical layers: cyto-, myelo-, receptor- and synaptic architecture in human cortical areas. Neuroimage 197, 716–741. doi: 10.1016/j.neuroimage.2017.08.035

PubMed Abstract | Crossref Full Text | Google Scholar

Papp, E. A., Leergaard, T. B., Calabrese, E., Johnson, G. A., and Bjaalie, J. G. (2014). Waxholm Space atlas of the Sprague Dawley rat brain. Neuroimage 97, 374–386. doi: 10.1016/j.neuroimage.2014.04.001

PubMed Abstract | Crossref Full Text | Google Scholar

Paxinos, G., and Watson, C. (2014). The Rat Brain in Stereotaxic Coordinates, 7th Edn. San Diego, CA: Elsevier Academic Press.

Google Scholar

Rogers, L. J., Vallortigara, G., and Andrew, R. J. (2013). Divided Brains: The Biology and Behaviour of Brain Asymmetries. Cambridge: Cambridge University Press.

Google Scholar

Ronneberger, O., Fischer, P., and Brox, T. (2015). U-Net: convolutional networks for biomedical image segmentation. arXiv [preprint]. doi: 10.1007/978-3-319-24574-4_28

Crossref Full Text | Google Scholar

Štajduhar, A., Lipić, T., Lončarić, S., Judaš, M., and Sedmak, G. (2023). Interpretable machine learning approach for neuron-centric analysis of human cortical cytoarchitecture. Sci. Rep. 13:5567. doi: 10.1038/s41598-023-32154-x

PubMed Abstract | Crossref Full Text | Google Scholar

Stringer, C., Wang, T., Michaelos, M., and Pachitariu, M. (2021). Cellpose: a generalist algorithm for cellular segmentation. Nat. Methods 18, 100–106. doi: 10.1038/s41592-020-01018-x

PubMed Abstract | Crossref Full Text | Google Scholar

Tobet, S. A., Roca, A. L., and Crandall, J. E. (1993). Cellular organization in rat somatosensory cortex: effects of sex and laterality. Exp. Neurol. 121, 65–76. doi: 10.1006/exnr.1993.1072

PubMed Abstract | Crossref Full Text | Google Scholar

Wagstyl, K., Larocque, S., Cucurull, G., Lepage, C., Cohen, J. P., Bludau, S., et al. (2020). BigBrain 3D atlas of cortical layers: cortical and laminar thickness gradients diverge in sensory and motor cortices. PLoS Biol. 18:e3000678. doi: 10.1371/journal.pbio.3000678

PubMed Abstract | Crossref Full Text | Google Scholar

Warr, W. B., de Olmos, J. S., and Heimer, L. (1981). “Horseradish peroxidase,” in Neuroanatomical Tract-Tracing Methods, eds. L. Heimer, and M. J. Robards (Boston, MA: Springer US), 207–262.

Google Scholar

West, M. J. (1999). Stereological methods for estimating the total number of neurons and synapses: issues of precision and bias. Trends Neurosci. 22, 51–61. doi: 10.1016/S0166-2236(98)01362-9

PubMed Abstract | Crossref Full Text | Google Scholar

West, M. J., and Gundersen, H. J. G. (1990). Unbiased stereological estimation of the number of neurons in the human hippocampus. J. Comp. Neurol. 296, 1–22. doi: 10.1002/cne.902960102

PubMed Abstract | Crossref Full Text | Google Scholar