Embedded-deep-learning-based sample-to-answer device for on-site malaria diagnosis

Improvements in digital microscopy are critical for the development of a malaria diagnosis method that is accurate at the cellular level and exhibits satisfactory clinical performance. Digital microscopy can be enhanced by improving deep learning algorithms and achieving consistent staining results. In this study, a novel miLab™ device incorporating the solid hydrogel staining method was proposed for consistent blood film preparation, eliminating the use of complex equipment and liquid reagent maintenance. The miLab™ ensures consistent, high-quality, and reproducible blood films across various hematocrits by leveraging deformable staining patches. Embedded-deep-learning-enabled miLab™ was utilized to detect and classify malarial parasites from autofocused images of stained blood cells using an internal optical system. The results of this method were consistent with manual microscopy images. This method not only minimizes human error but also facilitates remote assistance and review by experts through digital image transmission. This method can set a new paradigm for on-site malaria diagnosis. The miLab™ algorithm for malaria detection achieved a total accuracy of 98.86% for infected red blood cell (RBC) classification. Clinical validation performed in Malawi demonstrated an overall percent agreement of 92.21%. Based on these results, miLab™ can become a reliable and efficient tool for decentralized malaria diagnosis.

1 Introduction

With over 249 million reported cases and 608,000 casualties as of 2022, malaria is a global public health concern (World Health Organization, 2023). Most malaria cases (exceeding 99%) are concentrated in low- and middle-income nations, with sub-Saharan Africa accounting for 95% of cases. Despite cost-effective rapid diagnostic tests (RDTs) and precise polymerase chain reaction (PCR), manual microscopy coupled with visual inspection by highly trained experts is widely used for malaria diagnosis because of its robustness and limited false positives and false negatives (Beck, 2022; Fitri et al., 2022). However, the smearing and staining quality, as well as the expertise of microscopists who read smeared blood slides, greatly affect the quality of microscopy-based diagnosis. Malaria eradication in malaria-endemic countries is hindered by the lack of adequate healthcare facilities, reagents, trained professionals, vector control, and surveillance systems (Oduola et al., 2018; Sori et al., 2018; Gaston and Ramroop, 2020).

To overcome the challenges of manual microscopic examinations in malaria diagnosis, researchers have proposed digital microscopy and computational image analysis algorithms (Mody et al., 2006). Diagnosis based on digital images can reduce human labor, aid local healthcare workers, and enable experienced experts in remote locations to review microscopic results. Advances in image analysis techniques based on deep learning (Krizhevsky et al., 2012; Shen et al., 2017) have rendered this approach more affordable because of its higher accuracy than that of traditional machine learning algorithms (Liang et al., 2016; Gopakumar et al., 2018; Rajaraman et al., 2019; Zhao et al., 2020; Li et al., 2021; Meng et al., 2022; Madhu et al., 2023). However, these studies have only performed cell-level evaluations on datasets without clinical tests (Liang et al., 2016; Gopakumar et al., 2018; Rajaraman et al., 2019; Molina et al., 2020; Zhao et al., 2020; Li et al., 2021; Meng et al., 2022; Madhu et al., 2023). Although studies have reported patient-level malaria diagnosis using image analysis in clinical settings, the accuracy of these methods is limited (Yoon et al., 2019; Das et al., 2022). Despite computer algorithms exhibiting greater consistency than that of manual readers, slide quality considerably affects the performance of these algorithms (Das et al., 2022). Therefore, algorithms should be scaled up using larger datasets to overcome their dependency on blood-film preparation quality and ensure adaptability to the variability of slides.

Traditional machine learning algorithms tend to saturate as the amount of training data increases. By contrast, deep learning algorithms can learn from larger datasets to train larger models to attain improved accuracy. For example, in 2012, the best model for image classification had 62 million trainable parameters (Krizhevsky et al., 2012), whereas in 2022, the number of parameters increased to 2.44 trillion (Wortsman et al., 2023). The required computational resources should be increased to accommodate larger models. Cloud computing or server-level computing is utilized to run current state-of-the-art models. However, in malaria-endemic countries, using large, extensive, deep models that require powerful computing capabilities or a consistently stable Internet connection is not feasible. By contrast, embedding an efficient deep learning model into a portable device can be prove useful in real clinical environments. Furthermore, consistent blood film preparation is essential to achieve high accuracy by reducing image variability.

In this study, we introduced miLab™, an embedded-deep-learning-based sample-to-answer device capable of automated blood film preparation and autofocused imaging using digital microscopy for on-site malaria diagnostics (Figure 1A). We provided a quick, inexpensive, and environment-friendly method to stain smears with dye using deformable staining patches. The automated preparation provided by miLab™ reduces differences in staining methods between technicians and institutions, resulting in consistent and high-quality preparation results for blood samples over a broad range of hematocrits. (Choi et al., 2021; Bae et al., 2023). The embedded deep learning algorithm was employed to analyze malaria-suspected morphology from the blood cells stained in this device, and the autofocused digital images were captured by the optical system of the same device. The proposed miLab™ device can perform continuous autofocus imaging, providing the same effect as continuous field-of-view (FoV) readings applied when examined under a microscope. Digital microscopy allows the scanning of more than 200,000 RBCs within 7–10 min without errors, which is more than the number of RBCs recommended by the Word Health Organization (WHO) guidelines. Therefore, digital microscopy is beneficial for samples from patients with low parasitemia (in cases of low infection). This innovative device allows on-site users to immediately analyze digital image data and obtain immediate diagnostic results without high-performance computing. Simultaneously, digital images can be sent to experts from remote locations to assist with diagnosis (Figure 1B). On-site users can immediately check the suspected morphology of malaria-infected cells through the screen mounted on the device (Figure 1C), and other experts can access the same digital images on web-based software to review the suspected morphologies and diagnostic results (Figure 1D). The embedded deep-learning-based on-site malaria diagnostic platform not only provides similar results as in-person microscopy examination in the laboratory but also enables remote diagnosis.

Figure 1

Figure 1. Schematic of the embedded deep learning based on-site malaria diagnosis. (A) The miLab™ device not only automates the process (automated blood staining without liquid handling and autofocused digital images) of malaria diagnosis through microscopic analysis but also incorporates deep learning algorithm directly into the device for on-site review. (B) A web-based software allows experts to access the digital images for remotely reviewing the result through the internet. (C) Photograph of the result page in miLab™ for Plasmodium falciparum (P. falciparum) positive patient specimens. Users can review and confirm the results in the miLab™ for sample-to-answer, on-site malaria diagnosis. (D) Photograph of the screen shot of the result page from the same patient specimens on the web-based software, accessing remotely digital images and raw data from miLab™. Other experts can remotely review and confirm the same results from miLab™.

2 Materials and methods

2.1 miLab™ platform

The miLab™ platform was designed to provide automated microscopic analysis, which is the standard for malaria diagnostics. Each cartridge [40 × 92 × 15 mm (width × length × height)] of the miLab™ device [212 × 390 × 244 mm (width × length × height)] prepares the blood film and three types of staining patches, facilitating consistent RBC smearing and staining (Supplementary Figure S1). The cells can be smeared and stained by moving 5 μL of a blood-loaded cartridge without controlling the aqueous solution. Thus, automated processes for blood films can be designed to effectively and appropriately stain the morphology of Plasmodium on-site. We utilized a solid staining method using hydrogels to eliminate the need to control complex equipment or maintain the liquid reagents (Choi et al., 2021). In this approach, the hydrogel is brought into contact with appropriately smeared and fixed cell surfaces, allowing the dye to efficiently stain the cells. Depending on the type of dye used, the dye from the hydrogel was applied to the smear in a short time, under 1 min (Bae et al., 2023). Furthermore, hydrogels without dye can absorb any remaining dye on the cell surface and attain suitable staining quality depending on the pH of the buffer in the hydrogel, which is similar to the role of the conventional buffer solution in blood cell staining (Oktiyani et al., 2023). After automated blood-film preparation, the optical system of the device captured the digital images of the stained blood cells. At least 200,000 RBCs and up to 500,000 RBCs were scanned in continuously obtained digital images from each blood film. The optical system was designed to capture multifocal images from the blood film by considering the size and location of the stained parasites within the RBCs.

Subsequently, a machine learning algorithm was applied to analyze the captured images and detect malarial parasites in the blood. To incorporate the machine learning model into the limited resources of the embedded hardware, all neural network architectures were designed to reduce computational complexity. We devised a two-step image analysis algorithm, including RBC detection and subsequent classification, to identify parasites that are rarely present in RBCs. The subimages were extracted from the image to obtain one RBC for each event (1st step: detection). Each of the cropped subimages was tested for the presence of malarial parasites (2nd step: classification). In the two-step algorithm, images are divided into meaningful units, and each unit is comprehensively analyzed instead of finding scarce malarial parasites directly in numerous images.

In the detection module of miLab™, a semantic segmentation algorithm is used to detect each RBC in the images. Because our target images were full of RBCs, we adopted a semantic segmentation algorithm inspired by the U-net instead of a general object detector to achieve compactness and efficiency (Ronneberger et al., 2015; Tran et al., 2019; Navya et al., 2022). The model has three up-sampling and three down-sampling layers to capture the visual features from the images in different spatial scales. Specifically, the model was designed to produce two outputs, namely, objectness and contours, as segmentation mask images. After running the semantic segmentation algorithm, the contour image is subtracted from the objectness image to separate the aggregated cells. The cell locations were determined by performing the connected component analysis from the resulting mask (Supplementary Figure S3A). The detected blood cells were cropped and passed through the classification module.

In the classification module, we used the fact that uninfected cells were the dominant component of many detected cells. Because passing all cells through the entire classification procedure is inefficient, we designed the classification module as two cascaded classifiers to reduce the computational burden imposed by the skewed ratio of parasite-positive to parasite-negative cells (Supplementary Figure S3B). The first classifier consists of only three convolutional layers for the maximum speed and screens for apparent “clean” uninfected cells. If a cell was clearly classified as uninfected, then it was not passed on to the second classifier. This prior “screening” classifier considerably reduces the number of cells delivered to the subsequent classifier. The second “main” classifier performs an in-depth examination of only those cells deemed by the first classifier to be possibly infected. The second classifier is based on the ResNet (He et al., 2016) architecture and equipped with a convolutional block attention scheme (Woo et al., 2018). We carefully designed the main classifier to have 23 layers with increasing numbers of channels along the depth for the balance between speed and accuracy. Similar to our segmentation network, the main classifier has a multi-task learning framework with two output branches: one branch that predicts the presence of malaria (infection branch) and the other branch that estimates its developmental stage (stage branch). In our verification study, only 17.8% of the RBCs passed the screening classifier and were delivered to the main classifier, whereas the conventional monolithic classifier examined all RBCs (Supplementary Figure S4). The classifier was trained to be robust to distracting elements, such as white blood cells (WBCs), which are categorized as malaria-negative. The current AI of miLab™ can detect malaria parasite-infected RBCs in the operator-assigned number (default: 200,000) of RBCs and classify developmental stages based on their morphology: P. falciparum and P. vivax ring stage, P. falciparum late stage (i.e., gametocyte), P. vivax trophozoite stage, and P. vivax late stage (i.e., schizont and gametocyte).

The algorithm of the device provides flexibility in the tradeoff between sensitivity and specificity. The output score of the infection branch was thresholded to determine whether the cells were positive or negative. Lowering the threshold yields higher sensitivity and raising the threshold provides higher specificity. Furthermore, patient-level recommendations are provided based on the number of suspected cells on a slide. In this clinical evaluation, we considered a slide with more than one positive RBC as malaria-positive.

2.2 Analysis of the blood film in miLab™

Whole blood samples were collected for research purposes, and the study was approved by the Institutional Review Boards (P01-202003-31-007 and GCL-2020-1011-01) of the Korea National Institute for Bioethics Policy (Seoul, Korea) and GCLabs (Yongin, Korea). The hematocrit of the blood sample was measured using a hematocrit-measuring instrument to analyze the blood film (Boditech Med Inc., FPRR005). The blood film was analyzed based on the number of RBCs and color value of the prepared stained RBCs. Images of stained RBCs prepared using the specimens were captured in the device, and the field of views (FoVs), including stained RBCs, were acquired. The number of RBCs was analyzed using a verified image analysis tool. Color values (red, green, and blue (RGB)) were obtained from the pixels of the segmented RBCs using a verified, self-made image analysis tool. To examine the morphological features of Plasmodium, four types of typical stages and species (rings of early trophozoites, late trophozoites from P. vivax, and gametocytes from P. falciparum and P. vivax) were collected. A manual Giemsa slide was prepared with the same blood samples and observed under a microscope (CX33 with a ×100 objective lens, Olympus) to compare the staining quality of Plasmodium in miLab™. The same morphology of Plasmodium in the blood film prepared from miLab™ was observed using the same device and under a microscope (CX33 with a ×50 objective lens, Olympus) to compare the image quality of miLab™.

2.3 System verification of miLab™

To evaluate the reproducibility of blood films smeared using miLab™, a verified, self-made image analysis tool was employed to determine the number of segmented RBCs in each FoV. The mean RBC counts from 50 FoVs of 20 replicate slides for seven specimens were used to examine the precision of the blood smear. To evaluate the reproducibility of blood film staining by miLab™, a verified, a similar self-made image analysis tool was used to determine the mean RGB value of segmented RBCs from 200 FoVs. Two hundred FoVs per specimen were used to examine the color of the specimen by calculating the mean, standard deviation, and coefficient of variance.

A receiver operating characteristic (ROC) curve was drawn using the binary classification result of 4,005 normal RBCs and 7,713 malaria positive cells. The classification performance at the cellular level for Plasmodium (ring and gametocytes) was represented using a confusion matrix (2 × 2), where the maximum accuracy was obtained. The detection rate of Plasmodium (ring and gametocyte) for the entire embedded deep-learning algorithm was verified using the Plasmodium ratio among the total number of RBCs per FoV obtained from the blood film in miLab™. In total, 3,000 FoVs from 15 malaria-positive clinical specimens (200 FoVs each) from the blood film in miLab™ were labeled by an experienced microscopist. A total of 3,290 Plasmodium (ring, gametocyte) were confirmed in 1,751 FoVs. The detection rate of Plasmodium, determined by the embedded deep-learning algorithm in miLab™ (test group), was compared and fitted to that observed by the human eye on the same images (control group).

2.4 Clinical evaluation

Clinical samples (n = 555) analyzed using the two miLab™ devices were collected from April 2022 to November 2022 and used in the clinical validation study approved by the Institutional Review Board (IRB00003905) of the National Health Sciences Research Committee (Ministry of Health, Malawi). After explaining the purpose of the study, procedure, possible benefits, risks, and rights of the participants, all participants were first requested to sign informed consent forms. The results obtained by miLab™ were compared to those of microscopic examination by an experienced local microscopist in Malawi and alongside with RDT (CareStart™ Malaria Pf (HRP2) Ag RDT, AccessBio, NJ). Blood samples (∼250 µL) were collected into blood capillary tubes by a finger prick through a sterile lancet and stored in an anticoagulant tube with ethylenediaminetetraacetic acid (EDTA). Five microliters of collected blood was loaded onto the miLab™ cartridge and 5 μL of the collected blood was used to prepare a thick and a thin blood film each for microscopic examination. The blood film was stained with a mixture of eosin and methylene blue using Giemsa staining. Local microscopists examined the Giemsa-stained slides using an Olympus CX33 microscope based on the standard microscopy methods of the WHO (World Health Organization, 2015). Parasitemia determined using miLab™ (test group) was compared and fitted linearly to that measured from the thick film of the manual Giemsa slide under microscopic examination by a local microscopist (control group).

2.5 Statistical analysis

GraphPad Prism (Ver.7, GraphPad Inc., San Diego, CA, United States) was used to perform all statistical analyses. To evaluate the linear correlation for the detection rate in the system verification and parasitemia in clinical evaluation, linear least squares analysis was performed at the 95% confidence interval (CI) of each variable, and the Pearson correlation coefficient (r) was calculated. Differences between the test and control groups were examined using Student’s unpaired t-test. A two-sided test, and the results were considered statistically significant at p < 0.05.

3 Results

3.1 Characterization of the blood film in miLab™

To address the unique characteristics of malaria patients with lower RBC counts, we implemented a two-speed smearing process on a spreader with a consistent angle, screening an adequate number (at least more than 200 RBCs per FoV) of RBCs in low hematocrit samples, ranging from 20% to 35%. This method provided two zones for detecting appropriate RBCs, depending on the hematocrit of the samples (Figure 2A). When we investigated the correlation between the average RBC counts per FoV and the hematocrit from the 37 clinical specimens, Zones A and B revealed a linear correlation between RBC counts and hematocrit (Figure 2B; Supplementary Figure S5). The RBC counts of samples with low (<30%) and middle/high hematocrit (>30%) were selected from Zones B and A, respectively. To increase the efficiency of RBC screening, RBCs from low-hematocrit samples can be screened in Zone B (average 200–400 RBC counts per FoV) rather than in Zone A (average 100–300 RBC counts per FoV).

Figure 2

Figure 2. Characterization of the blood film in the miLab™. (A) Photograph of the prepared blood films from the miLab™ using a patient specimen with low hematocrit and high hematocrit from Malawi. Low hematocrit samples to be read in Zone B instead of Zone A, where high hematocrit samples were read. The miLab™ device automatically detects an appropriate area to observe RBCs in a monolayer. (B) Correlation of average RBC counts per FoV depending on the hematocrit of the clinical specimens (n = 37) was shown with open dots (Zone A) and close dots (Zone B). (C) Schematic of blood staining using three distinct staining patches in the cartridge and pictures of stained blood cells with Plasmodium-infected RBCs (black arrow) from each step of the staining procedure. The scale bars = 10 μm. (D) Comparison of microscopic cell image with the miLab™ blood film acquired from miLab™, 50x olympus microscopy with miLab™ blood film, and 100x microscopy with conventional Giemsa slides. The scale bars = 5 μm.

Depending on the types of staining patches that included various types of Romanowsky stains, such as eosin, methylene blue, and azure B, the color of RBCs and the morphology of parasites were observed at each staining step (Figure 2C). The transparent patch was used for absorbing the excess dye left on the slide and optimizing the stained colors of the cells, whereas the dye-containing patches were used to deliver dyes to the cells (Choi et al., 2021; Bae et al., 2023). To confirm that the morphological characteristics of Plasmodium were revealed in the blood film prepared in miLab™, four types of typical stages and species (rings for early trophozoites, late trophozoites from P. vivax, and gametocytes from P. falciparum and P. vivax) were collected from patient samples (Figure 2D). The stained blood film provided a clear morphology of Plasmodium at each stage, similar to conventional microscopic examination using Giemsa staining for 100x microscopy images. When comparing the images of Plasmodium morphologies stained by the cartridge with manually focused images at the same resolution using a 50x microscope, the distinctions in morphology and species were discernible, even in the auto-focused images within the device. Both automatically stained cells and their autofocused digital images not only allow the embedded deep-learning algorithm to detect malaria on-site but also enable experts to distinguish between types and stages of malaria through the result screen in the device or viewer at remote locations.

3.2 System verification of miLab™

To ensure an accurate diagnosis, digital images that demonstrate reproducible automated preparation processes and effectively depict the morphology of Plasmodium are crucial. Therefore, when performing automated preparation processes in miLab™, the blood film was verified by the consistency of RBC counts for smearing and the color difference for staining. The screening of RBCs performed on various hematocrit samples, including 20%–50%, resulted in a coefficient of variation (CV%) of less than 10%, even in 20 replicates of seven clinical specimens (Figure 3A). Because three-color values (red, green, and blue (RGB) levels) are the most common color spaces for segmenting parasites and RBCs from thin blood films (Fong Amaris et al., 2022), each color value was extracted from segmented RBCs to quantitatively analyze the appropriate blood smear and staining. Thus, hydrogel staining precisely controls the color of Plasmodium (ring, gametocyte) and performs blood film preparation efficiently and reproducibly. Figure 3B displays the color values of the stained images. The %CV of the three-color values over 4,000 FoVs was less than 5% and was maintained across 200 FoVs for 20 specimens (Supplementary Figure S6).

Figure 3

Figure 3. System verification of the miLab™ device. (A) Reproducibility of blood smear was represented with the box plot using RBC counts per FoV in seven clinical specimens (n = 20). Average RBC counts per FoV were demonstrated with low, middle, and high hematocrits. The RBC counts of the samples with the low (<30%) and the middle/high hematocrits (>30%) were selected from Zones B and A, respectively. (B) Reproducibility of blood staining was represented with the box plot using the red, green, and blue color value, which was obtained from the stained RBCs in FoVs of clinical specimens. The RGB color values of each RBC was conserved across FoVs (n = 4,000). (C) Cellular level classification performance for Plasmodium (ring, gametocytes) was represented with the ROC curve. The area under the curve (AUC) was 0.999 with 95% confidence interval in the range of 0.9986–0.9994. The confusion matrix was calculated at the optimal point where maximum accuracy was obtained. (D) Correlation of the detection rate of malaria positives (ring, gametocytes) between the deep learning algorithm (test group: miLab™) and naked eyes (control group: Microscopy) with the Pearson’s correlation coefficient (r) of 0.96 (p < 0.0001, n = 3,000).

The ROC curve in Figure 3C displays the cellular-level performance of Plasmodium binary classification. The area under the curve (AUC) was 0.999, indicating that the proposed classifier was highly accurate. The magnified graph displays the trade-off relationship between the true positive and false positive rates. We selected an optimal point on the curve that achieved the maximum accuracy and calculated the sensitivity, specificity, and accuracy at that point as 99.25%, 98.1%, and 98.86% (95% CI: 98.65%–99.04%), respectively. The infected RBC detection performance was verified using blood films prepared from 15 malaria-positive clinical specimens. We randomly selected 200 FoVs for each blood film and compared the proportion of infected RBCs from miLab™ to microscopic results obtained visually. The classifier infection branch applied an empirically determined threshold value to the output. A cell is classified as malaria-positive if its output value surpasses a specified threshold. An excellent correlation existed between the results from both methods, with a Pearson’s correlation coefficient (r) of 0.96 (p < 0.0001; Figure 3D).

3.3 Evaluation of clinical performance in Malawi

The clinical performance of miLab™ was evaluated using 555 patients who were selected from patients with fever and visited the clinical study site, Mzuzu Health Center in Malawi. The predominant species was P. falciparum (Gaston and Ramroop, 2020; U.S. President’s Malaria Initiative, 2020). Clinical information of the patients is summarized in Supplementary Tables S1, S2. Figure 4A illustrates the study design for the clinical validation of miLab™ based on the approved study protocol. Since the miLab™ diagnoses is similar as that of microscopy examination, the protocol of this study was primarily to compare the results of a local microscopist with the results of the miLab™. However, given the widespread use of RDTs in local workflows, we identified 115 positive and 386 negative samples out of 555 total enrolled samples when we selected samples where the RDT provided the same diagnosis as the microscopic result on the Gisma slide. Additionally, to ensure the reliability of local microscopy results, a third expert microscopist re-examined the same Giemsa slides. After excluding 67 samples with discrepancies, the final evaluation of miLab™ was conducted on the remaining 488 confirmed samples. On analysis of 488 clinical specimens by miLab™, an overall percent agreement (OPA) of 92.21% (95% CI; 89.48%–94.43%), positive percent agreement (PPA) of 95.15% (95% CI; 89.03%–98.41%), and negative percent agreement (NPA) of 91.43% (95% CI; 88.17%–94.03%) was observed (Figure 4B).

Figure 4

Figure 4. Clinical validation of miLab™ in Malawi. (A) Design for a clinical study. A total of 555 clinical specimens were enrolled and subjected to microscopy and analysis by miLab™ for comparison with the reference tests (both local microscopy examination and RDT). Yellow cells indicate samples discordant with the reference test. (B) Agreement of analysis by miLab™ with the reference tests (microscopy and RDT). Based on the concordance of microscopy and RDT, overall percent agreement (OPA), positive percent agreement (PPA), and negative percent agreement (NPA) were 92.21%, 95.15%, and 91.43% respectively. (C) Correlation of the parasitemia level between microscopy and the miLab™ on a logarithmic scale. The Pearson’s correlation coefficient (r) is 0.8259 (95% CI: 0.7518–0.8794).

Figure 4C presents the correlation of parasitemia between manual microscopic examination by a local microscopist and miLab™. The mean parasitemia of positive samples is approximately 26,000 (parasites per µL) from the miLab™ and approximately 22,000 (parasites per µL) from the local microscopist. Because miLab™ quantifies parasitemia assuming 5,000,000 RBCs per microliter of the blood, parasitemia in patients with abnormal RBCs or out-of-range WBCs differs from quantification based on 8,000 WBCs per microliter of blood through the conventional microscopy examination. Nevertheless, the Pearson’s correlation coefficient (0.8259, 95% CI: 0.7518–0.8794) of parasitemia determined by miLab™ exhibited excellent consistency with the quantification results obtained from Giemsa slides by the microscopist. According to the results of the parasitemia level, 200 clinical samples (100 positive, 100 negative) were randomly selected, and the limit of detection (LOD) for the point at which a positive result is more than 95% probable using probit regression is approximately 31 parasites per μL.

4 Discussion

On-site sample-to-answer malaria diagnosis in miLab™ enables blood film preparation for embedded deep learning-based malaria detection using digital microscopy images. In miLab™, hydrogel-stained patches are applied to blood film generation in a highly reproducible manner. In this on-site diagnostic platform with an A4 paper-sized footprint, ethanol-based fixation is applied to avoid the use of methanol and staining patches to reduce liquid waste without maintaining reagents for user safety and convenience. Thus, a sophisticated, well-equipped central laboratory is not required for on-site diagnosis. The miLab™ device functions as a stand-alone unit that can be used in resource-limited environments. System validation revealed excellent reproducibility for blood film preparation (Figure 3; Supplementary Figure S5). The accuracy of deep-learning-based analysis at the cellular level was comparable to that of other research groups (Liang et al., 2016; Gopakumar et al., 2018; Rajaraman et al., 2019; Zhao et al., 2020; Li et al., 2021; Meng et al., 2022; Madhu et al., 2023). Among those researches, only Zhao et al. and the proposed method are integrated into the mobile platform. Also, the works of Li et al., Ment et al., and ours can detect multiple stages of malaria infection. Detailed comparison is summarized in Supplementary Table S3. The consistent performance and higher accuracy of miLab™ eliminated the dependency on technicians for manual microscopy-based malaria diagnosis by providing both blood film preparation and automated analysis compared with other products (Yoon et al., 2019; Das et al., 2022).

In this clinical performance evaluation, 555 clinical data samples were registered during the study period, but only 488 samples were used for performance evaluation, excluding 67 samples. Since miLab™ utilizes a microscopy-based diagnostic approach, microscopy examination is considered the ground truth. Although the locally prevalent RDT is reported to have issues such as false negatives due to hrp2/3 deletions and false positives due to hrp2 antigen persistence (Poti et al., 2020; Hosch et al., 2022), this study has also considered RDT results to enhance the reliability of clinical evaluations. Nonetheless, analyzing miLab™ results for 67 cases where there were discrepancies among the three different diagnostic outcomes reveals the following: For the 50 cases where RDT was positive and local microscopy results were negative, miLab™ diagnosed 41 cases as negative. Conversely, for the four cases where RDT was negative and microscopy results were positive, miLab™ diagnosed three cases as positive. Additionally, for the 12 cases where both RDT and local microscopy diagnosed positive but expert microscopy diagnosed negative, miLab™ diagnosed nine cases as positive. For the case where both RDT and local microscopy diagnosed negative but expert microscopy diagnosed positive, miLab™ diagnosed it as negative. In conclusion, the analysis of miLab™ results for the 67 discrepant cases shows that miLab™ results align more closely with microscopy results than with RDT results. Furthermore, miLab™ results tend to agree more with the diagnosis in which two out of the three diagnostic results concurred. In fact, in regions of Ethiopia where hrp2/3 deletions occur, miLab™ has demonstrated the capability to diagnose samples that RDT has falsely identified as negative due to hrp2/3 deletions (Ewnetu et al., 2024).

The clinical performance evaluation of miLab™ using 488 clinical specimens revealed an OPA of 92.21% before user reviews. Among the 488 samples analyzed, 38 (7.79%) were discordant with the reference tests as false-negatives or false-positives. The miLab™ device displays “review needed” or “suspected” morphology of the parasites on its display to users who aim to review the results either on the device or web-based software. In this clinical validation study, users were required to review the raw data and confirm the diagnostic results for 131 of the 488 patient specimens (including 98 true positives and 33 false positives). To achieve optimal results expected from the device, users can set a cellular-level threshold based on the situation, such as the presence of an expert. Although we attained high cellular-level accuracy, a special strategy is required to provide patient-level recommendations. Because the number of normal RBCs is greater than the number of infected RBCs, setting a cellular level threshold for high specificity can reduce reviewing effort. However, considering the five samples identified as false negatives, it is crucial that any single cell requiring review is displayed on the results page. Notably, four out of the five false-negative samples were diagnosed by expert microscopists as having low parasitemia levels, and lowering the threshold could have allowed miLab™ to flag three of these samples as suspected morphology. Therefore, to detect Plasmodium in patients with low parasitemia or to reduce false negatives, it is important not only to increase the sensitivity of the miLab™ algorithm but also to guide users to review as many cells as possible. To achieve this, enhancing the training dataset, lowering the threshold sensitively, or applying post-processing algorithms may be necessary. By screening more cells or guiding users to review high-scoring cells, effectively detecting samples with low parasitemia levels, and also improving the accurate quantification of parasitemia level are possible. Consequently, all suspected cells, even a single cell need to be shown on the Results section. These strategies can be included in patient-level recommendations.

The first generation of malaria diagnosis using miLab™ was based on the characteristic morphology of P. falciparum. However, the results obtained by a local microscopist confirmed that four patients were infected with P. vivax and Plasmodium malariae in addition to P. falciparum. On reviewing the digital images of miLab™, experts observed various distinct morphologies of Plasmodium species. The current stage classification performance was examined using 11,718 single-cell images acquired from blood films prepared using miLab™, where the stages were determined independently by expert microscopists. Figure 5B presents the confusion matrix from the verification study for multistage malaria parasite classification. The total accuracy for the classification of infected RBCs was 98.83% (95% CI: 98.62%–99.01%) and that for the classification of Plasmodium species and stages was 97.82% (95% CI; 97.53%–98.06%). The miLab™ algorithm revealed excellent performance in malaria detection, with the identification of multiple stages of the parasite’s life cycle in this analytical validation study. If the algorithm is optimized at patient-level estimation, then the performance of miLab™ can be improved, allowing the classification or detection of other Plasmodium species. A deep learning algorithm trained with other types of infected blood cells would improve the performance of malaria diagnosis by reducing the interference caused by WBCs, platelets, and other types of parasites (e.g., Trypanosoma cruzi).

Figure 5

Figure 5. Performance of detecting stages and species in the miLab™. The confusion matrix represented stage and species classification using embedded deep learning algorithm with additional patient-level estimation. (p.f gameto; gametocyte, p. v trop.; trophozoite, p. v late; schizonts or gametocyte).

The miLab™ device can provide on-site, sample-to-answer diagnostics for malaria and enables decentralized patient care in resource-limited settings, especially low- and middle-income countries. In future work, the miLab™ cartridge can be applied to the morphological detection of WBCs because similar methods derived from Romanowsky-type staining are used for staining for both WBCs and malaria (Supplementary Figure S7A)(Choi et al., 2021). Moreover, miLab™ can prepare various types of sample slides by modifying the composition of the hydrogels in the cartridge. These slides included cytology slides with Papanicolaou staining (Supplementary Figure S7B) and tissue sections stained with hematoxylin and eosin (H&E) for histopathology (Supplementary Figure S7C) (Chin et al., 2022; Kim et al., 2023). The miLab™ device can be utilized to diagnose other infections and the detection of intestinal parasites (Supplementary Figure S7D) that require microscopic examination without staining (wet preparations). The use of miLab™ can be extended to the diagnosis of multiple diseases using the external training of deep-learning algorithms on various types of digital images.

5 Conclusion

On-site sample-to-answer malaria diagnosis using miLab™ enables blood film preparation for embedded deep learning-based malaria detection using digital microscopy images. The miLab™ algorithm achieved 98.86% detection accuracy of infected RBCs. The clinical validation of miLab™ demonstrated an OPA of 92.21% in Malawi. This on-site malaria diagnostic platform can assist experts in evaluating the suspected morphology of Plasmodium in laboratories and remote locations and realize remote diagnosis of malaria, especially in resource-limited settings.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human specimens were approved by The National Health Sciences Research Committee (IRB00003905, Ministry of Health, Malawi). The studies were conducted in accordance with the local legislation and institutional requirements. Written informed consent for participation in this study was provided by the participants’ legal guardians/next of kin. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.

Author contributions

CYB: Data curation, Formal Analysis, Investigation, Methodology, Visualization, Writing–original draft, Writing–review and editing. YMS: Formal Analysis, Investigation, Methodology, Software, Visualization, Writing–original draft, Writing–review and editing. MK: Formal Analysis, Investigation, Methodology, Project administration, Visualization, Writing–original draft, Writing–review and editing. YS: Data curation, Formal Analysis, Visualization, Writing–original draft, Writing–review and editing. HJL: Data curation, Project administration, Software, Writing–review and editing. KHK: Conceptualization, Data curation, Methodology, Software, Writing–review and editing. HWL: Data curation, Project administration, Software, Writing–review and editing. YJK: Data curation, Project administration, Writing–review and editing. CK: Data curation, Writing–review and editing. DKL: Investigation, Supervision, Writing–review and editing. B-IK: Formal Analysis, Visualization, Writing–original draft. SH: Formal Analysis, Visualization, Writing–orignial draft. H-PB: Investigation, Supervision, Writing–review and editing. S-HC: Data curation, Writing–review and editing. BMW: Data curation, Formal Analysis, Writing–review and editing. CYL: Funding acquisition, Investigation, Resources, Writing–review and editing. K-HC: Investigation, Supervision, Validation, Writing–review and editing.

Funding

The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This study was supported by a grant from the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health and Welfare, Republic of Korea (grant number: HI18C1685).

Acknowledgments

We thank the following associates from Noul Co., Ltd. for their contributions to the development of hardware, software, and the verification of miLab™: H. Lee, S. Moon, J. Cho, Y. Shin, S. Hong, R. Choi, D. Ham, O. Bailo, M. J. Seol, Y. Hong, S. Yun, H. Hwang, E. Hwang, S. K. Beak, and J. K-HC. We thank J. Lee, H. Park, and J. Shin for monitoring the IRB project and the Mzuzu Health Centre study team for collecting the clinical data. We thank H. H. Cho for reviewing and editing the manuscript.

Conflict of interest

Authors CYB, YMS, MK, YS, HJL, KWK, HWL, YJK, B-IK, SH, S-HC, BMW, CYL, K-HC were employed by Noul Co. Ltd. H-PB, DKL were a member of Scientific Advisory Board (Technical Consultant) in Noul Co., Ltd.

The remaining author declares that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fbioe.2024.1392269/full#supplementary-material

References

Bae, C. Y., Esmaeili, H., Zamin, S. A., Seol, M. J., Hwang, E., Beak, S. K., et al. (2023). Quantification of solution-free red blood cell staining by sorption kinetics of Romanowsky stains to agarose gels. Anal. Methods. 15, 5369–5379. doi:10.1039/D3AY01431B

PubMed Abstract | CrossRef Full Text | Google Scholar

Beck, H.-P. (2022). Digital microscopy and artificial intelligence could profoundly contribute to malaria diagnosis in elimination settings. Front. Artif. Intell. 5, 510483. doi:10.3389/frai.2022.510483

PubMed Abstract | CrossRef Full Text | Google Scholar

Chin, L. K., Li, H., Choi, J.-H., Iwamoto, Y., Oh, J., Min, J., et al. (2022). Hydrogel stamping for rapid, multiplexed, point-of-care immunostaining of cells and tissues. ACS Appl. Mater. Interfaces 14, 27613–27622. doi:10.1021/acsami.2c05071

PubMed Abstract | CrossRef Full Text | Google Scholar

Choi, J.-H., Chin, L. K., Woo, B. M., Song, Y., Seol, M. J., Hong, Y., et al. (2021). Hydrogel-based stamping technology for solution-free blood cell staining. ACS Appl. Mater. Interfaces 13, 22124–22130. doi:10.1021/acsami.0c22521

PubMed Abstract | CrossRef Full Text | Google Scholar

Das, D., Vongpromek, R., Assawariyathipat, T., Srinamon, K., Kennon, K., Stepniewska, K., et al. (2022). Field evaluation of the diagnostic performance of EasyScan GO: a digital malaria microscopy device based on machine-learning. Malar. J. 21, 122. doi:10.1186/s12936-022-04146-1

PubMed Abstract | CrossRef Full Text | Google Scholar

Ewnetu, Y., Badu, K., Carlier, L., Vera-Arias, C. A., Troth, E. V., Mutala, A., et al. (2024). A digital microscope for the diagnosis of Plasmodium falciparum and Plasmodium vivax, including P. falciparum with hrp2/hrp3 deletion. PLOS Glob. public health 4, e0003091. doi:10.1371/journal.pgph.0003091

PubMed Abstract | CrossRef Full Text | Google Scholar

Fitri, L. E., Widaningrum, T., Endharti, A. T., Prabowo, M. H., Winaris, N., and Nugraha, R. Y. B. (2022). Malaria diagnostic update: from conventional to advanced method. J. Clin. Lab. Anal. 36, e24314. doi:10.1002/jcla.24314

PubMed Abstract | CrossRef Full Text | Google Scholar

Fong Amaris, W. M., Martinez, C., Cortés-Cortés, L. J., and Suárez, D. R. (2022). Image features for quality analysis of thick blood smears employed in malaria diagnosis. Malar. J. 21, 74. doi:10.1186/s12936-022-04064-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Gaston, R. T., and Ramroop, S. (2020). Prevalence of and factors associated with malaria in children under five years of age in Malawi, using malaria indicator survey data. Heliyon 6, e03946. doi:10.1016/j.heliyon.2020.e03946

PubMed Abstract | CrossRef Full Text | Google Scholar

World Health Organization (2023). World malaria report 2023. Geneva: World Health Organization.

Google Scholar

Gopakumar, G. P., Swetha, M., Sai Siva, G., and Sai Subrahmanyam, G. R. K. (2018). Convolutional neural network-based malaria diagnosis from focus stack of blood smear images acquired using custom-built slide scanner. J. Biophot. 11. doi:10.1002/jbio.201700003

PubMed Abstract | CrossRef Full Text | Google Scholar

He, K., Zhang, X., Ren, S., and Sun, J. (2016). “Deep residual learning for image recognition,” in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, June 27–30, 2016 (IEEE), 770–778.

CrossRef Full Text | Google Scholar

Hosch, S., Yoboue, C. A., Donfack, O. T., Guirou, E. A., Dangy, J., Mpina, M., et al. (2022). Analysis of nucleic acids extracted from rapid diagnostic tests reveals a significant proportion of false positive test results associated with recent malaria treatment. Malar. J. 21, 23. doi:10.1186/s12936-022-04043-7

PubMed Abstract | CrossRef Full Text | Google Scholar

Kim, J., Choi, W., Yoo, D., Kim, M., Cho, H., Sung, H.-J., et al. (2023). Solution-free and simplified H&E staining using a hydrogel-based stamping technology. Front. Bioeng. Biotechnol. 11, 1292785. doi:10.3389/fbioe.2023.1292785

PubMed Abstract | CrossRef Full Text | Google Scholar

Krizhevsky, A., Sutskever, I., and Hinton, G. E. (2012). ImageNet classification with deep convolutional neural networks. Adv. Neural Inf. Process Syst. 60, 84–90. doi:10.1145/3065386

CrossRef Full Text | Google Scholar

Li, S., Du, Z., Meng, X., and Zhang, Y. (2021). Multi-stage malaria parasite recognition by deep learning. Gigascience 10, giab040. doi:10.1093/gigascience/giab040

PubMed Abstract | CrossRef Full Text | Google Scholar

Liang, Z., Powell, A., Ersoy, I., Poostchi, M., Silamut, K., Palaniappan, K., et al. (2016). “CNN-based image analysis for malaria diagnosis,” in 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM). Shenzhen, China, 2016 Dec 15–18 (IEEE), 493–496.

CrossRef Full Text | Google Scholar

Madhu, G., Mohamed, A. W., Kautish, S., Shah, M. A., and Ali, I. (2023). Intelligent diagnostic model for malaria parasite detection and classification using imperative inception-based capsule neural networks. Sci. Rep. 13, 13377. doi:10.1038/s41598-023-40317-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Meng, X., Ha, Y., and Tian, J. (2022). Neighbor correlated graph convolutional network for multi-stage malaria parasite recognition. Multimed. Tools Appl. 81, 11393–11414. doi:10.1007/s11042-022-12098-6

CrossRef Full Text | Google Scholar

Mody, R. M., Murray, C. K., Dooley, D. P., Hospenthal, D. R., Horvath, L. L., Moran, K. A., et al. (2006). The remote diagnosis of malaria using telemedicine or e-mailed images. Mil. Med. 171, 1167–1171. doi:10.7205/MILMED.171.12.1167

PubMed Abstract | CrossRef Full Text | Google Scholar

Molina, A., Alférez, S., Boldú, L., Acevedo, A., Rodellar, J., and Merino, A. (2020). Sequential classification system for recognition of malaria infection using peripheral blood cell images. J. Clin. Pathol. 73, 665–670. doi:10.1136/jclinpath-2019-206419

PubMed Abstract | CrossRef Full Text | Google Scholar

Navya, K. T., Prasad, K., and Singh, B. M. K. (2022). Analysis of red blood cells from peripheral blood smear images for anemia detection: a methodological review. Med. Biol. Eng. Comput. 60, 2445–2462. doi:10.1007/s11517-022-02614-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Oduola, A. O., Obembe, A., Adelaja, O. J., Adeneye, A. K., Akilah, J., and Awolola, T. S. (2018). Outcome of capacity building intervention for malaria vector surveillance, control and research in Nigerian higher institutions. Malar. J. 17, 193. doi:10.1186/s12936-018-2344-z

PubMed Abstract | CrossRef Full Text | Google Scholar

Oktiyani, N., Muhlisin, A., Roebiakto, E., Norsiah, W., and Mahpolah, M. (2023). Utilization of alternative buffer solutions for staining thin blood smears by the Giemsa, Wright stain and Romanowsky method. Trop. Health Med. Res. 5, 34–45. doi:10.35916/thmr.v4i1.76

CrossRef Full Text | Google Scholar

Poti, K. E., Sullivan, D. J., Dondorp, A. M., and Woodrow, C. J. (2020). HRP2: transforming malaria diagnosis, but with caveats. Trends Parasitol. 36, 112–126. doi:10.1016/j.pt.2019.12.004

PubMed Abstract | CrossRef Full Text | Google Scholar

Rajaraman, S., Jaeger, S., and Antani, S. K. (2019). Performance evaluation of deep neural ensembles toward malaria parasite detection in thin-blood smear images. PeerJ 7, e6977. doi:10.7717/peerj.6977

PubMed Abstract | CrossRef Full Text | Google Scholar

Ronneberger, O., Fischer, P., and Brox, T. (2015). “U-Net: convolutional networks for biomedical image segmentation,” in 2015 international Conference on Medical image Computing and Computer-Assisted Intervention (MICCAI). Munich, Germany, 2015 Oct 5–9. 9351.

CrossRef Full Text | Google Scholar

Shen, D., Wu, G., and Suk, H.-I. (2017). Deep learning in medical image analysis. Annu. Rev. Biomed. Eng. 19, 221–248. doi:10.1146/annurev-bioeng-071516-044442

PubMed Abstract | CrossRef Full Text | Google Scholar

Sori, G., Zewdie, O., Tadele, G., and Samuel, A. (2018). External quality assessment of malaria microscopy diagnosis in selected health facilities in Western Oromia, Ethiopia. Malar. J. 17, 233. doi:10.1186/s12936-018-2386-2

PubMed Abstract | CrossRef Full Text | Google Scholar

Tran, T., Minh, L. B., Lee, S.-H., and Kwon, K.-R. (2019). Blood cell count using deep learning semantic segmentation. doi:10.20944/preprints201909.0075.v1

CrossRef Full Text | Google Scholar

U.S. President’s Malaria Initiative (2020). Malawi malaria operational plan FY. Available at: https://d1u4sg1s9ptc4z.cloudfront.net/uploads/2022/01/FY-2022-Malawi-MOP.pdf (Accessed February 25, 2023).

Google Scholar

Woo, S., Park, J., Lee, J.-Y., and Kweon, I. S. (2018). CBAM: convolutional block attention module. Comput. Sci. doi:10.48550/arXiv.1807.06521

CrossRef Full Text | Google Scholar

World Health Organization (2015). Microscopy for the detection, identification and quantification of malaria parasites on stained thick and thin blood films in research settings (version 1.0): procedure: methods manual. Geneva, Switzerland: World Health Organization.

Google Scholar

Wortsman, M., Ilharco, G., Gadre, S. Y., Roelofs, R., Gontijo-Lopes, R., Morcos, A. S., et al. (2023). Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time. ArXiv 2203, 05482. doi:10.48550/arXiv.2203.05482

CrossRef Full Text | Google Scholar

Yoon, J., Kwon, J. A., Yoon, S. Y., Jang, W. S., Yang, D. J., Nam, J., et al. (2019). Diagnostic performance of CellaVision DM96 for Plasmodium vivax and Plasmodium falciparum screening in peripheral blood smears. Acta. Trop. 193, 7–11. doi:10.1016/j.actatropica.2019.02.009

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhao, O. S., Kolluri, N., Anand, A., Chu, N., Bhavaraju, R., Ojha, A., et al. (2020). Convolutional neural networks to automate the screening of malaria in low-resource countries. PeerJ 8, e9674. doi:10.7717/peerj.9674

PubMed Abstract | CrossRef Full Text | Google Scholar