Ranking Methodology for Evaluating Region-Wise COVID-19 Testing Performance in India

For containment of COVID-19, most countries are following the isolate, test, treat and trace approach. Following the approach, India scaled up COVID-19 testing from about 5,000 tests per day at the end of March 2020 to more than 1 M tests per day in September 2020. Testing scale-up has seen a huge variation across states and union territories (UTs) with respect to growth rates, testing strategies, testing infrastructure, and deployment of various kit types (RT-PCR, Antigen, CBNAAT, etc). To understand the gaps in testing and prioritize appropriate interventions, it is important for national stakeholders to evaluate and rank states/UTs based on their testing performance. Indicators like total samples tested, total samples positive, tests per million, and positives per million are currently being used by epidemiologists and researchers for comparing the performance of various regions. This article proposes a robust ranking methodology to rank the states/UTs in India based on a comprehensive scoring developed by combining multiple variables for evaluating the testing performance of states/UTs. Since RT-PCR testing is considered the gold standard for COVID-19 testing, the composite score for testing performance in this article is defined by the ability of states/UTs to successfully deploy RT-PCR testing and reduce its positivity over time. Evaluating region-wise ranking can enable the identification of areas requiring immediate attention in poorly performing regions, thus channelizing efforts and resources in the right direction.

Introduction

The novel coronavirus COVID-19 originated in China in December 2019. It rapidly spread across the globe, eventually becoming a pandemic in March 2020 (1). WHO recommended governments isolate, test, and treat every case to quell and control the pandemic effectively (2). India detected its first case of COVID-19 on 30th Jan 2020 (3). Following the same approach, India augmented its testing capacity more than 200 times and is testing more than 1 million samples per day (4). Augmenting the capacity and scaling up of COVID-19 testing has played a crucial role in India's response to the pandemic. Despite huge disparities across states and union territories (UTs), this scale-up has been achieved in terms of testing volumes, infrastructure, product mix, strategies, and timelines. Going forward, states/UTs in India need to continue the scale-up in the positive direction to keep the pandemic in check. To accomplish this, the national stakeholders in India must continuously evaluate the state/UT testing performance in order to identify areas for development and achieve optimal testing performance.

The COVID-19 testing varies across states and UTs with respect to growth rates, testing strategies, testing infrastructure, and deployment of various kit types (RT-PCR, Antigen, CBNAAT, etc). Different countries, such as India (5) and USA (6, 7), among others, have ranked their states/UTs based upon the total samples tested and total samples positives (5–8). Indicators like total samples tested, total samples positive, tests per million, and positives per million are currently being used by epidemiologists and researchers for comparing the performance of various regions. For example, to measure COVID-19 testing performance, commonly used indicators include tests per million population, positivity rate, and rate of change of these indicators. A report from Niti Aayog (9) in 2019 describes creating a composite health index wherein scaling of indicators was done to compute a final composite score. For COVID-19, no composite indicators have been reported at a national level to date. This article aims to describe a comprehensive ranking methodology for evaluating the testing performance of states/UTs in India.

Methodology

Data Sources

1. Testing and positivity data: The total COVID-19 tests and positives detected in each state bifurcated by kit type (RT-PCR, Antigen, and Overall) for five consecutive weeks: 22–28 October 2020, 29 October−5 November 2020, 6–12 November 2020, 13–19 November 2020, and 20–26 November 2020. This aggregated data is taken from the ICMR's COVID-19 database (4) containing a line list of all COVID-19 tests conducted in the country.

2. State population: State-wise projected population as of 31st May 2020 by Unique Identification Authority of India (10).

Variables for Ranking

The following variables have been calculated at a state level for the scoring process:

1. % RT-PCR tests in the selected week are defined as:

$\begin{array}{l}\%RTPCRtests=\frac{TotalsamplestestedusingRTPCR}{Totalsamplestested}*100\end{array}$

2. RT-PCR positivity (%) in the selected week is defined as:

$\begin{array}{l}\%RTPCRpositivity=\frac{TotalpositivesdetectedusingRTPCR}{TotalsamplestestedusingRTPCR}\\ \ast 100\end{array}$

3. RT-PCR tests per million (TPM) in the selected week are defined as:

$\begin{array}{l}RTPCRtestspermillion(TPM)\\ =\frac{TotalsamplestestedusingRTPCR}{Populationofthestate}\ast 1,000,000\end{array}$

4. Change in RT PCR positivity (%) is defined as:

$\begin{array}{l}\%ChangeinRTPCRpositivity\\ =\frac{\text{RTPCRpositivity}(\%)(\text{selectedweek-previousweek})}{\text{RT\hspace{0.17em}PCR\hspace{0.17em}positivity\hspace{0.17em}}(\%)\text{\hspace{0.17em}previous\hspace{0.17em}week}}\ast 100\end{array}$

5. Change in RT PCR TPM (%) is defined as:

$\begin{array}{l}\%ChangeinRTPCRTPM\\ =\frac{\text{RT\hspace{0.17em}PCR\hspace{0.17em}TPM\hspace{0.17em}}(\%)(\text{selected\hspace{0.17em}week\hspace{0.17em}-\hspace{0.17em}previous\hspace{0.17em}week})}{\text{RT\hspace{0.17em}PCR\hspace{0.17em}TPM\hspace{0.17em}}(\%)\text{\hspace{0.17em}previous\hspace{0.17em}week}}\ast 100\end{array}$

Ranking Methodology

The ICMR data analysis portal (4) has been used to collect state-level numbers for each of the five variables stated above, and the standardized scores at the state level for each of these variables have been calculated using the formula below.

$\begin{array}{l}Z=\frac{x-\mu }{\sigma }\end{array}$

Where Z = standardized score, x = observed value for the variable and state, μ = mean of the variable, and σ = standard deviation of the variable.

For variables RT-PCR positivity (%) in the last week and change in RT-PCR positivity (%), a higher Z-value reflects a poor score for the two variables i.e., RT-PCR positivity (%) in the last week and change in RT-PCR positivity (%), whereas a higher value reflects a higher score for the other three variables i.e., % RT-PCR tests, RT-PCR tests per million (TPM), and change in RT PCR TPM (%). Therefore, the variables RT-PCR positivity (%) in the last week and change in RT-PCR positivity (%) have been multiplied by −1 to get the final standardized scores, which reflect the fact that higher scores imply higher ranking across the five variables.

After calculating standardized scores, each of the standardized variables has been given equal weights (20%), and a composite score for each state has been calculated by adding the sum of the standardized scores and the weights of the five variables.

$\begin{array}{l}Compositescore=\sum (\text{Weight }*Standardizedscore)\end{array}$

Before assigning equal weights to all the five variables, a correlation matrix has been made for standardized scores of all variables for all last 5 weeks. Ideally, variables with high correlation (Pearson correlation coefficient > 0.7) should not be assigned the same weights (11) in creating the composite score since the combination of highly correlated variables can skew the composite score. For correlated variables, different methods as described by Becker et al. can be used (12) for decomposition that minimizes the effect caused by correlations of variables for assigning the weights based on the dataset. The correlation matrix for all the 5 weeks showed that none of the five variables were substantially correlated with each other, therefore equal weighting was justified.

Results

For the practical demonstration, the state-wise ranking of testing performed for the week 20^th-26th November 2020 is shown in Table 1. The top five states/UTs in order of testing performance were S1, S2, S3, S4, and S5 (as shown in Table 1), according to composite scores obtained for each state for the selected week: 20th−26th November 2020. The top five states/UTs have a high percent of RT-PCR tests, low RT-PCR positivity, high RT-PCR tests per million, declining RT-PCR positivity (percent), and increasing RT-PCR tests per million, as seen in Table 1. The bottom five states, on the other hand, exhibit reverse behavior for most of the variables and require actions to enhance testing metrics where they lag the others.

TABLE 1

Table 1. State-wise ranking of Indian states on testing performance by the composite index for the week 20–26 November 2020.

However, if we compare consecutive 4-week composite indexes for all the states, 19 states increased their ranking from the week of 20–6 November 2020 to the week of 29 October−5 November 2020. The states have moved up in the rankings owing to improvement in testing numbers and vice versa. The state S4, for example, was ranked 31 in the week of October 29th to November 5th, despite being ranked 9 in the week of November 20th to 26th. The testing metrics have been improved in the multiple ways: (a) increase in % RT PCR tests from 27 to 43%; (b) decrease in RT PCR positivity from 26 to 22%; (c) increase in RT PCR TPM from 5,976 to 8,862; (d) week over week % change in RT PCR positivity from +44 to −23%; (e) week over week % change in RT PCR TPM from +9 to +42%. Because of a high percent of RT-PCR tests (>98%), low positive (3.2%), high tests per million (2–2.5 times the national average), and decreasing positivity week after week, the state S3 has been ranked third for the last 4 weeks. Likewise, in the last 4 weeks, State S35 has regularly been in the bottom half of the rankings, owing to a low percentage of RT-PCR tests (11–14%) and low tests per million (26–38% of the national average), with tests per million decreasing week after week.

Discussion and Conclusion

This article describes a composite index that will be calculated considering the testing done in the previous week. The states/UTs can be ranked based on the composite index. Further, once an intervention is implemented by the states, a change in the ranking of the states in the consecutive weeks can help the stakeholders in analyzing the effect of their interventions. Thus, the ranking methodology discussed in this article can be used to evaluate and rank states/UTs/districts based on their testing performance across the several metrics and provide a directional sense of the gaps that would require focused efforts to improve the testing outcomes. For e.g., the States/UTs with an RT-PCR TPM that is less than the national average and RT-PCR positivity greater than the national average should be prime targets for improving RT-PCR coverage. This methodology can also be used to compare the performance of regions or facilities on key programmatic indicators in other national and state programs. The publication of these rankings on a regular basis can promote healthy competition among states/UTs to improve their performance across critical indicators, which will enable them to attain the optimum programmatic outcomes. A limitation of this methodology is that it just aims to rank the states and UTs based on their current testing performance and does not take into account the qualitative factors, such as governance, IEC activities, population demographics, and social behavior, which are also important in determining testing outcomes. In future work, the ranking of the states/UTs using this proposed ranking methodology can be used for optimizing the testing performance in poorly performing & low resource regions by integrating a model and channelizing efforts & resources in the right direction (13).

Data Availability Statement

The original contributions presented in the study are included in the article/supplementary material, further inquiries can be directed to the corresponding author/s.

Ethics Statement

The study has been approved by the Director-General, Indian Council of Medical Research. The study does not require taking informed consent from all patients as no patient is sampled exclusively for the study.

Author contributions

HS and JasmK have conceptualized the work and devised the methodology. JasmK and JaslK have written the manuscript. HS has critically reviewed the manuscript. All authors listed have made a substantial, direct, and intellectual contribution to the work and approved it for publication.

Funding

This study has been funded by the Indian Council of Medical Research, New Delhi (No- ISRM/14(01)/TF/Genom/2017).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

Special thanks to Chirag Sidana and Shashwat Shivam, William J. Clinton Foundation, India for their feedback and suggestions.

References

1. Mackenzie JS, Smith DW. (2020). COVID-19: a novel zoonotic disease caused by a coronavirus from China: what we know and what we don't. Microbiol Aust. 41:45. doi: 10.1071/MA20013

PubMed Abstract | CrossRef Full Text | Google Scholar

2. WHO Director-General's Opening Remarks at the Media Briefing on COVID-19. (2020). Available online at: https://www.who.int/director-general/speeches/detail/who-director-general-s-opening-remarks-at-the-media-briefing-on-covid-19-−18-march-2020 (accessed June 2, 2022).

Google Scholar

3. Andrews MA, Areekal B, Rajesh KR, Krishnan J, Suryakala R, Krishnan B, et al. First confirmed case of COVID-19 infection in India: a case report. Indian J Med Res. (2020) 151:490. doi: 10.4103/ijmr.IJMR_2131_20

PubMed Abstract | CrossRef Full Text | Google Scholar

4. ICMR COVID-19 Analytics Portal. Available online at: https://cvanalytics.icmr.org.in/ (accessed June 2, 2022).

Google Scholar

5. India: Number of COVID-19 Tests by State 2021. Statista. Available online at: https://www.statista.com/statistics/1111063/india-coronavirus-covid-19-testing-pre-million-by-state/ (accessed June 2, 2022).

Google Scholar

6. All State Comparison of Testing Efforts. Johns Hopkins Coronavirus Resource Center. Available online at: https://coronavirus.jhu.edu/testing/states-comparison (accessed June 2, 2022).

Google Scholar

7. States Ranked by Number of People Tested for COVID-19. Available online at: https://www.beckershospitalreview.com/rankings-and-ratings/states-ranked-by-number-of-people-tested-for-covid-19.html (accessed June 2, 2022).

Google Scholar

8. COVID-19, Testing, Ranked State by, State. newsadvance.com. Available online at: https://newsadvance.com/html_0ee7c256-b77c-5837-8eb7-167c7c13684a.html (accessed June 2, 2022).

Google Scholar

9. Healthy States Progressive India - Report on the Ranks of States and Union Territories. Niti Aayog (2019). Available online at: http://social.niti.gov.in/uploads/sample/health_index_report.pdf

Google Scholar

10. Unique Identification Authority of India. State Population Data. (2020). Available online at: https://uidai.gov.in/images/state-wise-aadhaar-saturation.pdf

Google Scholar

11. Pearson Correlation - an Overview. ScienceDirect Topics. Available online at: https://www.sciencedirect.com/topics/computer-science/pearson-correlation (accessed June 2, 2022).

Google Scholar

12. Becker W, Saisana M, Paruolo P, Vandecasteele I. Weights and importance in composite indicators: closing the gap. Ecol Indic. (2017) 80:12–22. doi: 10.1016/j.ecolind.2017.03.056

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Cherian P, Krishna S, Menon GI. Optimizing testing for COVID-19 in India. PLoS Comput Biol. (2021) 17:e1009126. doi: 10.1371/journal.pcbi.1009126

PubMed Abstract | CrossRef Full Text | Google Scholar