Anwesha Dhal Samanta
Rishima Borah
Gayatri Saberwal- Institute of Bioinformatics and Applied Biotechnology, Bengaluru, India
Introduction: Clinical trial registries serve a key role in tracking the trial enterprise. We are interested in the record of trials sites in India. In this study, we focused on the European Union Clinical Trial Registry (EUCTR). This registry is complex because a given study may have records from multiple countries in the EU, and therefore a given study ID may be represented by multiple records. We wished to determine what steps are required to identify the studies that list sites in India that are registered with EUCTR.
Methods: We used two methodologies. Methodology A involved downloading the EUCTR database and querying it. Methodology B used the search function on the registry website.
Results: Discrepant information, on whether or not a given study listed a site in India, was identified at three levels: (i) the methodology of examining the database; (ii) the multiple records of a given study ID; and (iii) the multiple fields within a given record. In each of these situations, there was no basis to resolve the discrepancy, one way or another.
Discussion: This work contributes to methodologies for more accurate searches of trial registries. It also adds to the efforts of those seeking transparency in trial data.
Introduction
Clinical trial registries are databases that serve a key role in keeping track of the trial enterprise, thereby promoting transparency and accountability in medical research. These registries were originally set up to meet two requirements. First, to list ongoing studies that patients may wish to participate in. And second, to avoid a bias in the literature that will arise if trial results are reported only if they are positive (1). If all trials are registered, it becomes difficult to hide those that do not yield a positive result. Aside from these two original aims, data in these registries have been used for many other purposes, such as tracking the advancement of cutting-edge science through trials, analyzing whether studies have been in compliance with the law, holding journals to account, analyzing the participation of developing country professionals in international studies, etc. (2).
Around the world, there are several public registries, and the prominent ones are ClinicalTrials.gov of the United States (US) and 17 registries that the World Health Organization (WHO) recognizes as Primary Registries (3). A given registry holds records for studies run in a country (such as Clinical Research Information Service, CriS, for the Republic of Korea), a region (the Pan African Clinical Trial Registry, PACTR, for all African nations), or in any part of the world (ClinicalTrials.gov). Clinical Trials Registry-India (CTRI) is one of the 17 primary registries. Mainly, CTRI holds records of studies that had sites in India, although it does accept records from countries that do not have their own Primary Registry (4).
Various stakeholders have pushed for the mandatory registration of all trials. Illustratively, since 2005, the International Committee of Medical Journal Editors, or ICMJE (5), has required the registration of each trial before the recruitment of the first participant. Also, there are laws in various countries or regions of the world that require that trials be registered. Examples include (a) the US’s Food and Drug Administration’s Amendments Act, or FDAAA, which, since 2007, has required the registration of a wide variety of studies (6), and (b) since 15 June 2009, it has been mandatory to register trials running in India with CTRI (7).
Our group is based in India and we are interested in the record of studies run locally. For instance, several of our studies have been concerned with how CTRI records could be improved (8–13).
Particularly relevant to this study is one where, using a modeling approach, we demonstrated that a few tens or hundreds of trials were not registered with CTRI, although the law required it, but were registered with ClinicalTrials.gov (9). This speaks to the issue of the “findability” of trials with sites in India.
In order to determine what steps need to be taken in order to identify every trial with sites in the country, we wished to study all major public registries. In unpublished work, we studied ClinicalTrials.gov. This is not a Primary Registry, but is the largest among these public registries, and is termed a data provider to WHO (14). Next, we wished to identify such trials that were registered in non-Indian Primary Registries. In unpublished work, we covered 15 of these registries but excluded the European Union Clinical Trial Registry (EUCTR). EUCTR is complex because a given study may have records from multiple countries in the EU, and therefore a given study ID may be represented by multiple records. This is illustrated by trial 2014-002275-28, which has records from 22 countries (15). Given this complexity, in this work we focused on EUCTR alone and determined what steps are required to identify the studies that listed sites in India that are in this registry.
Methods
We provide an outline of the methods here, with further details in Supplementary material S1 and files referenced therein (Supplementary material S2–S7). Broadly, we used two methodologies. Methodology A involved downloading the EUCTR database and querying it for trials that had the keyword “India.” Since studies registered with CTRI have a trial ID beginning with “CTRI,” and such trials may be cross-referenced in EUCTR, we also queried EUCTR for studies containing “CTRI.” Methodology B used the search function on the website of EUCTR (16), searching for “India” and “CTRI.”
Each step in the Methodology was performed by two authors, independently.
Results
The several steps used to identify the trials of interest by Methodologies A and B are outlined in Figures 1, 2, and are detailed in Supplementary material S1–S7.
Figure 1. The steps used to identify the trials of interest by Methodology A.
Figure 2. The steps used to identify the trials of interest by Methodology B.
Here, we summarize the key results obtained by the two methodologies:
Methodology A: This method yielded 7,534 records (with 1,343 unique IDs), all of which mentioned India in Trial involving sites outside the EEA (that is, outside the European Economic Area). There were 39 other records (13 IDs) that had information in other fields indicating that India had hosted the study. Overall, 7,573 records (1,352 IDs) indicated that the study had sites in India.
Methodology B: This method yielded 7,533 records (1,342 IDs) that listed India in Trial involving sites outside the EEA. There were 32 other records (10 IDs) that had information in other fields indicating that the trial had run in India. Overall, 7,565 records (1,348 IDs) indicated that the study had sites in India.
In summary, Methodology A identified four IDs (eight records) that Methodology B did not pick up.
These IDs were 2020-001335-28, 2008-007784-16, 2008-007762-39, and 2006-000156-40. In one case, 2020-001335-28-NL, India had been listed as a country of recruitment at the time of this work, although it is no longer listed. The other seven cases were as follows: 2008-007784-16, 2008-007762-39, and 2006-000156-40 (in the last in Sweden, Great Britain, Hungary, Germany, and Czechoslovakia). In each of these cases, no country was listed as a site of recruitment. However, India was listed elsewhere, in a manner that indicated that the trial had run in the country.
As a final step, for the 1,352 IDs identified by Methodology A, which included all those identified by Methodology B, we assessed how often India was mentioned in the multiple records per ID. Only 849 IDs (63%) had complete consensus on this matter, where all the records mentioned India. Other IDs had discrepant records. For example, trial ID 2021-005184-42 had four records, only one of which listed India (Supplementary material S8). As such, it had a discrepancy of three. The number of discrepancies ranged from 1 to 19 countries per ID, with the distribution captured in Table 1. The discrepancies have been decreasing over the years, from 2004 to 2022, and in a linear regression of years against the number of discrepant records, the best-fit line yields an R2 value of 0.7 (Supplementary material S8).
Table 1. For 1,352 EUCTR trial IDs, discrepancies between “The total number of countries per ID” and “The number of countries that listed India.”
Discussion
Our study was based on the idea that various stakeholders should be able to determine which clinical trials have run in the country. In parallel, this issue has come up in the United Kingdom (UK) as well. Following an important review of commercial clinical trials in the UK, the Lord O’Shaughnessy review recommended that the government of the UK should establish a single platform called clinicaltrials.gov.uk (17). This platform should track all phase 1 to phase 4 clinical trials conducted within the country. Our effort has been a more limited one, directed at identifying all trials running in India that have been registered with EUCTR. This follows our earlier work in which we posed the same question to the 15 other non-Indian Primary Registries.
We used two methodologies to search the EUCTR database for studies that had sites in India, and identified slight discrepancies in the numbers found by these two methods. As such, the most obvious way of looking for records of interest, Methodology B, may not provide the correct answer. Further, the discrepancies in the multiple records per study ID caused some confusion since there was no basis to decide which record was correct, that is, whether or not the trial had sites in India. Finally, in several records we found that another field, that one would not normally inspect in one’s search for relevant trials, indicated that the study had sites in India. The fact that the most relevant field, Trial involving sites outside the EEA, did not list India but that some other field did, is confusing. In summary, discrepant information was identified at three levels: (i) the methodology of examining the database; (ii) the multiple records of a given study ID; and (iii) the multiple fields within a given record. In each of these situations, there was no basis to resolve the discrepancy.
It is important that such discrepancies be resolved. Recently, it was pointed out that about half of biomedical research is not published (18), and therefore, at least for clinical trials, the data in registries becomes important. Nevertheless, speaking more broadly, there are, again, various levels at which one can fail to identify relevant data.
First, not finding the record at all. Several individuals and groups (19, 20) have been concerned with how to find all relevant trials. For instance, the Cochrane Collective recommends that even though ClinicalTrials.gov supplies data to the International Clinical Trial Registry Platform (ICTRP), one needs to search both ClinicalTrials.gov and ICTRP to find relevant records (21). However, that, too, may be inadequate (22).
Second, other researchers have found discrepancies in the records of a given study registered with more than one registry. Illustratively, in a 2018 study that looked at almost 10,500 studies that were registered with both ClinicalTrials.gov and EUCTR, one-third of the EU trials were found to have a “completion status” different from that in ClinicalTrials.gov (23). And in a 2021 study of almost 200 trials that were registered with multiple registries, primarily ClinicalTrials.gov and EUCTR, researchers found discrepancies in sponsors, funding sources, primary outcomes, sample size, etc. (24).
And third, aside from discrepancies across the records of a given study, previous work has described discrepancies within various fields of a given record (8). Illustratively, and most pertinently, in earlier work, we noted that discrepant data in the record of a trial registered with CTRI may cause confusion as to whether the study had run only in India or across multiple countries (including India) (8). In the current study, seven of the eight discrepant records concern India being listed in a field other than Trial involving sites outside the EEA. Nevertheless, Methodology B should have picked them up since it did pick up other cases where India was in fields other than the Trial involving sites outside the EEA field. We are not sure why this did not happen. It may have to do with the specific algorithm used by the search function. To be noted, these records pertained to the 2006–2008 time frame. Given that the number of discrepancies have come down over the years, the source of this particular discrepancy may have been addressed. We are also not sure why the record in 2020 was not picked up by the search function. Since India is no longer listed as a country of recruitment, it is possible that it was dropped in the small time difference between when we ran Methodology A and Methodology B.
Although tips for searching the EU trial database are available,2 this study provides a word of caution about using only the obvious Methodology B for interrogating a trial database and also for searching obvious fields for a given piece of information, in this case Trial involving sites outside the EEA. In order to accurately catalogues the studies that may have listed sites in India, we need to use the more cumbersome methodology of downloading the entire database and then check every study ID for which at least one record mentions India, and within such records, every field that mentions it.
Our findings echo those of other researchers who have found that using filters can exclude relevant studies, such as when interventional studies have been mislabeled as observational studies (18). As such, this work adds to efforts to provide methodologies for more accurate searches (21). It also adds to the efforts of those seeking transparency (25–26) in trial data. We note that information available in this database has been provided by the relevant national competent authorities, to whom the sponsor had provided the details. The European Medicines Agency, which manages the database, is not responsible for the veracity of the information inputted to its database (27). We also note that the database is not specifically designed to answer our research question. Nevertheless, it may be possible to provide further guidance to the sponsors or use logic rules that would decrease the occurrence of some of the discrepancies. Only accurate data meets the goals of transparency (28). And as a US Senator remarked, “Public-facing websites run by the government should be accurate. That’s not asking much.” (29).
Limitations
In this work, we only studied the issue of whether or not a given study had sites in India. Although there may be discrepancies in other fields of data, we cannot extrapolate this work to those fields. Also, because we cannot verify information in a trial record with other documentation of the study, we cannot be sure of the veracity of all the inputted information.
Therefore, we may have both false positives (records indicating that India was a country of recruitment, although it was not) and false negatives (records that failed to indicate that India had been a country of recruitment, although it was). Finally, we have searched only for “India,” not for “IN” or for any misspellings of the word. It is possible that we missed records that did mention India, but in a different way.
In conclusion, in order to determine what steps need to be taken to identify every trial that listed sites in India that is registered with EUCTR, we used two methodologies to search the database. We also examined the multiple records, from multiple countries, of the same study ID. Finally, we examined not only the most obvious field, Trial involving sites outside the EEA, but also other fields for information that indicated that the study had sites in India. At all three levels there were discrepancies, which varied from small to large. It is unclear which data is reliable, and therefore it is important that the data be cleaned up and ways found to prevent such confusion in the future.
Transparency is meaningful only if the data is accurate.
Data availability statement
The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.
Author contributions
ADS: Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – review & editing. RB: Data curation, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – review & editing. GS: Conceptualization, Formal analysis, Funding acquisition, Investigation, Methodology, Project administration, Resources, Supervision, Validation, Visualization, Writing – original draft, Writing – review & editing.
Funding
The author(s) declare that financial support was received for the research, authorship, and/or publication of this article. This work was entirely supported by the Department of Electronics, IT, BT, and S&T of the Government of Karnataka, India.
Conflict of interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher’s note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary material
The Supplementary material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fmed.2024.1357930/full#supplementary-material
Footnotes
References
1. Dickersin, K, and Rennie, D. Registering clinical trials. JAMA. (2003) 290:516–23. doi: 10.1001/jama.290.4.516
2. Saberwal, G. The many uses of data in public clinical trial registries. Curr Sci. (2021) 120:1686–91. doi: 10.18520/cs/v120/i11/1686-1691
3. WHO (n.d.). Primary registries in the WHO registry network. Available at: https://www.who.int/clinical-trials-registry-platform/network/primary-registries. (Accessed June 26, 2024).
4. ICMR-National Institute of Medical Statistics (n.d.). CTRI—Frequently Asked Questions. Available at: https://ctri.nic.in/Clinicaltrials/faq.php (Accessed December 18, 2023).
5. De Angelis, C, Drazen, JM, Frizelle, FA, Haug, C, Hoey, J, Horton, R, et al. International Committee of Medical Journal Editors. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. N Engl J Med. (2004) 351:1250–1. doi: 10.1056/NEJMe048225
6. Zarin, DA, Tse, T, Williams, RJ, and Carr, S. Trial reporting in ClinicalTrials.Gov—the final rule. N Engl J Med. (2016) 375:1998–2004. doi: 10.1056/NEJMsr1611785
7. Pandey, A, Aggarwal, A, Maulik, M, Gupta, J, and Juneja, A. Challenges in administering a clinical trials registry: lessons from theClinical Trials Registry-India. Pharm Med. (2013) 27:83–93. doi: 10.1007/s40290-013-0009-3
8. Pillamarapu, M, Mohan, A, and Saberwal, G. An analysis of deficiencies in the data of interventional drug trials registered with Clinical Trials Registry—India. Trials. (2019) 20:535. doi: 10.1186/s13063-019-3592-0
9. Kumari, S, Mohan, A, and Saberwal, G. Hidden duplicates: 10s or 100s of Indian trials, registered with ClinicalTrials.Gov, have not been registered in India, as required by law. PLoS One. (2020) 15:e0234925. doi: 10.1371/journal.pone.0234925
10. Venugopal, N, and Saberwal, G. A comparative analysis of important public trial registries, and a proposal for an interim ideal one. PLoS One. (2021) 16:e0251191. doi: 10.1371/journal.pone.0251191
11. Chakraborty, I, Shreya, A, Mendiratta, J, Bhan, A, and Saberwal, G. An analysis of deficiencies in the ethics committee data of interventional trials registered with the Clinical Trials Registry–India. PLOS Global Public Health. (2022) 2:e0000617. doi: 10.1371/journal.pgph.0000617
12. Mendiratta, J, Pillamarapu, M, and Saberwal, G. In Clinical Trials Registry-India, the classification of sponsors needs to be standardised. Indian J Med Ethics. (2023) IX:11–8. doi: 10.20529/IJME.2023.071
13. Bhalla, IS, Ravindranath, AG, Vaswani, R, and Saberwal, G. Data from the Indian drug regulator and from Clinical Trials Registry-India does not always match. Front Med. (2024) 11:1346208. doi: 10.3389/fmed.2024.1346208
15. EU Clinical Trials Register. 2016-003897-41. Available at: https://www.clinicaltrialsregister.eu/ctr-search/search?query=2016-003897-41 (Accessed December 18, 2023).
16. EU Clinical Trials Register (2023). Clinical Trials Home & Search. Available at: https://www.clinicaltrialsregister.eu/ctr-search/search (Accessed December 18, 2023).
17. UK Government (2023). Commercial clinical trials in the UK: the Lord O’Shaughnessy review—final report. Available at: https://www.gov.uk/government/publications/commercial-clinical-trials-in-the-uk-the-lord-oshaughnessy-review/commercial-clinical-trials-in-the-uk-the-lord-oshaughnessy-review-final-report (Accessed December 18, 2023).
18. Hunter, KE, Webster, AC, Page, MJ, Willson, M, McDonald, S, Berber, S, et al. Searching clinical trials registers: guide for systematic reviewers. BMJ. (2022) 377:e068791. doi: 10.1136/bmj-2021-068791
19. Tang, W, Fukuzawa, M, Ishikawa, H, Tsutani, K, and Kiuchi, T. Review of the registration of clinical trials in UMIN-CTR from 2 June 2005 to 1 June 2010—focus on Japan domestic, academic clinical trials. Trials. (2013) 14:333. doi: 10.1186/1745-6215-14-333
20. Alqaidoom, Z, Nguyen, P-Y, Awadh, M, and Page, MJ. Impact of searching clinical trials registers in systematic reviews of pharmaceutical and non-pharmaceutical interventions: reanalysis of meta-analyses. Res Synth Methods. (2023) 14:52–67. doi: 10.1002/jrsm.1583
21. Banno, M, Tsujimoto, Y, and Kataoka, Y. Using the Cochrane Central Register of Controlled Trials to identify clinical trial registration is insufficient: a cross-sectional study. BMC Med Res Methodol. (2020) 20:200. doi: 10.1186/s12874-020-01083-y
22. Glanville, JM, Duffy, S, McCool, R, and Varley, D. Searching ClinicalTrials.Gov and the International Clinical Trials Registry Platform to inform systematic reviews: what are the optimal search approaches? J Med Libr Assoc. (2014) 102:177–83. doi: 10.3163/1536-5050.102.3.007
23. Fleminger, J, and Goldacre, B. Prevalence of clinical trial status discrepancies: a cross-sectional study of 10,492 trials registered on both ClinicalTrials.Gov and the European Union Clinical Trials Register. PLoS One. (2018) 13:e0193088. doi: 10.1371/journal.pone.0193088
24. Speich, B, Gloy, VL, Klatte, K, Gryaznov, D, Taji Heravi, A, Ghosh, N, et al. Reliability of trial information across registries for trials with multiple registrations: a systematic review. JAMA Netw Open. (2021) 4:e2128898. doi: 10.1001/jamanetworkopen.2021.28898
25. Viergever, RF, Karam, G, Reis, A, and Ghersi, D. The quality of registration of clinical trials: still a problem. PLoS One. (2014) 9:e84727. doi: 10.1371/journal.pone.0084727
26. House of Commons, UK Parliament, Science and Technology Committee (2018). Research integrity: clinical trials transparency. Available at: https://publications.parliament.uk/pa/cm201719/cmselect/cmsctech/1480/1480.pdf (Accessed December 18, 2023).
27. EU Clinical Trials Register. About the EU Clinical Trials Register. Available at: https://www.clinicaltrialsregister.eu/about.html (Accessed May 28, 2024).
28. Bruckner, T, and Vidal, J (2021). Access to clinical trial data in Europe. Lessons from EudraCT for Eudamed and the Clinical Trials Information System. Health Action International. Available at: https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&cad=rja&uact=8&ved=2ahUKEwjJ1NCIycz0AhWvSGwGHYBlB3oQFnoECAcQAQ&url=https%3A%2F%2Fhaiweb.org%2Fwp-content%2Fuploads%2F2021%2F09%2FLessons-Eudamed-and-CTIS-2021.pdf&usg=AOvVaw1fmvJU49tdIw_hodvxL5ML (Accessed December 18, 2023).
29. Piller, C. (2020). FDA and NIH let clinical trial sponsors keep results secret and break the law. Science. Available at: https://www.science.org/content/article/fda-and-nih-let-clinical-trial-sponsors-keep-results-secret-and-break-law (Accessed December 18, 2023).
Keywords: clinical trial registries, EUCTR, country of recruitment, data integrity, data quality, search strategies, trial registry metaresearch
Citation: Samanta AD, Borah R and Saberwal G (2024) Three levels of discrepancies in the records of trial sites in India, registered with the European Union Clinical Trials Register. Front. Med. 11:1357930. doi: 10.3389/fmed.2024.1357930
Edited by:
Hubert G. Leufkens, Utrecht University, NetherlandsReviewed by:
Shereen Cox, University of Oslo, NorwaySamruddhi Yerunkar, Charité University Medicine Berlin, Germany
Copyright © 2024 Samanta, Borah and Saberwal. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Gayatri Saberwal, Z2F5YXRyaUBpYmFiLmFjLmlu
†These authors have contributed equally to this work and share first authorship