Common Evaluations of Disease Activity in Rheumatoid Arthritis Reach Discordant Classifications across Different Populations

Objectives: The classification of disease activity states in rheumatoid arthritis (RA) can be achieved through disease activity indices, such as the Disease Activity Score in 28 joints erythrocyte sedimentation rate (DAS28-ESR), the Simplified Disease Activity Index (SDAI), and the Clinical Disease Activity Index (CDAI). Subjective measurements, such as patient reported outcomes have been incorporated into several of these indices alongside more objective assessments, such as increases in the ESR and C-reactive protein. Moreover, while they use similar criteria, different indices weight these criteria to different extents. Therefore, the classifications based on each evaluation may not always be the same. We aim to compare the performance of the three indices and their individual components in two different populations.

Methods: Data from Dutch and Portuguese adherent centers were extracted from the METEOR database, a multinational collaboration on RA. We included a total of 24,605 visits from Dutch centers (from 5,870 patients) and 20,120 visits from Portuguese centers (from 3,185 patients). We compared the disease activity states as evaluated by the DAS28-ESR, CDAI, and SDAI across the two populations. In addition, we analyzed the individual components of each evaluation, including their respective contributions to the outcome, in each population.

Results: We found significant differences in the disease activity states classified with the DAS28-ESR between the two populations. SDAI and CDAI had more congruous results. While the proportion of visits to Dutch and Portuguese centers that were classified as “in remission” was very similar between the CDAI and SDAI, the DAS28-ESR gave discordant results. Dutch patients had lower ESRs, which is more heavily weighted in the DAS28-ESR. In addition, even though the mean physicians’ global assessment values did not vary significantly for Dutch vs Portuguese physicians, we found that doctors at Portuguese centers overall scored the physician’s global assessment lower than Dutch physicians for patient visits classified by disease activity state.

Conclusion: While the CDAI and SDAI assigned disease activity states that were largely similar, the DAS28-ESR was often discordant across the two populations. Moreover, we found that physicians, more than patients, evaluated disease activity differently among the Portuguese and Dutch populations.

Introduction

Rheumatoid arthritis (RA) is a chronic, inflammatory disease that can affect joints, resulting in pain and discomfort. Patient reported outcomes (PROs) are increasingly being included in clinical trials and clinical practice to evaluate pain, function, and quality of life. PROs have also been incorporated into several of the major disease activity indices alongside more objective assessments, such as increases in the erythrocyte sedimentation rate (ESR) and C-reactive protein (CRP). Specifically, the Disease Activity Score in 28 joints (DAS28-ESR) (1), the Simplified Disease Activity Index (SDAI) (2), and the Clinical Disease Activity Index (CDAI) (3) are the disease activity indices that are most frequently used in RA, which generate specific cut-off values that are used to classify RA as in remission or in a low, moderate, or high activity state (4–6).

DAS28-ESR (1) is the most commonly used evaluation in daily practice, as well as in clinical trials, where it has also been validated for assessment of treatment response (4, 7, 8). In contrast, SDAI (2) and CDAI (3) were developed later than DAS28-ESR (5), although the SDAI is included in the American College of Rheumatology (ACR)/European League Against Rheumatism (EULAR) remission criteria (9) and more often used in clinical trials. All three evaluations rely on composite scores and include similar individual components; however, the components are weighed differently in each assessment (3). In addition, the evaluations incorporate both objective and subjective components, including both patient and physician perspectives, which are not always the same. As such, there is the potential for variability between these indices, and our previous work, along with that of others, has reported disagreements (10, 11). This could be concerning, as physicians use the scores obtained to make decisions regarding proper treatment and dosing, although they also take into account the patient’s clinical history and perspectives, as well as their own professional observations.

Therefore, the primary objective of this study was to compare the performance of the three indices and their individual components in two different populations. Since patient and physician perspectives can vary based on factors such as education and culture, we compared these different measures in two distinct populations, using the Measurement of Efficacy of Treatment in the Era of Rheumatology (METEOR) multinational database.

Materials and Methods

Patients and Visits

Data from Dutch and Portuguese adherent centers were extracted from the METEOR database, a multinational collaboration on RA, from 2008 until 2013 (12). The database provides data on patient- and physician-reported outcome measures in RA. De-identified data have been longitudinally collected in the central database. Data collection from both countries started in 2008 (12, 13). Currently, the tool is used worldwide and includes more than 50,000 patients. Data from Dutch centers were directly inserted in METEOR, by physicians and clinical nurses. Netherlands data included patients from one university hospital (the majority) and several other centers of secondary care from the western part of the Netherlands. Conversely, Portuguese visits were initially registered in Reuma.pt, the nationwide Portuguese Rheumatic Diseases Register, from the Portuguese Society of Rheumatology that comprises contribution from 90% of Portuguese rheumatology centers (academic centers, public, and private hospitals) (13), and then were posteriorly exported to METEOR. Data acquisition was made by rheumatologist or rheumatology nurses. All RA patients fulfilled the 1987 ACR criteria for the diagnosis of RA (14). Our data set contained information from 9,055 RA patients for a total of 44,725 visits (24,605 visits from 5,870 Dutch patients and 20,120 visits from 3,185 Portuguese patients). Reuma.pt was approved by the Portuguese National Data Protection Board and the ethics committees of the participating hospitals. Patients provided written informed consent for registry participation. METEOR was approved by the local ethics committees and is adherent with European General Data Protection Regulation guidelines. All study procedures were in accordance with the Declaration of Helsinki. The METEOR tool used only de-identified data and all personal information was encrypted locally.

Measurements

Information was obtained about tender joint count based on 28-joint assessment (TJC28), swollen joint count based on 28-joint assessment (SJC28), physician’s global assessment of RA disease activity on a 100 mm visual analog scale (MDGA), patient’s global assessment of activity on a 100 mm analog scale, ESR and CRP (mg/l). Patients’ characteristics and number of visits were also collected (gender, age, disease duration, and disease diagnosis). The three clinical activity disease indices, the DAS28-ESR, SDAI, and CDAI were calculated as previously described (1–3).

Disease Activity Definitions

The three disease activity indices (DAS28-ESR, CDAI, and SDAI) and their respective validated cut-offs were used to define remission, low disease activity, moderate disease activity, and high disease activity as described in Table 1 (4–7).

TABLE 1

Table 1. Cutoff values for different disease activity states.

For remission definition, the 2011 ACR/EULAR Boolean remission criteria were also applied in this study. In the 2011 ACR/EULAR Boolean remission criteria, all TJC28, SJC28, CRP (mg/dl), and PGAvalues should be ≤1 (9).

Statistical Analysis

The demographic and clinical characteristics of the two populations were compared using chi-squared tests and t-tests, respectively, for discrete and continuous variables. DAS28-ESR, CDAI, and SDAI scores were calculated and at each visit the disease activity state was classified according to previously established cutoffs (4–6). We calculated the relative contribution of the individual components of DAS28-ESR, CDAI, and SDAI scores. Descriptive statistics for the RA core set measures were calculated. Z-tests for equality of two independent proportions and for equality of two independent means were used to compare the proportions of visits and the score means, respectively, at each disease activity state for each index and population. Correlation between TJC28, SJC28, ESR, and CRP was done using Pearson correlation test. Missing data were not imputed.

To eliminate the effect of the number of visits per patient, we performed a sensitivity analyses where we analyzed a subset of data containing one visit per patient from each population. For that, we wrote a Visual Basic for Applications (VBA) script that computed the total number of visits for each patient and, for each one of them, generated a random number between one and the total number of visits obtained. Then, for each patient, we selected the visit corresponding to the randomly generated number. In this sample of one random visit per patient, we independently performed the same analyses described above for the group with all visits. All available visits were analyzed using Stata or R programming. Statistical significance was determined when p-values were less than 0.05.

Results

A total of 44,725 rheumatology visits, from 9,055 RA patients, were included in our analysis. Of those, 5,870 patients were seen in Dutch centers, accounting for 24,605 visits, and 3,185 patients were seen in Portuguese centers, accounting for 20,120 visits. Information on the patients and their visits is described in Table 2.

TABLE 2

Table 2. Characteristics of patients and their visits.

The individual components of the disease activity scores, including acute phase reactants, were significantly higher for the Portuguese visits, with the only exception being the MDGA, which did not differ between groups.

We next performed an analysis of each disease activity category as classified by the DAS28-ESR, the CDAI, and the SDAI. Table 3 presents the proportion of Dutch and Portuguese visits in which the patient was classified into each disease activity category. We found that the proportion of patient visits in each disease category as classified by the DAS28-ESR was significantly different between the two populations. Specifically, significantly more visits to Portuguese centers involved patients classified as moderate or high disease activity by the DAS28-ESR than the Dutch centers. In comparison, the CDAI and SDAI results had a more homogenous distribution across visits. In addition, for the remission category, no significant differences were detected between the two populations according to the CDAI, SDAI, and ACR/EULAR 2011 remission criteria (9).

TABLE 3

Table 3. The number and percentage of visits in each population according to disease activity state and corresponding z-tests for equality of two independent proportions.

We also compared the individual components of the DAS28-ESR, CDAI, and SDAI between the two populations at each disease activity state (Tables 4 and 5). With the exception of the high disease activity category, we found ESR to be the individual parameter that differed the most between the Dutch and Portuguese visits. In addition, even though the mean physician’s global assessment values did not vary significantly for Dutch vs Portuguese physicians, we found that doctors at Portuguese centers overall scored the physician’s global assessment lower than Dutch physicians for patient visits with same disease activity state.

TABLE 4

Table 4. The mean scores for the individual components of each disease activity evaluation, according to disease activity category.

TABLE 5

Table 5. Comparison of the mean scores from the individual components of each disease activity evaluation between the Dutch and Portuguese populations.

In Dutch centers, SJC28 was positively correlated with TJC28 (r = 0.56; p < 0.001), with ESR (r = 0.23; p < 0.001) and with CRP (r = 0.32; p < 0.001). TJC28 was also positively correlated with ESR (r = 0.15; p < 0.001) and CRP (r = 0.17; p < 0.001). The same findings were found for the visits made in Portuguese centers, where SJC28 were positively correlated with TJC28 (r = 0.66; p < 0.001), ESR (r = 0.28; p < 0.001), and CRP (r = 0.25; p < 0.001). TJC28 was also positively correlated with ESR (r = 0.21; p < 0.001) and CRP (r = 0.19; p < 0.001). For both countries, these correlations are weak with the exception of the correlation between SJC28 and TJC28 where a moderate/good correlation was found.

The relative contribution of each individual component (independent of the weight of the variable, given by the standard calculation formula) was also determined. For the CDAI, we found that the contribution of global assessments of disease activity was decreased, and the contribution of joint counts was increased, in the high disease activity states compared to remission. We identified similar results with the SDAI; however, we also found that CRP (not included in the CDAI) had less of a contribution in high disease states compared to remission. With respect to the DAS28-ESR, the relative contribution of the individual components was very similar across populations and across all disease activity categories. Finally, the relative contribution of the PGA was higher for visits in which the CDAI and SDAI classified the patient as in remission than when remission was determined by DAS28-ESR. As such, higher PGA values were reported for visits in which patients were determined to be in remission by the DAS28-ESR than by the CDAI/SDAI, and less than 50% of “DAS28-ESR remission” visits were also in the CDAI or SDAI remission category.

To eliminate the effect of the number of visits per patient, we performed a sensitivity analysis where one visit was selected at random to analyze per patient. For this, we found that while the sample size was reduced, the results were generally unchanged (Tables 6 and 7).

TABLE 6

Table 6. Patients’ characteristics and disease activity assessment using a randomly selected visit per patient.

TABLE 7

Table 7. Comparison of the mean of the components of the disease activity indices within each disease activity state between Dutch and Portuguese populations using one randomly selected visit per patient.

Discussion

Here, we have compared the DAS28-ESR, CDAI and SDAI indices in clinical practice in both Portuguese and Dutch populations using a large sample of RA patients retrieved from the multinational METEOR database. These populations had different sociodemographic characteristics (the Portuguese population included more women and had a longer disease duration); however, our primary aim was to compare the performance of the three indices and their individual components, not the disease activity within each population. Nevertheless, we found differences in disease activity between Portuguese and Dutch populations, with the exception for MDGA values.

We observed that use of the DAS28-ESR to characterize patients’ disease activity states resulted in larger differences in the proportion of patients assigned to each state between the Dutch and Portuguese populations, compared to the CDAI and SDAI assessments. We also found that ESR which is weighted more substantially in the DAS28-ESR than the other evaluations, differed the most between the two populations, which might explain the level of disagreement observed between the Dutch and Portuguese populations when using this assessment (15). The higher levels of ESR in RA patients seen in Portuguese centers may be explained by the higher prevalence of obesity found in Portuguese population as compared to the Dutch (16). However, in our study, we did not have data to confirm this hypothesis. For the CDAI and SDAI, PGA was the most influential measure across all disease activity states, except for high disease activity. This could be due to the subjective nature of the PGA, as patients’ assessments of disease activity may vary depending on their individual characteristics, such as personality, sociodemographic factors, or culture (7, 17–20). Taken together, our results suggest that, by weighting the individual components of the assessment differently, the DAS28-ESR and the CDAI/SDAI may result in different classifications of RA disease activity.

In addition, there is an ongoing discussion about the appropriateness of cut-off values for therapy response criteria, as debated in a paper by González-Álvaro and colleagues (21). Remission assessed by both the CDAI and the SDAI were found to be more stringent than the DAS28-ESR score, but less so than the ACR/EULAR remission criteria. This is in agreement with previous findings (11, 22) and suggests that the DAS28-ESR value of 2.6 may not be the most appropriate remission cut-off point (23). Moreover, in our study, the mean PGA value for patients with a DAS28-ESR score of less 2.6 is approximately twice than the PGA value for patients classified in remission according to the ACR/EULAR Boolean criteria. The low impact that a high PGA has in the DAS28-ESR seemed to be the primary reason for the significantly higher percentage of DAS28-ESR visits classified as in remission in the Dutch clinics, compared to Portuguese. Conversely, other reports have investigated potential problems stemming from the inclusion of the PGA as a component of the ACR/EULAR remission criteria (24), and these same issues likely extend to the CDAI and SDAI, due to the heavy contribution of the PGA in these indices.

Another interesting observation of this study was that the Portuguese MDGA was lower than that of the Dutch, regardless of disease activity state, and was usually discordant from ESR and joint counts. This surprising result was also found in the Quantitative Standard Monitoring of Patients with RA (QUEST-RA) registry that included data from 30 countries, where significant intercenter variation for MDGA and ESR was present (25).

There are some limitations to our analysis that must be considered when interpreting our results. METEOR is a large multinational database that gathers information on daily clinical practice, allowing for comparisons across different RA populations. However, there were missing values for some variables, namely for CRP, patients’ therapies and MDGA (in particular in Dutch population), which limited our analysis. In addition, some relevant variables were not included in the METEOR database, such as comorbidities, body mass index and level of education. Therefore, we were unable to include an assessment of these factors.

Historically, disease activity states were defined based on the physician’s decision and the drugs that were prescribed (26). However, now physicians rely on evaluations of disease activity states to decide the appropriate treatment and management strategies (27). Therefore, an understanding of the disease activity indices used in clinical practice is critical and may result in novel insights that can be used to develop new evaluations and/or improve the existing composite indices, such as by establishing new disease activity cutoffs that may be more appropriate for different populations. Here, we report that the DAS28-ESR and the CDAI/SDAI weight their components differently, sometimes resulting in discordant assessments of RA disease activity. Therefore, these indices should not be used interchangeably. In addition, these tests incorporate subjective assessments as well as objective measurement, such as PROs and MDGA, which may be influenced by cultural and educational backgrounds. Moreover, a physician’s evaluation may differ from the patient’s perspective and, as was observed in this study, may also differ from objective assessments such as swollen joint counts. Since the concept of “treat-to-target” depends on well-defined targets, it is necessary to revise the targets in RA and achieve a standardized and consistent evaluation method before this concept can be applied successfully.

Conclusion

Here, we used the METEOR multinational database to analyze data from a total of 24,605 RA visits to Dutch and Portuguese clinics. We compared the outcomes of three methods to assess RA disease activity states; specifically, the DAS28-ESR, the CDAI, and the SDAI. We found that the percentage of Dutch and Portuguese visits classified as “in remission” was very similar when using the CDAI, the SDAI and the ACR/EULAR remission criteria. However, use of the DAS28-ESR resulted in a significantly higher proportion of remission classifications at Dutch clinics. In addition, we found that Portuguese physicians tended to classify patients into lower disease activity states than Dutch physicians.

Taken together, our results indicate that the DAS28-ESR and the CDAI/SDAI weights their individual components differently, which sometimes caused discordant assessments of RA disease activity. Based on our findings, a more consistent and standardized approach for classifying RA disease activity may be necessary, and the evaluations used may need to be adapted to better suit differences between individual populations.

Ethics Statement

Reuma.pt was approved by Portuguese National Data Protection Board (CNDP) and participant hospitals Ethics Committee. METEOR was approved by local Ethics Committees. All study procedures were in accordance with Declaration of Helsinki.

Author Contributions

HC, AR, JG, MS, JF, and JS contributed to the conception, design and organization of the study and critique of the manuscript. HC, AR, MG, SD, JG, MS, AF, JC, CA, EG, DH, PM, JB, JF, and JS contributed to the acquisition of data and/or the statistical analyses. HC, AR, MG, JG, MS, EG, PM, JF, and JS contributed to data interpretation and drafted the manuscript. All authors reviewed and accepted the final version of the manuscript.

Conflict of Interest Statement

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Acknowledgments

We would like to thank the METEOR Board for all of their support.

Funding

This work has received an unrestricted research grant from the Meteor Foundation. http://www.meteorfoundation.com.

References

1. van Gestel AM, Haagsma CJ, van Riel PL. Validation of rheumatoid arthritis improvement criteria that include simplified joint counts. Arthritis Rheum (1998) 41:1845–50. doi:10.1002/1529-0131(199810)41:10<1845::AID-ART17>3.0.CO;2-K

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Smolen JS, Breedveld FC, Schiff MH, Kalden JR, Emery P, Eberl G, et al. A simplified disease activity index for rheumatoid arthritis for use in clinical practice. Rheumatology (Oxford) (2003) 42:244–57. doi:10.1093/rheumatology/keg072

CrossRef Full Text | Google Scholar

3. Aletaha D, Nell VP, Stamm T, Uffmann M, Pflugbeil S, Machold K, et al. Acute phase reactants add little to composite disease activity indices for rheumatoid arthritis: validation of a clinical activity score. Arthritis Res Ther (2005) 7:R796–806. doi:10.1186/ar1740

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Fransen J, van Riel PL. The Disease Activity Score and the EULAR response criteria. Rheum Dis Clin North Am (2009) 35:745–57. doi:10.1016/j.rdc.2009.10.001

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Aletaha D, Smolen J. The Simplified Disease Activity Index (SDAI) and the Clinical Disease Activity Index (CDAI): a review of their usefulness and validity in rheumatoid arthritis. Clin Exp Rheumatol (2005) 23(5 Suppl 39):S100–8.

PubMed Abstract | Google Scholar

6. Aletaha D, Ward MM, Machold KP, Nell VP, Stamm T, Smolen JS. Remission and active disease in rheumatoid arthritis: defining criteria for disease activity states. Arthritis Rheum (2005) 52:2625–36. doi:10.1002/art.21235

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Prevoo ML, van’t Hof MA, Kuper HH, van Leeuwen MA, van de Putte LB, van Riel PL. Modified disease activity scores that include twenty-eight-joint counts. Development and validation in a prospective longitudinal study of patients with rheumatoid arthritis. Arthritis Rheum (1995) 38:44–8. doi:10.1002/art.1780380107

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Vander Cruyssen B, Van Looy S, Wyns B, Westhovens R, Durez P, Van den Bosch F, et al. DAS28 best reflects the physician’s clinical judgment of response to infliximab therapy in rheumatoid arthritis patients: validation of the DAS28 score in patients under infliximab treatment. Arthritis Res Ther (2005) 7:R1063–71. doi:10.1186/ar1787

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Felson DT, Smolen JS, Wells G, Zhang B, van Tuyl LH, Funovits J, et al. American College of Rheumatology/European League against rheumatism provisional definition of remission in rheumatoid arthritis for clinical trials. Ann Rheum Dis (2011) 70:404–13. doi:10.1136/ard.2011.149765

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Martins FM, da Silva JAP, Santos MJ, Vieira-Sousa E, Duarte C, Santos H, et al. DAS28, CDAI and SDAI cutoffs do not translate the same information: results from the Portuguese Rheumatic Diseases Register, Reuma.pt. Rheumatology (Oxford) (2015) 54(2):286–91. doi:10.1093/rheumatology/keu313

CrossRef Full Text | Google Scholar

11. Gülfe A, Aletaha D, Saxne T, Geborek P. Disease activity level, remission and response in established rheumatoid arthritis: performance of various criteria sets in an observational cohort, treated with anti-TNF agents. BMC Musculoskelet Disord (2009) 10:41. doi:10.1186/1471-2474-10-41

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Koevoets R, Allaart CF, van der Heijde DM, Huizinga TW. Disease activity monitoring in rheumatoid arthritis in daily practice: experiences with METEOR, a free online tool. J Rheumatol (2010) 37:2632–3. doi:10.3899/jrheum.100552

CrossRef Full Text | Google Scholar

13. Canhão H, Faustino A, Martins F, Fonseca JE; Rheumatic Diseases Portuguese Register Board Coordination, Portuguese Society of Rheumatology. Reuma.pt – the rheumatic diseases Portuguese register. Acta Reumatol Port (2011) 36:45–56.

PubMed Abstract | Google Scholar

14. Arnett FC, Edworthy SM, Bloch DA, McShane DJ, Fries JF, Cooper NS, et al. The American Rheumatism Association 1987 revised criteria for the classification of rheumatoid arthritis. Arthritis Rheum (1988) 31:315–24. doi:10.1002/art.1780310302

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Fleischmann R, van der Heijde D, Koenig AS, Pedersen R, Szumski A, Marshall L, et al. How much does Disease Activity Score in 28 joints ESR and CRP calculations underestimate disease activity compared with the Simplified Disease Activity Index? Ann Rheum Dis (2015) 74(6):1132–7. doi:10.1136/annrheumdis-2013-204920

CrossRef Full Text | Google Scholar

16. World Health Organization Regional Office for Europe. Nutrition, Physical Activity and Obesity. Netherlands: WHO Global Health Observatory Data Repository (2013). Available from: http://www.euro.who.int/__data/assets/pdf_file/0018/243315/Netherlands-WHO-Country-Profile.pdf

Google Scholar

17. Gvozdenović E, Volterbeek R, Allaart CF, Brenol C, Dougados M, Emery P, et al. Assessment of global disease activity in rheumatoid arthritis by patients and physicians: differences across countries in the METEOR database. J Clin Rheumatol (2015) 1(7):349–54. doi:10.1097/RHU.0000000000000296

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Putrik P, Ramiro S, Hifinger M, Keszei AP, Hmanouchi I, Dougados M, et al. In wealthier countries, patients perceive worse impact of the disease although they have lower objectively assessed disease activity: results form the cross-sectional COMARA study. Ann Rheum Dis (2016) 5(4):715–20. doi:10.1136/annrheumdis-2015-207738

CrossRef Full Text | Google Scholar

19. Stenov V, Willaing I. Patient perspectives of sharing experiences in group-based diabetes education: a second-order analysis. PSYCH (2016) 7(1):12–8. doi:10.4236/psych.2016.71003

CrossRef Full Text | Google Scholar

20. Haizlip J, May N, Schorling J, Williams A, Plews-Ogan M. Perspective: the negativity bias, medical education, and the culture of academic medicine: why culture change is hard. Acad Med (2012) 7(9):1205–9. doi:10.1097/ACM.0b013e3182628f03

PubMed Abstract | CrossRef Full Text | Google Scholar

21. González-Álvaro I, Castrejón I, Ortiz AM, Toledano E, Castañeda S, García-Vadillo A, et al. Cutoffs and response criteria for the Hospital Universitario La Princesa Index (HUPI) and their comparison to widely-used indices of disease activity in rheumatoid arthritis. PLoS One (2016) 11(9):e0161727. doi:10.1371/journal.pone.0161727

CrossRef Full Text | Google Scholar

22. Lin C, Karlson EW, Canhao H, Miller TA, Dligach D, Chen PJ, et al. Automatic prediction of rheumatoid arthritis disease activity from the electronic medical records. PLoS One (2013) 16(8):e69932. doi:10.1371/journal.pone.0069932

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Landewé R, van der Heijde D, van der Linden S, Boers M. Twenty-eight-joint counts invalidate the DAS28 remission definition owing to the omission of the lower extremity joints: a comparison with the original DAS remission. Ann Rheum Dis (2006) 65:637–41. doi:10.1136/ard.2005.039859

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Masri KR, Shaver TS, Shahouri SH, Wang S, Anderson JD, Busch RE, et al. Validity and reliability problems with patient global as a component of the ACR/EULAR remission criteria as used in clinical practice. J Rheumatol (2012) 39:1139–45. doi:10.3899/jrheum.111543

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Khan NA, Spencer HJ, Nikiphorou E, Naranjo A, Alten R, Chirieac RM, et al. Intercentre variance in patient reported outcomes is lower than objective rheumatoid arthritis activity measures: a cross-sectional study. Rheumatology (2017) 56(8):1395–400. doi:10.1093/rheumatology/kex076

CrossRef Full Text | Google Scholar

26. van der Heijde DM, van’t Hof MA, van Riel PL, Theunisse LA, Lubberts EW, van Leeuwen MA, et al. Judging disease activity in clinical practice in rheumatoid arthritis: first step in the development of a disease activity score. Ann Rheum Dis (1990) 49:916–20. doi:10.1136/ard.49.11.916

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Aletaha D, Machold KP, Nell VP, Smolen JS. The perception of rheumatoid arthritis core set measures by rheumatologists. Results of a survey. Rheumatology (Oxford) (2006) 45:1133–9. doi:10.1093/rheumatology/kel074

PubMed Abstract | CrossRef Full Text | Google Scholar