Reproducibility and agreement of radiographic assessment of carpal deformities in foals

Introduction: The Pivot Point (PP) method is commonly used in the radiographic assessment of carpal deformities in young foals, as the range of deviation may influence treatment choice. The aims of this study were to assess the intra- and interobserver reproducibility of the PP method and subjective radiographic evaluation without line drawing to establish the anatomical site responsible for carpal deviation in foals and to evaluate the agreement between these two techniques.

Material and methods: Anonymized radiographs of foals presented for investigation of carpal deformity or prematurity were retrospectively and independently reviewed by six readers. Readers were first asked to subjectively identify the origin of the deviation and then apply the PP method and calculate the angle of deviation (PP_Angle). A second reading in a different randomized order was performed at least two weeks after the first reading. The carpi with the highest variability in PP_Angle measurements were reviewed in consensus by two other radiologists who did not perform the measurements.

Results: A total of 52 radiographs from 25 foals were selected. Good intraobserver reproducibility was observed for all variables, with no significant differences between the first and second readings by the same reader. Measurement of the angle of deviation using the PP method had a high intraobserver reproducibility (correlation coefficient of 0.93, p < 0.05). PP and subjective evaluations revealed strong intraobserver reproducibility for the origin of deviation (Cramer coefficients of 0.4 and 0.5, respectively; p < 0.05). There was strong agreement between PP and subjective evaluation for establishing the origin of deviation for all readers (Cramer coefficient 0.41; p < 0.05). Conversely, interobserver reproducibility for PP and subjective evaluations was low (Kappa values of 0.26 and 0.20, respectively; p < 0.05). Higher variability of PP_Angle was found in limbs with lateral bowing of the distal radius.

Discussion: The results of this study suggest that the PP method can reliably be used by the same reader for follow-up of carpal deformities and that there is no need to draw lines if the only required information is the origin of deviation, while measurements by different readers on the same patient may be misleading.

Introduction

Carpal deformities represent a prevalent and well-known syndrome in foals, characterized by abnormal alignment and rotation of bone segments constitutive of the carpus (1). These deformities can be present at birth or manifest during the initial months of life (2). The 3 primary reported causes of carpal deformities are ligament hyperlaxity, incomplete ossification of cuboid carpal bones and asymmetric development of the distal radial physis (2). These osteoarticular abnormalities are typically assessed in the dorsal plane with description of valgus or varus and origin of the deviation. Less commonly, malalignment in the sagittal plane, referred to as flexural carpal deformities, along with concurrent joint flexion or extension, is also described (2, 3).

Due to the rapid growth experienced in early life, limb deformities can rapidly develop. The growth plates of the long bones typically close in the first year of life, and cuboid carpal bones are fully developed about 18 months of age (4). Therefore early diagnosis and treatment are crucial for successful management of carpal deformities (3).

Conservative management or surgical treatment are used to correct these growth disorders otherwise leading to permanent limb deformity. Treatment approaches are based on the patient clinical condition but also depend on the degree of the angular limb deformity (2, 3, 5). Carpal deviation in the frontal plane less than 4° is considered normal in neonates (6), while when deviation exceeds 4°, a medical or surgical management should be established rapidly. While ligament hyperlaxity and incomplete ossification can often be managed through physiotherapy and/or application of splints, surgical intervention can be recommended for uneven growth plate development (3, 5). Surgery is recommended if a conservative approach fails and/or when deviation is superior to 10° (2, 7).

Radiography is the method currently used to determine the origin and the degree of deviation and is used to monitor its evolution (2, 5). The PP method is the most commonly used method to determine the anatomical site responsible for deviation and to measure the angle of deviation on dorsopalmar radiographs of the carpus that include radial and metacarpal/tarsal diaphysis (5, 8). It consists in drawing 2 lines bisecting the radius and the third metacarpal bone respectively, with the intersection representing the origin of the deviation and the angle between the 2 lines its amplitude/range.

Nonetheless, while intended to offer greater objectivity than subjective visual assessment, this technique remains imperfect (8). Discrepancies in radiographic projection have the potential to lead to erroneous measurement, a concern that becomes critical when deciding the opportunity of surgical treatment based on the 10° cut-off (2, 7).

Therefore, the objectives of this study were first, to assess the intra- and interobserver reproducibility in the radiographic analysis (measurements and origin) of carpal deviation, both subjectively and using the PP method; then, to investigate the agreement of the PP method with subjective radiographic evaluation without line drawing for identification of the site of origin. It was hypothesized that intraobserver reproducibility of the PP method would be good to excellent, while interobserver reproducibility would be lower, particularly between observers of different experience (either students versus veterinarian or radiologists versus surgeons), and that agreement between subjective evaluation and PP method for identification of the origin of the deviation would be good.

Materials and methods

Selection and description of subjects

This was a retrospective, secondary analysis study. Case records from the University of Liège from 2014 to 2022 were searched retrospectively for foals less than 6 months of age presented for investigation of carpal deformity or prematurity. Radiographic examination should include at least one dorsopalmar view of one or both carpi. Post-operative radiographs were excluded. Radiographs of foals with a skeletal ossification grade of 1 or 2 out of 4, according to Adams and Poulos (9), were also excluded.

Radiographic analysis and data recording

Dorsopalmar radiographs were acquired with a phosphide plate detector (Agfa CR with Musica system, Agfa HealthCare NV) or direct digital detector (Agfa DR 10e C with Musica system, Agfa HealthCare NV). Radiographs of each included carpus were randomized, anonymized, and compiled within a file document on a free DICOM medical image viewer (Horos Project, Geneva, Switzerland). Six independent observers, representing 3 groups of different experience (2 ECVDI-certified veterinary radiologist, 2 ECVS-certified surgeons, and 2 final year veterinary students) assessed the radiographs independently. All radiographs were presented to the reader oriented with the lateral aspect on the right.

For each radiograph, observers subjectively assessed image quality, categorizing it as poor, suboptimal or correct (grade 0, 1 or 2 respectively). Subsequently, they were asked to subjectively identify the site origin of the deviation (Orig_dev) as absent (no deviation), located at the radial diaphysis, at the radial metaphysis/epiphysis, articular or in the metacarpal region (Figure 1A). Following this subjective assessment, radiographs underwent geometric evaluation employing the PP method (10). Observers were asked to draw 2 lines bisecting the radius and the third metacarpal bone respectively, measure the angle between the 2 lines and identify their point of intersection (PP), representing the origin of the deviation. The angle formed by these 2 intersecting lines was labeled as the pivot point angle (PP_Angle) (Figure 1B). The PP method was demonstrated to the students on one carpus.

Figure 1

Figure 1. Dorsopalmar radiographic view of the left carpus of a 45 days hanovrian foal. (A) Four different categories for the location of the origin of the deviation. 1, radial diaphysis; 2, radial metaphysis or epiphysis; 3, articular; 4, metacarpus. (B) Application of the Pivot Point (PP) method. PP denotes the origin of the deviation, corresponding to the intersection of two lines bisecting the radius and the third metacarpal bone, respectively. The angle formed by these two lines is the pivot point angle (PP_Angle).

A second radiographic assessment was performed by each observer, at least 2 weeks after the initial reading. Radiographs were presented in a different randomized order.

For each carpus, minimal and maximum angle, mean angle and standard deviation were calculated among all readings. To assess a possible cause of high interobserver discrepancies in PP_Angle, carpi with the highest variability in PP_Angle measures, demonstrated by a standard deviation of more than 2.5°, were reviewed in consensus by 2 other radiologists (one ECVDI resident and one ECVDI-certified veterinary radiologist), who did not perform the measurements.

Statistics

The normality of the continuous variable PP_Angle was confirmed with a Shapiro test. For the discrete variables image quality, Orig_dev and PP, Kruskal-Wallis and Cramer tests were used to evaluate intraobserver and interobserver reproducibility and agreement between all readers and groups of readers. A Kappa or Chi-square test was used to weight the different levels of agreement between readers. The continuous variable PP_Angle was evaluated with ANOVA and Pearson correlation tests. Statistical tests were repeated after exclusion of radiographs with highest discrepancies in PP_Angle between readers (standard deviation >2.5°). All analyses were conducted with R files (version 3.6.2, R Core Team 2019) and on Microsoft Excel by a statistician. A p-value <0.05 was used as criterion for statistical significance.

Results

A total of 52 radiographs from 25 foals (including 16 males and 9 females) were selected. Median ages of horses was 25 days (range 1–182 days). Various breeds were included, with warmbloods being the most represented (n = 14).

The majority (84% of readings—524/624) of the images were considered to have optimal or sub-optimal quality.

A valgus deformity was identified in 35/52 foals, a varus in 3/52 foals, and the deviation was considered irrelevant (<4°) in the remaining 14 foals, based on the mean angle of all readers. Origin of the deviation was identified in the radial metaphysis/epiphysis for almost half of the readings (44 and 49% for subjective and PP evaluation, respectively).

A good intraobserver reproducibility was observed for all variables with no significant differences between first and second reading of the same reader, confirmed by a very strong association of each discrete variable Orig_dev and PP between both readings (Cramer coefficient 0.5 and 0.4, respectively; p < 0.05). For angle measurement, Pearson’s test showed a very strong correlation between the two readings (correlation coefficient of 0.93, p < 0.05).

Between both readings, radiologists changed significantly less their opinion about Orig_dev and PP compared with surgeons and veterinary students (Chi-square value of 18 for both parameters; p < 0.05). When a change occurred, it was not correlated to radiographic quality (p < 0.05).

For all readers, there was a significant association between Orig_dev and PP (Cramer coefficient = 0.41 with p < 0.05). Both parameters had a significant effect on PP_Angle (p = 0.01): carpi with no deviation or metacarpal origin of the deviation having small angles and carpi with radial diaphyseal, epiphyseal or articular origin of the deviation having higher angle measurements. Exclusion of images considered with poor quality (grade 0) by radiologists did not significantly influence other variables between readers or groups of readers.

Interobserver reproducibility was low with only fair agreement for PP and Orig_dev (Kappa value 0.26 and 0.20, respectively; p < 0.05). When considering PP_Angle of all readers, maximal angle differences ranged from 2 to 9,58°. Standard deviation from the mean of the measured angle ranged from 0.7° to 3.5°. Discrete variables image quality, Orig_dev and PP demonstrated a fair agreement between readers (respective kappa coefficient of 0.24, 0.25, and 0.20, p < 0.05).

Comparing readers’ group, significant differences were identified only for image quality and PP. In contrary there was no significant difference between groups for the subjective Orig_dev and PP_Angle.

The highest discrepancies in PP_Angle (standard deviation >2.5°) were found in 5 foals. Reviewing of the radiographs of these foals and their comparison with the others subjects by the non-observers radiologists revealed that all the carpi with high disagreement presented very asymmetric slopes of medial and lateral radial diaphyseal cortices, with a more pronounced concavity of the lateral cortex (Figures 2A–F). When excluded from statistical analysis, no significant differences for PP_Angle was observed between readers.

Figure 2

Figure 2. Dorsopalmar radiographic views of carpi of six different foals. (A–C) Are carpi with the highest disagreement in PP_Angle between readers. Notice the asymmetric slopes of the medial and lateral radial diaphysis in these three radiographs. (D–F) For comparison, are carpi with the highest agreement in PP_Angle between readers.

Discussion

The PP method is commonly used to assess and quantify carpal deformity in foals (5, 8). The present study aimed to assess intra- and interobserver reproducibility of the PP method and agreement between the PP method and a subjective evaluation to identify the site origin of the deviation.

Intraobserver reproducibility was found to be good. Because there was minimum 2 weeks interval between the readings of the same observer in the present study, good intraobserver reproducibility indicates that the reader has a clear personal understanding of how to draw the lines in the PP method and then applies the same criteria consistently each time. This is consistent with previous studies showing good reproducibility of the PP method and radiographic measurement of carpal parameters (8, 11). Olusa and coauthors showed high intraclass correlation coefficient for all the 14 investigated parameters, including PP_Angle (called medial carpal angle in their study) (11).

Intraobserver reproducibility of radiologists was higher compared to other readers. Experience in radiographic interpretation may explain the more constant opinion of radiologists who have extensive experience in consistently using and interpreting medical images and accurately identifying landmarks for measurements (12, 13).

Interobserver reproducibility was found to be poor. This differs from a previous study demonstrating an excellent correlation between investigators (8). This discrepancy is likely due to that fact that a precise and written guidance about how to trace the lines was not given to the readers in the present study. Therefore, without clear, standardized guidelines, each observer has interpreted the line placement differently, although an “internal standardization” of the technique and the reference points to trace the lines has likely occurred leading to consistency in each reader’s repeated measurements. The article of Brauer and colleagues do not explain how the guidelines were given to the readers so the possibility that the difference interobserver reproducibility due to an imaged and detailed explanation of the PP method given to the readers remains speculative. Visual demonstrations or examples would have likely increased the chances to have a better uniformity in the present study.

The subjective comparison of carpi of higher angle discrepancies with carpi of lower discrepancies suggests that most-likely cause is the inter-individual differences of methods in the placement of the line parallel to the long axis of the radius. This hypothesis is suggested by the fact that all cases with a high disagreement concerned foals with prominent curvature of the lateral cortex of the radius which may have complicated the estimation of the bone axis and/or the bisecting line between the medial and lateral radial cortex. In these cases, readers obtaining the highest measures likely placed the radial line parallel to the medial diaphysis while readers obtaining the lowest angles attempted to place the line in a position bisecting the angle between medial and lateral slopes.

Another technique named individual joint angle method has been proposed for radiographic evaluation of carpal deviations in foals, with excellent interobserver correlation (8). This technique consists in measuring the angle of deviation at the physis and at each carpal joints and then adding these angles to determine the total angle of deviation. Reproducibility of this technique has not been assessed in the present study as the PP method is the most used in equine practice, likely because the individual joint angle technique is more time-consuming. However, we can speculate that because total summated angle is the result of 4 individual joint angles, this method may have reduced interobserver discrepancies in the determination of the total angle of deviation.

Quality of a radiograph is subjective and depends on several factors such as exposure, centering, axis, collimation and patient positioning (4). In the present study it was asked to the reader to grade the “overall quality” of the radiographs, in terms of usability for assessing the carpal deviation, using a three-level semiquantitative scale without any given criteria. Using this overall subjective quality scale, radiographs considered with poor or suboptimal quality had limited influence on other parameters. However, some obliquity of radiographs may potentially have induced an over or under-estimation of the angle of deviation. Limb positioning was not precisely evaluated in the present study. In human medicine, variations of limb position, especially rotation, significantly influence the alignment from standing knee radiographs and standardized radiographs with established landmarks appears essential in recognition of positional error (11, 14). Amount of loading of the limb can also have significant effect on PP_Angle in radiographic assessment of the equine carpal joint of adult horses with increasing of loading lead to progressive increase in PP_Angle (15). Foal positioning is challenging and may have had influence in discrete and continuous variables of the present study.

Why significant differences of several parameters (Orig_Dev & PP_Angle) disappeared when considering groups rather than individual readers remains speculative. Most likely the reason is presence of relatively large differences between the 2 readers of each group and the reduction of individual differences by grouping and averaging of the measures.

The subjective identification of the site of origin of the deviation and the origin determined by the crossing lines (PP) agreed for the same reader in the present study. This is likely due to both methods relying on the same visual markers as landmarks. Since subjective visual markers guide both the subjective judgment and the objective measurement, the consistency in identifying these landmarks ensures that both methods yield similar results for the same reader. Moreover, lines’ placement was performed just after subjective assessment of the origin of the deviation. Therefore, the order of the assessments may have influenced one another with possible increase effect on association between the 2 methods. Even if delayed from subjective assessment, the fact that the initial impression remains present, a cognitive bias named anchoring in daily Medical Imaging practice (16, 17), has likely influenced the PP geometric technique.

Finally, because of the retrospective study design, radiographic measurements were made without any clinical consideration. This is not what happens in authentic clinical practice where visual assessment of the limb deformity on the foal precedes and potentially influences the radiographic interpretation, particularly in surgeons directly dealing with the patient and directly thinking about management and treatment while looking to radiographs.

In conclusion, the results of this study suggest than the PP method can reliably be used by the same reader for follow-up of carpal deformities, while measurements by different readers on the same patient may be misleading. Moreover, the agreement between the subjective assessment and the PP method, suggests that when the measure of the angle is not required, line drawing is not needed to identify the origin of the deviation. Finally high interobserver discrepancy suggests that when the angle is measured for decision making on type of treatment, multiple measurements by different readers, obviously interpreted in conjunction with the clinical assessment of the foal, should be used for final decision to reduce the margin of error.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation. Requests to access these datasets should be directed to YWxleGFuZHJlLmNoYXJsZXNAdWxpZWdlLmJl.

Ethics statement

Ethical approval was not required for the studies involving animals in accordance with the local legislation and institutional requirements because this study is a retrospective analysis of anonymized radiographs of client-owned animals obtained in a clinical context. Written informed consent was not obtained from the owners for the participation of their animals in this study because the use of the data is authorized by the owner in the hospitalization contract as included in the terms of sale.

Author contributions

AC: Conceptualization, Formal analysis, Writing – original draft, Writing – review & editing. XP: Writing – original draft. CV: Conceptualization, Writing – review & editing. MV: Writing – review & editing. LE: Writing – review & editing. VB: Conceptualization, Supervision, Validation, Writing – review & editing.

Funding

The author(s) declare no financial support was received for the research, authorship, and/or publication of this article.

Acknowledgments

The authors would like to thank the surgeons Alexandra Salciccia and Geoffroy de la Rebière de Pouyade as well as the students Damien Charlet and Mathilde Simon for the acquisiton of data (reading of radiographs).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. García-López, JM. Angular limb deformities: growth augmentation. Vet Clin North Am Equine Pract. (2017) 33:343–51. doi: 10.1016/j.cveq.2017.03.005

PubMed Abstract | Crossref Full Text | Google Scholar

2. Auer, JA. Angular limb deformities. In: JA Auer, JA Stick, editors. Equine urgery 4th ed. (St. Louis, MO: Elsevier Mosby) (2012), p. 1201–21.

Google Scholar

3. Trumble, TN. Orthopedic disorders in neonatal foals. Vet Clin North Am Equine Pract. (2005) 21:357–85. doi: 10.1016/j.cveq.2005.04.008

PubMed Abstract | Crossref Full Text | Google Scholar

4. Butler, JA, Colles, CM, Dyson, SJ, Kold, SE, Poulos, PW, and Puchalski, S. Clinical radiology of the horse. Fourth edition. Chapter 1 general principles. Wiley Blackwell: Chichester, West Sussex (2017). 798 p.

Google Scholar

5. McCarrel, TM. Angular limb deformities. Vet Clin North Am Equine Pract. (2017) 33:353–66. doi: 10.1016/j.cveq.2017.03.006

Crossref Full Text | Google Scholar

6. Auer, JA, Martens, RJ, and Williams, EH. Periosteal transection for correction of angular limb deformities in foals. J Am Vet Med Assoc. (1982) 181:459–66.

PubMed Abstract | Google Scholar

7. Auer, JA, and von Rechenberg, B. Treatment of angular limb deformities in foals. Clin Tech Equine Pract. (2006) 5:270–81. doi: 10.1053/j.ctep.2006.09.001

Crossref Full Text | Google Scholar

8. Brauer, TS, Booth, TS, and Riedesel, E. Physeal growth retardation leads to correction of intracarpal angular deviations as well as physeal valgus deformity. Equine Vet J. (1999) 31:193–6. doi: 10.1111/j.2042-3306.1999.tb03171.x

PubMed Abstract | Crossref Full Text | Google Scholar

9. Adams, R, and Poulos, P. A skeletal ossification index for neonatal foals. Vet Radiol. (1988) 29:217–22. doi: 10.1111/j.1740-8261.1988.tb01503.x

Crossref Full Text | Google Scholar

10. Fretz, PB. Angular limb deformities in foals. Vet Clin North Am Large Anim Pract. (1980) 2:125–50. doi: 10.1016/S0196-9846(17)30178-7

Crossref Full Text | Google Scholar

11. Olusa, TAO, Ismail, SMY, Murray, CM, and Davies, HMS. Radiographic assessment of carpal conformation in the horse: technique development and validation of the consistency of measurements. Anat Histol Embryol. (2021) 50:284–99. doi: 10.1111/ahe.12627

PubMed Abstract | Crossref Full Text | Google Scholar

12. Lesgold, A, Glaser, R, Rubinson, H, Feltovich, P, Klopfer, D, and Wang, Y. Expertise in a complex skill: diagnosing X-ray pictures In: MTH Chi, R Glaser, and MJ Farr, editors. The nature of expertise. Hillsdale, NJ: Psychology Press (1988)

Google Scholar

13. Sheridan, H, and Reingold, EM. The holistic processing account of visual expertise in medical image perception: a review. Front Psychol. (2017) 8:1620. doi: 10.3389/fpsyg.2017.01620

PubMed Abstract | Crossref Full Text | Google Scholar

14. Cooke, TDV, and Sled, EA. Optimizing limb position for measuring knee anatomical axis alignment from standing knee radiographs. J Rheumatol. (2009) 36:472–7. doi: 10.3899/jrheum.080732

Crossref Full Text | Google Scholar

15. Olusa, TAO, Murray, CM, and Davies, HMS. Radiographic assessment of the equine carpal joint under incremental loads and during flexion. Comp Exerc Physiol. (2019) 15:359–70. doi: 10.3920/CEP180044

Crossref Full Text | Google Scholar

16. Strack, F, and Mussweiler, T. Explaining the enigmatic anchoring effect: mechanisms of selective accessibility. J Pers Soc Psychol. (1997) 73:437–46. doi: 10.1037/0022-3514.73.3.437

Crossref Full Text | Google Scholar

17. Busby, LP, Courtier, JL, and Glastonbury, CM. Bias in radiology: the how and why of misses and misinterpretations. Radiographics. (2018) 38:236–47. doi: 10.1148/rg.2018170107

PubMed Abstract | Crossref Full Text | Google Scholar