Byron Field
Gordon Waddington
Andrew McKune
Roland Goecke
Andrew J. Gardner- 1Research Institute for Sport and Exercise, Faculty of Health, University of Canberra, Canberra, ACT, Australia
- 2Discipline of Biokinetics, Exercise, and Leisure Sciences, School of Health Sciences, University of KwaZulu Natal, Durban, South Africa
- 3Research Institute for Sport and Exercise, Faculty of Science and Technology, University of Canberra, Canberra, ACT, Australia
- 4Sydney School of Health Sciences, Faculty of Medicine and Health, The University of Sydney, Camperdown, NSW, Australia
Background: To better understand the biomechanical profile of direct head impacts and the game scenarios in which they occur in Rugby Union, there is a need for an on-field validation of a new instrumented mouthguard (IMG) against the reference standard. This study considers the potential of a combined biomechanical (IMG) and video analysis approach to direct head impact recognition, both of which in isolation have limitations. The aim of this study is to assess the relationship between an instrumented mouthguard and video analysis in detection of direct head impacts in rugby union.
Design: Pilot Study - Observational Cohort design
Methods: The instrumented mouthguard was worn by ten (3 backs, 7 forwards) professional Rugby Union players during the 2020–21 Gallagher Premiership (UK) season. Game-day video was synchronized with timestamped head acceleration events captured from the instrumented mouthguard. Direct Head Impacts were recorded in a 2 × 2 contingency table to determine sensitivity. Impact characteristics were also collected for all verified head impacts to further the understanding of head biomechanics during the game.
Results: There were 2018 contact events that were reviewed using video analysis. Of those 655 were categorized as direct head impacts which also correlated with a head acceleration event captured by the IMG. Sensitivity analysis showed an overall sensitivity of 93.6% and a positive predictive value (PPV of 92.4%). When false positives were excluded due to ball out of play, mouthguard removal or handling after a scoring situation or stoppage, PPV was improved (98.3%). Most verified head impacts occurred in and around the ruck contest (31.2%) followed by impacts to the primary tackler (28.4%).
Conclusion: This pilot validation study demonstrates that this IMG provides a highly accurate measurement device that could be used to complement video verification in the recognition of on-field direct head impacts. The frequency and magnitude of direct head impacts derived from specific game scenarios has been described and allows for greater recognition of high-risk situations. Further studies with larger sample sizes and in different populations of Rugby Union players are required to develop our understanding of head impact and enable strategies for injury mitigation.
Introduction
There is growing interest regarding the acute and chronic health effects of concussive and sub-concussive impacts in contact and collision sports (1, 2). As a result, there has been an increased motivation to better understand the biomechanical profile of head impacts and the game scenarios in which they occur (3, 4). A better understanding of the frequency, magnitude and game characteristics may assist with the time-critical decision of removing a player from participation to reduce subsequent injury risk.
A variety of head impact sensors are now commercially available, with most of the initial validation research arising from studies using instrumented helmets in American football (5, 6). Previous research has commonly combined the use of impact sensors inserted into helmets or caps or mounted on skin with game video analysis to verify the accuracy of impacts recorded by the wearable technology (7–9). However, coupling the sensor to movement of the skull is crucial for accurate detection of linear and rotational accelerations, thus overcoming errors caused by non-adherence to soft tissue mounted or head gear mounted sensors (10). Mouthguards with embedded inertial measurement devices are now available (6). This technology can be used to determine accelerations (via accelerometers) and angular accelerations (via gyroscopes) experienced at the head (10). This approach has been adopted to describe head-related impacts in amateur rugby union (6), head acceleration events (HAE) in rugby league (11), and to enable future research into the effects of impact exposure in soccer (12). The primary motivation among researchers is to provide evidence-based recommendations that may lead to improved player safety. The same technology is yet to be validated in professional rugby union.
To advance the understanding of the potential effects of acute and repetitive head impacts, on-field exposure data is critical to identify the characteristics specific to the game of rugby union. Given the widespread accepted use of mouthguards and the increasing ability to offer minimally intrusive instrumentation, IMGs offer the opportunity to improve the understanding of head impacts and their short, medium and long-term effects. As there has been an appreciation of the effect of load on injury in sport for many years with technology now common place in tracking output predominantly using wearables such as GPS devices, the IMG has the potential to progress the field of head impact load measurement, though requires on-field validation. One approach to validation is the use of video verification of observable impacts.
Developments in wearable instrumented technologies offer the potential to provide time-sensitive real-time feedback in a sporting environment to inform exposure monitoring, identify potentially injurious impacts and to provide objective quantification to aid performance and coaching (13). The major international contact sports (Rugby, Rugby league, ice hockey, American football, Australian football) have benefited from introducing video footage analysis (14) or GPS devices (15) for improving their ability to detect head injury. The intention of these approaches has been to quantify the type, frequency, intensity, and immediate effects of collision events to support clinical decision making.
Further, a clear understanding of the face validity is critical when considering whether head impact telemetry systems, such as the IMG, could be used as an adjunct to a clinical assessment. If the IMG were to be considered as an adjunct to clinical decision making, then having high false positive or high false negative results may undermine the confidence of the practitioner. The consequences of a device that over-estimates impacts could result in a loss of compliance or an overly conservative approach to head impact recognition while under reporting could have more significant consequences whereby a player may remain on the field despite a potentially injurious head impact. Similarly, these compliance and clinical recognition issues may also arise where the IMG is considered to be unreliable in the context of it being used to understand the cumulative kinematic loading sustained throughout the course of a given athletic exposure period (i.e., impacts or head accelerations during training and match play).
Possessing quantitative biomechanical head impact information may add additional support to the clinical assessment (16) and further develop our understanding of the relationship between the biomechanical transfer of force to the head and injury incidence and severity (17). The aim of this study is to assess the accuracy of a new instrumented mouthguard (IMG) for face validity in an on-field setting, in professional Rugby Union players. Given the unique impact profile of rugby union, IMG derived direct head impacts will be considered in the context of game characteristics using standardised definitions (18). This research has the potential to progress understanding of head impact exposure in Rugby union through a field-based assessment of accuracy of an emerging piece of wearable technology.
Method
Participants
Data were prospectively collected from Gallagher Premiership teams (the premier club Rugby Union competition in England) during the 2020–21 season. A total of 10 players (mean ± SD: age 25 yr ± 2 yr, body mass 110 kg ± 14 kg, height 191 cm ± 6 cm) across two clubs participated in the study. The participants included 3 backs and 7 forwards. There were 24 games requiring video verification of contact events with an average of 3 players per game using the IMG. All participants provided written informed consent to participate in the season-long study. The study was approved by the University of Canberra Human Research Ethics Committee (HREC No 20216912).
Mouthguard specifications
Prior to the start of the season, players were supplied with a dental impression-based custom fit, IMG (Nexus A9, HitIQ Pty Ltd, Melbourne Australia). The players were afforded the ability to have their unique IMG modified to ensure comfort and to optimise fit. The IMG contained three low-power, high-g triaxial accelerometers each sampling at approximately 3,200 Hz with a 200 g maximum. The IMG also contained a 16-bit triaxial gyroscope with a sampling rate of approximately 800 Hz capable of sampling at 3,200 Hz (down-sampled to 800 Hz). The IMG recorded linear and rotational acceleration, impact location, and duration. The time history incorporated three axes (x, y, z) of acceleration with an “event” recorded if a trigger threshold of 10 g of linear acceleration was reached. The trigger is based on the absolute value across one (ACCO) of the linear accelerometer's three axes. The subsequent “event” was recorded for 20 ms before and for 80 ms after the trigger was reached (19). If another trigger was identified during this recording period a further 80 ms was added (19). This allowed for multiple acceleration events to be captured resulting from a single impact.
Due to individual variation within linear accelerometer sampling rates, time series for each axis of the three linear accelerometer sensors were resampled to 3,200 Hz and the gyroscope up-sampled from 800 Hz to 3,200 Hz (19). To identify and remove signals associated with vocalisation and high frequency noise (19) the normed signal from the left linear accelerometer was low pass filtered at 300 Hz using a 4th order, non-phase corrected low-pass Butterworth filter. The IMG captures were filtered using a phase shift corrected, 4th order 300 Hz low-pass Butterworth filter and linearly transformed to the centre of gravity of the head based on the 50th percentile hybrid III headform (20). This laboratory-based validation, demonstrated a strong correlation between the IMG and reference headform (LCCC- .997 for PLA) (20). A classification algorithm described by Goodin et al. (19) was then used to identify head acceleration events which were subsequently included in the video verification process. IMGs were returned by the player at the end of each game and once in the charging unit, the data having been stored in onboard memory, was downloaded prior to use at the next game.
If an IMG recorded HAE did not exceed 10 g the data was not included. This threshold was based on the evidence that impacts below this level are considered negligible given impacts below this threshold have been previously considered as non-contact events (21) (e.g., landing, running). This approach is consistent with data acquisition limits used in previous studies (6) and acknowledges an acceleration threshold set too low will lead to a large number of false positive results and one set too high will likely miss potential true impacts (22).
Video analysis
The full game video analysis for all games during the season was conducted at the conclusion of the season by the lead author using Catapult Vision (Melbourne, Victoria, Australia). The video review process consisted of adjusting angles, pausing, replaying, and using slow motion as required. A world-clock was time stamped at the beginning of each video, so that video events could synchronised to time-stamped sensor events. Game footage, sampling at 25 frames per second with a resolution of 1,280 × 720 pixels, was initially analysed using three camera angles (wide, tight, broadcast) independently of the IMG data, before these two datasets were compared with one another. Video of each game was reviewed and played back at an appropriate speed to verify whether the player was involved in a contact event (Figure 1). The verification process involved identifying a contact event, verifying whether it resulted in direct head impact and whether a head acceleration event (HAE) was captured concurrently by the IMG (Figure 2). Such instances were classified as verified head impacts (VHIs). A contact event was defined as any contact from another player or the ground. Further, the contact event was categorised based on whether it was a direct head impact or not. A direct head impact was defined as any contact above the shoulder with any connection with the head/neck made to the player wearing an IMG by another player or the playing surface.
Figure 1. Flow diagram of video verified contact events, direct head impacts and IMG captured head acceleration events.
Figure 2. Examples of a contact event, direct head impact and unclear event. (A) Contact Event (B) Direct Head Impact – yellow arrow (C) Unclear (the player in the rear –black arrow obscured from view by the player in the foreground – grey arrow).
During a secondary analysis, VHIs were also characterised using standardised descriptors and definitions where possible (18). These included game specific situations including a scrum, maul, tackle, carrying the ball, ruck and whether the impact was to the primary or secondary carrier (supporting carrier) or to a primary or secondary tackler (supporting tackler). Pre-contact characteristics included the speed of the player involved in contact [fast (e.g., sprint), moderate (e.g., jog), slow (e.g., stationary or walk)] or body position (upright, medium e.g., bent at the hip, or low). VHIs were also characterised by the direction of contact with the head (i.e., front, angle, side, back, above), whether the impact was anticipated or not and when a tackle was made whether the actions of the tackler were active (attempting to drive the carrier backwards), passive (not attempting to drive the player backwards) or a wrap tackle (arms wrap the ball carrier and no ground gained nor conceded.
The lead author, trained in video analysis, completed the verification process. To enable a 2 × 2 contingency table, if the video of a direct head impact matched a head acceleration event captured on the IMG a “true positive” was recorded. Direct head impacts observed on video that did not match a captured head acceleration event on the IMG were identified as false negatives. A contact event with no direct head impact observed on video and no HAE captured was classified as a true negative. Following the initial round of screening for TPs, TNs, and FNs, captured HAEs that did not match with a contact event or direct head impact were identified as false positives.
In certain game circumstances the video of the player in question was obstructed by other players or officials. To fully explore the data, we evaluated a “best case” scenario (assuming the “Unclear” are direct head impacts), a “worst case” scenario (assuming the “Unclear” are not direct head impacts), and the “visually verified scenario” (only those that could be visually verified were considered). This approach was undertaken to ensure full consideration was given to all events, including those that were “unclear.” All the unclear events were included in separate accuracy analysis, first, entering them all as true positive events, and second, entering them all as false positive events. As a definitive decision against the video verification standard could not be made, unclear events were not included in analysis of game characteristics.
To assess inter-rater reliability two reviewers independently completed a verification process on one randomly selected game. Inter-rater reliability was determined using Cohen's K coefficient based on agreement under a two-way mixed effects model comparing the recorded time points separately. The agreement between the two raters when identifying contact events was 84.6% and for direct head impacts was 76.1%. They were blinded to the IMG capture. To assess the intra-rater reliability, the lead author reviewed the same game two weeks apart. The intra-rater reliability was also assessed to ensure consistency across time – for direct head impacts the IRR was 86%.
The initial analysis was then carried out on all contact events observed on video regardless of whether a head impact occurred. The sensitivity (TP/(TP + FN) and positive predictive value (PPV = TP/(TP + FP) of the IMG were calculated.
Data analysis
All data collected were entered into an Excel spreadsheet and analysed with SPSS v22.0.0 (SPSS Inc). The impact variables related to the characteristics of the impact were assessed for normality to inform the appropriate analyses using a Shapiro-Wilk test. Linear and rotational acceleration were non-normally distributed (W = .48 and W = .46 p < 0.001 respectively). Based on this outcome and a visual inspection of the resultant histograms IQR, median and non-parametric tests were used to describe the data. For each separate impact, peak linear acceleration (PLA) (g's) and peak rotational acceleration (PRA) (rad/s2) were defined as the maximum values of the resultant time-series data for linear and rotational accelerations respectively (20). PLA and PRA were considered by game scenario as well as by “athlete exposure” defined as a participation by a player using an IMG in a game within the study period (23).
Results
During the video verification process, 2018 contact events were observed in 24 games (61 athlete exposures) across the season, equating to approximately 3,660 playing minutes. There were 257 contact events that were deemed to be unclear and mainly the result of a maul (84.1%) or the player of interest being obscured during a ruck (5.5%). Of those contact events, 95 (37%) had IMG data recorded with a median of 21.4 g and 1,465 rad/s2. The impact magnitudes (PLA and PRA) of unclear results were not statistically different (p = 1 and p = 0.051 respectively)) from verified head-impacts. When using the “visually verified” standard, as shown in the 2 × 2 contingency analysis (Table 1), an overall sensitivity of 93.6% and a PPV of 92.4% was demonstrated. When all FPs were excluded due to ball out of play mouthguard removal or handling after a scoring situation or stoppage (n = 43/54), as is recommended (24), PPV was improved (98.4%). There were 655 verified head impacts (VHIs) where video observed direct head impacts were matched to a >10 g head acceleration event captured by the IMG. This corresponds to 10.7 VHIs and 29.0 contact events per athlete exposure. The impact magnitudes were skewed to the lower values with a median (IQR) linear acceleration of 21.5 g (15.4–33.9.) and a rotational acceleration of 1,702 rad/s2 (1,170.0–2,772.0 rad/s2).
Table 1. 2 × 2 contingency table with various video confirmation scenarios.
A total of 54 false positive impacts were observed, 85% of those occurred when the ball was not in play, for example a player removing their mouthguard. Those remaining had full visualisation though with no identified impact. The impact magnitudes for the FP impacts were 62.6 g (24.6–106.7) for PLA and 6,168 rad/s2 (2,174–10,690 rad/s2) for PRA which were statistically different (p < 0.001) from verified head impacts. There were 1,061 contact events where a direct head impact was not identified nor was a head acceleration event captured from the IMG. These occurred mostly through players being involved in scrums (29.0%), rucks (26.0%, or as the primary tackler (17.7%). There were 45 potential false negatives that occurred wherein a direct head impact was identified though no head acceleration event was captured.
Of the 655 VHIs, 328 (median 21.2 g, 1,692 rad/s2) were recorded in the 1st half of games and 327 (21.8 g, 1,707 rad/s2) in the second half (Table 2). There was no statistical difference between linear and rotational accelerations sustained in each half (p > 0.05). Backs (1,880 rad/s2) sustained higher PRAs than forwards (1,631 rad/s2), which was statistically significant (p = 0.03) though the PLA between backs (20.3 g) and forwards (21.7 g) was not statistically significant (p > 0.05). Unexpected impacts (23.1 g, 1,745 rad/s2) had higher PLA than expected impacts (20.9 g, 1,638 rad/s2) (p = .018). The majority (41.8%) of unexpected contacts were sustained during ruck involvements.
Table 2. Game impacts linear and rotational accelerations stratified by half, player position, and expected or non-expected impact.
The majority of VHIs occurred in and around the ruck contest (31.2%) followed by impacts to the primary tackler (28.4%) and primary ball carrier (13.8%), the remaining (26.6%) were either to the secondary tackler, secondary ball carrier or during scrum and mauls. The highest peak linear accelerations were recorded during maul impacts whereas the highest rotational accelerations were during a secondary tackle (Table 3).
Table 3. Peak linear acceleration and peak rotational acceleration for VHIs by game scenario, direction of contact, and the intent of the tackler.
Tackler speed, and speed differential into contact are known risk factors for head impact (25–27). The highest PLA and PRA occurred when the players were running fast (sprinting) immediately prior to making a secondary tackle [27 g, (95% CI: 19.9–46.1 g) (2,469 rad/s2 1,494–3,100)]. A similar speed related pattern was also seen with moderate arrival speed to a ruck generating higher PLAs and PRAs (27.3 g, 1,878 rad/s2). There was a difference in PLAs and PRAs if the intent of the primary tackler was that of a driving or “active tackle” [21 g, (95% CI: 15.7–34.7 g and 1,672 rad/s2), (95% CI: 1,280–2,857 rad/s2)] whereas a “passive” tackle was consistently lower (17.6 g, 95% CI: 13.2–26.9 g and 1,540 rad/s2, 1,024–2,642.3 rad/s2).
Discussion
This study is one of the first to examine the relationship between video verified direct head impacts and head acceleration events (HAE) captured concurrently by an IMG in Professional Rugby Union Players. A reliable kinematic measurement tool with appropriately set algorithms, should minimise noise (non-impacts) and optimize the quantification of incidence, frequency and severity of “true” head impacts. Combined with a strong internal validation (LCCC – .997) the IMG demonstrated a sensitivity of 93.6% and a PPV of 98.3% during “active minutes” which is indicative of an accurate wearable sensor and comparable to previously analysed instrumented head impact sensors (24).
Previous studies that have relied on head impact sensors in isolation to confirm head impacts have been shown to have high occurrences of false positives resulting in an overestimation of head impact exposure due to the sensitivity and inaccuracies of the sensor used (28). Conversely, video verification of contact events and head impacts, are qualitative in nature and open to intra-rater and inter-rater reliability issues in the interpretation of even the most well-designed consensus video analysis framework. This study independently compared the video verification of direct head impacts to that of the head acceleration events captured on the IMG. Consistent with previous research (29), this complimentary approach to combined datasets enabled this study to determine how well this new IMG performed during real-world conditions. In this study for a head acceleration event reaching a trigger threshold of 10 g, there is a 98.3% chance it is the result of a direct head impact. Similarly, while the sample size in this pilot study was small, few issues arose with the comfort of the IMG or in the players reported ability to communicate on the field.
This IMG did not have a proximity sensor to filter out recordings where the IMG was not properly placed on the upper dentition. Featuring proximity sensors, such as an infrared sensor to determine contact of the IMG with the teeth, has reportedly been beneficial in reducing the proportion of false-positives (30). Despite not having a proximity sensor, the reason for the low false positive rates in this study is likely 3-fold. The 10 g trigger threshold was relatively high compared to other IMGs, this may have resulted in less false-positives due to lower-magnitude contact events and erroneous noise not being captured. Further, the decision to assess only direct head impacts rather than indirect or inertial loads may have resulted in lower-magnitude events not being captured. It is also possible that the custom-fit nature of the IMG by a dentist experienced in IMG fabrication, as well as the ability to offer adjustments once fabricated, was crucial to both comfort and in ensuring firm coupling to the upper dentition (29) resulting in relatively low false-positives recorded in this study. Selection bias was also likely an issue with players that opted into the study likely to remain compliant and adherent to instructions around avoiding removal and adjustment of the IMG where possible. In studying larger sample sizes than those used in this pilot study, proximity sensors would likely be essential to filter out recordings where the IMG was not coupled to the upper dentition resulting in a high proportion of false positives.
The approach was to match IMG captures to video verified events such that head acceleration events have a high likelihood of being matched with direct head impacts. Conversely non-impacts would have a low likelihood of including an IMG capture. In this study, false negatives were considered a direct head impact that did not correspond to a head acceleration event. Although used in the 2 × 2 contingency table to calculate specificity limitations remain due to the challenges that exist in objectively estimating impact magnitude from video review and the set trigger threshold of the IMG. False positives were collected as a secondary analysis. Given the relatively low number, the authors were able to review and consider whether FPs were generated during “ball out of play” time as is usually the case due to players removing their IMGs at a break in play or to talk (24). Consistent with previous studies that have excluded ball out of play (24), the calculated PPV was higher when they were removed and would be a more accurate reflection on the FP rate. Those that remained after the “out of play” occurrences were removed were possibly due to sudden changes in direction, chewing or by moving the IMG device around in their mouth (12).
Several attempts to define true negatives using machine learning classification have been made (22, 31) as has the issue of classifying true or false negatives due to a lack of kinematic reference measurement on-field (24). The intent of the “true negative” in this study was to provide the reader with an understanding of video verified “contact events” that do not have “direct head impact.” The context being the value of describing the incidence of contact events (without direct head impacts) compared to verified head impacts. This description would potentially add to the literature around repeated sub-concussive impacts and the consideration of using the IMG for the broad description of contact exposure.
Players in the current study were exposed to an average of 10 verified impacts per match. A comparison of athlete exposure is relatively problematic due to the relatively new technology, the lack of on-field validation studies using IMGs and the lack of reporting using a standard definition of “athlete exposure”. A comparison to behind ear patches demonstrated an equivalent 10 impacts per player per match (7) across four games in junior Rugby Union, 14 impacts per player per match (32) were reported across 9 games in female Rugby League, while in American college football players experienced 14 head impacts per game. However, the issues related to decoupling and over-estimation of head impacts effect any worthwhile comparison to skin-patches (10, 33), while IMGs also have unique trigger thresholds and specific filtering algorithms specific to each IMG rendering comparison between studies impossible (34). It is also likely that each sport has unique kinematic head impact characteristics and scenarios likely to cause head acceleration events. Contact events commonly encountered in rugby union would be specific to the speed profile of players, game duration, player size and sex which would need to be considered in any cross-sport comparison of athlete exposure or different population groups participating in Rugby Union.
The median PLA of 21.5 g (15.6–34.1) was similar to those previously reported kinematic studies completed in Rugby Union (24). Kieffer et al. (24). reported PLA values with a mean of 15.2 g in men's and women's Club level rugby, while King et al. reported a mean PLA of 22.2 g (6), though a markedly higher PRA of 3,847 rad/s2 compared to 1,719 rad/s2 in the current study. This difference may be explained by either the potential of skin patches to over-predict linear and rotational acceleration (10) or the lack of video verification of impacts that is critical to the processing and interpretation of data (28).
The frequency and magnitude of verified head impacts for various game characteristics have also been reported. The finding of the highest incidence of exposure of verified head impacts occurring during the ruck, an event unique to Rugby Union, has not been previously reported. The unexpected nature of many of these VHIs would seem logical given the players usual body position in the ruck dictates the hips be bent and the head often down contesting or protecting the ball in such a way where anticipation of an impending impact is less likely. Such unexpected impacts at the ruck were shown to be significantly greater than expected events. Previous studies have cited high incidence of concussion during a ruck as well as impact readiness (27) as associated with injury risk. This finding would necessitate a review of the process in place to protect players in ruck situations, including consideration of regulations that limit approach speed to a ruck along with coaching interventions that enhance awareness around the ruck area may be worthy of further consideration. The high PLAs attributed to the maul were an unexpected finding given their relatively stationary nature. However, many of the head impacts were the result of the player rapidly accelerating into the maul using their head and shoulders to move opposition bodies.
The findings related to the tackle contest provide additional quantitative context to several earlier published findings (25–27). Although most concussions are associated with the tackler rather than the ball carrier (26), the current data suggests that higher PLAs are sustained by the ball carrier than the tackler. However, with higher PRAs for the tackler than the ball carrier, it is likely that the increased risk remains with the tackler due to the association between concussion and brain tissue deformation as a result of rotational accelerations and their increased shearing forces (35). This is potentially supported by the findings of risk associated with tackler speed and having a speed differential into contact (25–27). The highest PLA and PRA of all game characteristics in this study occurred when players were running fast (sprinting) immediately prior to making a secondary tackle. Instances of this on the field were seen when a secondary tackler accelerated into a stationary player that was already being held upright in a tackle as well as from kick return or set-piece plays. The addition of quantitative data to known risk-inducing game scenarios could be considered valuable in weighing up the benefit of rule changes to limit second-person in tackles or to inform training methods such as focused training on deceleration ability, agility elements to contact drills and evasive skills at speed to prevent potential injury.
This pilot validation study contributes important knowledge to head impact recognition strategies expanding into various levels of contact sport. The IMG evaluated in this study represents an improvement over existing instrumentation including helmet based or skin mounted devices. The complimentary approach of independent video verification and comparison to the data captured from an IMG enabled this study to determine how well this new IMG performed during real-world conditions. The accuracy of this IMG means that it could be considered to support the video verification process which lacks the ability to capture the magnitude of impacts. It could also be used as an alarm system where video verification is not readily available or for those discrete impacts obscured from view, which due to the congested nature and game characteristics of rugby union, has been shown to be not insignificant. In the future, advancing technology may enable the IMG to be used in real-time to inform pitch side-clinicians of a potentially injurious head impact by providing reliable data in a timely manner that complements the standard medical assessment and video verification process. This effective recognition, using the best available information and technology is critical to support the removal of a player from training or competition for further assessment.
Limitations
In this study, active minutes or ball in play time was assessed manually to provide two accounts of FP recordings. To limit time waste and the potential for error through data handling, proximity sensing should be added to the IMG to determine whether the IMG is mounted on the teeth. This would enable rejection of data when a player removes the IMG during a break in play or to make a particular tactical call.
Secondly, the trigger threshold used in IMG studies varies though has regularly been based on linear acceleration thresholds alone. In setting the appropriate trigger threshold, researchers should consider for example, recent reviews noting everyday activities ranging from .33 to 13.8 g while rotational accelerations ranged from 13 to 1,375 rad/s2 (36). This underlines a fundamental concept in the use of IMGs with greater consideration required as to whether an IMG can be considered as both a tool for detecting potentially injurious direct head impacts, in which case setting the threshold too low may increase the number of false positives, as well as a tool capable of detecting inertial loads during non-contact events, in which case setting the threshold too high may increase the number of false negatives. This development may be aided by the inclusion of rotational acceleration triggers which have been shown to be effective and should be considered in future research (35) particularly given recent research questioning the triggering biases associated with linear acceleration, in particular the suggestion that specific directions of impact were being overlooked due to the placement of sensors (37).
Thirdly, the validation of the mouthguard is limited to the research environment used and as such its generalisability limited to a reasonably specific set of parameters, namely male, professional, rugby union players. Its performance has not been assessed during practise training sessions, with female players or at various levels of age-based competition which are likely to result in different exposure profiles and magnitudes of impact. Further, the small sample size made it impossible to compare by player position groupings, an essential factor in Rugby Union given the highly unique positional roles.
Fourthly, this is the first study to consider the effect of including and excluding “unclear” data. Given the proposed accuracy of recording head impacts and contact events, it is likely that recordings from moments where vision is obstructed, such as during a ruck, scrum or maul, where players are obscured for lengthy periods included potential head impacts that were obscured from view. This is supported by the lack of a statistical significance difference between the unclear and verified head impact groups. Despite three camera angles throughout the study, issues remained with obtaining the appropriate viewing angle and on occasion resolution. For transparency, reporting of how unclear events are managed in research is critical given the known limitations outlined in this study and due to the congested nature of Rugby Union and multiple sources of impact beyond just the tackle contest. The data points considered “unclear” provide insight to the potential utility of an accurate IMG as well as supporting the previously stated need for multiple camera angles to assist with accuracy in recognising impacts and potential video signs of concussion (38).
Finally, to ensure the consistent application of the accepted descriptors and definitions used in Rugby Union, only one reviewer viewed the approximate 32 h of game video. This labour-intensive process may benefit in the future by the inclusion and use of new technologies. Research in professional sports continues to explore the benefits of machine learning algorithms and statistical modelling, with the aim of improving decision making and minimizing error due to over-reliance on human expertise (39) - in this case the coding of contact events, direct head impacts and specific game characteristics.
Conclusion
This study highlights the importance of a combined biomechanical (IMG) and qualitative (video analysis) approach for furthering the understanding of game specific head biomechanics in Rugby Union. Further, video verification in isolation lacks the ability to comment on the magnitude of impact and has the potential to overlook discrete, multiple or obscured impacts. Conversely, biomechanical instrumented mouthguard approaches alone without video verification may be prone to false-positive readings and are limited to their predetermined filtering algorithms and trigger thresholds (40). Given the demonstrated accuracy of the IMG and limitations of video verification, further exploration of the combination of these assessment tools is warranted. Previous findings relating to injury risk and speed of the tackler or speed differentials are supported by this study, though new areas for further investigation based on these findings include the magnitude of forces sustained during unexpected events particularly while contesting the ruck and the incidence of VHIs at the ruck compared to other game scenarios An extensive exploration with larger sampling in other populations of interest including women's Rugby Union, Rugby sevens, and in adolescents is required. Additionally, examining information regarding medically diagnosed concussion alongside biomechanical characteristics of impacts is a logical next step.
Practical implications
• Understanding the accuracy of any technology is critical to an assessment of clinical utility. This instrumented mouthguard is a sensitive tool for identifying direct head impacts and could be used at the elite level of Rugby Union in conjunction with a video verification process to improve the clinician's ability to recognise head impacts;
• When evaluating validation studies, consideration should be given to how the inclusion or exclusion of head impacts that cannot be discerned through the reference standard (video review) are considered.
• The obscured nature of some contact events and the inability of video to consider the magnitude of an impact should ensure that further feasibility studies are completed into the use of IMGs.
• At levels of community sport where video verification is not readily available, the use of IMGs should be explored though caution exercises when interpreting the IMG data given the potential issues with false negative results.
Data availability statement
The raw data presented in this article is not readily available because of privacy considerations. The aggregated data presented in the study are included in the article and further inquiries should be directed to Byron Field E: u113947@uni.canberra.edu.au.
Ethics statement
The studies involving humans were approved by University of Canberra Human Research Ethics Committee. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study. Written informed consent was obtained from the individual(s) for the publication of any potentially identifiable images or data included in this article.
Author contributions
BF was the primary author and was responsible for the collection and analysis of the data and subsequent results. RG collaborated with the organisation of the database and statistical analysis. AG provided content expertise and overall guidance on the concept and design of the study. GW and AM supervised the project revision and provided editorial guidance throughout the submission process. All authors contributed to the article and approved the submitted version.
Conflict of interest
AG serves as a scientific advisor for hitIQ, Ltd. He has a clinical practice in neuropsychology involving individuals who have sustained sport-related concussion (including current and former athletes). He has been a contracted concussion consultant to Rugby Australia since July 2016. He has received travel funding or been reimbursed by professional sporting bodies, and commercial organisations for discussing or presenting sport-related concussion research at meetings, scientific conferences, workshops, and symposiums. Previous grant funding includes the NSW Sporting Injuries Committee, the Brain Foundation (Australia), an Australian-American Fulbright Commission Postdoctoral Award, a Hunter New England Local Health District, Research, Innovation and Partnerships Health Research & Translation Centre and Clinical Research Fellowship Scheme, and the Hunter Medical Research Institute (HMRI), supported by Jennie Thomas, and the HMRI, supported by Anne Greaves. He has current philanthropic support from the Nick Tooth Foundation.
The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
References
1. Stuart S, Hickey A, Morris R, O'Donovan K, Godfrey A. Concussion in contact sport: a challenging area to tackle. J Sport Health Sci. (2017) 6(3):299–301. doi: 10.1016/j.jshs.2017.03.009
2. Koh JO, Cassidy JD, Watkinson EJ. Incidence of concussion in contact sports: a systematic review of the evidence. Brain Inj. (2003) 17(10):901–17. doi: 10.1080/0269905031000088869
3. Waltzman D, Sarmiento K, Devine O, Zhang X, DePadilla L, Kresnow MJ, et al. Head impact exposures among youth tackle and flag American football athletes. Sports Health. (2021) 13(5):454–62. doi: 10.1177/1941738121992324
4. Williams RM, Dowling M, O'Connor KL. Head impact measurement devices. Sports Health. (2016) 8(3):270–3. doi: 10.1177/1941738116641912
5. O'Connor KL, Rowson S, Duma SM, Broglio SP. Head-impact–measurement devices: a systematic review. J Athl Train. (2017) 52(3):206–27. doi: 10.4085/1062-6050.52.2.05
6. King D, Hume PA, Brughelli M, Gissane C. Instrumented mouthguard acceleration analyses for head impacts in amateur rugby union players over a season of matches. Am J Sports Med. (2015) 43(3):614–24. doi: 10.1177/0363546514560876
7. King DA, Hume PA, Gissane C, Clark TN. Similar head impact acceleration measured using instrumented ear patches in a junior rugby union team during matches in comparison with other sports. J Neurosurg Pediatr. (2016) 18(1):65–72. doi: 10.3171/2015.12.Peds15605
8. Hecimovich M, King D, Dempsey A, Murphy M. Head impact exposure in junior and adult Australian football players. J Sports Med (Hindawi Publ Corp). (2018) 2018:8376030. doi: 10.1155/2018/8376030
9. Cecchi NJ, Monroe DC, Fote GM, Small SL, Hicks JW. Head impacts sustained by male collegiate water polo athletes. PLoS ONE. (2019) 14(5):e0216369. doi: 10.1371/journal.pone.0216369
10. Wu LC, Nangia V, Bui K, Hammoor B, Kurt M, Hernandez F, et al. In vivo evaluation of wearable head impact sensors. Ann Biomed Eng. (2016) 44(4):1234–45. doi: 10.1007/s10439-015-1423-3
11. Tooby J, Weaving D, Al-Dawoud M, Tierney G. Quantification of head acceleration events in rugby league: an instrumented mouthguard and video analysis pilot study. Sensors (Basel), 2022. 22(2):584. doi: 10.3390/s22020584
12. Rich AM, Filben TM, Miller LE, Tomblin BT, Van Gorkom AR, Hurst MA, et al. Development, validation and pilot field deployment of a custom mouthpiece for head impact measurement. Ann Biomed Eng. (2019) 47(10):2109–21. doi: 10.1007/s10439-019-02313-1
13. Adesida Y, Papi E, McGregor AH. Exploring the role of wearable technology in sport kinematics and kinetics: a systematic review. Sensors (Basel, Switzerland). (2019) 19(7):1597. doi: 10.3390/s19071597
14. Davis G, Makdissi M. Use of video to facilitate sideline concussion diagnosis and management decision-making. J Sci Med Sport. (2016) 19(11):898–902. doi: 10.1016/j.jsams.2016.02.005
15. Chambers RM, Gabbett TJ, Gupta R, Casey J, Bown R, Stridgeon P, et al. Automatic detection of one-on-one tackles and ruck events using microtechnology in rugby union. J Sci Med Sport. (2019) 22(7):827–32. doi: 10.1016/j.jsams.2019.01.001
16. Gardner A, Kohler R, McDonald W, Fuller GW, Tucker R, Makdissi M. The use of sideline video review to facilitate management decisions following head trauma in super rugby. Sports Med Open. (2018) 4(1):1–8. doi: 10.1186/s40798-018-0133-4
17. Iverson GL, Gardner AJ, Terry DP, Ponsford JL, Sills AK, Broshek DK, et al. Predictors of clinical recovery from concussion: a systematic review. Br J Sports Med. (2017) 51(12):941–8. doi: 10.1136/bjsports-2017-097729
18. Hendricks S, Till K, den Hollander S, Savage TN, Roberts SP, Tierney G, et al. Consensus on a video analysis framework of descriptors and definitions by the rugby union video analysis consensus group. Br J Sports Med. (2020) 54(10):566–72. doi: 10.1136/bjsports-2019-101293
19. Goodin P, Gardner AJ, Dokani N, Nizette B, Ahmadizadeh S, Edwards S, et al. Development of a machine-learning-based classifier for the identification of head and body impacts in elite level Australian rules football players. Front Sports Act Living. (2021) 3:725245. doi: 10.3389/fspor.2021.725245
20. Stitt D, Draper N, Alexander K, Kabaliuk N. Laboratory validation of instrumented mouthguard for use in sport. Sensors. (2021) 21(18):6028. doi: 10.3390/s21186028
21. King D, Hume P, Gissane C, Brughelli M, Clark T. The influence of head impact threshold for reporting data in contact and collision sports: systematic review and original data analysis. Sports Med (Auckland). (2015) 46(2):151–69. doi: 10.1007/s40279-015-0423-7
22. Wu LC, Kuo C, Loza J, Kurt M, Laksari K, Yanez LZ, et al. Detection of American football head impacts using biomechanical features and support vector machine classification. Sci Rep. (2017) 8(1):855. doi: 10.1038/s41598-017-17864-3
23. Swenson AG, Pritchard NS, Miller LE, Urban JE, Stitzel JD. Characterization of head impact exposure in boys’ youth ice hockey. Res Sports Med. (2021) 31(4):1–11. doi: 10.1080/15438627.2021.1989433
24. Kieffer EE, Begonia MT, Tyson AM, Rowson S. A two-phased approach to quantifying head impact sensor accuracy: in-laboratory and on-field assessments. Ann Biomed Eng. (2020) 48(11):2613–25. doi: 10.1007/s10439-020-02647-1
25. Tucker R, Hester B, Cross M, Kemp S, Raftery M. Tackle technique and body position of the tackler and ball carrier significantly influences head injury risk in rugby union. Br J Sports Med. (2017) 51(11):A70. doi: 10.1136/bjsports-2016-097270.182
26. Cross MJ, Tucker R, Raftery M, Hester B, Williams S, Stokes KA, et al. Tackling concussion in professional rugby union: a case–control study of tackle-based risk factors and recommendations for primary prevention. Br J Sports Med. (2019) 53(16):1021–5. doi: 10.1136/bjsports-2017-097912
27. Hendricks S, O'Connor S, Lambert M, Brown JC, Burger N, Mc Fie S, et al. Video analysis of concussion injury mechanism in under-18 rugby. BMJ Open Sport Exerc Med. (2016) 2(1):e000053. doi: 10.1136/bmjsem-2015-000053
28. Patton DA, Huber CM, McDonald CC, Margulies SS, Master CL, Arbogast KB. Video confirmation of head impact sensor data from high school soccer players. Am J Sports Med. (2020) 48(5):1246–53. doi: 10.1177/0363546520906406
29. Bartsch AJ, Hedin D, Alberts J, Benzel EC, Cruickshank J, Gray RS, et al. High energy Side and rear American football head impacts cause obvious performance decrement on video. Ann Biomed Eng. (2020) 48(11):2667–77. doi: 10.1007/s10439-020-02640-8
30. Patton DA, Huber CM, Jain D, Myers RK, McDonald CC, Margulies SS, et al. Head impact sensor studies in sports: a systematic review of exposure confirmation methods. Ann Biomed Eng. (2020) 48(11):2497–507. doi: 10.1007/s10439-020-02642-6
31. DiCesare CA, Green B, Yuan W, Diekfuss JA, Barber Foss KD, Dudley J, et al. Machine learning classification of verified head impact exposure strengthens associations with brain changes. Ann Biomed Eng. (2020) 48(12):2772–82. doi: 10.1007/s10439-020-02662-2
32. King D, Hume P, Gissane C, Clark T. Head impacts in a junior rugby league team measured with a wireless head impact sensor: an exploratory analysis. J Neurosurg Pediatr. (2017) 19(1):13–23. doi: 10.3171/2016.7.Peds1684
33. Flao EL, Lenetsky S, Siegmund G, Borotkanics R. Capturing head impacts in boxing: A video-based comparison of three wearable sensors. Research Square (2023). doi: 10.21203/rs.3.rs-2940480/v1
34. Jones B, Tooby J, Weaving D, Till K, Owen C, Begonia M, et al. Ready for impact? A validity and feasibility study of instrumented mouthguards (iMGs). Br J Sports Med. (2022) 54(20):1171–9. doi: 10.1136/bjsports-2022-105523
35. Lota KS, Malliaropoulos N, Blach W, Kamitani T, Ikumi A, Korakakis V, et al. Rotational head acceleration and traumatic brain injury in combat sports: a systematic review. Br Med Bull. (2022) 141(1):33–46. doi: 10.1093/bmb/ldac002
36. Miller LE, Urban JE, Whelan VM, Baxter WW, Tatter SB, Stitzel JD. An envelope of linear and rotational head motion during everyday activities. Biomech Model Mechanobiol. (2020) 19(3):1003–14. doi: 10.1007/s10237-019-01267-6
37. Wang T, Kenny R, Wu LC. Head impact sensor triggering bias introduced by linear acceleration thresholding. Ann Biomed Eng. (2021) 49(12):3189–99. doi: 10.1007/s10439-021-02868-y
38. Carey L, Terry DP, McIntosh AS, Stanwell P, Iverson GL, Gardner AJ. Video analysis and verification of direct head impacts recorded by wearable sensors in junior rugby league players. Sports Med Open. (2021) 7(1):66. doi: 10.1186/s40798-021-00353-3
39. Chmait N, Westerbeek H. Artificial intelligence and machine learning in sport research: an Introduction for non-data scientists. Front Sports Act Living. (2021) 3:682287. doi: 10.3389/fspor.2021.682287
Keywords: head impact, rugby union, mouthguard, validation, concussion
Citation: Field B, Waddington G, McKune A, Goecke R and Gardner AJ (2023) Validation of an instrumented mouthguard in rugby union—a pilot study comparing impact sensor technology to video analysis. Front. Sports Act. Living 5:1230202. doi: 10.3389/fspor.2023.1230202
Received: 28 May 2023; Accepted: 25 October 2023;
Published: 20 November 2023.
Edited by:
Calvin Kuo, University of British Columbia, CanadaReviewed by:
Adam Clansey, University of British Columbia, CanadaEnora Le Flao, Stanford University, United States
© 2023 Field, Waddington, McKune, Goecke and Gardner. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Byron Field u113947@uni.canberra.edu.au