Cortney Armitano-Lago
Dominic Willoughby
Adam W. Kiefer- 1Department of Exercise and Sport Science, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
- 2Department of Exercise Science, Elon University, Elon, NC, United States
Markerless motion capture systems are promising for the assessment of movement in more real world research and clinical settings. While the technology has come a long way in the last 20 years, it is important for researchers and clinicians to understand the capacities and considerations for implementing these types of systems. The current review provides a SWOT (Strengths, Weaknesses, Opportunities, and Threats) analysis related to the successful adoption of markerless motion capture technology for the assessment of lower-limb musculoskeletal kinematics in sport medicine and performance settings. 31 articles met the a priori inclusion criteria of this analysis. Findings from the analysis indicate that the improving accuracy of these systems via the refinement of machine learning algorithms, combined with their cost efficacy and the enhanced ecological validity outweighs the current weaknesses and threats. Further, the analysis makes clear that there is a need for multidisciplinary collaboration between sport scientists and computer vision scientists to develop accurate clinical and research applications that are specific to sport. While work remains to be done for broad application, markerless motion capture technology is currently on a positive trajectory and the data from this analysis provide an efficient roadmap toward widespread adoption.
Introduction
Markerless motion capture systems have emerged as a promising tool to assess movement in both research and clinical settings. As we continue to develop and evolve the current standard of 3D motion capture, the emergence of markerless systems has provided the means to accurately measure patterns of motion in a manner that is neither invasive nor cumbersome, and that also limit the risk of measurement-induced artifacts common with marker-based systems (Mündermann et al., 2006a). Continued advancements in motion capture capabilities have supported the development of more accessible and cost-effective motion capture systems that can uniquely target a wide range of questions surrounding human movement. In the area of sports medicine and movement science, markerless motion capture offers a technological solution for evaluating unrestricted sport-specific movement patterns to better inform an athlete's risk of injury, assess rehabilitative progression, and evaluate their readiness to return to play, as well as refine skilled performance through efficient motor behavior.
One of the most exciting aspects of markerless motion capture is that can facilitate a new understanding of human movement by removing the environmental constraints of marker-based data collections (Mündermann et al., 2006a) and enable the cultivation of truly large databases of human movement (Mathis et al., 2020). In the realm of sports medicine, for example, changes in movement patterns due to injury can have a profound impact on the progression of musculoskeletal pathology, as well as the treatment of such pathologies—e.g., changes in gait mechanics following ACL injury and subsequent reconstruction have been shown to influence the progression and severity of knee osteoarthritis (Andriacchi et al., 2004; Pietrosimone et al., 2016). The ability to provide a robust kinematic assessment to address current and emerging clinical questions about factors that influence normal patterns of movement, and particularly sport-related movement, would further our understanding of human movement and provide researchers and clinicians the information necessary to enhance injury prevention, rehabilitation and general training programs.
As the accessibility of markerless motion capture software and systems seems to be markedly increasing, it is important for both researchers and clinicians to understand the up-to-date capacities of these technologies as well as areas that may require additional consideration for implementation—e.g., the computational and methodological approaches that have been taken to successfully achieve markerless motion capture. A SWOT (Strengths, Weaknesses, Opportunities, and Threats) analysis is a tool developed for strategic analysis that serves to reveal the internal strengths and weaknesses of a given entity and evaluate the external factors (opportunities and threats) the entity will face (Scholes et al., 2002). Appropriately, SWOT is an acronym for strengths, weaknesses, opportunities, and threats and is an analysis framework commonly employed to inform decision making and development. While a SWOT analysis is most commonly associated with applications in business, this framework has been applied to healthcare (Helms et al., 2008), athlete training (Düking et al., 2018), and rehabilitative technologies (Rizzo and Kim, 2005). A structured examination using a SWOT analysis of markerless motion capture systems and their application in the field of sports medicine would provide guidance and direction to the implementation of this technology.
In this SWOT analysis, the various markerless motion capture approaches used to assess lower extremity biomechanics were collectively examined to provide a broad discussion on the use and application of portable, low-cost markerless motion capture in the field of sports medicine. Ultimately, the goal of this analysis is to provide clarity on what is currently available and to identify areas for development to ensure a successful future for this technology. SWOT analyses are often criticized for their subjectivity (Pickton and Wright, 1998), and while this may be a limitation of this type of analysis, each SWOT factor has been extensively investigated through a thorough review of the current literature. To ensure a systematic review of the literature was conducted to build the foundation of the SWOT analysis, the following electronic databases were searched for relevant studies from their inception through February 2021, and a second time through December 2021: PubMed, ProQuest Health & Medical Collection, CINAHL, and Google Scholar. These electronic databases were searched using combinations of key words related to the scope of the review (search terms: lower extremity, kinematics, ankle, knee, hip, pelvis, markerless motion capture) and Boolean operators OR and AND were used to combine search terms. Of note, based on the two searches conducted, there has been a 24% increase in the research conducted within this specific scope within the last year (134 articles found during the first search in February 2021 and an additional 33 articles found in the December 2021 search), illustrating the exponential progress in this area of research. The inclusion criteria for this review were studies (a) with full- text articles available, (b) published in peer-reviewed journals, (c) in English, (d) utilizing a quantitative study design, excluding systematic reviews, (e) with human participants, (f) that evaluated the validity and/or reliability of a markerless motion capture system against a marker-based system and/or clinical assessment tool, (g) assessed lower extremity (pelvis/hip, knee, and/or ankle) kinematics, and (h) the total cost of the markerless system, cameras and software must be under $5,000.00. Thirty-one articles met all the inclusion criteria and were reviewed by the authors for the content to build and conduct the SWOT assessment (see Table 1 for study characteristics from literature review and Figure 1 for summary of SWOT analysis).
Table 1. Characteristics of validation studies.
Figure 1. Summary of SWOT analysis for markerless motion capture.
Strengths
Agreement Between Markerless and Marker-Based Systems
In assessing the validity and limits of markerless motion capture, many studies have concurrently compared markerless systems to marker-based systems (Clark et al., 2013; Sandau et al., 2014; Mentiplay et al., 2015; Perrott et al., 2017; Harsted et al., 2019; Tanaka et al., 2019; Tipton et al., 2019; Wochatz et al., 2019; Drazan et al., 2021). In such studies, markerless motion capture has shown great promise. Specifically, focusing on the ability to detect lower extremity movement, multiple studies have indicated that markerless motion capture can efficiently capture spatiotemporal joint kinematic variables (Clark et al., 2013; Sandau et al., 2014; Mentiplay et al., 2015; Rocha et al., 2018) with moderate-to-high agreement during tasks such as a single leg squat (Perrott et al., 2017; Kotsifaki et al., 2018; Tipton et al., 2019), vertical jump (Drazan et al., 2021), countermovement jump (Kotsifaki et al., 2018), stair climbing (Ogawa et al., 2017), walking (Ceseracciu et al., 2014; Sandau et al., 2014; Kanko et al., 2021; Pagnon et al., 2021; Stenum et al., 2021; Takeda et al., 2021; Vafadar et al., 2021), running (Corazza et al., 2006; Macpherson et al., 2016; Pagnon et al., 2021), gymnastics tasks (Corazza et al., 2006, 2010; Mündermann et al., 2007), and clinical evaluations (Eltoukhy et al., 2017; Mauntel et al., 2021). To date, the highest accuracy with markerless motion capture has been achieved when fitting a prior articulated model to a 3D surface visual hull reconstruction using matching algorithms (Corazza et al., 2006, 2007, 2008, 2010; Mündermann et al., 2006b, 2007). More recently, however, the application of deep learning algorithms, keypoint detection approaches for biomechanical assessment are beginning to show similar or greater accuracies and illustrate significant promise for the future of markerless motion capture in the sports medicine domain (Drazan et al., 2021; Kanko et al., 2021; Needham et al., 2021; Pagnon et al., 2021; Stenum et al., 2021; Vafadar et al., 2021). It is important to note that the majority of validation studies utilizing markerless motion capture to assess joint kinematics evaluate relatively slow movements such as walking, or single plane motions such as the sagittal plane during jumping. To continue to verify the utility of these approaches for sport applications, a thorough evaluation of quicker, sport-specific movements—such as rapid change in directions, and non-linear movements—is necessary to confirm the applicability to a broad range of sports. In addition, it is important to note that with this agreement between systems, there has been some evidence to suggest that during trials using marker-based and markerless motion capture systems concurrently, the reflective markers used for tracking marker-based assessments may distort the results gleaned from cameras used for markerless motion capture (Naeemabadi et al., 2018), therefore systems compared to marker-based approaches may be better than we currently realize.
Elimination of Marker Dependency and Environmental Restrictions
A prominent source of measurement error when using marker-based motion capture systems is skin movement artifact (Leardini et al., 2005). Soft tissue movement introduces errors of similar frequency to the actual bone movements and therefore it is difficult to parse the movement artifacts through filtering and smoothing of the data (Leardini et al., 2005). A systematic review by Peters and colleagues revealed that this artifact can be as great as 30 mm on body segments, such as the thigh, when compared to more precise methods, such as intra-cortical bone pins or X-ray radiation (Peters et al., 2010). Artificial stimuli information is also introduced with marker-based systems. Methods such as wrapping limb segments to secure clusters on the thigh or shank, the insertion of bone pins, or the attachment of numerous reflective markers introduce an artificial stimulus to the neurosensory system that can yield changes that deviate from a natural movement pattern that may mask underlying movement deficits (Mündermann et al., 2006b). The application of machine learning pose estimation algorithms offers the promise of reducing experimental error(s) either due to soft tissue movement artifact, as described above, or variability of marker placements (Szczerbik and Kalinowska, 2011), which may lead to more accurate data. Through the implementation of sophisticated pose-estimation algorithms, rather than the use of markers or clusters on the body, markerless motion capture offers a means to eliminate soft tissue artifact without requiring the use of potentially more invasive marker-based techniques (e.g., the use of intra-cortical bone pins). This affords more seamless data capture and may lead to faster and more ecologically valid data collections. In short, markerless motion capture affords the measurement of natural movement patterns that could lead to a more robust analysis of human kinematics.
Use of Machine Learning Algorithms
One of the greatest challenges of markerless motion capture is the complexity of acquiring accurate three-dimensional kinematics without the spatial correspondence that markers deliver to marker-based systems. Fortunately, machine learning-based discriminative algorithms provide an avenue to estimate human motion. Previously captured data are used to inform a specific model (the representation of the human body) or to train the machine learning algorithm. The captured data are then input into the machine learning algorithm that is used to extract specific features from the captured image(s) to deduce explicit motions. A wide variety of machine learning algorithms have been proposed to estimate human motion (Gavrila and Davis, 1996; Bregler and Malik, 1998; Deutscher et al., 2000; Grauman et al., 2003; Baker and Kanade, 2005; Corazza et al., 2006; Moeslund et al., 2006; Poppe, 2007; Ionescu et al., 2013). Unfortunately, the research into the development and improvement of these algorithms is outside the scope for most biomechanics research and, thus, the applicability of several of these algorithms for biomechanical use is still widely unknown (Mündermann et al., 2006a; Colyer et al., 2018). That being said, more recently researchers have begun to evaluate the more popular human pose estimation algorithms (e.g., OpenPose, DeepLabCut, AlphaPose) for their accuracy when applying them to the human movement sciences domain (Drazan et al., 2021; Needham et al., 2021; Pagnon et al., 2021; Stenum et al., 2021; Takeda et al., 2021) and perhaps with the growing popularity of applying markerless motion capture to the sports medicine domain, there will be a push for more biomechanically accurate algorithms. The benefit of using machine learning-based algorithms for estimating human movement is that these algorithms can be continually refined. Specifically, data can be used to train and refine the existing algorithms to drive more accurate estimations of joint kinematics. In addition, the application of machine learning pose estimation algorithms offers the promise of reducing experimental error(s) either due to variability of marker placements (Szczerbik and Kalinowska, 2011) or skin movement artifact (see previous section Elimination of soft tissue artifacts and artificial stimuli) which, in turn, would result in more accurate data.
Application to Clinical Settings
One of the goals of having a markerless system is the ease of use for compatible devices in a clinical setting to provide a valid clinical measurement tool. For a clinician, the important concern is implementation and integration into practice. If this is not supported by the technology, these systems will not be utilized. As such, researchers have investigated the ability of these systems to capture clinically relevant information for tests, such as the sit-to-stand (Otte et al., 2016), Landing Error Scoring System test (LESS; Mauntel et al., 2017, 2021), and the Star Excursion Balance Test (SEBT; Eltoukhy et al., 2017). For example, Eltoukhy et al. (2017) validated the performance of a markerless motion capture system on its ability to measure consistent results compared to manual assessment of SEBT reach results, as well as agreement with a marker-based motion capture system. The results of this study showed that the markerless system provided high agreement (ICC < 0.90) when assessing reach distance, with an error of <2 cm between systems. The conclusion was that this technology was efficient for the assessment the SEBT (Eltoukhy et al., 2017). An additional advantage of these systems for clinical assessments is their potential to provide an automated and consistent means of rapidly and accurately identifying aberrant movement patterns relevant to the clinician. For example, Mauntel et al. (2017) assessed the reliability of a markerless system used concurrently with movement assessment software in scoring the LESS as compared to scores rated by expert raters. The results showed that the markerless system was able to reliably score the LESS test and provided consistently accurate results (Mauntel et al., 2017). This finding is of great clinical relevance as it demonstrates a means toward greater throughput to clinically relevant movement assessments by (a) limiting the amount of time for the clinician to conduct such assessments (especially for an assessment such as the LESS) and (b) reducing analysis and, thus, evaluation times. These gains can also facilitate individualized interventions and enhance the quality of clinician-patient contact during treatment.
Enhanced Ecological Validity
One of the biggest criticisms of marker-based motion capture research conducted within a laboratory setting is degree of relevance or similarity that a laboratory-based assessment has to natural movements performed on-field/on-court. While traditional laboratory- and marker-based motion capture has provided invaluable information about the characteristics of movement and movement-related deficits, the question remains whether these assessments can accurately determine if an athlete is able to return to sports without the risk of injury due to aberrant movement patterns. Research has shown that athletes can improve mechanics based on these assessments and related motion capture methods to, ultimately, reduce their risk of second injury (e.g., Paterno et al., 2010; Ardakani et al., 2019). However, the greatest predictor of a new injury is a history of previous injury (e.g., Guskiewicz et al., 2000; Salmon et al., 2005; Paterno et al., 2010; Roos et al., 2017; Losciale et al., 2019), and this indicates existing assessments and tools may not provide the most clear indication of “real world” movement function. Markerless motion capture affords the possibility to design studies to assess unrestricted movement, and to incorporate real world task contexts by assessing these movements in a sport setting. There are a number of examples illustrating the application of markerless motion capture that enhance the ecological validity by assessing movement in relevant functional environments (Parsons and Alexander, 2012; Abrams et al., 2014; Moon et al., 2019). In addition, with the continued technological advances and improved algorithms, the experimental rigor required for accurate biomechanical analysis can still be maintained while integrating the complex challenges experienced by athletes in a more natural environment (Alderson, 2015; Moon et al., 2019; Nakano et al., 2020). Thus, the data captured using markerless motion capture derived from more natural settings could have greater clinical relevance for the types of movements required of athletes on the field or court.
Cost Efficacy—Expanding Motion Capture Accessibility
Two of the deterrents of marker-based 3D motion capture systems are the financial and time-related costs associated with such systems. Research-grade marker-based motion capture systems can cost in the tens to hundred thousands of dollars, not including the annual maintenance contracts for system hardware and software, or the employment of researcher assistants or personnel (e.g., students or technicians) who are trained to use these systems. Markerless systems provide a low-cost alternative for motion capture, providing researchers and clinicians with motion capture capabilities previously not available to them. For example, a single depth-sensing camera (often termed RGB-D camera for its capability to capture color and depth)—the latter a type of technology that comprises the Microsoft Kinect sensor (Microsoft, Redmond, WA, USA) and available on an Intel RealSense camera (Intel Corp., Santa Clara, CA, USA), for example—can be purchased for anywhere between $100 and $200. This is significantly less expensive technology that can greatly expand access to motion capture capabilities outside of the traditional laboratory setting. For example, portable markerless systems that use RGB-D cameras, or in other circumstances an off-the-shelf camera or mobile phone camera, can be deployed outside a laboratory to sport performance enhancement domains such as rehabilitation clinics or even in-home to facilitate telerehabilitation. Further, with less infringement on their personal space, patients, or participants can feel more comfortable during testing, which may lead to more diverse populations that would not have otherwise participated in motion capture research. By eliminating the preparatory time needed with marker-based systems, assessments are more time-efficient such that clinicians could feasibly utilize these technologies within their practice (Mündermann et al., 2006b), and may even be preferable in a situation with a high volume of participants [e.g., pre-season injury risk screening (Kotsifaki et al., 2018)].
The Power of Data
One of the greatest potentials for markerless motion capture for biomechanics research lies in the potential for building data-driven tools for inferencing/predicting based on large databases (Mathis and Mathis, 2020), instead of providing observational quantitative results from relatively small samples (as seen in marker-based studies thus far). Markerless motion capture has the potential for developing truly large databases of human movement (i.e., with N in the thousands), which may lead to better statistical and predictive models of human movement (Schmidhuber, 2015; Litjens et al., 2017).
Weaknesses
Not All Markerless Systems Are Equivalent I—Hardware and Software Considerations
Movement analysis for clinical application requires accurate representations of joint-specific information. While markerless motion capture has been widely applied to surveillance and gaming industries, its application to the biomechanical, clinical, and sport performance enhancement domains have been limited by the accuracy of the current methods. For instance, while the studies referenced above in the “Strengths” section (Agreement between markerless and marker-based systems section) have reported good to high agreement between markerless and marker-based motion capture systems, the range of the error recorded within the literature suggest that markerless motion capture systems overall are still not compatible with accurate biomechanical analysis. That said, there have been some systems that have demonstrated sufficient biomechanical accuracy, especially with respect to joint positions and sagittal plane joint angles validated against marker-based systems, but this accuracy is at times plane dependent (e.g., Macpherson et al., 2016; Yeung et al., 2021).
These differences could be explained by the differences in the methods and tools used for markerless motions capture. There are different computational and methodological approaches that have been taken to successfully achieve markerless motion capture, however whether these approaches are biomechanically and clinically applicable remains an open question. There are five components that need to be considered when evaluating a markerless motion capture system: (1) the number of cameras utilized, (2) the features of the captured image(s) used, (3) the model utilized to define a human body, (4) the machine learning algorithm employed to determine the desired variables from the body model (Colyer et al., 2018), and (5) as a result of components 3 and 4, errors due to variability in anthropometrics. Given the great variety of camera configurations, model types, and algorithms that have been proposed, several variations of these markerless motion capture features can be found throughout the literature. As discussed in the single vs. multi-camera configurations section below, it can be expected that more cameras allows for more features to be tracked and will lead to better biomechanical results. However, this limits the generalizability of studies as it limits our ability to compare data across different markerless systems, presenting an additional challenge of clarifying the advantages of specific configurations.
Not All Markerless Systems Are Equivalent II—Methodological Considerations for System Accuracy
There are additional methodological factors that can influence the accuracy of a markerless system that need to be taken into consideration. Specifically, four areas in particular should be highlighted. These include: (1) lighting, (2) camera range/resolution/positioning, (3) task complexity, and (4) collection environment. Current hardware limitations across available markerless systems limit their application to sports biomechanics due to a need to either (1) a need for controlled lighting conditions—e.g., markerless systems that emit light information for the acquisition and representation of human movement (Mündermann et al., 2006b), or (2) an inability to accurately capture data in direct sunlight [e.g., depth cameras such as the Kinect 1 and 2 (Zennaro et al., 2015)]. While more recent camera technologies (and algorithms) are becoming more robust to these limitations, the problem is not completely solved. An additional hardware limitation of systems such as Kinect-based markerless motion capture is that their effectiveness is only within a limited capture range (e.g., the Kinect 2 has a range of 0.5–4.5 m). Outside of the hardware limitations, several studies have noted a reduced accuracy, of the Kinect specifically, as task complexity increases (Mündermann et al., 2007; Tipton et al., 2019; Wochatz et al., 2019; Ressman et al., 2020), limiting the ability to capture and assess natural sport-specific movements of athletes. Another consideration is camera resolution—with greater resolution comes better tracking and keypoint detection. One of the weaknesses of greater resolution is the increase in the cost of the camera used as well as the processing costs. Ideally, there is a balance that needs to be met with regards to resolution and the hardware and processing costs associated. Thirdly, there is the limited generalizability of markerless motion capture models to include all body types. A challenging issue with markerless motion capture and the use of discriminative algorithms is simply if the available data is insufficient, the 2D and 3D reconstructed poses and motion trajectories will not be suitably represented. While more sophisticated models and reconstructions have been described (Corazza et al., 2010), there is a tradeoff in accuracy and processing time—increasing the accuracy also typically increases the computational burden (more time required for offline processing). Finally, one of the most attractive aspects of markerless motion capture for sports medicine is the ability to capture movement, non-invasively, during normal training environments. However, there are still environmental considerations with some markerless motion capture technology that need to be considered: (1) sunlight and (2) access to power supply and remote access such as the cloud. As previously mentioned, one consideration is sunlight. Due to its infrared properties, sunlight can introduce noise into capture devices that utilize an infrared camera (e.g., the Kinect). On a similar note, another issue would be darkness—RGB cameras are not able to track a person if it is too dark. The second environmental consideration when outside of the laboratory setting are access to a power supply for associated hardware and access to a portable hotspot in order to store data on the cloud. The latter point is going to become increasingly more important as camera resolution and subsequent data size continue to increase exponentially. As the field of sports medicine and sports science aims to leverage and validate markerless motion capture technology, these are all considerations that need to be taken into account and assessed.
Single vs. Multi-Camera Configurations
While the most accurate markerless systems tend to have a multi-camera configuration, much literature surrounding the use of markerless motion capture for biomechanical and/or clinical assessment often only consider a single dimension via a single-camera setup [predominantly, the Microsoft Kinect sensor (Pfister et al., 2014; Xu et al., 2015; Eltoukhy et al., 2017; Guess et al., 2017; Mauntel et al., 2017, 2021; do Carmo Vilas-Boas et al., 2019; Tanaka et al., 2019)]. Undoubtedly, ease of use surrounding this system relative to other similar single-camera systems (e.g., the set-up, preparation, and data acquisition of these systems) is the driving factor behind researchers trying to determine its application in the sports medicine arena; however, the single camera feature might be one of the leading obstacles for these systems matching the accuracy of marker-based systems. Single-camera systems like the Microsoft Kinect were designed to capture human movement for activities performed within a limited space (capture volume of the Kinect v2 ranges from 0.5 to 4.5 m) with the human facing the device. The efficiency of these systems is thus highly dependent on camera placement relative to the subject being captured (Chakraborty et al., 2020). Accordingly, the Microsoft Kinect seems to produce comparable kinematic data [results ≤5° in the sagittal plane are assumed by this review to be clinically negligible (McGinley et al., 2009); it is important to note that not all kinematic errors are equivalent and thus the kinematic plane should be considered when evaluating accuracy] to a marker-based system when performing tasks within the optimal capture volume such as squats (Schmitz et al., 2015; Perrott et al., 2017; Mentiplay et al., 2018), or a Functional Reach Test (Tanaka et al., 2019). Movement that is outside of this optimal capture volume leads to greater difficulties for this system. Specifically, previous studies have found large differences for the estimated joint kinematics captured between a Kinect system and marker-based motion capture systems during walking (Pfister et al., 2014; Mentiplay et al., 2015; Xu et al., 2015; Guess et al., 2017; do Carmo Vilas-Boas et al., 2019), and jumping tasks (Mentiplay et al., 2018; Harsted et al., 2019; Tipton et al., 2019). Due to the complex and highly variable nature of human movement, a single camera is not properly equipped to provide sufficient 3D pose information, providing a challenge when presented with self-occlusion, or identification of another occluding object within the environment (Mündermann et al., 2006b). When compared to a single-camera system, multi-camera systems have demonstrated improved agreement and reliability in capturing the dynamic characteristics of human movement (Núñez et al., 2017; Ryselis et al., 2020). In addition, by re-identifying positions across multiple view-points, the use of multi-camera capture has shown to increase classification rates (i.e., the accuracy of the systems in identifying movements) to more than 90% (Huang et al., 2012). The robustness of markerless systems can then be increased by increasing the number of cameras. Increasing the number of cameras increases the data available to solve the given number of degrees of freedom and provide more biomechanically accurate assessments (Mündermann et al., 2006b). Algorithms are starting to more accurately identify 3D kinematics from a 2D image, but cannot replace biomechanically accurate kinematic data from markered systems.
Opportunities
Enhancing Injury Risk Evaluation and Return-to-Play Assessments
Computer vision-based machine learning approaches provide a powerful framework that allow for automated inferences for multiple variations of postures and movements. There is a need for the development of markerless motion capture that is easy for clinicians and biomechanists to implement and apply. The use of markerless motion capture in a natural sport environment could allow training and rehabilitation specialists on the field (such as skill coaches and athletic trainers) to determine if an athlete is at risk of injury during practice or a game situation. For example, fatigue during sport performance is associated with compensatory movement patterns believed to predispose athletes to an increased risk of injury (Shaw et al., 2008; Small et al., 2010; De Ste Croix et al., 2015; Schütte et al., 2018). The implementation of markerless motion capture could index such fatigue related risk and apply this information to individualized injury prevention and training protocols.
Telerehabilitation
Telemedicine has emerged over the past century as a means to extend patient care and provide access to healthcare beyond a doctor's office. Broadly, the term telemedicine encompasses a wide range of telecommunications and information technologies used to facilitate the access of provider-patient (and provider-provider) health information, heath care, health education, and health care-based administrative services (Bashshur, 1995). One of the components within telemedicine is telerehabilitation—providing a range of rehabilitative services (therapeutic intervention, progression monitoring, education) to individuals without easy access to rehabilitation specialists (Theodoros et al., 2008) and can provide individualized rehabilitation outside of a hospital setting, allowing for continuous monitoring of patient progress. Research into the use of markerless motion capture systems for telerehabilitation has shown tremendous promise (Antón et al., 2013, 2016; Vukićević et al., 2015; Eichler et al., 2019; Steiner et al., 2020). To highlight a few, Antón et al. (2013) have presented a telerehabilitation system called KiReS (Kinect Rehabilitation System), a Kinect-based telerehabilitation system that allows for rehabilitation specialists to record exercises for a patient to perform and the patient can receive immediate feedback on their performance of the defined exercise (Antón et al., 2013). Vukićević et al. (2015) proposed a telerehabilitation platform based on Internet of things (IoT)—a contemporary technology aimed at improving healthcare by revamping classical methods of medical care (Jog et al., 2015)—that utilizes markerless motion capture to track and detect body movement (Vukićević et al., 2015). There is currently a gap in the literature with respect to lower limb telerehabilitation applications, with the majority of these studies only assessing posture and/or upper limb motor tasks. This may be due to the fact that current portable markerless motion capture systems seem to have better accuracy with detecting upper limb motor tasks over lower limb motor tasks (Capecci et al., 2016). In addition, despite the great potential markerless motion capture can have for telerehabilitation, the transition of telerehabilitation systems from proof-of-concept application into healthcare solutions has been challenging. One of the main challenges, highlighted by Tsiouris et al. (2020), is the lack of interoperability of these telerehabilitation platforms, beginning with the limited evidence supporting its efficacy. In order to utilize markerless motion capture to its full potential in telerehabilitation, a strong focus on developing advanced analytics for more precise outcomes and treatment plans in a user-friendly product is needed to optimize home-based rehabilitation. Markerless motion capture offers a window into addressing clinical and biomechanical challenges associated with prevention and recovery, as well as an opportunity to increase accessibility for patients to high-quality healthcare from home.
Emerging Technologies
The recent advancements over the past decade in 3D motion capture technology and the availability of low-cost devices that afford these capabilities (e.g., Microsoft Kinect) has made the collection of 3D data more feasible than ever. This increase in 3D data has encouraged researchers to take advantage of this richer content and address several computer vision problems. For instance, the eventual hope of markerless motion capture is to have real-time 3D reconstructions of the captured data for clinical application. One of the largest obstructions technologically to real-time 3D reconstruction using markerless motion capture is local processing power capabilities. While the processing power of a computer is able to handle single camera 2D reconstructions without issue, tracking movement from multiple video camera streams is a significant challenge in the computer vision domain. The processing power bottle neck can be an issue with multicamera solutions and high-FPS single camera requirements. In both instances, solving the pose estimation may take place at lower frame rates than needed, especially when capturing movements at >90 Hz. There are a couple of strong trends, however, toward technological developments that improve markerless motion capture performance with either local or remote processing power.
The first is the push toward improving onboard processing with computer graphics processing units (GPUs). Historically, the primary drive for GPU development has been for videogame applications or animation and graphical rendering (Luebke, 2008; McClanahan, 2010). This makes GPUs ideal for computer vision applications (Greengard, 2016) and this has helped drive relevant enhancements in the markerless motion capture space (e.g., the development of CUDA by NVIDIA in 2007; Sanders and Kandrot, 2010). This has been accelerated by a number of secondary market factors, and the rapid evolution of local GPU performance, including high-end mobile GPUs, will continue to allow researchers to exploit the possibilities of 3D motion capture technologies: improving 3D object classification, 3D object recognition, and 3D shape retrieval (Ioannidou et al., 2017) on a local device.
Another alternative is the push for the use of cloud computing to run computer vision algorithms when limitations in local processing power do not allow for real-time 3D reconstructions. Cloud computing development over the past decade has gained considerable attention and has provided a platform for massive data processing and data-intensive computing (Lin et al., 2013). The fundamental concept of cloud computing is that computing takes place in the “cloud;” i.e., referring to a network of services accessed over the internet rather than from the local infrastructure of one system. This reduces the need of purchasing the physical infrastructure while providing access to data storage, computing resources, and processing capabilities that would be otherwise unattainable even with the latest in current physical technology. Efficient cloud computing requires the availability of high bandwidth network communication through which the cloud architecture provides services for large data storage and large-scale data processing. This is an important step for markerless motion capture given the computational burden required for a processing system to detect, recognize, track, and retrieve 3D data for real-time processing. Companies such as Microsoft (Microsoft Azure), Amazon (Amazon Web Services, or AWS), and Google (Google Cloud Computing Services) all provide cloud services that emphasize machine learning with strong video-based analysis options. An important benefit of these services are the reduced storage costs for longer-term (aka cold) storage (i.e., once videos are processed, they can be stored long-term if they are only accessed occasionally), as these large scale platforms have a greater capability to subsidize storage costs than local or university-wide servers. Alternatively, one of the current downsides of these cloud platforms is the potential for ongoing subscription-level service costs associated with processing time. In addition, these platforms are less user-friendly to set up and manage for those not familiar with this technological infrastructure. However, these companies are working quickly to democratize these platforms and reduce such barriers. Overall, cloud computing platforms provide an avenue to improve 3D data processing by allowing for more rapid processing with less expensive devices, while also facilitating efficient deployment outside research and development spaces and to the end-user.
Interdisciplinary Collaborations
The driving force behind the development of markerless motion capture originates from computer vision and machine learning fields for character animation, virtual reality, smart surveillance, and the identification, recognition and tracking of human motion (Wang et al., 2003). That is to say, pose estimation algorithms were not built around biomechanical analysis or sports in general. As demonstrated by this manuscript, researchers and clinicians have expressed the value of such an application to sport or clinical settings; however, the original applications tried to fit computer-vision-based human motion analysis to non-optimized settings. Thus, the earlier pose estimation systems generally do not have the fidelity necessary for the resolution of motion capture tracking required for accurate and clinically valid biomechanical analysis. This has provided and continues to provide a unique opportunity for biomechanists and rehabilitation scientists to partner with the field of computer vision to enhance the current pose estimation solutions to increase the fidelity for the types of motions that are most pertinent to sports medicine and performance. Collaborations with computer vision specialists are critical for the development of biomechanically accurate algorithms as these professionals would have the expertise to understand the various pose estimation algorithms and models that could allow for refinement in accuracy and optimizing the relevant performance capabilities of these systems. This, in turn, would afford greater accessibility of these systems to clinicians and researchers and allow them to focus on their areas of expertise without concern for algorithmic and hardware-specific details of the technology itself. Additionally, these interdisciplinary partnerships are crucial and will continue to be very important as users of markerless motion capture begin to face new challenges, such as the organization and storage of very large video databases, building efficient database structures for human movement data, processing and reprocessing large amounts of data, and storing video data as protected health data. Such an interdisciplinary partnership would make these solutions more applicable, and may help catalyze meaningful development in this space.
Threats
Ethical Challenges
The feasibility of assessing athletes' movements during a sporting event or practice has radically advanced in the last decade, and it is expected to continue to evolve for the foreseeable future. Along with these technological advances in the sports medicine domain come ethical considerations that are critical to the use of technology and the end-users. As the adoption of this technology increases, scientists and health care providers will face many challenges related to information privacy and confidentiality. Specifically, the ethical dilemma becomes the protection and confidentiality of the information that can be attained through the 2D video that enables markerless motion capture. Researchers and clinicians must be equipped with the appropriate safeguards to protect and maintain the storage of video recordings, and the transmission of images and other patient record information to avoid privacy violations. It is paramount for the considerations to be discussed as this technology evolves to ensure that the personal data collected using markerless motion capture must be protected from misuse and HIPPA violations. Importantly, a few of the cloud services discussed in the previous section are well-equipped to handle these issues through the implementation of face-filter blurring, and data security practices that meet HIPPA and, in some cases, national security level approvals. Regardless, this is an important consideration for those who wish to adopt and implement this technology.
Data Ownership and Legal Considerations
Along a similar thread as the foreseeable ethical challenges mentioned above, as these systems become more widely used in sports settings, the legal considerations surrounding ownership of the data obtained from markerless motion capture technology should be examined. In the United States, while a few states have state regulations regarding biometric privacy [e.g., California: California Consumer Privacy Act (CCPA) and the California Privacy Rights Act (CPRA); New York: Stop Hacks and Improve Electronic Data Security (SHIELD) Act; Illinois: Biometric Information Privacy Act (BIPA)], there is no comprehensive federal law regulating the collection and use of biometric data. The European Union and the United Kingdom have done a better job with the regulations they have in place (for more detail, see Tikkinen-Piri et al., 2018), however, as markerless motion capture becomes feasible and applied in sports, the issue of data ownership will be a challenging hurdle. Because this information holds considerable interest to teams and stakeholders, this is a particularly relevant threat to collegiate and professional athletes. The collection of such data raises unprecedented concerns surrounding confidentiality and data privacy, raising important questions such as who owns the data, who has access to that data, and how will this information impact an athlete's career (Karkazis and Fishman, 2017).
Acceptance by Sports Medicine Researchers
From a researcher perspective the cost of these systems can prove quite economical, but the current computational burden and expense for biomechanically accurate markerless motion capture may be a deterrent for biomechanists. The priority for technological progression at this moment is in the honing of algorithmic techniques for markerless motion capture to enhance pose estimation accuracy at a resolution that enables the detection of subtle variations required by many biomechanical applications. This requires buy-in from the field of computer vision and sports medicine to see the potential of this application and begin those collaborations mentioned in the “Opportunities” section. Markerless motion capture is the future for human movement analysis; however, the speed at which we get there is strongly dependent on interprofessional collaboration and investment into the research and development that integrates biomechanical accuracy and pose estimation algorithms.
Challenging Error Sources and the Potential for Misuse
One of the advancements that come with markerless motion capture systems is the elimination of soft tissue artifacts and errors due to marker placement found when using marker-based systems (Mündermann et al., 2006b). However, the caveat is that measurement errors in markerless data are more challenging to detect, discern, and study than those from marker-based systems. The accuracy of the machine learning algorithms is decisively determined by the choice of the underlying model as to its accuracy to functional movement (Begon et al., 2018). This includes biases that arise from training datasets, and other unknown biases due to the “black box” nature of machine learning algorithms (Mathis et al., 2020). Additionally, with the reduced cost and increased accessibility of markerless motion capture, it allows biomechanical data to be obtained, used, and interpreted by users with insufficient technical backgrounds. The lack of an adequate understanding, skill, and experience in biomechanics could propagate to errors in the reporting of kinematics and kinetics potentially leading to biomechanical data and findings that lack proper scientific rigor.
Clinical Cost/Benefit
While the research findings of markerless motion capture offers several benefits for adoption into rehabilitative and preventative programs, these systems first must prove their value to rehabilitation specialists for adoption into everyday practice. Specifically, cost- and time-effective movement assessments are ideal so that clinicians can quickly and accurately identify movements that place an individual at a greater risk of injury or that impede recovery progress. While several existing camera and camera-like devices provide a cost-effective component of markerless motion capture, the concern comes with time efficiency of these devices in the set-up, acquisition and dissemination of movement information. The development of applications that allow clinicians to use markerless motion capture for specific movement assessments would prove quite beneficial and encourage the initial adoption of these systems. For instance, Mauntel et al. (2017) applied markerless motion capture technology to automate scoring of the LESS, a tool used to identify individuals at risk of lower extremity injury. Such an application was observed to reliably assess the LESS as expert raters and reduce the time requirements of a clinician conducting this assessment (Mauntel et al., 2017). However, the paucity of studies similar to this one limits the current understanding of clinical costs and benefits and, at present, negatively impacts mainstream markerless motion capture adoption.
Limitations
The purpose of this SWOT analysis was to provide clarity surrounding the currently available markerless motion capture approaches and identify specific areas for future development for this technology with regards to lower extremity biomechanical assessments in sports. However, this review is not without limitations that should be considered. First, as was previously mentioned, SWOT analyses are often criticized for their subjectivity (Pickton and Wright, 1998). However, the SWOT-analysis was developed as a tool for strategic analysis, as such, each factor within this review has been thoroughly reviewed by each author. A second consideration lies within the current subject matter—markerless motion capture and its related technologies is an active area of research that progresses rapidly, thus devices and their implementation techniques can quickly become outdated. In a similar thread, there are markerless systems that have been developed that have yet been evaluated for biomechanical accuracy (e.g., the Intel3D athlete system). Therefore, while this review is based on the current technologies to date, this is an important consideration. Finally, the discussion of this review is limited to those included within the inclusion criteria framework implemented. There may be additional markerless systems available or in development that were missed based on the scope of the current review.
Summary
Markerless motion capture systems show considerable promise for enhancing our understanding of human movement, and specifically providing unrestricted movement assessments in natural sport contexts. The emergent theme from this SWOT analysis is that despite nearly 20 years of development and discussion, markerless motion capture is still in its development stage for full application to the field of sports medicine. The success of several variants of system configurations and the encouraging initial results as well as clinical applications serve as the foundation for the future of biomechanically accurate markerless motion capture. Certain limitations still exist regarding accuracy, however these do not threaten the viability of this technology when considering the opportunities that this technology provides in the long run. The existing threats are not catastrophic as they are addressable and serve to provide valuable insight as markerless motion capture continues to develop. With thoughtful system design grounded in multidisciplinary collaborations, markerless motion capture will develop accurate clinical and research applications to expand current motion capture capabilities as well as its reach. The trajectory of this technology is positive and the future remains bright.
Author Contributions
CA-L and AK contributed to the writing and editing of the current manuscript. CA-L led the review. DW contributed to the review of the literature and the writing of the manuscript. All authors contributed to the article and approved the submitted version.
Funding
This work was supported, in part by an award (No. 1R21 EB027865 to AK) from the National Institute for Biomedical Imaging and Bioengineering.
Conflict of Interest
The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
Publisher's Note
All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.
Supplementary Material
The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fspor.2021.809898/full#supplementary-material
References
Abrams, G. D., Harris, A. H., Andriacchi, T. P., and Safran, M. R. (2014). Biomechanical analysis of three tennis serve types using a markerless system. Br. J. Sports Med. 48, 339–342. doi: 10.1136/bjsports-2012-091371
Alderson, J. (2015). A markerless motion capture technique for sport performance analysis and injury prevention: toward a ‘big data', machine learning future. J. Sci. Med. Sport 19:e79. doi: 10.1016/j.jsams.2015.12.192
Andriacchi, T. P., Mündermann, A., Smith, R. L., Alexander, E. J., Dyrby, C. O., and Koo, S. (2004). A framework for the in vivo pathomechanics of osteoarthritis at the knee. Ann. Biomed. Eng. 32, 447–457. doi: 10.1023/B:ABME.0000017541.82498.37
Antón, D., Goñi, A., Illarramendi, A., Torres-Unda, J. J., and Seco, J. (2013). “KiReS: a KINECT-based telerehabilitation system,” in 2013 IEEE 15th International Conference on e-Health Networking, Applications and Services (Healthcom 2013) (Lisbon).
Antón, D., Nelson, M., Russell, T., Goñi, A., and Illarramendi, A. (2016). Validation of a Kinect-based telerehabilitation system with total hip replacement patients. J. Telemed. Telecare 22, 192–197. doi: 10.1177/1357633X15590019
Ardakani, M. K., Wikstrom, E. A., Minoonejad, H., Rajabi, R., and Sharifnezhad, A. (2019). Hop-stabilization training and landing biomechanics in athletes with chronic ankle instability: a randomized controlled trial. J. Athl. Train. 54, 1296–1303. doi: 10.4085/1062-6050-550-17
Baker, S., and Kanade, T. (2005). Shape-from-silhouette across time part II: applications to human modeling and markerless motion tracking. Int. J. Comput. Vis. 63, 225–245. doi: 10.1007/s11263-005-6879-4
Bashshur, R. L. (1995). On the definition and evaluation of telemedicine. Telemed. J. 1, 19–30. doi: 10.1089/tmj.1.1995.1.19
Begon, M., Andersen, M. S., and Dumas, R. (2018). Multibody kinematics optimization for the estimation of upper and lower limb human joint kinematics: a systematized methodological review. J. Biomech. Eng. 140:030801. doi: 10.1115/1.4038741
Bregler, C., and Malik, J. (1998). “Tracking people with twists and exponential maps,” in Proceedings 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No. 98CB36231) (Santa Barbara, CA). doi: 10.1109/CVPR.1998.698581
Capecci, M., Ceravolo, M. G., Ferracuti, F., Iarlori, S., Longhi, S., Romeo, L., et al. (2016). “Accuracy evaluation of the Kinect v2 sensor during dynamic movements in a rehabilitation scenario,” in 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC) (Orlando, FL). doi: 10.1109/EMBC.2016.7591950
Ceseracciu, E., Sawacha, Z., and Cobelli, C. (2014). Comparison of markerless and marker-based motion capture technologies through simultaneous data collection during gait: proof of concept. PLoS ONE 9:e87640. doi: 10.1371/journal.pone.0087640
Chakraborty, S., Nandy, A., Yamaguchi, T., Bonnet, V., and Venture, G. (2020). Accuracy of image data stream of a markerless motion capture system in determining the local dynamic stability and joint kinematics of human gait. J. Biomech. 104:109718. doi: 10.1016/j.jbiomech.2020.109718
Clark, R. A., Bower, K. J., Mentiplay, B. F., Paterson, K., and Pua, Y.-H. (2013). Concurrent validity of the Microsoft Kinect for assessment of spatiotemporal gait variables. J. Biomech. 46, 2722–2725. doi: 10.1016/j.jbiomech.2013.08.011
Colyer, S. L., Evans, M., Cosker, D. P., and Salo, A. I. (2018). A review of the evolution of vision-based motion analysis and the integration of advanced computer vision methods towards developing a markerless system. Sports Med. Open 4, 1–15. doi: 10.1186/s40798-018-0139-y
Corazza, S., Gambaretto, E., Mündermann, L., and Andriacchi, T. P. (2008). Automatic generation of a subject-specific model for accurate markerless motion capture and biomechanical applications. IEEE Trans. Biomed. Eng. 57, 806–812. doi: 10.1109/TBME.2008.2002103
Corazza, S., Muendermann, L., Chaudhari, A., Demattio, T., Cobelli, C., and Andriacchi, T. P. (2006). A markerless motion capture system to study musculoskeletal biomechanics: visual hull and simulated annealing approach. Ann. Biomed. Eng. 34, 1019–1029. doi: 10.1007/s10439-006-9122-8
Corazza, S., Mündermann, L., and Andriacchi, T. (2007). A framework for the functional identification of joint centers using markerless motion capture, validation for the hip joint. J. Biomech. 40, 3510–3515. doi: 10.1016/j.jbiomech.2007.05.029
Corazza, S., Mündermann, L., Gambaretto, E., Ferrigno, G., and Andriacchi, T. P. (2010). Markerless motion capture through visual hull, articulated icp and subject specific model generation. Int. J. Comput. Vis. 87:156. doi: 10.1007/s11263-009-0284-3
De Ste Croix, M., Priestley, A. M., Lloyd, R. S., and Oliver, J. (2015). Acl injury risk in elite female youth soccer: changes in neuromuscular control of the knee following soccer-specific fatigue. Scand. J. Med. Sci. Sports 25, e531–e538. doi: 10.1111/sms.12355
Deutscher, J., Blake, A., and Reid, I. (2000). “Articulated body motion capture by annealed particle filtering,” in Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662) (Hilton Head, SC). doi: 10.1109/CVPR.2000.854758
do Carmo Vilas-Boas, M., Choupina, H. M. P., Rocha, A. P., Fernandes, J. M., and Cunha, J. P. S. (2019). Full-body motion assessment: concurrent validation of two body tracking depth sensors versus a gold standard system during gait. J. Biomech. 87, 189–196. doi: 10.1016/j.jbiomech.2019.03.008
Drazan, J. F., Phillips, W. T., Seethapathi, N., Hullfish, T. J., and Baxter, J. R. (2021). Moving outside the lab: markerless motion capture accurately quantifies sagittal plane kinematics during the vertical jump. J. Biomech. 125:110547. doi: 10.1016/j.jbiomech.2021.110547
Düking, P., Holmberg, H.-C., and Sperlich, B. (2018). The potential usefulness of virtual reality systems for athletes: a short SWOT analysis. Front. Physiol. 9:128. doi: 10.3389/fphys.2018.00128
Eichler, S., Salzwedel, A., Rabe, S., Mueller, S., Mayer, F., Wochatz, M., et al. (2019). The effectiveness of telerehabilitation as a supplement to rehabilitation in patients after total knee or hip replacement: randomized controlled trial. JMIR Rehabil. Assistive Technol. 6:e14236. doi: 10.2196/14236
Eltoukhy, M., Kuenze, C., Oh, J., Wooten, S., and Signorile, J. (2017). Kinect-based assessment of lower limb kinematics and dynamic postural control during the star excursion balance test. Gait Posture 58, 421–427. doi: 10.1016/j.gaitpost.2017.09.010
Gavrila, D. M., and Davis, L. S. (1996). “3-D model-based tracking of humans in action: a multi-view approach,” in Proceedings CVPR IEEE Computer Society Conference on Computer Vision and Pattern Recognition (San Francisco, CA). doi: 10.1109/CVPR.1996.517056
Grauman, K., Shakhnarovich, G., and Darrell, T. (2003). “Inferring 3D structure with a statistical image-based shape model,” in ICCV (Nice). doi: 10.1109/ICCV.2003.1238408
Gray, A. D., Willis, B. W., Skubic, M., Huo, Z., Razu, S., Sherman, S. L., et al. (2017). Development and validation of a portable and inexpensive tool to measure the drop vertical jump using the microsoft kinect v2. Sports Health: A Multidisciplinary Approach 9, 537–544. doi: 10.1177/1941738117726323
Guess, T. M., Razu, S., Jahandar, A., Skubic, M., and Huo, Z. (2017). Comparison of 3D joint angles measured with the kinect 2.0 skeletal tracker versus a marker-based motion capture system. J. Appl. Biomech. 33, 176–181. doi: 10.1123/jab.2016-0107
Guskiewicz, K. M., Weaver, N. L., Padua, D. A., and Garrett, W. E. (2000). Epidemiology of concussion in collegiate and high school football players. Am. J. Sports Med. 28, 643–650. doi: 10.1177/03635465000280050401
Harsted, S., Holsgaard-Larsen, A., Hestbæk, L., Boyle, E., and Lauridsen, H. H. (2019). Concurrent validity of lower extremity kinematics and jump characteristics captured in pre-school children by a markerless 3D motion capture system. Chiropr. Man. Therap. 27, 1–16. doi: 10.1186/s12998-019-0261-z
Helms, M. M., Moore, R., and Ahmadi, M. (2008). Information technology (IT) and the healthcare industry: a SWOT analysis. Int. J. Healthc. Information Syst. Informatics 3, 75–92. doi: 10.4018/jhisi.2008010105
Huang, Q., Yang, J., and Qiao, Y. (2012). “Person re-identification across multi-camera system based on local descriptors,” in 2012 Sixth International Conference on Distributed Smart Cameras (ICDSC) (Hong Kong).
Ioannidou, A., Chatzilari, E., Nikolopoulos, S., and Kompatsiaris, I. (2017). Deep learning advances in computer vision with 3d data: a survey. ACM Comput. Surveys 50, 1–38. doi: 10.1145/3042064
Ionescu, C., Papava, D., Olaru, V., and Sminchisescu, C. (2013). Human3. 6m: large scale datasets and predictive methods for 3d human sensing in natural environments. IEEE Trans. Pattern Anal. Mach. Intelligence 36, 1325–1339. doi: 10.1109/TPAMI.2013.248
Jog, Y., Sharma, A., Mhatre, K., and Abhishek, A. (2015). Internet of things as a solution enabler in health sector. Int. J. Bio Sci. Bio Technol. 7, 9–24. doi: 10.14257/ijbsbt.2015.7.2.02
Kanko, R. M., Laende, E. K., Davis, E. M., Selbie, W. S., and Deluzio, K. J. (2021). Concurrent assessment of gait kinematics using marker-based and markerless motion capture. J. Biomech. 127:110665. doi: 10.1016/j.jbiomech.2021.110665
Karkazis, K., and Fishman, J. R. (2017). Tracking US professional athletes: the ethics of biometric technologies. Am. J. Bioethics 17, 45–60. doi: 10.1080/15265161.2016.1251633
Kotsifaki, A., Whiteley, R., and Hansen, C. (2018). Dual kinect v2 system can capture lower limb kinematics reasonably well in a clinical setting: concurrent validity of a dual camera markerless motion capture system in professional football players. BMJ Open Sport Exerc. Med. 4, 1–9. doi: 10.1136/bmjsem-2018-000441
Leardini, A., Chiari, L., Della Croce, U., and Cappozzo, A. (2005). Human movement analysis using stereophotogrammetry: part 3. Soft tissue artifact assessment and compensation. Gait Posture 21, 212–225. doi: 10.1016/j.gaitpost.2004.05.002
Lin, C., Su, W., Meng, K., Liu, Q., and Liu, W.-D. (2013). Cloud computing security: architecture, mechanism and modeling. Chin. J. Comput. 36, 1765–1784. doi: 10.3724/SP.J.1016.2013.01765
Litjens, G., Kooi, T., Bejnordi, B. E., Setio, A. A. A., Ciompi, F., Ghafoorian, M., et al. (2017). A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88. doi: 10.1016/j.media.2017.07.005
Losciale, J. M., Zdeb, R. M., Ledbetter, L., Reiman, M. P., and Sell, T. C. (2019). The association between passing return-to-sport criteria and second anterior cruciate ligament injury risk: a systematic review with meta-analysis. J. Orthopaedic Sports Phys. Ther. 49, 43–54. doi: 10.2519/jospt.2019.8190
Luebke, D. (2008). “GPU architecture: implications & trends,” in SIGGRAPH 2008: Beyond Programmable Shading Course Materials (Los Angeles, CA).
Macpherson, T. W., Taylor, J., McBain, T., Weston, M., and Spears, I. R. (2016). Real-time measurement of pelvis and trunk kinematics during treadmill locomotion using a low-cost depth-sensing camera: a concurrent validity study. J. Biomech. 49, 474–478. doi: 10.1016/j.jbiomech.2015.12.008
Mathis, A., Schneider, S., Lauer, J., and Mathis, M. W. (2020). A primer on motion capture with deep learning: principles, pitfalls, and perspectives. Neuron 108, 44–65. doi: 10.1016/j.neuron.2020.09.017
Mathis, M. W., and Mathis, A. (2020). Deep learning tools for the measurement of animal behavior in neuroscience. Curr. Opin. Neurobiol. 60, 1–11. doi: 10.1016/j.conb.2019.10.008
Mauntel, T. C., Cameron, K. L., Pietrosimone, B., Marshall, S. W., Hackney, A. C., and Padua, D. A. (2021). Validation of a commercially available markerless motion-capture system for trunk and lower extremity kinematics during a jump-landing assessment. J. Athl. Train. 56, 177–190. doi: 10.4085/1062-6050-0023.20
Mauntel, T. C., Padua, D. A., Stanley, L. E., Frank, B. S., DiStefano, L. J., Peck, K. Y., et al. (2017). Automated quantification of the Landing Error Scoring System with a markerless motion-capture system. J. Athl. Train. 52, 1002–1009. doi: 10.4085/1062-6050-52.10.12
McGinley, J. L., Baker, R., Wolfe, R., and Morris, M. E. (2009). The reliability of three-dimensional kinematic gait measurements: a systematic review. Gait Posture 29, 360–369. doi: 10.1016/j.gaitpost.2008.09.003
Mentiplay, B. F., Hasanki, K., Perraton, L. G., Pua, Y.-H., Charlton, P. C., and Clark, R. A. (2018). Three-dimensional assessment of squats and drop jumps using the Microsoft Xbox One Kinect: reliability and validity. J. Sports Sci. 36, 2202–2209. doi: 10.1080/02640414.2018.1445439
Mentiplay, B. F., Perraton, L. G., Bower, K. J., Pua, Y.-H., McGaw, R., Heywood, S., et al. (2015). Gait assessment using the Microsoft Xbox One Kinect: concurrent validity and inter-day reliability of spatiotemporal and kinematic variables. J. Biomech. 48, 2166–2170. doi: 10.1016/j.jbiomech.2015.05.021
Moeslund, T. B., Hilton, A., and Krüger, V. (2006). A survey of advances in vision-based human motion capture and analysis. Comput. Vision Image Understand. 104, 90–126. doi: 10.1016/j.cviu.2006.08.002
Moon, G., Chang, J. Y., and Lee, K. M. (2019). “Camera distance-aware top-down approach for 3d multi-person pose estimation from a single rgb image,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (Seoul). doi: 10.1109/ICCV.2019.01023
Mündermann, L., Corazza, S., and Andriacchi, T. P. (2006a). The evolution of methods for the capture of human movement leading to markerless motion capture for biomechanical applications. J. Neuroeng. Rehabil. 3, 1–11. doi: 10.1186/1743-0003-3-6
Mündermann, L., Corazza, S., and Andriacchi, T. P. (2007). “Accurately measuring human movement using articulated ICP with soft-joint constraints and a repository of articulated models,” in 2007 IEEE Conference on Computer Vision and Pattern Recognition (Minneapolis, MN). doi: 10.1109/CVPR.2007.383302
Mündermann, L., Corazza, S., Chaudhari, A. M., Andriacchi, T. P., Sundaresan, A., and Chellappa, R. (2006b). “Measuring human movement for biomechanical applications using markerless motion capture,” in Three-Dimensional Image Capture and Applications VII (San Jose, CA). doi: 10.1117/12.650854
Naeemabadi, M., Dinesen, B., Andersen, O. K., and Hansen, J. (2018). Investigating the impact of a motion capture system on Microsoft Kinect v2 recordings: a caution for using the technologies together. PLoS ONE 13:e0204052. doi: 10.1371/journal.pone.0204052
Nakano, N., Sakura, T., Ueda, K., Omura, L., Kimura, A., Iino, Y., et al. (2020). Evaluation of 3D markerless motion capture accuracy using OpenPose with multiple video cameras. Front. Sports Active Living 2, 1–9. doi: 10.3389/fspor.2020.00050
Needham, L., Evans, M., Cosker, D. P., Wade, L., McGuigan, P. M., Bilzon, J. L., et al. (2021). The accuracy of several pose estimation methods for 3D joint centre localisation. Sci. Rep. 11, 1–11. doi: 10.1038/s41598-021-00212-x
Núñez, J. C., Cabido, R., Montemayor, A. S., and Pantrigo, J. J. (2017). Real-time human body tracking based on data fusion from multiple RGB-D sensors. Multimed. Tools Appl. 76, 4249–4271. doi: 10.1007/s11042-016-3759-6
Ogawa, A., Mita, A., Yorozu, A., and Takahashi, M. (2017). Markerless knee joint position measurement using depth data during stair walking. Sensors 17:2698. doi: 10.3390/s17112698
Otte, K., Kayser, B., Mansow-Model, S., Verrel, J., Paul, F., Brandt, A. U., et al. (2016). Accuracy and reliability of the kinect version 2 for clinical measurement of motor function. PLoS ONE 11:e0166532. doi: 10.1371/journal.pone.0166532
Pagnon, D., Domalain, M., and Reveret, L. (2021). Pose2Sim: an end-to-end workflow for 3D markerless sports kinematics—part 1: robustness. Sensors 21:6530. doi: 10.3390/s21196530
Parsons, J. L., and Alexander, M. J. (2012). Modifying spike jump landing biomechanics in female adolescent volleyball athletes using video and verbal feedback. J. Strength Condition. Res. 26, 1076–1084. doi: 10.1519/JSC.0b013e31822e5876
Paterno, M. V., Schmitt, L. C., Ford, K. R., Rauh, M. J., Myer, G. D., Huang, B., et al. (2010). Biomechanical measures during landing and postural stability predict second anterior cruciate ligament injury after anterior cruciate ligament reconstruction and return to sport. Am. J. Sports Med. 38, 1968–1978. doi: 10.1177/0363546510376053
Perrott, M. A., Pizzari, T., Cook, J., and McClelland, J. A. (2017). Comparison of lower limb and trunk kinematics between markerless and marker-based motion capture systems. Gait Posture 52, 57–61. doi: 10.1016/j.gaitpost.2016.10.020
Peters, A., Galna, B., Sangeux, M., Morris, M., and Baker, R. (2010). Quantification of soft tissue artifact in lower limb human motion analysis: a systematic review. Gait Posture 31, 1–8. doi: 10.1016/j.gaitpost.2009.09.004
Pfister, A., West, A. M., Bronner, S., and Noah, J. A. (2014). Comparative abilities of Microsoft Kinect and Vicon 3D motion capture for gait analysis. J. Med. Eng. Technol. 38, 274–280. doi: 10.3109/03091902.2014.909540
Pickton, D. W., and Wright, S. (1998). What's swot in strategic analysis? Strategic Change 7, 101–109. doi: 10.1002/(SICI)1099-1697(199803/04)7:2<101::AID-JSC332>3.0.CO;2-6
Pietrosimone, B., Blackburn, J. T., Harkey, M. S., Luc, B. A., Hackney, A. C., Padua, D. A., et al. (2016). Greater mechanical loading during walking is associated with less collagen turnover in individuals with anterior cruciate ligament reconstruction. Am. J. Sports Med. 44, 425–432. doi: 10.1177/0363546515618380
Poppe, R. (2007). Vision-based human motion analysis: an overview. Comput. Vision Image Understand. 108, 4–18. doi: 10.1016/j.cviu.2006.10.016
Ressman, J., Rasmussen-Barr, E., and Grooten, W. J. A. (2020). Reliability and validity of a novel Kinect-based software program for measuring a single leg squat. BMC Sports Sci. Med. Rehabil. 12, 1–12. doi: 10.1186/s13102-020-00179-8
Rizzo, A. S., and Kim, G. J. (2005). A SWOT analysis of the field of virtual reality rehabilitation and therapy. Presence Teleoperators Virtual Environ. 14, 119–146. doi: 10.1162/1054746053967094
Rocha, A. P., Choupina, H. M. P., Vilas-Boas, M. d. C., Fernandes, J. M., and Cunha, J. P. S. (2018). System for automatic gait analysis based on a single RGB-D camera. PLoS ONE 13:e0201728. doi: 10.1371/journal.pone.0201728
Roos, K. G., Kerr, Z. Y., Mauntel, T. C., Djoko, A., Dompier, T. P., and Wikstrom, E. A. (2017). The epidemiology of lateral ligament complex ankle sprains in National Collegiate Athletic Association sports. Am. J. Sports Med. 45, 201–209. doi: 10.1177/0363546516660980
Ryselis, K., Petkus, T., BlaŽauskas, T., Maskeliunas, R., and Damaševičius, R. (2020). Multiple Kinect based system to monitor and analyze key performance indicators of physical training. Human Centric Comput. Information Sci. 10, 1–22. doi: 10.1186/s13673-020-00256-4
Salmon, L., Russell, V., Musgrove, T., Pinczewski, L., and Refshauge, K. (2005). Incidence and risk factors for graft rupture and contralateral rupture after anterior cruciate ligament reconstruction. Arthroscopy J. Arthroscopic Related Surg. 21, 948–957. doi: 10.1016/j.arthro.2005.04.110
Sandau, M. (2015). Applications of markerless motion capture in gait recognition. Med. Eng. Phys. 37, 948–955.
Sandau, M., Koblauch, H., Moeslund, T. B., Aanæs, H., Alkjær, T., and Simonsen, E. B. (2014). Markerless motion capture can provide reliable 3D gait kinematics in the sagittal and frontal plane. Med. Eng. Phys. 36, 1168–1175. doi: 10.1016/j.medengphy.2014.07.007
Sanders, J., and Kandrot, E. (2010). CUDA by Example: An Introduction to General-Purpose GPU Programming. Addison-Wesley Professional.
Schmidhuber, J. (2015). Deep learning in neural networks: an overview. Neural Netw. 61, 85–117. doi: 10.1016/j.neunet.2014.09.003
Schmitz, A., Ye, M., Boggess, G., Shapiro, R., Yang, R., and Noehren, B. (2015). The measurement of in vivo joint angles during a squat using a single camera markerless motion capture system as compared to a marker based system. Gait Posture 41, 694–698. doi: 10.1016/j.gaitpost.2015.01.028
Scholes, K., Johnson, G., and Whittington, R. (2002). Exploring Corporate Strategy. Financial Times Prentice Hall.
Schütte, K. H., Seerden, S., Venter, R., and Vanwanseele, B. (2018). Influence of outdoor running fatigue and medial tibial stress syndrome on accelerometer-based loading and stability. Gait Posture 59, 222–228. doi: 10.1016/j.gaitpost.2017.10.021
Shaw, M. Y., Gribble, P. A., and Frye, J. L. (2008). Ankle bracing, fatigue, and time to stabilization in collegiate volleyball athletes. J. Athl. Train. 43, 164–171. doi: 10.4085/1062-6050-43.2.164
Small, K., McNaughton, L., Greig, M., and Lovell, R. (2010). The effects of multidirectional soccer-specific fatigue on markers of hamstring injury risk. J. Sci. Med. Sport 13, 120–125. doi: 10.1016/j.jsams.2008.08.005
Steiner, B., Elgert, L., Saalfeld, B., Schwartze, J., Borrmann, H. P., Kobelt-Pönicke, A., et al. (2020). Health-enabling technologies for telerehabilitation of the shoulder: a feasibility and user acceptance study. Methods Inf. Med. 59:e90. doi: 10.1055/s-0040-1713685
Stenum, J., Rossi, C., and Roemmich, R. T. (2021). Two-dimensional video-based analysis of human gait using pose estimation. PLoS Comput. Biol. 17:e1008935. doi: 10.1371/journal.pcbi.1008935
Szczerbik, E., and Kalinowska, M. (2011). The influence of knee marker placement error on evaluation of gait kinematic parameters. Acta Bioeng. Biomech. 13, 43–46.
Takeda, I., Yamada, A., and Onodera, H. (2021). Artificial Intelligence-Assisted motion capture for medical applications: a comparative study between markerless and passive marker motion capture. Comput. Methods Biomech. Biomed. Engin. 24, 864–873. doi: 10.1080/10255842.2020.1856372
Tanaka, R., Ishikawa, Y., Yamasaki, T., and Diez, A. (2019). Accuracy of classifying the movement strategy in the functional reach test using a markerless motion capture system. J. Med. Eng. Technol. 43, 133–138. doi: 10.1080/03091902.2019.1626504
Theodoros, D., Russell, T., and Latifi, R. (2008). Telerehabilitation: current perspectives. Stud. Health Technol. Inform. 131, 191–210.
Tikkinen-Piri, C., Rohunen, A., and Markkula, J. (2018). EU general data protection regulation: changes and implications for personal data collecting companies. Comput. Law Secur. Rev. 34, 134–153. doi: 10.1016/j.clsr.2017.05.015
Tipton, C. C., Telfer, S., Cherones, A., Gee, A. O., and Kweon, C. Y. (2019). The use of Microsoft Kinect™ for assessing readiness of return to sport and injury risk exercises: a validation study. Int. J. Sports Phys. Ther. 14:724. doi: 10.26603/ijspt20190724
Tsiouris, K. M., Gatsios, D., Tsakanikas, V., Pardalis, A. A., Kouris, I., Androutsou, T., et al. (2020). Designing interoperable telehealth platforms: bridging IoT devices with cloud infrastructures. Enterprise Information Syst. 14, 1194–1218. doi: 10.1080/17517575.2020.1759146
Vafadar, S., Skalli, W., Bonnet-Lebrun, A., Khalifé, M., Renaudin, M., Hamza, A., et al. (2021). A novel dataset and deep learning-based approach for marker-less motion capture during gait. Gait Posture 86, 70–76. doi: 10.1016/j.gaitpost.2021.03.003
Vukićević, S., Stamenković, Z., Murugesan, S., Bogdanović, Z., and Radenković, B. (2015). A new telerehabilitation system based on internet of things. Facta Universitatis Series Electron. Energet. 29, 395–405. doi: 10.2298/FUEE1603395V
Wang, L., Hu, W., and Tan, T. (2003). Recent developments in human motion analysis. Pattern Recognit. 36, 585–601. doi: 10.1016/S0031-3203(02)00100-0
Wochatz, M., Tilgner, N., Mueller, S., Rabe, S., Eichler, S., John, M., et al. (2019). Reliability and validity of the Kinect V2 for the assessment of lower extremity rehabilitation exercises. Gait Posture 70, 330–335. doi: 10.1016/j.gaitpost.2019.03.020
Xu, X., McGorry, R. W., Chou, L.-S., Lin, J.-h., and Chang, C.-c. (2015). Accuracy of the Microsoft Kinect™ for measuring gait parameters during treadmill walking. Gait Posture 42, 145–151. doi: 10.1016/j.gaitpost.2015.05.002
Yeung, L.-F., Yang, Z., Cheng, K. C.-C., Du, D., and Tong, R. K.-Y. (2021). Effects of camera viewing angles on tracking kinematic gait patterns using Azure Kinect, Kinect v2 and Orbbec Astra Pro v2. Gait Posture 87, 19–26. doi: 10.1016/j.gaitpost.2021.04.005
Keywords: SWOT, markerless, motion capture (Mocap), kinematics, sports medicine
Citation: Armitano-Lago C, Willoughby D and Kiefer AW (2022) A SWOT Analysis of Portable and Low-Cost Markerless Motion Capture Systems to Assess Lower-Limb Musculoskeletal Kinematics in Sport. Front. Sports Act. Living 3:809898. doi: 10.3389/fspor.2021.809898
Received: 05 November 2021; Accepted: 24 December 2021;
Published: 25 January 2022.
Edited by:
Pietro Picerno, University of eCampus, ItalyCopyright © 2022 Armitano-Lago, Willoughby and Kiefer. This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
*Correspondence: Adam W. Kiefer, awkiefer@email.unc.edu