Patient-reported outcome measures as an outcome variable in sports medicine research

Injury prevention and rehabilitation research often address variables that would be considered clinician-oriented outcomes, such as strength, range of motion, laxity, and return-to-sport. While clinician-oriented variables are helpful in describing the physiological recovery from injury, they neglect the patient perspective and aspects of patient-centered care. Variables that capture patient perspective are essential when considering the impact of injury and recovery on the lives of patients. The inclusion of patient-reported outcome measures (PROMs) as dependent variables in sports medicine research, including injury prevention and rehabilitation research, provides a unique perspective regarding the patient's perception of their health status, the effectiveness of treatments, and other information that the patient deems important to their care. Over the last 20 years, there has been a significant increase in the use of PROMs in sports medicine research. The growing body of work gives opportunity to reflect on what has been done and to provide some ideas of how to strengthen the evidence moving forward. This mini-review will discuss ideas for the inclusion of PROMs in sports medicine research, with a focus on critical factors, gaps, and future directions in this area of research. Important elements of research with PROMs, including instrument selection, administration, and interpretation, will be discussed and areas for improvement, consideration, and standardization will be provided.

Introduction

Recently, there have been efforts to give patients a voice in both clinical care and research. A focus on patient-centered care has permeated to sports medicine, providing opportunities for patients and families to be active collaborators in the development and implementation of their comprehensive care plan (1). In regards to sports medicine research, efforts to include patient-reported outcome measures (PROMs), a source of patient-oriented evidence, have been ongoing and are highlighted by calls by professional organizations and government entities, such as the Patient-Centered Outcomes Research Institute (PCORI) (2). A proliferation of published research on PROMs in the past decade further supports the increased utilization of these measures among sports medicine researchers. A database search using the terms patient reported outcome measures and sports medicine identified 562 citations in the years 2000–2009. This number increased to 4,250 between 2010 and 2019, with 3,217 publications already between 2020 and 2022. Unfortunately, issues still exist in the use of PROMs in sports medicine research, such as inconsistency and variability in choosing and implementing PROMs within investigations (3). This can introduce bias and influence the research findings of individual studies, as well as subsequent secondary analysis when conducting systematic reviews or meta-analyses if authors are aiming to synthesize a body of evidence, in which PROMs are not used similarly across individual studies (4).

Historically, sports medicine research, including research specific to injury prevention and rehabilitation, has focused on clinician-oriented outcomes, including strength, range of motion, and functional tests. While some patient-oriented outcomes have accompanied clinician-oriented outcomes in sports medicine research, the focus has been on variables such as patient perception of pain. Although an important outcome, pain offers a limited perspective on the patient's function and overall health status and is often assessed with a numeric pain rating scale that is unidimensional in nature and does not provide detailed information regarding the patient's perception of pain. Neglecting other important dimensions of a patient's health, including quality of life (e.g., Pediatric Quality of Life Inventory), psychological readiness (e.g., Anterior Cruciate Ligament-Return to Sport after Injury scale), and injury-related fear (e.g., Tampa Scale of Kinesiophopia) across different domains of disablement (e.g., activity, participation, and environmental factors) limits the comprehensiveness of a patient-centered approach. Current recommendations to embrace the biopsychosocial model of health requires the assessment of multidimensional health constructs including physical, psychological, interpersonal and contextual factors in a dynamic model (5). The inclusion of a broader set of multidimensional health constructs may allow clinicians and researchers to better understand the patient experience, appreciate patient perception of health, and assess additional domains of contemporary disablement models (6). While there has been a move to include more PROMs in research, there are still certain methodological issues with the inclusion of PROMs into sports injury prevention and rehabilitation research.

Current evidence suggests that many studies do not adequately use PROMs within sports medicine research (Table 1) (7–9). The adequate use of PROMs has been defined as using a PROM that measures what it claims to (content validity), has undergone appropriate statistical validation (reliability, validity, responsiveness), and is used to evaluate patients with conditions similar to those used in its validation (7). In a review of 349 articles, almost half were identified as having at least one irregularity or potential problem in PROM selection (9). For example, a significant number of studies used a PROM that was developed for a different patient population or used two or more PROMs containing identical questions, which suggests issues with instrument selection. Similarly, recent analyses of randomized controlled trials (RCT) in sports medicine (7, 8) also noted inadequate use of PROMs in more than half of studies. Instrument selection was a concern, with PROMs being used in populations they were not validated in, as was instrument administration, with studies not reporting the research protocol. Further, another challenge for sports medicine researchers is PROM interpretation given that there is limited information to guide interpretation across the large number of instruments available and there is an evolution of the science of meaningful change (11–14). Evidence suggests the appropriate use of PROMs for a research question of interest is associated with larger treatment effects as compared to studies with inadequate PROM use (7), highlighting the need to educate researchers on proper PROM selection and use to reduce variability and increase consistency between studies and better capture treatment effects within studies. Therefore, the purpose of this mini-review is to aid sports medicine researchers in the selection, administration, and interpretation of PROMs. In addition, national and international efforts to improve the inclusion of PROMs into rehabilitation research will be discussed.

TABLE 1

Table 1. Irregularities noted in studies of PROMs in sports medicine research (7–10).

Instrument selection

Sports medicine researchers integrating PROMs may find it challenging to decide which outcomes are most appropriate to use given the large number of PROMs available (10, 15–18). Patient-reported outcome measures can be designed as generic in focus which makes them applicable to evaluate health broadly in healthy or injured populations, or they can be designed to be specific to particular types of populations, regions, or conditions. Length is another consideration with some PROMs including a single item, like the Single Assessment Numeric Evaluation (SANE) (19), and others including multiple-items, such as the Lower Extremity Functional Scale (LEFS) (20). Use of generic vs. specific and single-items vs. multi-item PROMs will influence who the instruments can be administered to, time burden on researcher and patient, and depth and relevance of the information obtained from a population (21). Unfortunately, PROM selection is not a simple task and can be approached in different ways. A common approach is to vet a number of different PROMs according to general guidelines and identify the best fit for the study. Recent efforts to establish guidelines and checklists for including PROMs in research, particularly clinical trials, are helpful and should be used early in study development (22–25). For example, general guidelines to follow when selecting a PROM should: (1) identify relevant PROM domains of interest that align with research questions, (2) consider disease or condition-specific, population-specific, and region-specific PROMs that are likely to be influenced by the therapy or intervention being studied, (3) evaluate the psychometric properties (reliability, validity and responsiveness) of PROMs in the population being studied, and (4) consider practicalities of using the PROM in the study, such as respondent burden and mode of administration (17, 21, 24, 26, 27).

When identifying relevant PROM domains, frameworks, such as disablement models, may be helpful in choosing outcomes of interest most critical to patients (6). While pain is a common impairment studied for its importance to patients, researchers should consider other body structure and function impairments (e.g., fatigue, strength) and additional domains, such as function in terms of activity limitations and participation restrictions (e.g., throwing, running, playing sports, attending school) and psychological readiness. Selecting PROMs most likely to be influenced by the study's intervention is encouraged, and these tend to be disease or condition-specific, population-specific, and region-specific PROMs. Multidimensional PROMs that evaluate multiple social constructs, such as HRQOL, are important to understand in the context of the study's intervention, but are likely influenced by more aspects of the patient's life than the research study (24, 25). Therefore, using PROMs that evaluate HRQOL alone as an outcome is not advised if new interventions are being studied (25). Next, researchers should consider the PROM's psychometric properties. Specifically, researchers should consider how and in whom the PROM was developed (28), assess if the included items are relevant to the intended population, and determine if validity, reliability, and responsiveness of the PROM has been established for the intended population (10, 15–18, 29). Too often, modifications to PROMs are made to fit the population or study (e.g., wording altered, items removed or added, scoring adjusted), which compromises the scale's validity. In circumstances when researchers modify a PROM for their study, it is important to specify the modification in the protocol and recognize the psychometric properties may be compromised (25, 27). Validation of the modified PROM is suggested. Consideration of all of these factors may help to ensure that the selected PROMs are appropriate to for the population and to address the study aims and more likely to capture changes in health over time.

Another approach to selecting a PROM that researchers may find beneficial is to seek PROM recommendations from organizations vested in sports medicine research. Typically, these organizations have used experts to vet outcomes, ensure adequate measurement properties, and encourage synergy amongst researchers in the field. For example, the American Academy of Orthopedic Surgeons (AAOS) established a registry program in 2017 in collaboration with multiple specialty societies (30). One valuable result of a registry is that it can facilitate research aimed at a specific cause, such as increased patient safety, improving patient outcomes, and promoting best practices (30). The AAOS currently has five registries, including joint replacement, fracture and trauma, musculoskeletal tumor, shoulder and elbow, and spine, that offer resources and opportunities to access or contribute to research efforts (30).

Registries, often include required and suggested data elements and administration time points (30), also referred to as common data elements (CDE). A CDE is a set of variables, including PROMs, that are recommended in research relevant to the specific condition. The idea of the CDE is that it promotes consistency across studies in the way data are collected. Greater consistency can lead to more efficient cross-study comparisons and primary analyses as well as greater ability to perform secondary analyses, such as meta-analyses or systematic reviews. Another example of a collaborative, standardized data collection effort is the National Institute of Neurological Disorders and Stroke (NINDS) who has published several CDEs (31), including two related to traumatic brain injury and sport related concussion (32, 33). Selecting PROMs based on data sets that include CDEs not only streamlines the decision-making process but also creates opportunity to collaborate and align with research at a larger scale and for greater impact. However, the establishment of registries and CDEs is relatively recent and one may not exist yet for the condition or population of interest for all those engaged in sports medicine research.

Finally, given the importance of PROM selection, the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) group has created a toolkit for sports medicine researchers interested in conducting patient-oriented research (4). The COSMIN toolkit is designed to support the selection of PROMs and provides numerous resources including a taxonomy to better define measurement properties of PROMs, a checklist to evaluate the quality of studies on measurement properties of PROMs, and a databases of studies about the measurement properties of PROMs.

Instrument administration

After an appropriate PROM is selected, it is equally important to develop a protocol for how and when PROM administration will occur to ensure alignment with the research questions while also being cognizant of the administrative burden (2, 3, 25, 34). Topics to consider when developing the PROM administration protocol for a sports medicine study include establishing standardized time points for PROM administration, training the individuals who administer the PROM, and mode of PROM administration (3, 9, 25, 34).

Establishing standardized time points for PROM administration

Decisions about when to administer a PROM is a common challenge for researchers because there are no established best practice or standardized time point(s) to guide research design. Common time points for PROM administration, which often represent the minimum required to answer research questions, are at the beginning (initial evaluation) and end (discharge) of an injury rehabilitation or prevention intervention. Comparing outcomes between these two time points enables researchers to examine patients' perceptions about the effectiveness of interventions and establish the time course of treatment for a specific condition or population. However, to ensure the research is relevant to the real-world, careful consideration should be given to aligning administration to time points that are clinically relevant to the patient population and intervention, which may benefit from the insight of patient populations (3, 35, 36). The PROM selected for use may have instructions or wording of questions that can be used to inform administration time points, such as when questions ask patients to answer based on their experiences and perceptions over a time period (e.g., the past week, the previous month), or in comparison to a prior state (e.g., pre-injury level). These reflection periods must be considered when determining administration time points.

Further, while a simple strategy is to administer at the start and end of an intervention, there is evidence to suggest that administrating PROMS at intermittent time points has a clinical benefit to patient outcomes. For example, Werneke et al. (37) used a therapeutic outcomes database to examine the time points of PROM administration that resulted in the highest functional status at discharge from rehabilitation in patients treated for non-specific lumbar impairments. Of the 140,336 patients with completed PROMs, 83,101 (59%) did so only at the beginning and end of therapy and had slightly lower functional outcomes than patients who completed interim PROMs. The administration of an interim PROM, regardless of the time point, led to higher functional status outcomes at discharge compared to no interim administration. Similarly, two or more interim PROMs resulted in better outcomes than one interim PROM. The best result in patients' functional status outcomes occurred when at least one interim PROM was administered during the first 2 weeks after initial evaluation (37). This study highlights the importance of interim time points in promoting positive patient outcomes. What is unknown, yet a relevant consideration for future outcomes researchers, is what led to the better outcomes when intermittent PROMs were used. In theory, use of intermittent PROMs provides an opportunity to interject patient voice into the care process and for clinicians to respond and possibly adjust the strategy based on patient input. However, researchers and patients are often blinded to clinical findings during data collection to reduce bias and to add a higher level of control. More discussion is needed related to navigating research methods and designs used to collect patient-reported data in clinical research and whether point-of-care approaches (38) with less blinding may be appropriate for some research questions and study designs, especially when inclusion of the patient is a key element to the research. Further, identifying a standardized set of specific administration time points that can be shared across sports medicine researchers may strengthen the body of evidence from this research community.

Individuals administering PROMs

A component of sports medicine research protocols often overlooked involves the individuals who administer the PROMs to patients. These individuals may not have an extensive research training background or may not have been involved with the development of the research design, and they have reported a lack of clarity on alignment of the PROM to the research questions and expectations for how to administer these measures (25). Individuals responsible for administering PROMs require training related to the study protocol, purpose of the PROMs, informed consent procedures, and anticipated participant questions to provide firm guidance on executing the study protocol to support participant compliance and the situation of missing or incomplete data (25). This area of the study protocol development should involve the individuals administering the PROMs as research personnel and include practical aspects of administration such as workflow adjustments, identifying the primary administrator, and designing a secondary plan to maintain continuity in PROM collection if the primary administrator is unavailable (34). Ensuring a secondary administrator is trained and available promotes consistency across the collection of PROMs and reduces the risk of missed data points when the primary administrator is unavailable (34). It is also important to anticipate ways the administrator may need to navigate supporting patient needs while also adhering to a study protocol. For example, providing common or standardized ways of explaining items would prepare an administrator to respond to patients who ask questions while completing a PROM. Providing guidance to all research personnel, particularly individuals interfacing with patients during data collection, not only improves the quality of data collected, but also promotes a positive environment for the research team, reducing stress and burden and promoting professionalism, and a participant population that is more informed about the process and potentially more likely to adhere to the research protocol (25, 34).

Mode of PROM administration

Many PROMs were developed for paper-based administration, yet technological advances lead researchers to consider using electronic administration methods (e.g., RedCap, Qualtrics, OnlinePROMs). Research efforts comparing the paper-based and electronic modes of administration of patient responses (bias) has provided strong evidence demonstrating no difference in how patients respond when they complete PROMs via paper and electronic modes (2, 25, 39). While responses through paper-based or electronic methods are similar, there are considerations about response rate and cost of administration that may be factors in any assessment strategy (36, 40). Comprehensive strategies that combine electronic automated assessment, with human support such as checking for completeness of responses, reminders through postal mail or phone calls, and fielding email and phone inquiries from participants, have led to high response rates (36, 40). The cost-benefit of implementing a comprehensive strategy warrant more study. Regardless of how and where PROMs are administered, it is recommended that consistent administration methods of the same PROM be used within a study.

PROM analysis and interpretation

One of the primary considerations with the use of PROMs in research and patient care is how to interpret and analyze scale scores to determine differences in groups and changes within patients over time. Interpretation is important to give context to the scores and to support clinical decision making. However, there are nuances around PROM score interpretation, such as response option scale and scoring, lack of data to inform individual score interpretation, confusion with change score concepts (12), and differences in methodology related to calculating change scores (11, 12, 41–45), that warrant consideration by researchers and may benefit from standardization when using these tools in sports medicine research.

When considering the formal statistical analysis of PROM data, researchers must consider the fundamental components of the PROM, just as they would with any other measurement tool. Specifically, researchers should consider the structure of response options for individual items and how they are rated by patients as well as how overall or total scores are calculated based on those ratings (17). For example, many region-specific PROMs [e.g., FAAM (46), FAST (47), LEFS (20)] ask the patient to rate each item on an ordinal scale, with those responses then converted into a total score that exists on a ratio scale (e.g., 0–100%). Awareness of these calculations is important because the scale will influence the type of statistical analysis appropriate for the data (e.g., parametric vs. non-parametric methods). Similarly, researchers should take care to ensure that PROM data meet basic assumptions required of a statistical test. For instance, previous investigations (48–50) have reported that PROMs data often demonstrate a skewed distribution and violate the basic assumption of normality. Skewed PROMs data are a particular concern in sport injury and rehabilitation research as the patient population is generally young and healthy, resulting in PROM scores skewed toward better scores or better reports of health status. The degree to which the data are skewed will influence the presentation of basic descriptive statistics, such as measure of central tendency (51). Medians and interquartile ranges are more appropriate for skewed data than are means and standard deviations (51). Further, skewed data may require non-parametric statistical tests or more complex modeling methods than data that are not skewed (52).

From a more clinically meaningful perspective, interpretation of scale scores is important for a single-point in time, such as return-to-play, and over-time, such as the time between the start of an intervention and end of an intervention, for the PROMs to have meaning in clinical practice or research (44, 53). The methods and values used to support these types of interpretation differ. With individual scores, reference or normative values from large patient populations may be helpful to gain a general understanding of whether a score is similar to those reported in a like-population (29, 44, 54), although some researchers suggest that comparisons of individuals to group level data be avoided (55). Knowing the representative values in the population of interest can give context to scores and supports interpretation. To date, there is limited information to inform score interpretation across the wide variety of instruments used in healthcare, including commonly used PROMs in sports medicine (16). Research is needed that aims to provide clinical meaning to individual scores and links scores to the characteristics of the population of interest. This type of research will help inform score interpretation and the meaning of research results. While interpretation of a score at one point in time is important, it is also essential to be able to interpret changes in scores over time.

Research that evaluates interventions is often analyzed using between group comparisons. While between group comparisons may apply to study aims, within-person or group change may better inform whether the intervention is having an impact over the longer-term. Within-person change is a fundamental characteristic with any health measurement, including PROMs, because knowledge of how scores change over-time within people and groups is important to correctly identify improvement, deterioration, or no change as a result of treatment or intervention. Values such as substantial clinical benefit (SCB) (56), minimal clinically important difference (MCID) (57) or minimal important change (MIC) (12, 53), and meaningful change threshold have been used to describe change in health status. Substantial clinical benefit is reflective of a larger or sustained patient-perceived improvement in health status whereas minimal changes reflect small but meaningful patient-perceived improvements in health status. Another concept that relates to change over time, but contributes to some confusion regarding score interpretation, is the minimal detectable change (MDC) which is a value that reflects the statistical error within a measurement and does not inform the clinical meaningfulness of a change score like SCB or MIC attempt to do (58). The MDC is valuable in determining whether change is within or outside of measurement error and is essential with all measurement tools. When interpreting research results over a period of time, researchers should determine whether their findings exceed the measurement error of the instrument (i.e., the MDC). Minimal detectable change (MDC) is based on statistical distributions and does not account for clinical factors. Therefore, efforts have been made to identify values, like SCB or MIC, that help researchers determine the clinical meaningfulness of research results.

Identifying a useful indictor of meaningful change relevant to PROMs is important because, in theory, measures of clinical meaningfulness should tie a clinical marker of health to patient perception of health. As previously mentioned, there are numerous values used to define clinically meaningfulness, and a preferred concept reflective of the smallest amount of change patients' perceive as beneficial is MIC (12, 44), although the more common term in the literature is MCID. Terwee et al. (12) emphasize how the words “change” and “minimal” are purposeful in the naming of MIC. Change helps to explain a within-person phenomena that speaks to a longitudinal assessment as opposed to the word “difference” which suggests a cross-sectional contrast between groups (12). “Minimal” highlights the value in small but meaningful changes in health status. While both longitudinal and between groups comparisons are valuable in interpreting PROMs, evaluation over time aligns with the primary purpose of healthcare which is to produce a change in patient health status, such as occurs from the transition from an injured to rehabilitated health state (59). Researchers should consider how data are reported related to the MIC. Use of proportions that reflect the number of patients in a group who exceeded MIC values is one recommendation that helps to illustrate the individual effect of an intervention within a group of patients and is easy to interpret (60, 61).

A challenge with using MIC to interpret PROMs is that given the methods of calculation, there is not a single MIC value for any PROM. Methods of calculation can include ROC analysis (12), adjusted ROC analysis (11, 41), predictive modeling (12, 45), as well as vignette-based methods (12). Factors such as severity of the health condition at baseline (11, 41), time points of intermittent PROM assessment (e.g., 1 week, 2 week, 1 month post-injury, or return-to-play), deteriorating or improving health state, choice of anchor (e.g., pain scale rating, global rating of change score, SANE rating), and definition of “minimal” are all likely to influence the resulting values calculated (13). Researchers who calculate MIC values should be detailed in their methods for increased transparency and with a goal of use in appropriate populations. However, the numerous ways to calculate meaningful change and the various factors that influence the calculation have called into question whether the MIC is as helpful in interpreting change in health status as it was initially thought to be (11, 41, 42). Future efforts by leaders in sports medicine research are needed to provide guidance on best practices related to calculating meaningful change scores. While the ROC method has been appealing given the inclusion of a clinical anchor and the ease of interpretation, efforts to improve on this methodology, such as with the adjusted ROC method, are promising and may be a step toward advancing the science of PROM interpretation (11, 41).

Finally, while efforts are ongoing to increase the interpretability of the change in PROMs scores over time, there are outcome measures about “health state” that are used as primary outcomes in sports medicine research (62–65). For example, the Patient Acceptable Symptom State (PASS) (63, 64, 66) gathers patient perspective about the satisfaction a patient has with their current health state, considering an acceptable level of symptoms, activities of daily living, and function that results in a satisfactory assessment of health. There are various version of the PASS, but most use a dichotomous response scale of “yes” or “no.” Describing the level of satisfaction a patient has with their state of health at the end of care or at the end of participation in a research study warrants consideration as a clinically meaningful research outcome (65, 67).

Discussion

The future of sports medicine research is promising given the increased number of manuscripts with attention toward PROMs and the investigation in questions with patients at the center. The significant increase in attention toward person-centered variables such as quality of life, and efforts to evaluate health from a disablement model perspective, including evaluation of the impact of health conditions on social roles and environmental perspectives demonstrates the interest in conducting research with the patient at the center. Refinement of practices related to instrument selection, instrument administration, and data analysis and interpretation may support continued advancement in the quality, meaningfulness, and strength of the evidence used to promote the health of populations involved in sports medicine research (Figure 1). Additionally, increased awareness, collaboration, and adherence to national and international recommendations may support continued elevation of the quality of the science related to patient-oriented evidence (2, 4, 68). The National Institutes of Health (NIH), PCORI, and COSMIN group are three examples of entities that have prioritized patient-voice in research. Through their work, numerous lessons have been learned and advice and resources shared in an effort to improve collaborations and the quality of research using PROMs (Table 2) (2, 4, 68). Collectively, these leaders in healthcare have landed on similar recommendations, many of which highlight the critical importance of selecting PROMs and ensuring that they answer questions important to patients, that the assessment tools are administered within their limits, and that the outcomes have relevance and value to patients, clinicians, and researchers.

FIGURE 1

Figure 1. Considerations for instrument selection, administration, and analysis and interpretation for research using patient-reported outcome measures.

TABLE 2

Table 2. Resources for using PROMs in sports medicine research.

The notion of appropriateness of a PROM is a critical factor in advancing sports medicine research. Taking the time to select a PROM that has high relevance and meaningfulness to patients and clinicians may produce benefits to the success of the research. Using PROMs that are valid, reliable, and responsive to changes in patient care while also being of high relevance to patients and clinicians may strengthen adherence to the research protocol and the quality of the data collected. Over the last 20 years, there has been a significant increase in the body or research that include patient perspective and we look forward to the impact this research will have on the lives of patients.

Author contributions

AS, KH, and TV contributed to conception and design of the study and wrote the first draft of the manuscript. AS, KH, KL, and TV wrote sections of the manuscript. All authors contributed to manuscript revision, read, and approved the submitted version.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

1. New England Journal Medicine Catalyst,. What Is Patient-Centered Care? (2017). Available online at: https://catalyst.nejm.org/doi/full/10.1056/CAT.17.0559 (accessed September 20, 2022).

2. Bingham CO, Bartlett SJ, Merkel PA, Mielenz TJ, Pilkonis PA, Edmundson L, et al. Using patient-reported outcomes and promis in research and clinical applications: experiences from the pcori pilot projects. Qual Life Res. (2016) 25:2109–16. doi: 10.1007/s11136-016-1246-1

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Keeney T, Kumar A, Erler KS, and Karmarkar AM. Making the case for patient-reported outcome measures in big-data rehabilitation research: implications for optimizing patient-centered care. Arch Phys Med Rehabil. (2022) 103:S140–5. doi: 10.1016/j.apmr.2020.12.028

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Mokkink LB, Prinsen CA, Bouter LM, Vet HC, and Terwee CB. The consensus-based standards for the selection of health measurement instruments (cosmin) and how to select an outcome measurement instrument. Braz J Phys Ther. (2016) 20:105–13. doi: 10.1590/bjpt-rbf.2014.0143

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Lehman BJ, David DM, and Gruber JA. Rethinking the biopsychosocial model of health: understanding health as a dynamic system. Soc Personal Psychol Compass. (2017) 11:e12328. doi: 10.1111/spc3.12328

CrossRef Full Text | Google Scholar

6. Snyder AR, Parsons JT, Valovich McLeod TC, Curtis Bay R, Michener LA, and Sauers EL. Using disablement models and clinical outcomes assessment to enable evidence-based athletic training practice, Part I: Disablement models. J Athl Train. (2008) 43:428–36. doi: 10.4085/1062-6050-43.4.428

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Hansen CF, Jensen J, Brodersen J, Siersma V, Comins JD, and Krogsgaard MR. Are adequate proms used as outcomes in randomized controlled trials? An analysis of 54 trials. Scand J Med Sci Sports. (2021) 31:972–81. doi: 10.1111/sms.13896

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Jensen J, Hansen CF, Brodersen J, Comins JD, and Krogsgaard MR. Are proms used adequately in sports research? An analysis of 54 randomized controlled trials with proms as endpoint. Scand J Med Sci Sports. (2021) 31:982–90. doi: 10.1111/sms.13880

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Krogsgaard MR, Brodersen J, Jensen J, Hansen CF, and Comins JD. Potential problems in the use of patient reported outcome measures (proms) and reporting of prom data in sports science. Scand J Med Sci Sports. (2021) 31:1249–58. doi: 10.1111/sms.13888

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Hansen CF, Jensen J, Siersma V, Brodersen J, Comins JD, and Krogsgaard MR. A catalogue of proms in sports science: quality assessment of prom development and validation. Scand J Med Sci Sports. (2021) 31:991–8. doi: 10.1111/sms.13923

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Tenan MS, Simon JE, Robins RJ, Lee I, Sheean AJ, and Dickens JF. Anchored minimal clinically important difference metrics: considerations for bias and regression to the mean. J Athl Train. (2021) 56:1042–9. doi: 10.4085/1062-6050-0368.20

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Terwee CB, Peipert JD, Chapman R, Lai JS, Terluin B, Cella D, et al. Minimal important change (Mic): A conceptual clarification and systematic review of Mic estimates of promis measures. Qual Life Res. (2021) 30:2729–54. doi: 10.1007/s11136-021-02925-y

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Terwee CB, Roorda LD, Dekker J, Bierma-Zeinstra SM, Peat G, Jordan KP, et al. Mind the Mic: Large variation among populations and methods. J Clin Epidemiol. (2010) 63:524–34. doi: 10.1016/j.jclinepi.2009.08.010

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Wright A, Hannon J, Hegedus EJ, and Kavchak AE. Clinimetrics corner: A closer look at the minimal clinically important difference (Mcid). J Man Manip Ther. (2012) 20:160–6. doi: 10.1179/2042618612Y.0000000001

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Hansen CF, Obionu KC, Comins JD, and Krogsgaard MR. Patient reported outcome measures for ankle instability. An analysis of 17 existing questionnaires. Foot Ankle Surg. (2022) 28:288–93. doi: 10.1016/j.fas.2021.04.009

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Lam KC, Harrington KM, Cameron KL, and Snyder Valier AR. Use of patient-reported outcome measures in athletic training: common measures, selection considerations, and practical barriers. J Athl Train. (2019) 54:449–58. doi: 10.4085/1062-6050-108-17

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Lam KC, Marshall AN, and Snyder Valier AR. Patient-reported outcome measures in sports medicine: a concise resource for clinicians and researchers. J Athl Train. (2020) 55:390–408. doi: 10.4085/1062-6050-171-19

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Marshall AN, Root HJ, Valovich McLeod TC, and Lam KC. Patient-reported outcome measures for pediatric patients with sport-related injuries: a systematic review. J Athl Train. (2022) 57:371–84. doi: 10.4085/1062-6050-0598.20

PubMed Abstract | CrossRef Full Text | Google Scholar

19. O'Connor CM, and Ring D. Correlation of single assessment numeric evaluation (Sane) with other patient reported outcome measures (Proms). Arch Bone Jt Surg. (2019) 7:303–6.

PubMed Abstract | Google Scholar

20. Binkley JM, Stratford PW, Lott SA, and Riddle DL. The lower extremity functional scale (Lefs): Scale development, measurement properties, and clinical application. North American orthopaedic rehabilitation research network. Phys Ther. (1999) 79:371–83. doi: 10.1037/t35109-000

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Snyder AR, and Valovich McLeod TC. Selecting patient-based outcome measures. Athlet Ther Today. (2007) 12:12–5. doi: 10.1123/att.12.6.12

CrossRef Full Text | Google Scholar

22. Calvert M, Kyte D, Mercieca-Bebber R, Slade A, Chan AW, King MT, et al. Guidelines for inclusion of patient-reported outcomes in clinical trial protocols: the spirit-pro extension. JAMA. (2018) 319:483–94. doi: 10.1001/jama.2017.21903

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Equator Network,. Enhancing the Quality Transparency of Health Research. (2022). Available online at: https://www.equator-network.org/ (accessed September 20, 2022).

24. Luckett T, and King MT. Choosing patient-reported outcome measures for cancer clinical research–practical principles and an algorithm to assist non-specialist researchers. Eur J Cancer. (2010) 46:3149–57. doi: 10.1016/j.ejca.2010.08.002

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Mercieca-Bebber R, King MT, Calvert MJ, Stockler MR, and Friedlander M. The importance of patient-reported outcomes in clinical trials and strategies for future optimization. Patient Relat Outcome Meas. (2018) 9:353–67. doi: 10.2147/PROM.S156279

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Comins JD, Brodersen J, Siersma V, Jensen J, Hansen CF, and Krogsgaard MR. Choosing the most appropriate prom for clinical studies in sports medicine. Scand J Med Sci Sports. (2021) 31:1209–15. doi: 10.1111/sms.13906

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Snyder CF, Watson ME, Jackson JD, Cella D, Halyard MY, and Mayo FDAP-ROCMG. Patient-reported outcome instrument selection: designing a measurement strategy. Value Health. (2007) 10 Suppl 2:S76–85. doi: 10.1111/j.1524-4733.2007.00270.x

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Reeve BB, Wyrwich KW, Wu AW, Velikova G, Terwee CB, Snyder CF, et al. Isoqol recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res. (2013) 22:1889–905. doi: 10.1007/s11136-012-0344-y

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Snyder Valier AR, Welch Bacon CE, Bay RC, Molzen E, Lam KC, and Valovich McLeod TC. Reference values for the pediatric quality of life inventory and the multidimensional fatigue scale in adolescent athletes by sport and sex. Am J Sports Med. (2017) 45:2723–9. doi: 10.1177/0363546517714471

PubMed Abstract | CrossRef Full Text | Google Scholar

30. American Academy of Orththopaedic Surgeons. About the Aaos Registry Program. (2022). Available online at: https://www.aaos.org/registries/registry-program/about-the-aaos-registry-program/ (accessed September 20, 2022).

31. National Institute of Neurological Disorders Stroke. Ninds Common Data Elements. (2022). Available online at: https://www.commondataelements.ninds.nih.gov/ (accessed September 20, 2022).

32. Broglio SP, Kontos AP, Levin H, Schneider K, Wilde EA, Cantu RC, et al. National institute of neurological disorders and stroke and department of defense sport-related concussion common data elements version 1.0 recommendations. J Neurotrauma. (2018) 35:2776–83. doi: 10.1089/neu.2018.5643

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Wilde EA, Whiteneck GG, Bogner J, Bushnik T, Cifu DX, Dikmen S, et al. Recommendations for the use of common outcome measures in traumatic brain injury research. Arch Phys Med Rehabil. (2010) 91:1650–60.e17. doi: 10.1016/j.apmr.2010.06.033

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Mercieca-Bebber R, Calvert M, Kyte D, Stockler M, and King MT. The administration of patient-reported outcome questionnaires in cancer trials: interviews with trial coordinators regarding their roles, experiences, challenges and training. Contemp Clin Trials Commun. (2018) 9:23–32. doi: 10.1016/j.conctc.2017.11.009

PubMed Abstract | CrossRef Full Text | Google Scholar

35. McNeill M, Noyek S, Engeda E, and Fayed N. Assessing the engagement of children and families in selecting patient-reported outcomes (Pros) and developing their measures: a systematic review. Qual Life Res. (2021) 30:983–95. doi: 10.1007/s11136-020-02690-4

PubMed Abstract | CrossRef Full Text | Google Scholar

36. OME Cleveland Clinic Orthopaedics. Implementing a scientifically valid, cost-effective, and scalable data collection system at point of care: The Cleveland Clinic Ome Cohort. J Bone Joint Surg Am. (2019) 101:458–64. doi: 10.2106/JBJS.18.00767

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Werneke MW, Deutscher D, Fritz J, Kallen MA, Cook KF, Hayes D, et al. Associations between interim patient-reported outcome measures and functional status at discharge from rehabilitation for non-specific lumbar impairments. Qual Life Res. (2020) 29:439–51. doi: 10.1007/s11136-019-02314-6

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Lam KC, Bacon CEW, Sauers EL, and Bay RC. Point-of-care clinical trials in sports medicine research: identifying effective treatment interventions through comparative effectiveness research. J Athl Train. (2020) 55:217–28. doi: 10.4085/1062-6050-307-18

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Rutherford C, Costa D, Mercieca-Bebber R, Rice H, Gabb L, and King M. Mode of administration does not cause bias in patient-reported outcome results: a meta-analysis. Qual Life Res. (2016) 25:559–74. doi: 10.1007/s11136-015-1110-8

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Pronk Y, Pilot P, Brinkman JM, van Heerwaarden RJ, and van der Weegen W. Response rate and costs for automated patient-reported outcomes collection alone compared to combined automated and manual collection. J Patient Rep Outcomes. (2019) 3:31. doi: 10.1186/s41687-019-0121-6

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Boyer CW, Lee IE, and Tenan MS. All mcids are wrong, but some may be useful. J Orthop Sports Phys Ther. (2022) 52:401–7. doi: 10.2519/jospt.2022.11193

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Cook CE. Clinimetrics corner: the minimal clinically important change score (Mcid): A necessary pretense. J Man Manip Ther. (2008) 16:E82–3. doi: 10.1179/jmt.2008.16.4.82E

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Coon CD, and Cook KF. Moving from significance to real-world meaning: methods for interpreting change in clinical outcome assessment scores. Qual Life Res. (2018) 27:33–40. doi: 10.1007/s11136-017-1616-3

PubMed Abstract | CrossRef Full Text | Google Scholar

44. de Vet HC, Terwee CB, Mokkink LB, and Knol DL. Measurement in Medicine: A Practical Guide. Cambridge: Cambridge (2013).

Google Scholar

45. Terluin B, Eekhout I, Terwee CB, and de Vet HC. Minimal important change (Mic) based on a predictive modeling approach was more precise than mic based on roc analysis. J Clin Epidemiol. (2015) 68:1388–96. doi: 10.1016/j.jclinepi.2015.03.015

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Martin RL, Irrgang JJ, Burdett RG, Conti SF, and Van Swearingen JM. Evidence of validity for the foot and ankle ability measure (Faam). Foot Ankle Int. (2005) 26:968–83. doi: 10.1177/107110070502601113

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Huxel Bliven KC, Snyder Valier AR, Bay RC, and Sauers EL. The functional arm scale for throwers (Fast)-Part Ii: Reliability and validity of an upper extremity region-specific and population-specific patient-reported outcome scale for throwing athletes. Orthop J Sports Med. (2017) 5:2325967117700019. doi: 10.1177/2325967117700019

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Hsu JE, Nacke E, Park MJ, Sennett BJ, and Huffman GR. The disabilities of the arm, shoulder, and hand questionnaire in intercollegiate athletes: validity limited by ceiling effect. J Shoulder Elbow Surg. (2010) 19:349–54. doi: 10.1016/j.jse.2009.11.006

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Lam KC, and Markbreiter JG. The impact of knee injury history on health-related quality of life in adolescent athletes. J Sport Rehabil. (2019) 28:115–9. doi: 10.1123/jsr.2017-0226

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Matheny LM, Gittner K, Harding J, and Clanton TO. Patient reported outcome measures in the foot and ankle: normative values do not reflect 100% full function. Knee Surg Sports Traumatol Arthrosc. (2021) 29:1276–83. doi: 10.1007/s00167-020-06069-3

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Manikandan S. Measures of central tendency: median and mode. J Pharmacol Pharmacother. (2011) 2:214–5. doi: 10.4103/0976-500X.83300

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Schober P, and Vetter TR. Nonparametric statistical methods in medical research. Anesth Analg. (2020) 131:1862–3. doi: 10.1213/ANE.0000000000005101

PubMed Abstract | CrossRef Full Text | Google Scholar

53. van der Willik EM, Terwee CB, Bos WJW, Hemmelder MH, Jager KJ, Zoccali C, et al. Patient-reported outcome measures (Proms): Making sense of individual prom scores and changes in prom scores over time. Nephrology (Carlton). (2021) 26:391–9. doi: 10.1111/nep.13843

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Terwee CB, Bot SD, de Boer MR, van der Windt DA, Knol DL, Dekker J, et al. Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol. (2007) 60:34–42. doi: 10.1016/j.jclinepi.2006.03.012

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Cook CE, Wright A, Wittstein J, Barbero M, and Tousignant-Laflamme Y. Five recommendations to address the limitations of patient-reported outcome measures. J Orthop Sports Phys Ther. (2021) 51:562–5. doi: 10.2519/jospt.2021.10836

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Michener LA, Snyder Valier AR, and McClure PW. Defining substantial clinical benefit for patient-rated outcome tools for shoulder impingement syndrome. Arch Phys Med Rehabil. (2013) 94:725–30. doi: 10.1016/j.apmr.2012.11.011

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Jaeschke R, Singer J, and Guyatt GH. Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials. (1989) 10:407–15. doi: 10.1016/0197-2456(89)90005-6

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Michener LA. Patient- and clinician-rated outcome measures for clinical decision making in rehabilitation. J Sport Rehabil. (2011) 20:37–45. doi: 10.1123/jsr.20.1.37

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Streiner DL, and Norman GR. Health Measurement Scales: A Practical Guide to Their Development and Use. Oxford: Oxford University Press (2008).

PubMed Abstract | Google Scholar

60. Haley SM, and Fragala-Pinkham MA. Interpreting change scores of tests and measures used in physical therapy. Phys Ther. (2006) 86:735–43. doi: 10.1093/ptj/86.5.735

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Schmitt JS, and Di Fabio RP. Reliable change and minimum important difference (Mid) proportions facilitated group responsiveness comparisons using individual threshold criteria. J Clin Epidemiol. (2004) 57:1008–18. doi: 10.1016/j.jclinepi.2004.02.007

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Kvien TK, Heiberg T, and Hagen KB. Minimal clinically important improvement/difference (Mcii/Mcid) and patient acceptable symptom state (pass): what do these concepts mean? Ann Rheum Dis. (2007) 66 Suppl 3:iii40–1. doi: 10.1136/ard.2007.079798

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Su F, Allahabadi S, Bongbong DN, Feeley BT, and Lansdown DA. Minimal clinically important difference, substantial clinical benefit, and patient acceptable symptom state of outcome measures relating to shoulder pathology and surgery: a systematic review. Curr Rev Musculoskelet Med. (2021) 14:27–46. doi: 10.1007/s12178-020-09684-2

PubMed Abstract | CrossRef Full Text | Google Scholar

64. Tubach F, Ravaud P, Baron G, Falissard B, Logeart I, Bellamy N, et al. Evaluation of clinically relevant states in patient reported outcomes in knee and hip osteoarthritis: the patient acceptable symptom state. Ann Rheum Dis. (2005) 64:34–7. doi: 10.1136/ard.2004.023028

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Walsh JM, Huddleston HP, Alzein MM, Wong SE, Forsythe B, Verma NN, et al. The minimal clinically important difference, substantial clinical benefit, and patient-acceptable symptomatic state after medial patellofemoral ligament reconstruction. Arthrosc Sports Med Rehabil. (2022) 4:e661–e78. doi: 10.1016/j.asmr.2021.12.009

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Maksymowych WP, Richardson R, Mallon C, van der Heijde D, and Boonen A. Evaluation and validation of the patient acceptable symptom state (pass) in patients with ankylosing spondylitis. Arthritis Rheum. (2007) 57:133–9. doi: 10.1002/art.22469

PubMed Abstract | CrossRef Full Text | Google Scholar

67. Gordon D, Pines Y, Ben-Ari E, As R, Kwon YW, Zuckerman JD, et al. Minimal clinically important difference, substantial clinical benefit, and patient acceptable symptom state of promis upper extremity after total shoulder arthroplasty. JSES Int. (2021) 5:894–9. doi: 10.1016/j.jseint.2021.05.003

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Bennett AV, Jonsson M, Chen RC, Al-Khatib SM, Weinfurt KP, and Curtis LH. Applying patient-reported outcome methodology to capture patient-reported health data: report from an Nih collaboratory roundtable. Healthc (Amst). (2020) 8:100442. doi: 10.1016/j.hjdsi.2020.100442

PubMed Abstract | CrossRef Full Text | Google Scholar