Development of the Korean Medicine Core Outcome Set for Stroke Sequelae: Herbal Medicine Treatment of Elderly Patients With Stroke Sequelae in Primary Clinics

Objectives: We developed a Korean medicine core outcome set for stroke sequelae (COS-SS-KM) to evaluate the effectiveness and safety of herbal medicine (HM) for stroke sequelae, especially for elderly stroke patients in primary clinics.

Methods: We identified previously reported outcomes from a literature review and defined the list of outcomes and effect modifiers for the core outcome set (COS) questionnaire. Three rounds of modified Delphi consensus exercises with experts were conducted online for suitability assessment, and one round of a modified Delphi consensus exercise with primary clinicians was conducted for feasibility assessment.

Results: The review identified 17 outcomes and 16 effect modifiers; moreover, six outcomes and one effect modifier were suggested by the experts. The final COS comprised 8 outcomes and 12 effect modifiers for history taking, and experts listed 13 major symptoms of stroke sequelae for symptom assessment. The clinicians agreed on the feasibility of the COS.

Conclusion: This COS will help primary care researchers assess the effectiveness of pharmacotherapy, including HM, for elderly patients with stroke sequelae. Future studies should focus on reflecting the opinions of all stakeholders.

1 Introduction

Stroke-induced neurological injury leads to multiple sequelae throughout life. These post-stroke symptoms known as a result of the tissue injury are not easily resolved by time (Becker, 2016; De Doncker et al., 2018). A systematic review (SR) showed that 74% of patients with stroke have at least one unmet need after hospital discharge, including cognitive problems, emotional/mood problems, fatigue, and pain (Chen et al., 2019). Comprehensive sequelae of stroke lead patients to experience decreased quality of life and increased risk of additional hospitalization (Naess and Nyland, 2021).

Herbal medicine (HM) could have potential benefits on post-stroke symptoms, including aphasia and depression (Zeng et al., 2017, Zhang et al., 2018). A network meta-analysis showed that HM was effective for recovery after stroke (Han et al., 2017). A recent review reported that HM natural compounds including glycosides, flavonoids, phenols, and terpenes exhibited neuroprotective/neurorestorative effect against stroke (Zhu et al., 2022). For instance, Astragaloside IV from Astragalus membranaceus has been revealed to reduce neuronal apoptosis and infarct volume, leading to decreased neurological deficits (Zhang et al., 2019; Yin et al., 2020). Activation of neurogenesis-associated signaling pathways and down-regulating inflammation and apoptosis pathways are revealed as key mechanisms of HM in stroke sequelae treatment (Shen et al., 2015).

In Korea, HM based on Korean Medicine (KM) is the preferred primary care treatment for patients with stroke sequelae. However, studies on its therapeutic effect are still lacking. Despite their value in reflecting the realities of community care settings, clinical trials in primary or KM clinics have limitations (Lindbloom et al., 2004). Lack of manpower and equipment limits researchers in primary or KM clinics from conducting studies with the same research design as those in academic hospitals. This leads to low-quality studies investigating inadequate outcomes, thereby reducing the reliability of the studies’ conclusions.

In 2020, the Korean government launched a pilot project that aimed to support herbal medicinal treatment for patients (aged >65 years) with stroke sequelae in primary clinics. Healthcare insurance supports the cost of 10 days of HMs that meet the government’s standards of diagnosis and quality (No, 2020). However, large-scale pilot studies or standard outcome models for KM treatment have not yet been proposed.

This study aimed to develop a Korean medicine core outcome set for stroke sequelae (COS-SS-KM), to evaluate the safety and effectiveness of herbal KM treatments for stroke sequelae in primary clinics. A core outcome set (COS) refers to an agreed-upon, standardized set of outcomes recommended by the Core Outcome Measures for Effectiveness Trials (COMET) initiative. Adopting a suitable COS enhances the quality of clinical research (Williamson et al., 2012). A literature review and modified Delphi exercises with stakeholders were conducted to develop the COS.

2 Methods

We developed a COS with relevant scope and rationale for the aforementioned purpose (Table 1). In addition, effect modifiers were extracted to provide items for patient history-taking guidelines. The COS development process was guided by the COMET initiative and reported based on the COS standard guidelines (Cadogan et al., 2015; Kirkham et al., 2016; Rankin et al., 2018). COS development was conducted in three phases: Phase 1 (generation and refinement of a comprehensive list of outcomes and effect modifiers from a literature review), Phase 2 (modified Delphi consensus exercise by experts), and Phase 3 (modified Delphi consensus exercise by primary clinicians).

TABLE 1

TABLE 1. Recommendations of the Core Outcome Set Standards for Development (COS-STAD).

2.1 Phase 1: Generating and Refining a Comprehensive List of Outcomes and Effect Modifiers

In this phase, a project management group (PMG) was selected, and a comprehensive list of outcomes and effect modifiers were identified for the Delphi exercise in Phase 2. The PMG was composed of five researchers of the Korea Institute of Oriental Medicine (JC: literature review, outcome/effect-modifier extraction, removal of duplicated results; SK: literature review, outcome/effect-modifier extraction; PWK, HL, and SJ: result review). Prior to the literature review, the PMG clarified the scope of the COS.

First, a literature review was conducted to identify published SRs and randomized controlled trials (RCTs). One researcher (JC) conducted search on an English database [Medline (via PubMed)] and a Korean database (the Oriental Medicine Advanced Searching Integrated System). The search was on studies published from 2016 to 2020 and there was no restriction on language, publication country, or status. The following terms were used when searching PubMed: 1) (Traditional Chinese Medicine OR herbal medicine) AND geriatrics AND [systematicreview (Filter)], 2) [Traditional Chinese Medicine (MeSH) OR herbal medicine (MeSH)] AND geriatrics AND [systematicreview (Filter)] (Supplementary Material S1).

From the searched list, a researcher included SRs that contained trials of HM for geriatrics with stroke. Two researchers (JC and SK) independently assessed whether the searched articles met the inclusion criteria and hand-searched reference lists of the included SRs to identify the original RCTs and measured outcomes. They extracted the inclusion criteria, target symptoms, and outcomes from the included studies. In case of disagreement, they resolved it through discussions.

Second, two researchers examined the recommended outcomes and effect modifiers for stroke sequelae by reviewing books related to stroke and neurologic rehabilitation, such as “Stroke” (Korea Stroke Society, 2015), “Cardiovascular and Neurological Medicine in Korean Medicine II” (AKMPCNM, 2018), and “Stroke Rehabilitation” (Gillen, 2016). Third, two researchers examined outcomes for KM primary clinics by reviewing related articles and reports (Park, 2003; Lee, 2017). One researcher screened the extracted outcomes and removed duplicated studies and outcomes. Subsequently, the PMG reviewed the list and removed outcomes outside the scope of this COS to produce the final list.

2.2 Phase 2: Modified Delphi Consensus Exercise With Experts

The outcomes and effect modifiers derived from Phase 1 were condensed through three rounds of modified Delphi consensus exercises by the expert panel.

2.2.1 Recruitment of the Delphi Panel

Potential participants were sought through official contact by the PMG with the Society of Stroke on Korean Medicine, which recommended members for inclusion in the panel. Inclusion criteria for potential expert participants were as follows: being a professor of KM at a university, or being a clinician in a KM hospital specializing in the care of elderly patients with stroke.

Seven potential participants that met the criteria were recommended. The PMG introduced the aim and scope of the COS via email and invited them to partake in the modified Delphi consensus exercise. All of them replied to the email and participated in the Delphi consensus.

2.2.2 The Modified Delphi Questionnaire

This study included questionnaires (distributed from November to December 2020) with group feedback and meetings for consensus. The group feedback was embedded in subsequent questionnaires to inform respondents about the views of all participants. Each questionnaire along with the study’s background information was distributed via email. Participants had 1 week to complete each round.

In Round 1, participants received a list of outcomes and effect modifiers identified in Phase 1 and were asked to rate the importance of each. To avoid the tendency of choosing the middle, the PMG modified the Grading of Recommendations Assessment, Development and Evaluation working group scale recommended by the COMET guidelines to a 4-point numerical scale. Scores of 1, 2, 3, and 4 represented outcomes or effect modifiers of “rarely important,” “sometimes important,” “important but not critical,” and “critical,” respectively. During Round 1, participants were allowed to suggest additional outcomes or effect modifiers not presented in the list as well as ideas for optimizing the COS for KM primary care. Regarding the analysis of Round 1, the score distribution was calculated (i.e., the percentage of participants that rated each outcome and effect modifier at each score). Suggested outcomes or effect modifiers were reviewed by the PMG members to remove those beyond the scope of the COS.

Round 2 and Round 3 were conducted through videoconferencing. In Round 2, the panel defined the characteristics of the optimal outcomes for KM primary care and selected the final outcome candidates and effect modifiers. Round 2 questionnaires comprised all outcomes and effect modifiers included in Round 1, as well as suggested outcomes, effect modifiers, and ideas put forth by the panel in Round 1. Moreover, each outcome and effect modifier was re-discussed while considering the group’s responses and their previous responses. Final COS candidates included only those on which all participants had consensually agreed.

In Round 3, the participants reviewed the results of the previous round and discussed the outcomes or effect modifiers without reaching a consensus. Subsequently, they reviewed all outcomes and effect modifiers to determine those suitable for the scope of the COS. Discussion continued until a consensus was reached regarding the COS.

2.3 Phase 3: Modified Delphi Consensus Exercise With Primary Clinicians

The feasibility of derived outcomes and effect modifiers were reviewed through one round of modified Delphi consensus exercise by the primary clinicians, one of the key stakeholders.

2.3.1 Recruitment of the Delphi Panel

Potential participants were sought through official contact by the PMG with the Korean Medicine Specialist Association, and panelists were recruited among their members. All recommended participants were KM primary clinicians with at least 5 years of clinical experience.

Potential participants were initially contacted via an email that introduced the aim and scope of the COS and invited them to participate in the modified Delphi consensus exercise. All potential participants replied to the email. Eleven clinicians were selected to participate, of which six specialized in stroke and geriatric diseases.

2.3.2 Modified Delphi Questionnaire

We asked the clinicians to review the derived outcomes and modifiers and assess their feasibility in terms of collection in KM primary clinics. The questionnaire comprised five sections, as follows: 1) effect modifiers for general health, 2) effect modifiers in terms of stroke history, 3) modified Measure Yourself Medical Outcome Profile 2 (MYMOP2), 4) patient satisfaction, 5) and the five-level EuroQol 5-D (EQ-5D-5L), which includes the EuroQol Visual Analogue Scale (EQ-VAS). The clinicians responded to each question on a 9-point scale.

2.4 Data Analysis

2.4.1 Phase 2: Modified Delphi Consensus Exercise With Experts

The criteria for consensus in the COS were decided a priori as unanimity in Round 3. Consensus regarding the inclusion of an outcome in the COS (“consensus in”) was reached after Round 2 or Round 3 when a unanimous decision was reached. All other scores were classified as “no consensus;” such outcomes were considered ineligible for inclusion in the COS (Rankin et al., 2018).

2.4.2 Phase 3: Modified Delphi Consensus Exercise With Primary Clinicians

As a primary outcome, the content validity ratio (CVR) proposed by Lawshe (1975) was used to measure content validity and consensus formation. The CVR was calculated as follows:

$\mathrm{C}\mathrm{V}\mathrm{R}=\frac{N_e-\left(\frac{N}{2}\right)}{\frac{N}{2}}$

where N_e is the number of experts who select 6–9 points, and N is the total number of experts.

A consensus was considered to have been reached if the CVR was >0.5, and the CVR critical value (the minimum CVR value at which the CVR can be considered to not to be consensual by chance) was considered to exclude accidental results and to determine how many panel members must agree to decide whether an item is essential. According to an existing study, the CVR critical value suitable for 11 Delphi panelists—the number in this study—was ≥0.636 (Ayre and Scally, 2014).

As secondary outcomes, the degree of consensus and degree of convergence were used to indicate the degree to which a consensus was reached, supplementing the CVR.

The degree of consensus was calculated as follows:

$\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{u}\mathrm{s}=1- \frac{Q_3-Q_1 }{Median},Q1:\mathbf{1}\mathrm{s}\mathrm{t}\mathrm{q}\mathrm{u}\mathrm{a}\mathrm{r}\mathrm{t}\mathrm{i}\mathrm{l}\mathrm{e},Q3:\mathbf{3}\mathrm{r}\mathrm{d}\mathrm{q}\mathrm{u}\mathrm{a}\mathrm{r}\mathrm{t}\mathrm{i}\mathrm{l}\mathrm{e}$

The degree of convergence was calculated as follows:

$\mathrm{c}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{e}\mathrm{n}\mathrm{s}\mathrm{u}\mathrm{s}=\frac{Q_3-Q_1 }{2}$

In our study, when the degree of consensus was ≥0.75 and the degree of convergence was ≤0.5, agreement among panelists was considered to be achieved (Jang and Kwon, 2014).

2.5 Ethics and Consent

The Institutional Review Board of the Korea Institute of Oriental Medicine, Daejeon, Republic of Korea, provided an exemption for ethical approval. The participants provided their written informed consent to participate in this study.

3 Results

3.1 Phase 1: Generating and Refining a Comprehensive List of Outcomes and Effect Modifiers

During the literature review, 56 studies were searched from English and Korean databases. Three duplicated studies and 52 studies outside the inclusion criteria were excluded. The final selected one SR (Takayama and Iwasaki, 2017) included 5 RCTs (Supplementary Figure S1 and Supplementary Table S1). Furthermore, this SR (Takayama and Iwasaki, 2017) referred to the information about previous published SR (Siddiqui et al., 2013) which included 4 eligible RCTs (Supplementary Figure S1 and Supplementary Table S1). Therefore, a total of 9 RCTs was analyzed as listed in Supplementary Table S1.

The outcomes and detailed trial information were extracted from the 9 RCTs. In the PMG review, eight RCTs in which an inclusion criterion was stroke onset within 6 months were excluded, and outcomes from those trials were also excluded. Two outcomes (constipation scoring system and gas volume score) irrelevant to stroke were excluded. In total, five outcomes were included during the PMG review. During the review of books (Korea Stroke Society, 2015; Gillen, 2016; AKMPCNM, 2018), 11 outcomes and 16 effect modifiers related to stroke sequelae were extracted. All of these were included during the PMG review. From the articles on the development of appropriate outcomes for KM primary (Park, 2003; Lee, 2017), one outcome was extracted. This outcome was included during the PMG review.

In Phase 1, as a result, 17 outcomes and 16 effect modifiers were included in the consensus exercise (Phase 2). During the PMG review, the wording of these outcomes and effect modifiers were modified, as appropriate.

3.2 Phase 2: Modified Delphi Consensus Exercise With Experts

Seven experts in the fields of stroke and KM with ≥10 years of clinical experience were included in the Delphi panel. All participants responded to Round 1 (100%) and suggested a rationale for the COS (Table 1); moreover, the participants suggested six outcomes and one effect modifier.

Each additional outcome was discussed by the PMG for consideration for inclusion in Round 2. The Round 2 questionnaire comprised all additional outcomes and modifiers, along with all outcomes and modifiers included in Round 1. All respondents to Round 1 were invited to participate in Round 2, of whom four participated. In Round 2, five outcomes and six effect modifiers that were unanimously agreed on for inclusion were determined to be “consensus in,” and there was agreement that the items with no consensus would be revisited in Round 3. Accordingly, round 3 questionnaires comprised outcomes and effect modifiers without a reached consensus. As in Round 2, all respondents to Round 1 were invited to participate in Round 3, of whom four participated. Three outcomes and six effect modifiers were determined to be “consensus in.” In addition, experts suggested offering a supplementary list of major stroke sequelae symptoms to help patients answer the modified MYMOP2. A total of 18 symptoms were suggested, of which 13 were agreed upon for inclusion. Finally, eight outcomes and 12 effect modifiers met the consensus inclusion criteria for the COS. Table 2 and Figure 1 present the process involved for each round.

TABLE 2

TABLE 2. Summary of the consensus process during the three Delphi questionnaires.

FIGURE 1

FIGURE 1. Overview of the development of the Korean Medicine Core Outcome Set for Stroke Sequelae for (COS-SS-KM). The expert panels reached a consensus regarding the core outcome set through three rounds. In Round 2 and Round 3, the inclusion criteria for each outcome were unanimous. AST, Aspartate Transminase; ALT; Alanine Transaminase; BUN, Blood Urea Nitrogen; Cr, Creatinine; MYMOP2, Measure Yourself Medical Outcome Profile 2; TC, Total Cholesterol; TG, Triglyceride; LDL, Low Density Lipoprotein; HDL, High Density Lipoprotein; NIHSS, National Institute of Health Stroke Scale; mRS, modified Rankin Scale; mBI, modified Barthel Index; SS-QoL, Stroke Specific Quality of Life; EQ-5D-5L, five-level EuroQol 5-D; EQ-VAS, EuroQol Visual Analogue Scale; SF-36, 36-item Short Form health-survey questionnaire; FSS, Fatigue Severity Scale; GGT, Gamma-glutamyl transferase; PT-INR, Prothrombin Time International Normalized Ratio; FAS, Fatigue Assessment Scale.

3.3 Phase 3: Modified Delphi Consensus Exercise With Primary Clinicians

As the CVR of all questions exceeded the CVR critical value, a consensus was reached that all items were appropriate for inclusion in the COS and feasible for collection in KM primary clinics. The feasibility of collecting effect modifiers concerning stroke history and patient satisfaction met the criteria for the degree of consensus and convergence, indicating little deviation of opinion concerning collection of the effect modifiers. The feasibility of the modified MYMOP2, EQ-5D-5L, and EQ-VAS showed an insufficient degree of consensus and convergence, indicating relative controversy for collection (Table 3).

TABLE 3

TABLE 3. Results of the modified Delphi consensus exercise with primary clinicians.

4 Discussion

4.1 Summary of the Main Results

The aim of this study was to develop a COS of HM for elderly patients with stroke in primary clinics. According to the guidelines of the COMET initiative, a literature review by the PMG and a modified Delphi exercise by clinical experts in KM treatment for stroke were conducted. Delphi exercises by KM primary care clinicians were conducted to review the feasibility of the COS. As a result, effect modifiers were obtained for patient history taking. A consensus-based list of outcomes and supplementary symptoms were obtained. The final outcome set for COS-SS-KM is shown in Figure 2.

FIGURE 2

FIGURE 2. Conclusion of for the Korean Medicine Core Outcome Set for Stroke Sequelae (COS-SS-KM) development. Eight outcomes categorized in symptoms, drug safety, quality of life, and satisfaction were included in COS-SS-KM. For modified MYMOP2, symptom list suggested by experts will be used for response convenience. AST, aspartate transminase; ALT, alanine transaminase; BUN, blood urea nitrogen; Cr, creatinine; MYMOP2, Measure Yourself Medical Outcome Profile 2; EQ-5D-5L, EuroQol-5-Dimensions-5-Level; EQ-VAS, EuroQol Visual Analogue Scale.

4.2 Considerations for Core Outcome Set Development

With regards to outcome selection, we considered the characteristics of the elderly, chronic disease, and primary care.

Studies on the elderly should consider their unique physical and mental characteristics. Elderly individuals, especially those with physical or cognitive impairments, present with poor comprehension and increased fear for outcome assessment. Therefore, outcomes for the elderly should have a wide measurement scale, be simple and quick, and involve non-invasive measurement procedures that do not require much cooperation. Moreover, for complicated physical conditions of the elderly, measures for functional performance and quality of life (QOL) should be considered (Faes et al., 2007).

Although chronic diseases are slow and are not treated as an emergency, they affect the patient’s QOL by requiring long-term treatment. Traditional oriental medicinal treatment, including KM, seeks to improve both pathological symptoms and general health, including vitality, and is effective for chronic diseases. Therefore, as a post-KM-treatment outcome for chronic patients, it is recommended that QOL, overall health-related symptoms, and disease-specific symptomatic changes are assessed (Megari, 2013).

Approximately 83% of KM institutions are primary care clinics. Research conducted at primary care and KM clinics is valuable as it reflects the reality of community care (Kim and Kang, 2020). However, conducting clinical trials in primary care clinics, especially in KM clinics, has many limitations. Small primary care clinics and KM clinics are chronically short of manpower for crucial manual treatment. Primary care clinics lack equipment and policies regarding the authority of KM doctors and have limited access to modern diagnostic devices. Regarding feasibility to conduct, it requires an adequate research design and outcomes for primary care or KM clinics (Lindbloom et al., 2004). Simple but accurate outcomes with less disturbance to clinics were recommended for this COS.

4.3 Outcomes Included in the Core Outcome Set

Effect modifiers were recommended for history taking. Information regarding related diseases, alcohol ingestion, smoking, and concomitant drug use were included as basic history. Moreover, the history of stroke sequelae comprised stroke onset, stroke type, family history of stroke, and rehabilitation treatment. Additionally, since this study targeted elderly individuals with chronic diseases, we decided to record weight, height, and nutritional status, which substantially affect the health of the elderly.

According to a prospective observational study on drug-induced liver damage in patients taking HM, the risk of liver damage due to HM is lower than that with general drugs (Cho et al., 2017). For accurate drug safety evaluation, liver and renal functional markers were included in the COS. Blood aspartate transaminase, alanine transaminase, blood urea nitrogen, and creatinine levels were included. Gamma-glutamyl transferase was proposed to distinguish HM-induced liver damage but was excluded due to its difficulty to test in some KM clinics. Lipid factors, blood sugar, prothrombin time-international normalized ratio, and electrolytes were excluded due to difficulty to test in KM clinics and less relevance with the safety or pathological changes by HM. Since the COS is a minimal set of outcomes, KM doctors may include these outcomes at their discretion if available.

In the early stages after stroke, neurological impairment is dominant. Over time and with neurological recovery, patients tend to show more complicated and overall health problems with decreased neurological symptoms. Therefore, there is a need for a suitable tool that is tailored to the stroke phase. Most stroke-specific measurements, which mainly evaluate neurological impairment, have high reliability and sensitivity for acute or subacute patients but low sensitivity for patients in whom ≥6 months have elapsed since onset (Harrison et al., 2013). The Barthel index, which is a representative scale, is best suited for early stroke survivors requiring intensive rehabilitation. However, it has uncertain reliability in the elderly (Weimar et al., 2002), including those with chronic sequelae. Additionally, the modified Rankin scale (mRS) has limited reliability and substantial interobserver variability. Moreover, there may be a further significant decrease in the reliability of mRS when used by inexperienced primary care doctors (Sainsbury et al., 2005; Harrison et al., 2013). With long-term survival, assessing multifaceted function and the health-related quality of life (HRQoL) becomes important in the chronic stage (Harrison et al., 2013). The 36-item Short Form health-survey questionnaire and Stroke Specific Quality of Life Scale were considered for HRQoL assessment; however, they were excluded given the large numbers of questions that they contain. The EQ-5D-5L and EQ-VAS were selected since they are short and simple tools that have been validated in patients with stroke (Dorman et al., 1997).

It is difficult to specify the scope of stroke sequelae symptoms in the elderly given their diversity across individuals, including motor impairment, cognitive impairment, bladder and bowel dysfunction, infection, pressure sores, deep vein thrombosis, multiple pains, depression, and fatigue (Lui and Nguyen, 2018). MYMOP2, a patient-reported outcome measure that allows patients to select significant symptoms to report, is recommended to assess various symptoms in individuals. This brief and convenient questionnaire is a practical and sensitive measure for patients of complementary medicine and primary care (Paterson, 1996; Ralph and Webley, 2019). Experts selected a symptom evaluation method based on MYMOP2. They offered a list of major symptoms of stroke sequelae to help elderly patients select their symptoms and made each patient select two symptoms that they would like to improve with HM treatment. General health status was included in the symptom list, which made it possible to evaluate general symptoms caused by chronic illness (dyspepsia, sleep disturbance, etc.) and neurological symptoms directly caused by stroke (hemiplegia, dizziness, etc.). This list was composed by experts, and it covers symptoms commonly found in clinical practice. Patients can also select and respond to the symptoms they want to measure, allowing it to serve as a bottom-up survey. A 7-point scale was modified to a numeric rating scale to raise sensitivity. Additionally, a treatment satisfaction questionnaire was recommended to assess effectiveness.

4.4 Implications for Outcomes

KM-related outcomes, such as pulse or tongue diagnosis, were not included in the COS for assessing the effectiveness of KM herbal treatments. This might have been due to the difficulty in proving the mechanism of KM treatment and the fact that it is possible to indirectly determine the effect of KM treatment symptomatically. Traditional medicine-related outcomes tend to be overlooked in COS development, highlighting the need for improvements in the quality of traditional medicine outcome measures (Zhang et al., 2021). The development of quantitative measuring methods for KM outcomes may be helpful. Future studies should include sufficiently improved KM outcome measures to the COS in order to reflect crucial KM characteristics.

4.5 Implications for Development Process

Our study has several limitations. We performed an overview of SR and analyzed the included studies from two eligible SRs in the literature review process of Phase 1. Furthermore, grey literature was not included in this review scope. Therefore, we cannot be certain that all relevant trials were located. The second limitations include the paucity and often suboptimal quality of primary data. Although the included RCTs have a low risk of bias (Supplementary Figure S2), the analyzed SRs have a very low quality (Supplementary Table S2). Another limitation of this COS development process was the insufficiency of panel recruitment. Although panelists were recruited from a professional society, the number of panelists was insufficient when compared to that of other similar studies. While the involvement of all possible stakeholders in COS development is encouraged, our study involved only disease specialists and KM primary care clinicians. To supplement this, primary clinicians were asked to answer questions after applying the COS to patients. A semi-structured interview with patients and policy experts is recommended to refine the COS in further studies.

4.6 Conclusion

The COS-SS-KM is an outcome set for evaluating the effectiveness and safety of herbal KM treatment for stroke sequelae in elderly patients at primary clinics. This is the first COS developed for evaluating the effectiveness of herbal KM treatment in primary care and may evaluate the effectiveness of HM, which is difficult to capture with ordinary outcomes. This COS will be applied to evaluate health insurance pilot projects with continuous modifications for evaluating expanded projects. Future studies should supplement the COS by reflecting the opinions of other stakeholders, including patients and policy experts.

Data Availability Statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Ethics Statement

The studies involving human participants were reviewed and approved by the Institutional Review Board of Korea Institute of Oriental Medicine. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

Conceptualization: JC, SK and MSL. Data curation: JC. Formal analysis: MMK. Funding acquisition: SK and MSL. Investigation: JC and SK, Methodology: JC, SK, P-WK, HL, SJ and MSL. Project administration: SK and MSL. Resources: P-WK, HL and SJ. Supervision: MSL. Validation: MSL. Visualization: JC and SK. Writing-original draft: JC, SK and MSL.Writing-review and editing: P-WK, HL, MMC and SJ. All authors read and approved the final manuscript.

Funding

This study was funded by the Korea Institute of Oriental Medicine (KSN2022210) and Ministry of Health and Welfare of South Korea (ERT2111030). The funders had no role in the design or conduct of the study; in the analysis and interpretation of the data; or in the preparation or approval of the manuscript.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Acknowledgments

The authors thank all the participants who participated in the Delphi consensus exercise.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2022.868662/full#supplementary-material

Abbreviations

COMET, Core Outcome Measures for Effectiveness Trials; COS, core outcome set; COS-SS-KM, Korean medicine core outcome set for stroke sequelae; CVR, ontent validity ratio; EQ-5D-5L, ive-level EuroQol 5-D; EQ-VAS, EuroQol Visual Analogue Scale; HM, herbal medicine; HRQoL, health-related quality of life; KM, Korean medicine; mRS, modified Rankin scale; MYMOP2, Measure Yourself Medical Outcome Profile 2; PMG, project management group; QOL, quality of life; RCT, randomized controlled trial; SR, systematic review.

References

Association of Korean Medicine Professors for Cardiovascular and Neurological Medicine (2018). Cardiovascular and Neurological Medicine in Korean Medicine II. Seoul: Woori Medical Publisher.

Google Scholar

Ayre, C., and Scally, A. J. (2014). Critical Values for Lawshe's Content Validity Ratio. Meas. Eval. Couns. Dev. 47, 79–86. doi:10.1177/0748175613513808

CrossRef Full Text | Google Scholar

Becker, K. J. (2016). Inflammation and the Silent Sequelae of Stroke. Neurotherapeutics 13, 801–810. doi:10.1007/s13311-016-0451-5

PubMed Abstract | CrossRef Full Text | Google Scholar

Cadogan, C. A., Ryan, C., Francis, J. J., Gormley, G. J., Passmore, P., Kerse, N., et al. (2015). Improving Appropriate Polypharmacy for Older People in Primary Care: Selecting Components of an Evidence-Based Intervention to Target Prescribing and Dispensing. Implement. Sci. 10, 161. doi:10.1186/s13012-015-0349-3

PubMed Abstract | CrossRef Full Text | Google Scholar

Chen, T., Zhang, B., Deng, Y., Fan, J. C., Zhang, L., and Song, F. (2019). Long-term Unmet Needs after Stroke: Systematic Review of Evidence from Survey Studies. BMJ. Open. 9, e028137. doi:10.1136/bmjopen-2018-028137

PubMed Abstract | CrossRef Full Text | Google Scholar

Cho, J. H., Oh, D. S., Hong, S. H., Ko, H., Lee, N. H., Park, S. E., et al. (2017). A Nationwide Study of the Incidence Rate of Herb-Induced Liver Injury in Korea. Arch. Toxicol. 91, 4009–4015. doi:10.1007/s00204-017-2007-9

PubMed Abstract | CrossRef Full Text | Google Scholar

De Doncker, W., Dantzer, R., Ormstad, H., and Kuppuswamy, A. (2018). Mechanisms of Poststroke Fatigue. J. Neurol. Neurosurg. Psychiatry 89, 287–293. doi:10.1136/jnnp-2017-316007

PubMed Abstract | CrossRef Full Text | Google Scholar

Dorman, P. J., Waddell, F., Slattery, J., Dennis, M., and Sandercock, P. (1997). Is the EuroQol a Valid Measure of Health-Related Quality of Life after Stroke? Stroke 28, 1876–1882. doi:10.1161/01.str.28.10.1876

PubMed Abstract | CrossRef Full Text | Google Scholar

Faes, M., Van Iersel, M., and Olde Rikkert, M. (2007). Methodological Issues in Geriatric Research. J. Nutr. Health Aging 11, 254–259.

PubMed Abstract | Google Scholar

Gillen, G. (2016). Stroke Rehabilitation. 4th edition. Missouri: Mosby.

Google Scholar

Han, S. Y., Hong, Z. Y., Xie, Y. H., Zhao, Y., and Xu, X. (2017). Therapeutic Effect of Chinese Herbal Medicines for post Stroke Recovery: a Traditional and Network Meta-Analysis. Medicine (Baltimore) 96, e8830. doi:10.1097/MD.0000000000008830

PubMed Abstract | CrossRef Full Text | Google Scholar

Harrison, J. K., McArthur, K. S., and Quinn, T. J. (2013). Assessment Scales in Stroke: Clinimetric and Clinical Considerations. Clin. Interv. Aging 8, 201–211. doi:10.2147/CIA.S32405

PubMed Abstract | CrossRef Full Text | Google Scholar

Jang, S.-K., and Kwon, C.-S. (2014). Setting on Aims of Elementary Science Gifted Classes and Teaching Professionalism of Elementary Science Gifted Teachers through Delphi Method. J. Korean Soc. earth Sci. Educ. 7, 99–109. doi:10.15523/jksese.2014.7.1.099

CrossRef Full Text | Google Scholar

Kim, S. M., and Kang, D. T. (2020). National Health Insurance Statistical Annual Report. Seoul: Health Insurance Review and Assessment Service National Health Insurance Service.

Google Scholar

Kirkham, J. J., Gorst, S., Altman, D. G., Blazeby, J. M., Clarke, M., Devane, D., et al. (2016). Core Outcome Set–STAndards for Reporting: the COS-STAR Statement. PLOS. Med. 13, e1002148. doi:10.1371/journal.pmed.1002148

PubMed Abstract | CrossRef Full Text | Google Scholar

Korea Stroke Society (2015). Stroke. 2nd edition. Seoul: Beommun Education.

Google Scholar

Lawshe, C. H. (1975). A Quantitative Approach to Content Validity. Personnel Psychol. 28, 563–575. doi:10.1111/j.1744-6570.1975.tb01393.x

CrossRef Full Text | Google Scholar

Lee, H. S. (2017). Development and Validation of Patient-Centered Outcome Measure for Evaluation of Korean Medical Practice. Available at: https://doi.org/10.23000/TRKO201800005731 (Accessed November 01, 2021).

Google Scholar

Lindbloom, E. J., Ewigman, B. G., and Hickner, J. M. (2004). Practice-based Research Networks: the Laboratories of Primary Care Research. Med. Care 42, III45–9. doi:10.1097/01.mlr.0000119397.65643.d4

PubMed Abstract | CrossRef Full Text | Google Scholar

Lui, S. K., and Nguyen, M. H. (2018). Elderly Stroke Rehabilitation: Overcoming the Complications and its Associated Challenges. Curr. Gerontol. Geriatr. Res. 2018, 9853837. doi:10.1155/2018/9853837

PubMed Abstract | CrossRef Full Text | Google Scholar

Megari, K. (2013). Quality of Life in Chronic Disease Patients. Health Psychol. Res. 1, e27. doi:10.4081/hpr.2013.e27

PubMed Abstract | CrossRef Full Text | Google Scholar

Naess, H., and Nyland, H. (2021). Poststroke Fatigue and Depression Are Related to Mortality in Young Adults: a Cohort Study. BMJ Open 3, e002404. doi:10.1136/bmjopen-2012-002404

PubMed Abstract | CrossRef Full Text | Google Scholar

No, J. S. (2020). From the 20th, a Pilot Project of “Applied Medicine Health Insurance” to the Sick Sisa Journal. Available at: https://www.sisajournal.com/news/articleView.html?idxno=208211 (Accessed September 3, 2021).

Google Scholar

Park, J. B. (2003). Korean Clinic Based Outcome Measure Studies. J. Pharmacopuncture. 6, 35–36. doi:10.22253/jkss.2003.12.6.219

CrossRef Full Text | Google Scholar

Paterson, C. (1996). Measuring Outcomes in Primary Care: a Patient Generated Measure, MYMOP, Compared with the SF-36 Health Survey. BMJ 312, 1016–1020. doi:10.1136/bmj.312.7037.1016

PubMed Abstract | CrossRef Full Text | Google Scholar

Ralph, A., and Webley, G. (2019). A Prospective Audit of Pragmatic Herbal Treatment of Women Experiencing Menopausal Symptoms Using Measure Yourself Medical Outcome Profile (MYMOP2) Questionnaires. J. Herb. Med. 17–18, 100286. doi:10.1016/j.hermed.2019.100286

PubMed Abstract | CrossRef Full Text | Google Scholar

Rankin, A., Cadogan, C. A., Patterson, S. M., Kerse, N., Cardwell, C. R., Bradley, M. C., et al. (2018). Interventions to Improve the Appropriate Use of Polypharmacy for Older People. Cochrane. Database. Syst. Rev. 9, CD008165. doi:10.1002/14651858.CD008165.pub4

PubMed Abstract | CrossRef Full Text | Google Scholar

Sainsbury, A., Seebass, G., Bansal, A., and Young, J. B. (2005). Reliability of the Barthel index when Used with Older People. Age. Ageing. 34, 228–232. doi:10.1093/ageing/afi063

PubMed Abstract | CrossRef Full Text | Google Scholar

Shen, Y. C., Lu, C. K., Liou, K. T., Hou, Y. C., Lin, Y. L., Wang, Y. H., et al. (2015). Common and Unique Mechanisms of Chinese Herbal Remedies on Ischemic Stroke Mice Revealed by Transcriptome Analyses. J. Ethnopharmacol. 173, 370–382. doi:10.1016/j.jep.2015.07.018

PubMed Abstract | CrossRef Full Text | Google Scholar

Siddiqui, F. J., Venketasubramanian, N., Chan, E. S., and Chen, C. (2013). Efficacy and Safety of MLC601 (NeuroAiD®), a Traditional Chinese Medicine, in Poststroke Recovery: a Systematic Review. Cerebrovasc. Dis. 35 (Suppl. 1), 8–17. doi:10.1159/000346231

PubMed Abstract | CrossRef Full Text | Google Scholar

Takayama, S., and Iwasaki, K. (2017). Systematic Review of Traditional Chinese Medicine for Geriatrics. Geriatr. Gerontol. Int. 17, 679–688. doi:10.1111/ggi.12803

PubMed Abstract | CrossRef Full Text | Google Scholar

Weimar, C., Kurth, T., Kraywinkel, K., Wagner, M., Busse, O., Haberl, R. L., et al. (2002). Assessment of Functioning and Disability after Ischemic Stroke. Stroke 33, 2053–2059. doi:10.1161/01.str.0000022808.21776.bf

PubMed Abstract | CrossRef Full Text | Google Scholar

Williamson, P. R., Altman, D. G., Blazeby, J. M., Clarke, M., Devane, D., Gargon, E., et al. (2012). Developing Core Outcome Sets for Clinical Trials: Issues to Consider. Trials 13, 132–138. doi:10.1186/1745-6215-13-132

PubMed Abstract | CrossRef Full Text | Google Scholar

Yin, F., Zhou, H., Fang, Y., Li, C., He, Y., Yu, L., et al. (2020). Astragaloside IV Alleviates Ischemia Reperfusion-Induced Apoptosis by Inhibiting the Activation of Key Factors in Death Receptor Pathway and Mitochondrial Pathway. J. Ethnopharmacol. 248, 112319. doi:10.1016/j.jep.2019.112319

PubMed Abstract | CrossRef Full Text | Google Scholar

Zeng, L. F., Cao, Y., Wang, L., Dai, Y. K., Hu, L., Wang, Q., et al. (2017). Role of Medicinal Plants for Liver-Qi Regulation Adjuvant Therapy in Post-stroke Depression: A Systematic Review of Literature. Phytother. Res. 31, 40–52. doi:10.1002/ptr.5740

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhang, M., Zhang, J., Chua, H. Z., Feng, R., Lu, M., and Tian, Y. (2021). Core Outcome Set for Stable Angina Pectoris in Traditional Chinese Medicine (COS-SAP-TCM). Acupunct. Herb. Med. 1, 39–48. doi:10.1097/hm9.0000000000000007

CrossRef Full Text | Google Scholar

Zhang, T., Wu, X., Cao, S., and Park, S. (2018). Efficacy of the Oriental Herbal Medicine, Jie Yu Dan, for Alleviating post-stroke Aphasia: A Systematic Review and Meta-Analysis of Randomized Clinical Trials. Eur. J. Integr. Med. 24, 35–48. doi:10.1016/j.eujim.2018.10.012

CrossRef Full Text | Google Scholar

Zhang, Y., Zhang, Y., Jin, X. F., Zhou, X. H., Dong, X. H., Yu, W. T., et al. (2019). The Role of Astragaloside IV against Cerebral Ischemia/reperfusion Injury: Suppression of Apoptosis via Promotion of P62-LC3-Autophagy. Molecules 24, 1838. doi:10.3390/molecules24091838

PubMed Abstract | CrossRef Full Text | Google Scholar

Zhu, T., Wang, L., Wang, L. P., and Wan, Q. (2022). Therapeutic Targets of Neuroprotection and Neurorestoration in Ischemic Stroke: Applications for Natural Compounds from Medicinal Herbs. Biomed. Pharmacother. 148, 112719. doi:10.1016/j.biopha.2022.112719

PubMed Abstract | CrossRef Full Text | Google Scholar