Identification of high-risk patients for referral through machine learning assisting the decision making to manage minor ailments in community pharmacies

Background: Data analysis techniques such as machine learning have been used for assisting in triage and the diagnosis of health problems. Nevertheless, it has not been used yet to assist community pharmacists with services such as the Minor Ailment Services These services have been implemented to reduce the burden of primary care consultations in general medical practitioners (GPs) and to allow a better utilization of community pharmacists’ skills. However, there is a need to refer high-risk patients to GPs.

Aim: To develop a predictive model for high-risk patients that need referral assisting community pharmacists’ triage through a minor ailment service.

Method: An ongoing pragmatic type 3 effectiveness-implementation hybrid study was undertaken at a national level in Spanish community pharmacies since October 2020. Pharmacists recruited patients presenting with minor ailments and followed them 10 days after the consultation. The main outcome measured was appropriate medical referral (in accordance with previously co-designed protocols). Nine machine learning models were tested (three statistical, three black box and three tree models) to assist pharmacists in the detection of high-risk individuals in need of referral.

Results: Over 14′000 patients were included in the study. Most patients were female (68.1%). With no previous treatment for the specific minor ailment (68.0%) presented. A percentage of patients had referral criteria (13.8%) however, not all of these patients were referred by the pharmacist to the GP (8.5%). The pharmacists were using their clinical expertise not to refer these patients. The primary prediction model was the radial support vector machine (RSVM) with an accuracy of 0.934 (CI95 = [0.926,0.942]), Cohen’s kappa of 0.630, recall equal to 0.975 and an area under the curve of 0.897. Twenty variables (out of 61 evaluated) were included in the model. radial support vector machine could predict 95.2% of the true negatives and 74.8% of the true positives. When evaluating the performance for the 25 patient’s profiles most frequent in the study, the model was considered appropriate for 56% of them.

Conclusion: A RSVM model was obtained to assist in the differentiation of patients that can be managed in community pharmacy from those who are at risk and should be evaluated by GPs. This tool potentially increases patients’ safety by increasing pharmacists’ ability to differentiate minor ailments from other medical conditions.

Introduction

Community pharmacists’ scope of practice has been extending over time with the inclusion of services such as minor ailment services, medication therapy management or immunization are now under the pharmacists’ responsibility (Adams and Weaver, 2021; DiPietro Mager and Bright, 2022).

In Spain different pharmacy services are nationally defined and protocolized (ForoAF-FC, 2022), although not all of them are implemented or remunerated by the national health system. The minor ailment service (MAS) has been defined as the one “provided upon patient request in the pharmacy, when unsure of which medicinal product to acquire and upon requesting that the pharmacist provide the most appropriate remedy for a specific health problem” (ForoAF-FC, 2022).

In many other countries, such as the United Kingdom and Canada, MASs have been implemented to increase the capacity of primary care (Aly et al., 2018). Their main objectives are to reduce the burden of primary care consultations with general medical practitioners (GPs) allowing them to focus on more complex patients (Aly et al., 2018), to assist patients in choosing the appropriate level of care and to allow a better utilization of community pharmacist skills which extends their role in primary care (Butterworth et al., 2017; Bradley et al., 2018). A systematic review including studies from studies the United Kingdom, Europe, Australia, Canada, and Singapore, concluded that triage services in community pharmacy reduce the burden on other healthcare services by diverting those seeking treatment for minor ailments (Curley et al., 2016).

MAS includes clinical protocols for pharmacists to assist in the triage of patients that can be managed in situ or to those at risk that require referral to other healthcare professionals. One of the main characteristics of MASs is a series of clinical management pathways, which include referral criteria and red flags for minor ailments (Curley et al., 2016). Over sixty different minor ailments are included in protocols in international MASs in countries such as Scotland (gov.scot. NHS Pharmacy First Scotland: information for patients, 2020), England (Boots. NHS Minor, 2022), Wales (NHS. Common, 2019), Canada (CPhA, 2022) and Spain (Amador-Fernández et al., 2018). The protocols, that were mutually agreed with GPs, reflect the potential of community pharmacies to assess common problems that can frequently lead to consultations with GPs or emergency departments (Fielding et al., 2015).

MAS, with its underpinning clinical protocols, allows the differentiation of patients’ demands from patients’ needs in order to identify the most appropriate entry point into the healthcare system whilst ensuring patient safety (Robertson-Steel, 2006). There is evidence that pharmacist-led MASs improve healthcare outcomes (Amador-Fernández et al., 2019; Dineen-Griffin et al., 2021; Stämpfli et al., 2022).

More recently, technologies, such as machine learning, have been shown to be an important diagnostic tool assisting healthcare professionals (WHO, 2021). Machine learning is a field of artificial intelligence that focuses on the study of algorithms that learn from previous data (Mitchell, 1997). The World Health Organization has emphasized the future of artificial intelligence and its use in aiding decision-making by clinicians (WHO, 2021). Machine learning has already been used for aiding disease prognosis in cardiovascular diseases (Azmi et al., 2022), diagnosing neurodegenerative diseases (Myszczynska et al., 2020) and for classifying and predicting the progression of multiple sclerosis (Vázquez-Marrufo et al., 2022). The use of predictive models and their impact is said to be increasing exponentially (NCBI, 2016) and, although these models cannot replace GP diagnostic intelligence, it could augment it (Summerton and Cansdale, 2019).

Machine learning has been used in some pharmacy settings to detect adverse drug events (Bouzillé et al., 2019) and to evaluate patients’ adherence (Lo-Ciganic et al., 2015; Koesmahargyo et al., 2020) but it has not yet been used to assist community pharmacists provide triage services to appropriately identify high-risk individuals in need of referral. Therefore, there is a need of using advance technologies to assist pharmacists in their daily tasks of triaging patients consulting or demanding a treatment for a minor ailment in community pharmacy.

The aim of this study was to develop a predictive model for high-risk patients that need referral assisting community pharmacists’ triage through a MAS.

Materials and methods

Study design

Data for the development of the predictive model was derived from a pragmatic type 3 effectiveness-implementation hybrid study (Curran et al., 2012), which commenced in October 2020 and is ongoing, at a national level in Spanish community pharmacies. The study protocol was registered in ClinicalTrials.gov NCT05247333. A brief description of the study is as follows.

Study population

Six Pharmaceutical Associations and the Spanish Society of Clinical, Family and Community Pharmacy (SEFAC) invited their member community pharmacies/pharmacists to take part in the program through a number of channels. Patients included in the study were those who presented in the participating community pharmacies with symptoms or requested a medication for a minor ailment. The patient inclusion criteria were: aged ≥18 years, or younger if they were accompanied by a responsible adult, presenting one of the minor ailments listed below or other included at the discretion of the pharmacist. Clinical protocols were designed for the management of: upper respiratory tract related (nasal congestion, cold, cough); pain related (headache, joint and back pain, dental pain, sore throat, dysmenorrhea); gastroenterology (heartburn, flatulence, diarrhea, constipation, vomiting); dermatological (acne, mouth ulcers, dermatitis, soft tissue injuries, cold sore hyperhidrosis, bites and stings, athlete’s foot, burns, rashes); other ailments (acute stress disorder, fever, hemorrhoids, insomnia, red eye, dry eye, vaginal candidiasis, varicose veins). These protocols were agreed through a co-design process between medical and pharmacy associations and were facilitated by researchers from the University of Granada.

Description of the intervention

The pharmacist-patient intervention is described using the Template for Intervention Description and Replication or TIDieR (de Barra et al., 2019) template (Supplementary Appendix SA1). It included a standardized consultation protocol incorporating:

• A general procedure for the service (Pharmaceutical Care Forum in Community Pharmacy FA-F, 2019).

• Management protocols co-developed between GPs and community pharmacists for each specific minor ailment, including referral criteria, and recommended pharmacological and non-pharmacological treatment (Amador-Fernández et al., 2018).

• A web-based software program (SEFAC eXPERT^®) (SEFAC. SEFAC, 2014) that guided pharmacists

Educational training was provided covering topics such as service provision, service protocols, communication skills with the patient and other health professionals, web-based software use and data collection. Change agents (CAs) were used to conduct the training as well as periodic follow-ups in order to provide feedback and advice to the pharmacists as well as checking the fidelity to the intervention. CAs also received monthly ongoing training, implementation strategies (Damschroder et al., 2009) and feedback by the research group.

Study outcomes

The outcome variables measured related to the impact of MAS were (Amador-Fernández et al., 2022):

• Appropriate medical referral: patient referral by the pharmacist made in accordance with the co-designed protocols.

• Modification of direct product request.

• Symptom resolution: relief of symptoms from 1“not at all” to 10“completely” at 10-day follow-up.

• Re-consultation rate for the same minor ailment 10 days or after the pharmacist consultation.

This paper will report only on the medical referral model.

Ethics approval and consent to participate

The study was approved by the University of Granada Ethics Committee (registration number 253c23c678ee33a14d3dc16e585baf6a5a3da056). Participating pharmacists had to sign a commitment form to participate in the study. Patients or responsible adults were requested to provide written consent after being informed of the study.

Machine learning model

Data used for obtaining the model was extracted from the web-based consultation program. For the model development, the outcome was appropriate medical referral defined as a dichotomous qualitative variable (yes/no).

There were thirteen patient-related covariates: gender, age, province, other health problems, other treatments, minor ailment consulted, previous treatment of the minor ailment, minor ailment duration, referral criteria (due to the patient’s age, pregnancy/breastfeeding, red flags, symptom duration, other treatments, health problems and others), consultation type (presenting in CP with a consultation/requesting a medication for a minor ailment), quality of life (visual analogue scale from 0 to 100), physiological status (pregnancy/breastfeeding) and day of the consultation.

Pharmacists-related covariates included: gender, age, nationality and role in the CP. Pharmacies-related covariates were location (rural/urban), location size and number of pharmacists working in the CP.

Statistical analysis and design of the models

Quantitative covariates were described using the arithmetic mean, standard deviation, median, maximum, and minimum as appropriate. Near zero variance predictors were removed. Furthermore, two non-relevant variables with more than 65% of missing values were eliminated (Anatomical Therapeutic Chemical or ATC classification of the recommended treatments and whether the treatment requested was a prescription treatment) (Kang, 2013). Remaining variables were subjected to missing value imputation using k-Nearest Neighbors. Outliers were identified and treated using the Tukey’s boxplot method for the covariate symptom duration and Winsorization for existing underlying health problems and the number of treatments being used (Aguinis et al., 2013). In addition, a Yeo-Johnson transformation was used to stabilize variance for the three covariates (symptom duration, underlying health problems and the number of treatments being used) and additionally for EQVAS and number of pharmacists per community pharmacy (Yeo and Johnson, 2000). Finally, correlation between covariates was analyzed to avoid multicollinearity problems. 70.0% of the database was used for obtaining the models as a training cohort and 30.0% as a test cohort (Lantz, 2019).

For the main prediction model, an additive selection technique with three feature selection methods: Kohonen’s Learning Vector Quantization (LVQ) algorithm (Kohonen, 1995), Random Forest and RFE (recursive feature elimination) was used to select the number of predictors that should be included in the model (Kuhn and Johnson, 2013; Kuhn and Johnson, 2019).

Due to the high-class imbalance in the referral outcome variable (no: 91.5%, yes: 8.5%), the SMOTE (Synthetic Minority Over-sampling Technique) was applied to the training dataset to reduce the class imbalance (Chawla et al., 2002). Nine models were tested for the outcome “referral to the GP”: three statistical models (k-Nearest Neighbor or kNN, Naive Bayes or NB, Logistic Regression or RegLog), three black box models (Artificial Neural Network or ANN, Support Vector Machine Lineal or SVM. L, Support Vector Machine Radial or SVM. R) and three tree models (C5.0 Decision Tree or C5DT, Random Forest or RF, XGBoost or XGB).

To assess the primary prediction model, its performance was evaluated for the 25 most frequent patient profiles and for the 25 most frequent pharmacists’ profiles found in the study. Accuracy, Cohen’s kappa and recall were evaluated for each profile.

The software used for the analysis was R v4.2.2 with the package RStudio 2022.12.0 build 353.

Results

14,083 patients were included in the study with patient characteristics presented in Table 1. Most participant patients were female (68.1%, n = 9594), with a consultation about the minor ailment (84.6%, n = 11911), with no previous treatment for the minor ailment consulted (68.0%, n = 9588) and with no referral criteria (86.2%, n = 12146).

TABLE 1

TABLE 1. Patients’ characteristics.

Most patients did not present any referral criteria (86.2%, n = 12,116). Those with referral criteria presented a mean of 1.02 (SD = 0.56) criteria with a maximum of three criteria. Red flags were the most common referral criteria (39.5%, n = 846) (Table 1). Higher number of patients presented referral criteria (13.8%, n = 1937) than those who were ultimately referred (8.5%, n = 1,194). A synthetic minority oversampling technique (SMOTE) was used to solve the class imbalance.

For obtaining the machine learning model, 70.0% (n = 9,839) of the data set was used as a training cohort and 30.0% (n = 4,218) as a test cohort. Correlation between covariates was analyzed without finding any high correlations. A moderate correlation was found between the patients’ health problems and their medication (0.29); nevertheless, no additional correction was made due to the correlation being lower than 0.5.

The primary prediction model (Figure 1) was the radial support vector machine (RSVM) with an accuracy of 0.934 (CI95 = [0.926,0.942]), Cohen’s kappa of 0.630, recall equal to 0.975 and an area under the curve of 0.897 (Figure 2).

FIGURE 1

FIGURE 1. Comparative Roc curves.

FIGURE 2

FIGURE 2. Radial Support Vector Mchine Model:confusion matrix and Roc curve.

The confusion matrix for the primary prediction model (RSVM) (Figure 2) shows that the model could predict 95.2% (n = 3665) of the true negatives (patients that would not be referred to the GP according to the model and should not be referred) and 74.8% (n = 276) of the true positives (patients that would be referred to the GP according to the model and should be referred).

Twenty variables (out of 61 covariates evaluated) were finally included in the model, i.e., the variables that showed statistically significant difference (p ≤ 0.01) when tested compared to a random variable using a random forest model. These variables were listed according to relevance (Figure 3): referral criteria (red flags, symptom duration, health problems, treatments, patients’ age), symptom duration, CP population served, pharmacist’s age, physiological status, number of pharmacists working in the CP, CP’s location, pharmacist’s role in the CP, pharmacist’s gender, quality of life, previous treatment of the minor ailment, minor ailment consulted, month of the consultation and patient’s age.

FIGURE 3

FIGURE 3. Feature importance of the Variable (Boruta algorithems).

The first six covariates included in the primary prediction model were related to the patient presenting with criteria that might lead to referral (Figure 3). The importance of these referral criteria was compared to a random covariate. Red flags increase the odds of experiencing referral by 94.4% (IC95 = [91.8, 96.9]), symptom duration by 63.0% (IC95 = [61.7, 64.4]), other health problems unrelated to the minor ailment consulted by 55.4% (IC95 = [54.1, 56.6]), treatments to treat other health problems by 54.0% (IC95 = [53.0, 55.1]), other criteria by 34.9% (IC95 = [34.2, 35.5]) and patient’s age by 21.8% (IC95 = [21.4, 22.2]).

The 25 most prevalent patients’ profiles were extracted to evaluate performance of the model (Table 2). The model was considered appropriate (accuracy, Cohen’s kappa and recall over 0.62) for 56% (14/25) of the profiles. In addition, the 25 most prevalent pharmacists’ profiles were extracted (Table 3) and the model identified eight lower performance profiles (Cohen’s kappa <0.62) when triaging patients.

TABLE 2

TABLE 2. Accuracy of the Radial Support Vector Machine model for 25 patients’ profiles included in the study.

TABLE 3

TABLE 3. Accuracy of the Radial Support Vector Machine model for 25 pharmacists’ profiles included in the study.

Discussion

Machine learning technologies are increasingly being developed and used as a diagnostic tool assisting healthcare professionals. Nevertheless, its use is not extended in community pharmacy, and it has never been developed for assisting pharmacists while triaging patients with minor ailments. This study developed a machine learning model (RSVM) to assist community pharmacists when triaging patients to better identify high-risk patients in order to differentiate those that may be managed by a community pharmacist from those who should be referred and evaluated by a GP. The model highlighted the most important patient characteristics that could predict which patients have a high probability of referral (prediction of 95.2% of the true negatives and 74.8% of the true positives). It also allowed the detection of certain pharmacists’ profiles that would benefit from supplementary training.

No similar studies have been carried out in community pharmacy. Nevertheless, studies in healthcare have used machine learning techniques in other settings to assist diagnosis. A systematic review that compared machine learning techniques for cardiovascular disease prediction found models with an accuracy between 81% and 100% concluding that the random forest model produced the most significant outcomes in terms of accuracy (Azmi et al., 2022). Our RSVM model obtained better results in terms of accuracy, area under the curve and recall than those studies for the prediction of medication adherence (Koesmahargyo et al., 2020). The model included 20 variables that showed statistically significant difference when tested, similarly to those included in a model used to predict asthma exacerbation in an upcoming year (Jiao et al., 2022).

Implications for research and practice

Although, studies have reported that patients can be appropriately triaged in community pharmacy to GPs (Dineen-Griffin et al., 2020; Amador-Fernández et al., 2022); the use of predictive modelling at the time of the consultation would suggest, although not proven, a contribution to the increase in patient safety, clinical decision-making and governance. Future research should focus on developing artificial intelligence techniques to help clinicians in daily practice, always taking into consideration the legal and ethical aspects.

There are two main implications for practice: an increase in patients’ safety and improved community pharmacy practice. Interestingly, a study found that pharmacist self-perception to differentiate minor ailments from similar conditions has been found to have a low score when evaluating competencies to manage minor ailments such as recommendation of appropriate over the counter medicines (Makhlouf et al., 2021). Clinical protocols and guidelines together with machine learning models may increase pharmacists’ ability to differentiate minor ailments from other medical conditions. Secondly, a key element when triaging and/or diagnosing is not only the patients’ characteristics but also the health providers’ traits and their abilities to perform clinically. The model detected certain pharmacists’ profiles on which the CAs could focus their interventions, intensify training and perform more frequent follow-ups with the ultimate aim of improving patients’ clinical results.

Strengths and limitations

To our knowledge, this is the first study using machine learning to develop a predictive model that can assist community pharmacists delivering MASs to triage patients. A major strength is that all data used to construct the model emanated from data derived from actual practices. New data will be used for the continuing learning of the model, permitting further improvement of the predictive ability of the model (now it was considered appropriate for 56% of the 25 most common patients’ profiles included in the study).

One of the main limitations of the study was the lack of availability of individual patient clinical and medical history, as community pharmacies do not have access to patient clinical history. The model was therefore restricted to the data obtained during the pharmacist-patient consultation, reflecting usual practice. Model performance could potentially be improved by adding more specific biomarkers that allow patient differentiation, which currently is not feasible in real practice for community pharmacy.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation. Requests to access these datasets should be directed to bm9lbGlhLmFmQG91dGxvb2suY29t.

Ethics statement

The studies involving human participants were reviewed and approved by the University of Granada Ethics Committee. The patients/participants provided their written informed consent to participate in this study.

Author contributions

Conceptualization: NA-F, SB, VG-C, MG, VB, VC, and FM-M Data curation: NA-F, SB, and FM-M Formal analysis: NA-F, SB, RP-L, and JS-T Funding acquisition: NA-F, SB, VG-C, MG, and FM-M Investigation: NA-F, SB, and VC Methodology: NA-F, SB, VG-C, MG, EG, VB, VC, and FM-M Project administration: NA-F and SB Resources: NA-F, SB, and FM-M Software: NA-F, SB, and EG Supervision: NA-F and SB Validation: NA-F, SB, EG, RP-L, and JS-T Visualization: NA-F and SB Writing—Original draft preparation: NA-F and SB Writing—Review and editing: NA-F, SB, VG-C, MG, EG, RP-L, JS-T, VB, EP, JP, VC, RF, OG, and FM-M. All authors contributed to the article and approved the submitted version.

Funding

The funding for the study was provided by a national organization, the Spanish Society of Clinical, Family and Community Pharmacy and the Pharmaceutical Associations of Valencia, Madrid, Gipuzkoa, Malaga, Castellon and Valladolid. It was a contract signed by the University of Granada and the above name associations. The funders had no direct role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. They had no role in deciding to publish this information or the analysis of the data. Open access funding by University of Lausanne.

Acknowledgments

We thank all the Spanish community pharmacies and pharmacists who participated in the study for their time and commitment.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fphar.2023.1105434/full#supplementary-material

References

Adams, A. J., and Weaver, K. K. (2021). Pharmacists' patient care process: State "scope of practice" priorities for action. Ann. Pharmacother. 55 (4), 549–555. doi:10.1177/1060028020950193

PubMed Abstract | CrossRef Full Text | Google Scholar

Aguinis, H., Gottfredson, R. K., and Joo, H. (2013). Best-practice recommendations for defining, identifying, and handling outliers. Organ. Res. Methods 16 (2), 270–301. doi:10.1177/1094428112470848

CrossRef Full Text | Google Scholar

Aly, M., García-Cárdenas, V., Williams, K., and Benrimoj, S. I. (2018). A review of international pharmacy-based minor ailment services and proposed service design model. Res. Soc. Adm. Pharm. 14 (11), 989–998. doi:10.1016/j.sapharm.2017.12.004

CrossRef Full Text | Google Scholar

Amador-Fernández, N., Amariles, P., Baixauli-Fernández, V., Benrimoj, S., Climent-Catalá, M., Colomer-Molina, V., et al. (2018). Protocolos de Indicación Farmacéutica y Criterios de Derivación al Médico en Síntomas Menores [Protocols for the minor ailment service and referral criteria for minor ailments]. Granada: Editorial Técnica Avicam.

Google Scholar

Amador-Fernández, N., Benrimoj, S., Baixauli, V., Climent, M., Colomer, V., Esteban, O., et al. (2019). Colaboración farmacéutico-médico en la elaboración de protocolos consensuados para el tratamiento de síntomas menores: Programa ‘INDICA+PRO’. Farm. Comunitarios 11, 21–31. doi:10.5672/FC.2173-9218(2019/Vol11)00403

CrossRef Full Text | Google Scholar

Amador-Fernández, N., Benrimoj, S. I., Olry de Labry Lima, A., García-Cárdenas, V., Gastelurrutia, M., Berger, J., et al. (2022). Strengthening patients' triage in community pharmacies: A cluster randomised controlled trial to evaluate the clinical impact of a minor ailment service. PLoS One 17 (10), e0275252. doi:10.1371/journal.pone.0275252

PubMed Abstract | CrossRef Full Text | Google Scholar

Azmi, J., Arif, M., Nafis, M. T., Alam, M. A., Tanweer, S., and Wang, G. (2022). A systematic review on machine learning approaches for cardiovascular disease prediction using medical big data. Med. Eng. Phys. 105, 103825. doi:10.1016/j.medengphy.2022.103825

PubMed Abstract | CrossRef Full Text | Google Scholar

Boots NHS Minor (2022). Boots pharmacy. Available at: https://www.boots.com/nhs-services/minorailments.

Google Scholar

Bouzillé, G., Morival, C., Westerlynck, R., Lemordant, P., Chazard, E., Lecorre, P., et al. (2019). An automated detection system of drug-drug interactions from electronic patient records using big data analytics. Stud. health Technol. Inf. 264, 45–49. doi:10.3233/SHTI190180

CrossRef Full Text | Google Scholar

Bradley, F., Seston, E., Mannall, C., and Cutts, C. (2018). Evolution of the general practice pharmacist's role in England: A longitudinal study. Br. J. general Pract. J. R. Coll. General Pract. 68 (675), e727–e734. doi:10.3399/bjgp18X698849

CrossRef Full Text | Google Scholar

Butterworth, J., Sansom, A., Sims, L., Healey, M., Kingsland, E., and Campbell, J. (2017). Pharmacists' perceptions of their emerging general practice roles in UK primary care: A qualitative interview study. Br. J. general Pract. J. R. Coll. General Pract. 67 (662), e650–e658. doi:10.3399/bjgp17X691733

CrossRef Full Text | Google Scholar

Chawla, N., Bowyer, K., Hall, L., and Kegelmeyer, W. (2002). Smote: Synthetic minority over-sampling technique. J. Artif. Int. Res. 16 (1), 321–357. doi:10.1613/jair.953

CrossRef Full Text | Google Scholar

CPhA (2022). Common ailments Canadian pharmacists association. Available at: https://www.pharmacists.ca/advocacy/practice-development-resources/common-ailments/.

Google Scholar

Curley, L. E., Moody, J., Gobarani, R., Aspden, T., Jensen, M., McDonald, M., et al. (2016). Is there potential for the future provision of triage services in community pharmacy? J. Pharm. policy Pract. 9, 29. doi:10.1186/s40545-016-0080-8

PubMed Abstract | CrossRef Full Text | Google Scholar

Curran, G. M., Bauer, M., Mittman, B., Pyne, J. M., and Stetler, C. (2012). Effectiveness-implementation hybrid designs: Combining elements of clinical effectiveness and implementation research to enhance public health impact. Med. care 50 (3), 217–226. doi:10.1097/MLR.0b013e3182408812

PubMed Abstract | CrossRef Full Text | Google Scholar

Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., and Lowery, J. C. (2009). Fostering implementation of health services research findings into practice: A consolidated framework for advancing implementation science. Implement Sci. 4, 50. doi:10.1186/1748-5908-4-50

PubMed Abstract | CrossRef Full Text | Google Scholar

de Barra, M., Scott, C., Johnston, M., De Bruin, M., Scott, N., Matheson, C., et al. (2019). Do pharmacy intervention reports adequately describe their interventions? A template for intervention description and replication analysis of reports included in a systematic review. BMJ open 9 (12), e025511. doi:10.1136/bmjopen-2018-025511

PubMed Abstract | CrossRef Full Text | Google Scholar

Dineen-Griffin, S., Benrimoj, S. I., Rogers, K., Williams, K. A., and Garcia-Cardenas, V. (2020). Cluster randomised controlled trial evaluating the clinical and humanistic impact of a pharmacist-led minor ailment service. BMJ Qual. Saf. 29 (11), 921–931. doi:10.1136/bmjqs-2019-010608

PubMed Abstract | CrossRef Full Text | Google Scholar

Dineen-Griffin, S., Benrimoj, S. I., Williams, K. A., and Garcia-Cardenas, V. (2021). Co-design and feasibility of a pharmacist-led minor ailment service. BMC health Serv. Res. 21 (1), 80. doi:10.1186/s12913-021-06076-1

PubMed Abstract | CrossRef Full Text | Google Scholar

DiPietro Mager, N. A., and Bright, D. R. (2022). Promising practices and pockets of excellence: Community pharmacists supporting wellness for reproductive-age women. Health Serv. Res. 57 (6), 1384–1389. doi:10.1111/1475-6773.14057

PubMed Abstract | CrossRef Full Text | Google Scholar

Fielding, S., Porteous, T., Ferguson, J., Maskrey, V., Blyth, A., Paudyal, V., et al. (2015). Estimating the burden of minor ailment consultations in general practices and emergency departments through retrospective review of routine data in North East Scotland. Fam. Pract. 32 (2), 165–172. doi:10.1093/fampra/cmv003

PubMed Abstract | CrossRef Full Text | Google Scholar

ForoAF-FC (2022). Practical guide to clinical professional pharmacy services in community pharmacy. Spain GPCo, editor. Madrid: Foro de Atención Farmacéutica en Farmacia Comunitaria Foro AF-FC.

Google Scholar

gov.scot. NHS Pharmacy First Scotland: information for patients (2020). Scottish government. Available at: https://www.gov.scot/publications/nhs-pharmacy-first-scotland-information-patients/.

Google Scholar

Jiao, T., Schnitzer, M. E., Forget, A., and Blais, L. (2022). Identifying asthma patients at high risk of exacerbation in a routine visit: A machine learning model. Respir. Med. 198, 106866. doi:10.1016/j.rmed.2022.106866

PubMed Abstract | CrossRef Full Text | Google Scholar

Kang, H. (2013). The prevention and handling of the missing data. Korean J. Anesthesiol. 64 (5), 402–406. doi:10.4097/kjae.2013.64.5.402

PubMed Abstract | CrossRef Full Text | Google Scholar

Koesmahargyo, V., Abbas, A., Zhang, L., Guan, L., Feng, S., Yadav, V., et al. (2020). Accuracy of machine learning-based prediction of medication adherence in clinical research. Psychiatry Res. 294, 113558. doi:10.1016/j.psychres.2020.113558

PubMed Abstract | CrossRef Full Text | Google Scholar

Kohonen, T. (1995). Learning vector quantization. Self-organizing maps. Berlin: Springer, 175–189.

CrossRef Full Text | Google Scholar

Kuhn, M., and Johnson, K. (2013). Applied predictive modeling. New York: Springer.

Google Scholar

Kuhn, M., and Johnson, K. (2019). Feature engineering and selection: A practical approach for predictive models. New York: Chapman and Hall/CRC.

Google Scholar

Lantz, B. Machine learning with R: Expert techniques for predictive modeling. Birmingham2019: Publishing P.

Lo-Ciganic, W. H., Donohue, J. M., Thorpe, J. M., Perera, S., Thorpe, C. T., Marcum, Z. A., et al. (2015). Using machine learning to examine medication adherence thresholds and risk of hospitalization. Med. care 53 (8), 720–728. doi:10.1097/MLR.0000000000000394

PubMed Abstract | CrossRef Full Text | Google Scholar

Makhlouf, A. M., Ibrahim, M. I. M., Awaisu, A., Vyas, S. K., and Yusuff, K. B. (2021). Management of common minor ailments in Qatar: Community pharmacists' self-perceived competency and its predictors. PloS one 16 (8), e0256156. doi:10.1371/journal.pone.0256156

PubMed Abstract | CrossRef Full Text | Google Scholar

Mitchell, T. M. (1997). Does machine learning really work? AI Mag. 18 (3), 11. doi:10.1609/aimag.v18i3.1303

CrossRef Full Text | Google Scholar

Myszczynska, M. A., Ojamies, P. N., Lacoste, A. M. B., Neil, D., Saffari, A., Mead, R., et al. (2020). Applications of machine learning to diagnosis and treatment of neurodegenerative diseases. Nat. Rev. Neurol. 16 (8), 440–456. doi:10.1038/s41582-020-0377-8

PubMed Abstract | CrossRef Full Text | Google Scholar

NCBI. Medical subject headings: "Machine learning" https://pubmed.ncbi.nlm.nih.gov/: National Center for Biotechnology; 2016 [cited 2022 26.07.2022]. Available at: https://www.ncbi.nlm.nih.gov/mesh/2010029.

Google Scholar

NHS. Common Ailments scheme offers free over-the counter medicines ; 2019 [cited 2022 09.08.2022]. Available at: https://www.wales.nhs.uk/news/50454.

Google Scholar

Pharmaceutical Care Forum in Community Pharmacy FA-F (2019). Practical guide to pharmaceutical care services in community pharmacy. Madrid: General Pharmaceutical Council of Spain.

Google Scholar

Robertson-Steel, I. (2006). Evolution of triage systems. Emerg. Med. J. 23 (2), 154–155. doi:10.1136/emj.2005.030270

PubMed Abstract | CrossRef Full Text | Google Scholar

SEFAC. SEFAC eXPERT, software for the record of community pharmacy services. Madrid: Spanish Society of. Community Pharmacy; 2014 [cited 2022 28 February]. Available at: www.sefacexpert.org/.

Google Scholar

Stämpfli, D., Winkler, B. A., Vilei, S. B., and Burden, A. M. (2022). Assessment of minor health disorders with decision tree-based triage in community pharmacies. Res. Soc. Adm. Pharm. 18 (5), 2867–2873. doi:10.1016/j.sapharm.2021.07.003

CrossRef Full Text | Google Scholar

Summerton, N., and Cansdale, M. (2019). Artificial intelligence and diagnosis in general practice. Br. J. general Pract. J. R. Coll. General Pract. 69 (684), 324–325. doi:10.3399/bjgp19X704165

CrossRef Full Text | Google Scholar

Vázquez-Marrufo, M., Sarrias-Arrabal, E., García-Torres, M., Martín-Clemente, R., and Izquierdo, G. A systematic review of the application of machine-learning algorithms in multiple sclerosis. Neurologia (Barcelona, Spain). 2022.

Google Scholar

WHO (2021). Ethics and governance of artifical intelligence for health: WHO guidance. Geneva: World Health Organisation.

Google Scholar

Yeo, I-K., and Johnson, R. A. (2000). A new family of power transformations to improve normality or symmetry. Biometrika 87, 954–959. doi:10.1093/biomet/87.4.954

CrossRef Full Text | Google Scholar