Contemporary Educational Interventions for General Practitioners (GPs) in Primary Care Settings in Australia: A Systematic Literature Review

Background: The primary purpose of educational interventions is to optimize the clinical management of patients. General practitioners (GPs) play a major role in the detection and management of diseases. This systematic literature review will describe the type and outcomes of educational interventions designed for general practitioners (GPs) in the Australian context.

Methods: PubMed, CINHAL, and Scopus databases were systematically searched for studies on educational interventions conducted for GPs in Australia during 1st January 2008 to 11th June 2018. Data collected on the methodology of the interventions, GPs satisfaction regarding the educational intervention, changes in knowledge, confidence, skills and clinical behavior of the GPs. We also assessed whether the acquired clinical competencies had an impact on organizational change and on patient health.

Results: Thirteen publications were included in this review. The methods with which educational interventions were developed and implemented varied substantially and rigorous evaluation was generally lacking particularly in detailing the outcomes. The reported GP response rate varied between 2 and 96% across studies, depending upon the method of recruitment, the type of intervention and the study setting (rural vs. urban). The most effective recruitment strategy was a combination of initial contact coupled with a visit to GP practices. Nine of the studies reviewed reported improvement in at least one outcome measure: gaining knowledge, improving skills or change in clinical behavior which was translated into clinical practice. In the 3 pre- and post-intervention analysis studies, 90–100% of the participating GPs reported improvement in their knowledge and attitudes.

Conclusion: Education interventions for GPs in Australia had low response (recruitment) and retention (GPs that participated in follow-ups) rates, even when financial benefits or CPD points were used as incentives. Higher GP response rates were achieved through multiple recruitment strategies. Multifaceted interventions were more likely to achieve the primary outcome by improving knowledge, skills or changing practice, but the effect was often modest. Inconsistent results were reported in studies involving the use of multiple contact methods within an intervention and conducting online interventions.

Introduction

General practitioners (GP) play a vital role in the Australian health care system. About 87% of Australians visit their GP at least once a year (1). Recently, Harrison et al. (2), using a random sample of 8,707 patients at encounters with 290 general practitioners, estimated that about half (47.4%) of Australian patients consulting a GP and one-third (32.6%) of the Australian population have multi-morbidity. GPs are increasingly involved in chronic disease management and prevention (3), and are required to make timely and complex decisions for patient care. This in turn demands the attainment of up to date skills and knowledge regarding management of specific diseases, available treatment modalities and pathways to access them (1). GP trainees in Australia receive a combination of formal teaching and supervised practice managed by an accredited GP supervisor (4). Following the completion of training however, the practice of medicine requires regular appraisal and revalidation, and this is achieved through continuous medical education programmes (CME). These programmes are available in a variety of subject areas, formats and platforms. With evolving technology and evidence regarding the effectiveness of various medical education strategies, the format of these CME programmes has changed over the past years with more online and audio-visual content being integrated with conventional formats (1).

A survey conducted in 2012 involving 2,500 GPs, explored their preferences for CME. The study found that 80% of participants indicated that clinical practice was the key motivation for taking part. The preferred learning formats for GPs were: learning in a group rather than on their own (95%), face-to-face lecture-based formats (83%) and interactive discussion (70%). In contrast, the least preferred format was online self-education (55%) (5). GPs have the opportunity to select and follow an educational programme of their preference. It is the clinician's responsibility to update their knowledge, and participation in CME programmes is mandatory for ongoing registration with the Medical Board of Australia (1). While many interventional studies about GP education have been conducted in Australia little is known about the effectiveness of these interventions. Seminal research by Davis et al. (6) investigating CME strategies found that delivery methods such as conferences or activities without reinforcement have little direct impact on improving practice. Over two-decades later, with the introduction of online technology, this systematic review is interested in following up what changes in GP education have occurred in Australia. This systematic literature review aims to respond the following questions: what types of interventions have been used to educate GPs?; what is the response rate of such studies?; and what are the outcome measures used in these studies of education interventions?

The review was restricted to Australia as the research team plans to conduct a RCT with GPs using an education intervention. The educational intervention studies conducted during the past decade will be examined, focus on the recruitment strategies used, response rates and GP satisfaction with the educational intervention. This review will also examine the reported changes in GP knowledge, confidence, skills and clinical behavior, and assess whether the acquired clinical competencies had an impact on organizational change and on patient health outcomes.

Materials and Methods

Study Design and Participants

Eligible for this review were peer-reviewed studies (randomized control, quasi-experimental, cohort and qualitative) that described an educational intervention for GPs. The studies had to be conducted in Australia and published in the English language, between 1st of January 2008 to 11st June 2018. This time period was selected to capture the increasing availability of online technology that was integrated into the education system over the last decade (7). Studies that involved GPs and other healthcare professionals were included if the outcomes were reported specifically for GPs. Books, reviews, clinician review notes, case studies, clinical practice guidelines or recommendations, opinion pieces and commentaries were excluded. This review was registered with the Center for Reviews and Dissemination at the University of York (PROSPERO registration number CRD42018089492).

Systematic Review Protocol

Four researchers (PV, SS, IR, and JK) independently screened the titles of publications against the inclusion and exclusion criteria and selected potentially eligible publications for review. These selected publications were screened by reading their abstracts. Publications that fulfilled the criteria with their title and abstract were then selected for full text review. Any discrepancies in selecting an article were resolved by discussion among the researchers (SS, IR, and JK) and consulting two other researchers involved in the study (PV and CB).

Three researchers (CB, IR, and JK) assessed the methodological quality of eligible articles using a 14-item checklist adapted from the NIH Quality Assessment Tool for Observational Cohort and Cross-Sectional Studies (8). Disagreements were resolved by discussion among the researchers (JK, IR, and CB) and consultation with another independent researcher (PV).

Using a standardized data extraction sheet, four researchers (SS, IR, JK, and CB) summarized each article selected as “Good Quality” for the review, under the categories of: type of study, study duration, response rate, recruitment strategies, type of intervention used, methods of outcome assessment, outcome and conclusion. The quality of the RCTs were not assessed using the “Consolidated Standards of Reporting Trials” CONSORT guidelines, as the scope of this review was to identify strategies and response rates of recruitment, as well as the type outcome measures. Low response rate studies were not excluded to accurately demonstrate the success rate of these interventions.

Search Strategy and Data Sources

The systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines. The electronic search included the PubMed, CINAHL and Scopus databases based on inclusion and exclusion criteria defined for the review. The primary search in abstracts included two main terms and their variations (primary care, general pract^*, family pract^*, family physician, or community pract^*, and educat^*, train^*, or teach^*) and Australia. We refined our search yield by removing duplicates using the EndNote X7 software and IBM SPSS version 22.

Data Analysis

To review and synthesize the findings, a descriptive approach was utilized. Studies were classified according to the type of interventions used to educate GPs: randomized controlled trial, non-randomized intervention, pre- and post-intervention analysis, qualitative study, and mixed methods. The overall response rates of these studies were reported and examined by location; the outcome measures were examined and classified according to their impact in the improvement of knowledge, skills and translation into clinical practice.

Results

The initial search yielded 9,525 results (Figure 1). Following removal of duplicates, there were 8,414 abstracts for assessment of eligibility. Most titles were excluded because the manuscript did not describe an educational intervention for GPs (where GPs were the study's target participants). Eighty-eight articles were selected according to the inclusion and exclusion criteria. After reading the full text, 13 articles were selected for the final systematic review. Eleven were quantitative [six randomized controlled studies (9–14), one non–randomized community intervention (15), three pre- and post-intervention analysis (16–18), one cross-sectional (19)], one was a qualitative study (20), and one used mixed methods (21). The main characteristics of the studies are summarized in Table 1.

FIGURE 1

Figure 1. Flowchart of search and selection of articles for review.

TABLE 1

Table 1. Summary of the study characteristics included in this review.

GP Characteristics

The total number of study participants varied, ranging from 12 GPs in a qualitative study (20) to a maximum of 373 GPs in a cluster randomized controlled trial (9). The demographic characteristics of GPs were described in 4 studies (11, 13, 14, 20). Two studies (11, 13) reported the distribution of GPs by age ranges with the highest proportion of participants (35–37%) among the age range 45–54 years. A higher proportion (51–61%) of female GPs participated in three studies (11, 13, 14), while the remaining study reported a balanced participation of male and female GPs (20). Two studies reported the proportion of GPs who graduated in Australia [54% (14) and 70% (13)] and the proportion of GPs who graduated before 1989 [54% (13) and 80% (14)]. The working experience of GPs was reported by two studies (11, 14) with the majority of participants [56% (14) and 80% (11)] having <20 years of clinical practice.

Type of Interventions

Overall, interactive workshops were conducted in six studies (10, 12, 13, 15, 18, 21) educational material including guidelines, screening tools, information sheets, websites were provided in 8 studies (9–11, 13, 15, 17, 18, 20), academic detailing or facilitation visits were conducted in 4 studies (10, 11, 13, 15), feedback of GPs' performance was given in 5 studies (9, 11–13, 20), and online interventions were conducted in three studies (12, 20, 21). The facility to consult experts directly by the GPs had been provided in two studies (11, 12). Among the studies that achieved the expected primary outcome, one (13) had five contact times with GPs, two (11, 21) had 4 contact times, one had 2 (14) and one had a single (15) contact. Two studies (9, 10) with 4 and 3 contact times did not accomplish their primary outcome. Lastly, three studies (11, 13, 21) had nurses and/or other practice staff involved along with the GPs in the educational intervention and achieved their primary outcome with changes in clinical practice.

Randomized Controlled Trials

The six RCTs (9–14) reviewed targeted different chronic diseases and GP skill-sets, details are presented in Table 1. Briefly, in four studies (9–11, 13) researchers provided GPs with educational materials (e.g., guidelines, screening tools, and symptom assessment cards), additionally, one study conducted multiple academic detailing visits to educate GPs about the provided materials (10). In three RCTs (11, 13, 14) interventions were based on workshops: in one study a single small group training session based on clinical cases with presentations and role plays using simulated patients (11); another study conducted two small group interactive workshops using presentations and video demonstrations (14); the remaining RCT (13) conducted three interactive workshops with role plays, case discussions and video vignettes, and had provided the GPs with time to reflect on what they encountered during consultation with their patients.

Non-randomized Community Intervention

One non-randomized intervention (15) involved a programme to improve the awareness of physical and oral health care needs of individuals with severe mental illness (SMI) in public mental health, primary care, and non-governmental sectors (Table 1).

Pre- and Post-intervention Studies

Three pre- and post-intervention studies focused on the management of chronic hepatitis B (17), diabetes (16), and radiation oncology (18) (Table 1). Educational interventions included workshops with interactive peer-to-peer group learning (16) or interactive sessions with case discussions (18) and the distribution of treatment guidelines delivered by nurses (17).

Cross-Sectional Study

One cross-sectional study (19) mailed a package containing educational materials about dementia care, an adapted TOP 5 form for carers to write down top practical tips to care for patients with dementia; and a one-page survey response form for GPs to assess the acceptability of the educational materials (Table 1).

Qualitative Study

The qualitative study (20) reviewed included an online intervention aimed to improve GP's knowledge and practices regarding vitamin D via an online course with 8 modules. The course content was influenced by results of a baseline survey conducted to assess knowledge deficiencies and weaknesses in managing issues related to vitamin D and sun protection (Table 1).

Mixed Methods Study

This study (21) aimed to improve chronic disease management by GPs using four learning workshops, a follow-up workshop and plan-do-study-act cycles over a period of 8 months. Evaluation of the workshops was conducted at three-time points using semi-structured interviews (21) (Table 1).

Recruitment and Response Rate

Overall

The recruitment methods varied across the studies and often combined strategies to improve the response rate (Table 1). Recruitment methods varied from a single one-time invitation letter (9, 10, 16, 19, 21), or gratification points for professional development (CPD) (10, 16) and a combination of invitation letter, followed by a phone call and a visit by a research team member to clarify the study (14). An invitation letter followed by a phone call and a $250 retail voucher was used by another study (20). A study (12) targeting rural GPs utilized six rounds of invitation letters with a total of 10 occasions of contact via primer post card, mailed personalized letter, faxed personalized letter, promotional letter, provision of feedback about the current management of the disease of interest, provision of a brochure promoting online videos for education, and by offering jelly beans, CPD points and $200. One study (10) advertised for expression of interest in research involvement via newsletters, direct mail or phone calls, and an incentive of $1,000 per practice. Three studies did not clearly document their method of recruitment [15, 17, 18].

The response rate for most of the studies was low [varying between 2 and 24% (9, 12, 19, 21)] or was not reported (10, 13, 15–17). Higher response rates [ranging from 60 and 96% (11, 14, 18, 20)], were achieved in studies using a combination of recruitment strategies. The study(18) with the highest response rate (96%) did not state the method of recruitment. The second highest response rate (73.1%) was achieved in a study (14) in which members of the research team contacted GPs who expressed interest in the study and visited the practices. A cross-sectional study (19) that aimed to adapt the a program designed to provide five relevant and meaningful tips to assist people with dementia reported a low response by GPs (350 out of 4,827 GPs) but was highly accepted by participants (90%). GPs indicated they were very likely or moderately likely to assist family members and carers of patients with dementia to identify their top tips to personalize care.

By Location

The location of the intervention (rural vs. urban practice) has also been examined. In one study (10, 12) conducted in a rural area, 81% of practices agreed to a single educational session while 58% of practices agreed to participate in the entire programme (4 educational sessions) (10). Another rural study with an online intervention had a GP response rate of only 23% (12). Similarly two studies that were conducted in an urban setting reported response rates of 24% (11) and 73% (14). In one study (13) conducted in both urban and regional practices only 12% of practices agreed to participate in the study. Three studies (9, 18, 19)were conducted nationwide with response rates of 2% (9), 7% (19), and 96% (18). One of these studies (9) had obtained contact details of GPs from the data base of a medical publishing company, while the actual numbers of practicing GPs may be different, this may have led to inaccurate assessment of the response rate. In 4 studies (10, 15, 16, 20) the setting was not stated.

Outcome Measures

Knowledge or Skills Obtained

The knowledge retention or skills obtained by GPs was reported in 3 pre- and post-intervention analysis studies (16–18) in one qualitative study (20) and in one study using mixed methods (21). In the 3 pre- and post-intervention analysis studies (16–18), 90–100% of the participating GPs commented that their knowledge and attitudes improved following the intervention. This change persisted at the 3 month evaluation as increased confidence (71%) and change in behavior (87%) in one study (16). In another study, a 6 month post intervention review (18) reported persistence of increased confidence in 77% of attendees. However, in both studies the post intervention response rate was low, 21.8% (16) at the 3 month assessment and 37% (18) at the 6 months assessment. The remaining study did not assess the long term retention of confidence in the participants (16).

A qualitative study (20)and a study using mixed methods (21) evaluated computer assisted learning with 12 and 15 participants in each. In one study (20) 83% of participants rated the programme as very useful, while the mixed methods study (21) did not describe the proportion of GPs satisfied with the intervention. In the qualitative study (20), 11 of the 12 participants (92%) indicated that they would use the program if it was an accredited CME activity and 75% reported they would refer to the program when talking to patients about related topics. The mixed methods study (21) aimed to promote best practice in GP management of patients with chronic diseases using Medicare Item numbers and broadband-based service (cdmNet). The number of items created and completed initially increased, decreased and then increased again. These results were discussed during the workshops and GPs felt that this pattern occurred because most of the patients requiring a General Practice Management Plan (who were previously not considered for a GPMP) were identified and after that they were managing their “usual” throughput GPs evaluated the learning workshops as being of benefit.

Change in Behavior and/or Practice

Following the interventions, researchers expected to see an improvement in knowledge and a change in the behavior or attitudes of participating GPs (evidenced by adherence to guidelines, increased use of a particular software, and increased identification of patients, treatment initiation or specialist referrals).

Of the six RCTs, three studies (11, 13, 14) reported a change in the behavior of GPs at the end of the intervention, while two studies (9, 10) did not observe a difference in behavior. The remaining RCT (12) was withheld because of the low number of GPs recruited, which was inadequate to demonstrate a statistically significant change. The non-randomized control study (15) reported a change in behavior, with the GPs investing more time interviewing adults with chronic mental illness. All three pre- and post-intervention analysis (16–18) studies reported a change in the behavior of GPs, with referral of more patients to specialists (18), treatment initiation in one or more patients post intervention (16, 17) and adherence to guidelines in managing patients (16–18). In the cross-sectional study (19) GPs reported they would be very likely (51%) or moderately likely (38%) to support family members and carers to identify tips that assist in providing care for the person with dementia. The qualitative study did not evaluate a change in practice following the intervention (20). The mixed methods study reported an increase in the use of computer software introduced to the GPs (21).

In the cross-sectional study (19) the acceptability of the intervention by GPs was 90%, and 89% of GPs indicated that they were “very likely” or “moderately likely” to assist family members and carers to identify practical tips to care for dementia patients. In addition, more than a third of GPs (36%) received positive feedback from family and carers regarding the helpfulness of sharing their practical tips to personalize care for their loved ones with dementia. Three RCTs (9, 10, 12) did not achieve the primary outcomes expected.

Discussion

We reviewed contemporary studies involving GP educational interventions conducted in Australia during the past decade. While this review was restricted to Australia, other countries with similar health system and socio-economic development may face the similar challenges with regards to CME for GPs.

The interventional studies reviewed focused predominantly on improving specific GP competencies by means of improving their knowledge (11, 12, 14, 16, 18, 20, 21) or changing practice (9–11, 13–18, 21). The expected outcome of the interventions was that knowledge would be integrated into medical practice improving the management of specific conditions.

Despite the diverse efforts employed to recruit GPs, through multiple contacts and written materials, most studies had limited success in engaging GPs. Recruitment strategies varied from a single invitation to multiple different approaches including phone calls, financial incentives, CME points and practice visits. A onetime invitation letter was the most frequently used strategy to recruit GPs but resulted in poor response rates compared to other methods. Incorporating multiple recruitment strategies led to a more favorable response rate with the highest rates being recorded when researchers contacted GPs by telephone or by practice visits. Being able to communicate directly with the GPs to clarify their doubts and exchange ideas might have helped to engage GPs, resulting in a higher recruitment.

Offering incentives and CME points did not seem to be an effective strategy in the reviewed studies. This supports the conclusions of Chauhan et al. following an overview of 138 reviews comprising 3,502 individual studies. They found that financial incentives did not have an impact on the long term behavior or practice of family physicians in primary care centers (22).

Continuous professional development is the responsibility of each GP (23). Various circumstances may affect the decision to participate in a study. The driving force is self-motivation, being aware of a knowledge deficit and being driven to restore it. As Taylor and Hamdy describes, the first stage in adult learning, the dissonance phase, is when the learner recognizes a deficiency in his/her knowledge which motivates them to enroll in an educational programme. This challenge to knowledge can be internal; the learner being self-aware, or external, where another person highlights it to the learner (24). The latter strategy was employed in pre- and post-assessment studies where researchers make the participants aware of their level of knowledge prior to the intervention by administering a pre-evaluation. Providing GPs with feedback about their personal performance, lack of adherence to guidelines or impaired recognition of a condition may be a motivating factor for participation in the intervention (24). In a RCT (9) included in the review, researchers provided an audit and feedback incorporating patient responses within the first 6 months of the study, providing GPs with a summary of the prevailing situation. This is further supported by a systematic review conducted by Ivers et al. (25), which revealed that effectiveness of the audit and feedback is greater when the baseline performance is low. They found that the effect of audit and feedback is greater if it is provided by a supervisor or colleague, supplied at multiple instances, delivered in both verbal and written formats and when it includes an explicit target and action plan. Furthermore, according to previous studies, GPs were found to be more willing to participate if the topic or the study content was considered to be relevant and useful in their day to day practice (26).

The format or platform by which the intervention is conducted could be another reason for the choice of participation, retention and gaining expected outcomes in the intervention. Studies included in the review have used multifaceted interventions rather than conducting didactic workshops or distributing printed reading material only. Systematic reviews and meta-analyses show that CMEs conducting multifaceted or mixed interventions that include interactive lectures, case discussions, role plays, video demonstrations, printed educational materials, audits and feed-back are more effective and have a more substantial effect on physicians' performance than interventions involving a single mode of delivery (6, 22, 27–29). A review on behavior change interventions for health care professionals' showed that these interventions can improve knowledge, optimize patients' management, reduce adverse effects and improve patients' outcomes (22). Nine of the 13 studies included in our review used multifaceted interventions. The most common strategies were provision of printed educational materials (8 of 13 studies) and utilizing interactive workshops (6 of 13 studies). These workshops were based largely on clinical cases that GPs encounter in their practice and resulted in positive outcomes (e.g., improved knowledge, increased confidence of GP in prescribing insulin in the primary health setting, GPs investing more time investigating chronic mental issues, increasing referrals to specialists) in most studies.

Slotnick describes that physicians learn by applying the new knowledge to what they have observed and experienced in clinical practice (30), reinforcing the effectiveness of case-based discussions in physician education. Nine of the 13 studies in our review used clinical case discussions as a part of their workshops, academic detailing sessions, practice visits or online educational programmes, to emphasize key points to the GPs. A recent review by Mclean concludes that case-based learning is a method used worldwide in healthcare-related fields to enhance deeper learning in an individual (31).

Educational interventions with longer or multiple contact times were reported to be associated with a better outcome with regards to a behavioral change (28). This was not observed in the studies included in this review.

The meta-analysis conducted by Mansouri et al. notes that interventions targeted at a single discipline group with less participants were associated with better outcomes possibly related to the ability to deliver focused intervention and the opportunity for participants to be actively involved in discussions (28). In the current review, primary outcomes were achieved in all three studies that included nurses/practice staff in addition to GPs.

With advancing technology, internet based learning is recognized as a useful method of providing education, as participants can study remotely, avoiding traveling long distances to participate in workshops. It also provides flexibility, as lack of time or being overcommitted is a key reason given by GPs for withdrawal from a study (32). According to Australian statistics, 86% of households have access to the internet, but the proportion of users accessing the internet for formal educational activities is around 30%, in contrast to entertainment, banking and social networking (all 80%) (33). A study conducted in four GP divisions in Queensland, including two metropolitan, one provincial and one rural division, reported that 89% of GPs have access to the internet in their home or surgery (26). A recent systematic review of online CME programmes conducted in Australia suggests that positive outcomes can be expected in such interventions, particularly with respect to GP satisfaction, knowledge, and practice (34). The three studies included in this review had smaller numbers of GPs with two studies reporting pilot projects (20, 21) and the other study was withheld due to low GP recruitment (12). Therefore, we are unable to draw conclusions about the acceptability of online CME programmes and we believe that further studies are warranted.

Conclusion

Of the 13 studies we reviewed, favorable response rates were obtained when multiple recruitment strategies were employed. The GP response rate was not influenced by the setting (rural vs. urban), or by offering financial incentives or CME points. Multifaceted interventions resulted in improved knowledge and change in GP behavior. Inconsistent results were reported in studies involving the use of multiple contact methods within an intervention and conducting online interventions. Further study is needed to develop educational interventions that are feasible and able to achieve higher engagement of GPs.

Author Contributions

CB, PV, and EP: conceptualization. CB, IR, PV, JK, and SiS: data collection. CB, IR, JK, ShS, PV, and EP: formal analysis. CB, IR, PV, and EP: methodology. CB and IR: writing review and editing. CB, IR, JK, SiS, GM, ShS, PV, EP, JM, and JO: writing original draft. All authors have read and approved the final version of the manuscript.

Funding

PV was supported by an Australian Research Council Career Development Fellowship (No. 1083090). The contents of the published material are solely the responsibility of the Administering Institution, a Participating Institution or individual authors and do not reflect the views of the NHMRC.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

1. Britt H, Miller GC, Henderson J, Bayram C, Valenti L, Harrison C, et al. A Decade of Australian General Practice Activity 2005–06 to 2014–15. Sydney: Sydney University Press (2015).

Google Scholar

2. Harrison C, Henderson J, Miller G, Britt H. The prevalence of complex multimorbidity in Australia. Aust N Z J Pub Health. (2016) 40:239–44. doi: 10.1111/1753-6405.12509

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Ranson NE, Terry DR, Glenister K, Adam BR, Wright J. Integrated and consumer-directed care: a necessary paradigm shift for rural chronic ill health. Aust J Prim Health. (2016) 22:176–80. doi: 10.1071/PY15056

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Wearne S, Dornan T, W Teunissen P, Skinner T. General practitioners as supervisors in postgraduate clinical education: An integrative review. Med Educ. (2012) 46:1161–73. doi: 10.1111/j.1365-2923.2012.04348.x

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Yee M, Simpson-Young V, Paton R, Zuo Y. How do GPs want to learn in the digital era? Aust Fam Phys. (2014) 43:399–402.

PubMed Abstract | Google Scholar

6. Davis DA, Thomson MA, Oxman AD, Haynes RB. Changing physician performance: a systematic review of the effect of continuing medical education strategies. JAMA. (1995) 274:700–5. doi: 10.1001/jama.274.9.700

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Casebeer L, Brown J, Roepke N, Grimes C, Henson B, Palmore R, et al. Evidence-based choices of physicians: a comparative analysis of physicians participating in Internet CME and non-participants. BMC Med Educ. (2010) 10:42. doi: 10.1186/1472-6920-10-42

PubMed Abstract | CrossRef Full Text | Google Scholar

8. U.S. Department of Health and Human Services. Study Quality Assessment Tools. (2018). Available online at: https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools (accessed June 8, 2018).

Google Scholar

9. Almeida OP, Pirkis J, Kerse N, Sim M, Flicker L, Snowdon J, et al. A randomized trial to reduce the prevalence of depression and self-harm behavior in older primary care patients. Ann Fam Med. (2012) 10:347. doi: 10.1370/afm.1368

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Emery JD, Victoria G, Fiona MW, Shelley C, Emma JC, Terry S, et al. The improving rural cancer outcomes trial: a cluster-randomised controlled trial of a complex intervention to reduce time to diagnosis in rural cancer patients in Western Australia. Br J Cancer. (2017) 117:1459–69. doi: 10.1038/bjc.2017.310

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Harris MF, Parker SM, Litt J, van Driel M, Russell G, Mazza D, et al. Implementing guidelines to routinely prevent chronic vascular disease in primary care: the Preventive Evidence into Practice cluster randomised controlled trial. BMJ Open. (2015) 5:e009397. doi: 10.1136/bmjopen-2015-009397

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Paul CL, Piterman L, Shaw JE, Kirby C, Forshaw KL, Robinson J, et al. Poor uptake of an online intervention in a cluster randomised controlled trial of online diabetes education for rural general practitioners. Trials. (2017) 18:137. doi: 10.1186/s13063-017-1869-8

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Sanci L, Chondros P, Sawyer S, Pirkis J, Ozer E, Hegarty K, et al. Responding to young people's health risks in primary care: a cluster randomised trial of training clinicians in screening and motivational interviewing. PLoS ONE. (2015) 10:e0137581. doi: 10.1371/journal.pone.0137581

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Shah S, Sawyer SM, Toelle BG, Mellis CM, Peat JK, Lagleva M, et al. Improving paediatric asthma outcomes in primary health care: a randomised controlled trial. Med J Aust. (2011) 195:405. doi: 10.5694/mja10.11422

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Cameron CM, Cumsille Nazar J, Ehrlich C, Kendall E, Crompton D, Liddy AM, et al. General practitioner management of chronic diseases in adults with severe mental illness: a community intervention trial. Aust Health Rev. (2016) 41:665–71. doi: 10.1071/AH16151

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Deed G, Kilov G, Phillips P, Sharma A, Leow S, Arthur I, et al. Peer-to-Peer, Interactive GP education can reduce barriers to best practice in diabetes management. Research, treatment and education of diabetes and related disorders. (2016) 7:153–61. doi: 10.1007/s13300-016-0156-0

CrossRef Full Text | Google Scholar

17. Dev A, Nguyen JNH, Munafo L, Hardie D, Iacono L. Chronic hepatitis B: a clinical audit of gp management. Aust Fam Phys. (2011) 40:533–8.

PubMed Abstract | Google Scholar

18. Morris L, Gorayski P, Turner S. Targeting general practitioners: prospective outcomes of a national education program in radiation oncology. J Med Imaging Radiat Oncol. (2018) 62:270–5. doi: 10.1111/1754-9485.12685

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Westaway K, Frank O, Shute R, Moffat A, LeBlanc V, Rowett D, et al. Gathering tips from carers to support people with dementia: an adaptation of the TOP 5 program for community use. Int J Evid Based Healthcare. (2018) 16:128–35. doi: 10.1097/XEB.0000000000000136

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Bonevski B, Magin P, Horton G, Bryant J, Randell M, Kimlin MG. An internet based approach to improve general practitioners' knowledge and practices: The development and pilot testing of the “ABC's of vitamin D” program. Int J Med Inform. (2015) 84:413–22. doi: 10.1016/j.ijmedinf.2015.01.006

CrossRef Full Text | Google Scholar

21. Jones KM, Biezen R, Piterman L. Computer assisted chronic disease management: does it work? a pilot study using mixed methods. ISRN Fam Med. (2013) 2013:6. doi: 10.5402/2013/801723

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Chauhan B, Jeyaraman M, Mann A, Lys J, Skidmore B, Sibley K, et al. Behavior change interventions and policies influencing primary healthcare professionals' practice-an overview of reviews. Implement Sci. (2017) 12:3. doi: 10.1186/s13012-017-0568-x

CrossRef Full Text | Google Scholar

23. Medical Board of Australia. Registration Standard: Continuing Professional Development. MBA1608 01. (2016) Available online at: https://ama.com.au/careers/continuing-professional-development (accessed June 22, 2018).

Google Scholar

24. Taylor DCM, Hamdy H. Adult learning theories: Implications for learning and teaching in medical education: AMEE Guide No. 83. Med Teach. (2013) 35:e1561–e72. doi: 10.3109/0142159X.2013.828153

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. (2012) 13:CD000259. doi: 10.1002/14651858.CD000259.pub3

CrossRef Full Text | Google Scholar

26. Askew DA, Clavarino AM, Glasziou PP, Del Mar CB. General practice research: attitudes and involvement of Queensland general practitioners. Med J Aust. (2002) 177:74. doi: 10.5694/j.1326-5377.2002.tb04670.x

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Davis D, O'Brien MAT, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA. (1999) 282:867–74. doi: 10.1001/jama.282.9.867

CrossRef Full Text | Google Scholar

28. Mansouri M, Lockyer J. A meta-analysis of continuing medical education effectiveness. J Cont Educ Health Prof. (2007) 27:6–15. doi: 10.1002/chp.88

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Pippalla RS, Riley DA, Chinburapa V. Influencing the prescribing behaviour of physicians: a metaevaluation. J Clin PharmTher. (1995) 20:189–98. doi: 10.1111/j.1365-2710.1995.tb00648.x

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Slotnick BH. How doctors learn: the role of clinical problems across the medical school-to-practice continuum. Acad Med. (1996) 71:28–34. doi: 10.1097/00001888-199601000-00014

PubMed Abstract | CrossRef Full Text | Google Scholar

31. McLean SF. Case-based learning and its application in medical and health-care fields: a review of worldwide literature. J Med Educ Curric Dev. (2016) 3:377. doi: 10.4137/JMECD.S20377

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Williamson MK, Pirkis J, Pfaff JJ, Tyson O, Sim M, Kerse N, et al. Recruiting and retaining GPs and patients in intervention studies: the DEPS-GP project as a case study. BMC Med Res Method. (2007) 18:42. doi: 10.1186/1471-2288-7-42

CrossRef Full Text | Google Scholar

33. Australian Bureau of Statistics. Household Use of Information Technology, Australia, 2016-17. ABS Cat. No. 8146.0. Canberra, ACT: ABS (2018).

Google Scholar

34. Thepwongsa I, Kirby C, Schattner P, Piterman L. Online continuing medical education (CME) for GPs: does it work? A systematic review. Aust Fam Phys. (2014) 43:717–21.

PubMed Abstract | Google Scholar