Systematic Review of Health Economic Evaluation Studies Developed in Brazil from 1980 to 2013

Background: Brazil has sought to use economic evaluation to support healthcare decision-making processes. While a number of health economic evaluations (HEEs) have been conducted, no study has systematically reviewed the quality of Brazilian HEE. The objective of this systematic review was to provide an overview regarding the state of HEE research and to evaluate the number, characteristics, and quality of reporting of published HEE studies conducted in a Brazilian setting.

Methods: We systematically searched electronic databases (MEDLINE, EMBASE, Latin American, and Caribbean Literature on Health Sciences Database, Scientific Electronic Library Online, NHS Economic Evaluation Database, health technology assessment Database, Bireme, and Biblioteca Virtual em Saúde Economia da Saúde); citation indexes (SCOPUS, Web of Science), and Sistema de Informação da Rede Brasileira de Avaliação de Tecnologia em Saúde. Partial and full HEEs published between 1980 and 2013 that referred to a Brazilian setting were considered for inclusion.

Results: In total, 535 studies were included in the review, 36.8% of these were considered to be full HEE. The category of healthcare technologies more frequently assessed were procedures (34.8%) and drugs (28.8%) which main objective was treatment (72.1%). Forty-four percent of the studies reported their funding source and 36% reported a conflict of interest. Overall, the full HEE quality of reporting was satisfactory. But some items were generally poorly reported and significant improvement is required: (1) methods used to estimate healthcare resource use quantities and unit costs, (2) methods used to estimate utility values, (3) sources of funding, and (4) conflicts of interest.

Conclusion: A steady number of HEE have been published in Brazil since 1980. To improve their contribution to inform national healthcare policy efforts need to be made to enhance the quality of reporting of HEEs and promote improvements in the way HEEs are designed, implemented (i.e., using sound methods for HEEs) and reported.

Introduction

Brazil is an upper middle-income country with a population of 200 million citizens, largely urban (85%). It is a federative republic with 26 states, a federal district, and 5,564 municipalities. The 1998 Brazilian Constitution offered the right to health for all citizens, and created the Unified Health System (SUS): a public system directed at provision of universal, comprehensive, collective, and individual healthcare. The major SUS funders are federal, states, and cities governments, through taxes and social contributions. Public and private providers deliver services which are free at the point of delivery. The private sector covers approximately 25% of the population (48 million people), and dominated by an emergent health insurance market (1).

In 2014, Brazil had a gross domestic product (GDP) per capita of US$15,200, and approximately 9% of its GDP was spent on healthcare. The health expenditure per capita is US$1,109, and 46% of this is funded by public sources (2). The SUS public health financing is considered insufficient, and the equilibrium between public and private systems is challenging with considerable funds flowing from public coffers to private providers (3).

Resource scarcity is a reality in the Brazilian health system. Due to this scarcity, efficient allocation of resources is essential. Economic evaluation methods in healthcare have evolved as an important tool to assess the costs and benefits of health technologies and help decision-makers inform efficient allocation.

Brazil has sought to use economic evaluation to support decision making for rational management of the health system. The Ministry of Health, through the Department of Science and Technology (DECIT), has fostered the development of economic evaluation studies. Since 2006, DECIT has collaborated with the Committee on Incorporation of Technologies of the Ministry of Health (CITEC), an health technology assessment (HTA) body responsible for evaluating the incorporation of new technologies by SUS (4). In 2011, CITEC was replaced by the National Incorporation of Technologies in SUS, CONITEC, introduced the requirement for health economic evaluation (HEE) studies either to help inform policy recommendations for the adoption of new technologies or to review policy recommendations made by SUS (5, 6). The first Brazilian guideline for HEE studies was published in 2009, but the concept of a reference case has not been prescriptively adopted in Brazil yet. The revised HEE Brazilian guideline (7) issued by the Ministry of Health in 2014, presents some recommendations that broadly agree on many methodological specifications of the National Institute for Health and Care Excellence reference case.

In recent decades, a large amount of local HEE studies have been published. There is a strong evidence of the upward stream of blossoming in HEE publications and its acceleration (8). The evolution of scientific literature in health economics published in Brazil between 1986 and 2007 has been evaluated and reported in the published literature (9–11). Recently, Brazil appeared among the top 15 countries in HE research, accounting for 1.7% of identified records, and has been identified as the South American country that published the largest number of HEE studies (8, 12, 13).

Systematic reviews of country-specific HEE studies were conducted earlier in other countries including developed countries, Latin America, Asian, and African countries (14–28).

While a number of HEE have been conducted in Brazil, no study has systematically reviewed the quality of Brazilian HEE. The objective of this systematic review was to assess the state of the HEE research capacity development in Brazil and the ability to conduct good quality HEE. Specifically, this review evaluated the number, characteristics, and quality of reporting of published economic studies in a Brazilian setting.

Materials and Methods

This study followed the guidelines for systematic review of HEE studies published by the Centre for Reviews and Dissemination (CRD) and the preferred reporting items for systematic reviews and meta-analyses statement (29, 30). The protocol is available from the authors on request.

Systematic Search and Identification of Relevant Studies

A broad and exhaustive strategy search was formulated in order to identify all relevant studies published between January 1980 and December 2013. We systematically searched the following electronic data bases: MEDLINE (via PubMed), EMBASE, Latin American, and Caribbean Literature on Health Sciences Database, Scientific Electronic Library Online, NHS Economic Evaluation Database, HTA Database (CRD), Bireme, and Biblioteca Virtual em Saúde Economia da Saúde; citation indexes: SCOPUS, Web of Science, and the Sistema de informação da Rede Brasileira de Avaliação Tecnologia e Saúde (SISREBRATS). We also performed manual searches from the reference lists of included articles, and all issues of the Brazilian Journal of Health Economics (BJHE), a non-indexed journal in the previously mentioned databases in 2013.

The search strategy was reviewed by a librarian specialist and combined subject headings (MeSH and EMTREE) and free text terms (“Health Economics” OR “Economics, Hospital” OR “Economics, Medical” OR “Economics, Nursing” OR “Economics, Pharmaceutical” OR “Economics” OR “costs and cost analysis” OR “Cost” OR “Cost savings” OR “Cost of illness” OR “Analyses, Cost-Benefit” OR “Analysis, Cost-Benefit” OR “Cost-Benefit Analyses” OR “Cost Benefit Analysis” OR “Analyses, Cost Benefit” OR “Analysis, Cost Benefit” OR “Cost Benefit Analyses” OR “cost Effectiveness” OR “Effectiveness, Cost” OR “cost effectiveness analysis” OR “cost-Benefit Date” OR “cost Benefit Date” OR “Date, Cost-Benefit” OR “cost Benefit” OR “Benefits and Costs” OR “Costs and Benefits”) for “economic/cost” concept with subject headings (MeSH and EMTREE) and free text terms (“Brazil” OR “Brazilian” OR “Brazi*”) for “Brazil” concept. Keywords were matched to database specific indexing terms, taking into account the change in the indexing or classification of economic studies in different databases.

Eligibility: Selection Criteria

Articles were included if they were partial or full HEE according to internationally recognized criteria (31, 32), referred to the Brazilian setting, and at least one of the authors was Brazilian and affiliated to a Brazilian institution. Multicenter studies, where Brazil was one of the participating countries, as well as studies conducted on Brazil by foreign authors, were excluded.

Studies were considered partial HEE if they examined only costs (cost description), described costs of a particular disease to society (cost of illness), described costs and outcomes of a single service or program (cost-outcome description), described financial consequences of technology adoption [budget impact analysis (BIA)] or compared only costs of two or more interventions (cost analysis). Studies were considered full HEE if they compared costs and consequences of two or more healthcare interventions alternatives, including cost-consequences analysis (CCA), cost-minimization analysis (CMA), cost-effectiveness analysis (CEA), cost-utility analysis (CUA), and cost-benefit analysis (CBA).

Abstracts, editorials, letters, posters and congress communications, methodological, discussion and review articles, and economic evaluation of other than health technologies (for example, environment) were excluded.

The titles and abstracts of identified citations were screened for relevance independently by two reviewers (Tassia Cristina Decimoni and Roseli Leandro). Disagreements were resolved through discussion or through consultation with a third reviewer (Patrícia Coelho de Soárez). Full texts of selected and those for which inclusion was in doubt were retrieved and independently screened by both reviewers.

Data Extraction

Two reviewers (Tassia Cristina DecimoniTCD and Roseli Leandro) independently extracted data from each of the included studies on year and journal of publication, economic evaluation type, category of technology assessed (drugs, vaccines, equipment, clinical, surgical and diagnostic procedures, public health and health promotion programs), objective of the technology assessed (treatment, prevention, screening, and diagnosis), health problem studied (International statistical classification of diseases and related health problems, 10th revision, ICD-10) (33), first author affiliation (academy, government, research institutes, health organization, consulting, pharmaceuticals or equipment industry, international body), region of the first author, source of funding (research funding agencies, government, consulting, pharmaceuticals, or equipment industry), and authors’ conflict of interest. Conflict of interest was defined according to Valachis et al. (34) who argued that an author may need to declare having a conflict of interest in if she/he has received remuneration in payment or in kind (e.g., stocks or shares) from the manufacturer as a result of any of the following: research support or employment contract (salary, equipment, supply, reimbursement for participation in symposia, and other expenses), or consulting services.

In addition to the above, the standardized extraction form used also contained 17 questions from a systematic review of quality assessment tools (35), and the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) instrument (36). These questions were intended to assess the quality of reporting of full HEE. Quality was defined as the extent to which a study complied and reported items included in the quality assessment tool and CHEERS checklist mentioned above (35, 36). The quality of the sources of evidence used in the studies was assessed with the hierarchy proposed by Coyle and Lee (37, 38). Where data sources ranked 1 are considered to be the most appropriate source (highest quality), and those assigned a rank of 6 are considered the least appropriate (lowest quality) (39) (Table 1). Disagreements on the extracted data were resolved through discussion or through consultation with a third reviewer (Patrícia Coelho de Soárez).

TABLE 1

Table 1. Hierarchies of data sources for health economic evaluation studies modified from Coyle and Lee (37–39).

Data Summary

Data were summarized using qualitative narrative synthesis. The study characteristics are summarized in figures and summary tables.

Data Analysis

Data analysis was performed with descriptive statistics such as absolute frequencies (raw counts) for each category of the discrete variable, relative frequencies (proportions or percentages of the total number of observations), along with analytic statistics that included Pearson correlations to investigate the relationship between the quality of reporting of the full HEE and the publication time period (1980–2005, 2006–2009, and 2010–2013), conflict of interest and source of funding. The publication time periods were chosen because they represent three different stages of the HTA in Brazil (1980–2005: before the establishment of the General Coordination Office for HTA; 2006–2009: establishment of CITEC; 2010–2013; after the publication of the Brazilian guideline for HEE studies). Linear regression models were used to evaluate changes in study characteristics over time. Data analyses were conducted using STATA/SE version 12.1 (Stata Corp, College Station, USA). An alpha level of 5% was used for statistical significance (P ≤ 0.05).

Results

Literature Search

In total 11,841 records were identified from database searches, and 105 further articles were identified through hand-searching in BJHE, SISREBRATS, and other sources. Figure 1 depicts a flow diagram with full details of searches output, and reasons for inclusion/exclusion. We identified 9,304 non-duplicate citations, of which 721 were recognized as potentially relevant and full papers were retrieved. Out of the 721 studies 186 of them were excluded, reasons for exclusion included: thesis (50 studies), not HEE (88 studies), no Brazilian author (19 studies), reviews (18), and other (11 studies), see Figure 1 for detailed description. Scientific papers derived from excluded thesis and reports were included.

FIGURE 1

Figure 1. Flow diagram of health economic evaluation studies in Brazil, 1980–2013.

Study Characteristics

The publication of HEE studies in Brazil started in the 1980s. Since then, there has been an upward trend with a slight increase at the end of the 1990s, and a sharp increase in 2007 with the publication of 356 (67%, 356/535) articles (Figure 2). A total of 535 studies were identified as suitable for inclusion in this review and their characteristics are described in Table 2.

FIGURE 2

Figure 2. Number of health economic evaluation studies in Brazil, by type, 1980–2013.

TABLE 2

Table 2. Characteristics of health economic evaluation (HEE), according time period, Brazil, 1980–2013.

According to internationally agreed classifications of HEE studies (31, 32), more than half of included studies were partial HEE (63.2%, 338/535). Of these, the majority (66%, 223/338) were cost description followed by cost analysis studies (32.2%, 109/338). Of the 197 full HEE, 39.1% (77/197) were CEA, 20.3% (40/197) were CCA, 11% (21/197) were CEA and CUA, 8.6% (17/197) were CUA, 7.1% (14/197) were CMA, and 4.6% (9/197) were CBA. Nine percent (19/197) of the studies concurrently performed more than one type of analysis. CMA evaluated mainly medications (corticosteroids, antihistamines, antibiotics, biologics, monoclonal antibodies, tyrosine kinase inhibitors, chemotherapeutics, anticoagulants, etc.). Prior to 2008, the majority of published evaluations were partial HEE. From 2008 onward, there was an increase in the number of full HEE, and the distribution of full and partial HEE studies became almost equivalent. An initial increase in the number of CEA studies has been followed by a sharp rise in the number of CUA, these have almost quadrupled in the last 3 years. The proportion of CUA studies increased from 17.6% between 2004 and 2009 to 82.4% between 2010 and 2013 (P = 0.028) (Table 2; Figure 3). Out of the 535 included studies, nearly half (n = 248, 46.3%) did not report the type of HEE study performed. Among these, 228 were partial and 20 were full HEE.

FIGURE 3

Figure 3. Full health economic evaluation published in Brazil, 1980–2013. More than one: cost-effectiveness analysis (CEA) and budget impact analysis (BIA); cost-minimization analysis (CMA) and BIA; CMA, cost-benefit analysis (CBA), and cost-consequences analysis (CCA); ACU and ACB; CMA and CBA; CEA, cost-utility analysis (CUA), and BIA.

The review indicated an issue with the classification of study design in the identified HEE studies. According to international criteria (31, 32) of the 287 HEE that reported study type, 28.5% (82/287) had been classified incorrectly. Fifty-two (63.4%, 52/82) were partial analysis and 36.6% (30/82) full analysis. The most frequent misclassification among full HEE was to describe studies as CEA, where on investigation they were found to be CCA (53%, 16/30). Similarly, partial analyses described as cost analyses were cost descriptions (33%, 17/52). Some studies reported as CBA only performed cost analyses (29%, 15/52). Finally, some studies described as cost descriptions were cost analyses (15%, 8/52).

The categories of healthcare technologies that were most frequently assessed were procedures (34.8%, 186/535) and drugs (28.8%, 154/535). The proportion of studies that evaluate procedures increased from 4.3% during the 1980s to 39.2% between 2010 and 2013 (P = 0.002). Technologies assessed included treatment (72.1%, 386/535), prevention (8.6%, 46/535), and diagnostic and treatment (6%, 32/535).

The technologies evaluated in the studies were mainly related to the group of diseases of the Chapter I—certain infectious and parasitic diseases of ICD-10 (17.4%, 93/535), followed by Chapter IX—diseases of the circulatory system (12.9%, 69/535), Chapter II—neoplasms (10.3%, 55/535), and Chapter IV—endocrine, nutritional, and metabolic diseases (9.2%, 49/535).

Studies by Authorship, Journal, Funding Source, and Conflict of Interest

In most of the studies evaluated, the first authors were affiliated to academic institutions (65.1%, 348/535), followed by health organizations (19.8%, 106/535), public administration (5.8%, 31/535), consultancy firms (4.5%, 24/353), pharmaceuticals or equipment industry (2.8%, 15/535), research institutes (1.9%, 10/535), and international organizations (0.2%, 1/535). Although in the majority of publications, the first author was affiliated to an academic institution, there has been an increase of first authors affiliated to health organizations and consultancy firms (Figure 4).

FIGURE 4

Figure 4. Number of health economic evaluation studies in Brazil, by first author affiliation, 1980–2013.

The majority of Brazilian HEE studies were published in medical (55.5%, 297/535), and, public health (20%, 107/535) journals. Only 11.9% (64/535) were published in specialized health economics journals. Three hundred and eighty-eight studies (72.5%, 388/535) were published in Brazilian journals. Of those, 10.6% (41/388) were published in a non-indexed journal.

Regarding the geographical distribution of the first authors, southeast region stands out as a major producer of HEE (73.6%), followed by south (12.5%), and northeast (8.2%). The proportion of publications by regions remained constant during the study period. São Paulo and Rio de Janeiro were the Brazilian states more productive, 51.6 and 14.6%, respectively.

Two hundred and thirty-four studies (44%, 234/535) reported the funding source, among these, 12.4% (29/234) reported no funding and 87.6% (205/234) reported some funding source. Of these, 39% (80/205) were funded by research agencies, 32% (65/205) by industry, 15% (31/205) by the government, and 14% (29/205) had other or multiple funding sources.

Of the 535 studies included in the review, 36% (193/535) declared a conflict of interest, 82% (159/193) declared no conflict of interest. Of the 159 studies that declared no conflicts of interest, 13% (21/159) were considered (according to Valachis et al.) (34) to have a potential conflict of interest due to authors being industry or consultancy firm employees. Similarly, applying Valachis criteria to all 535 included studies, 84% (449/535) would be considered to not have a conflict of interest and 16% (86/535) could be considered to have a conflict of interest. Identified reasons for potential conflict of interest were: 49% (42/86) were developed by consultancy firms and industry; 45% (39/86) had at least one author contractually employed by the industry or funded by it; and 6% (5/86) were related to consultancy work.

Compliance with international recommendations for good reporting in the 197 studies identified as full HEE are presented in Figure 5. Most studies complied with the following items clearly stated: the research question (100%), competing alternatives (99%), primary outcome measure (95%), source of effectiveness estimates (94%), type of model (92%), and economic study design (90%).

FIGURE 5

Figure 5. Percentage of studies complying with recommendations for reporting of full health economic evaluation (n = 197), Brazil, 1980–2013.

Studies by Reporting Quality and Quality of the Sources of Evidence

Thirty-two percent (63/197) of the studies did not state a perspective for their analysis. Only 37% (73/197) described the methods for the estimation of total volume of healthcare resources used in a transparent manner (i.e., reporting quantities and prices/unit costs separately). Less than half (43%, 85/197) of the studies stated the discount rate applied. Other identified caveats associated with studies’ reporting were: failure to declare a conflict of interest, not stating a time horizon for the analysis, omitting to perform, and describe a sensitivity analysis. Of the 197 full HEE, 64.5% (127/197) were model-based economic evaluation. Of these, 92.1% (117/127) reported the type of model used: 41.9% (49/117) used Markov models, 34.2% (40/117) used decision trees, 6% (7/117) used Markov models with decision trees, 6% (7/117) dynamic models, and 12% (14/117) used other types of models.

The reporting quality association with the publication period was statistically significant (P < 0.001). The reporting quality increased progressively during the study period. Most recent periods showed better reporting quality. Studies published between 2010 and 2013 showed better reporting quality compared with those published between 2005 and 2009 and those published between 1980 and 2004. No association was found between the reporting quality and the study source of funding. Moreover, positive association was observed between the reporting quality and the variable conflict of interest. Studies with conflict of interest are associated with a better reporting quality (P < 0.001).

The quality of the sources of evidence used in the studies performed in the Brazilian setting was analyzed in depth using the hierarchy proposed by Coyle and Lee (37–39). Figure 6 presents a graphical representation of results from this analysis. Our findings suggest that poorer quality of information was available for estimating utilities values than costs, resource use, and clinical effect size. No study directly (e.g., via a health state preference evaluation exercise) or indirectly (e.g., used utility values from an alternative patient sample but with the disease of interest) performed utility assessment. Twenty-two percent of the studies estimated utilities parameters from a previous study, 37% had data source or method of elicitation unknown, 12% was based on expert opinion, and 29% was not possible to evaluate. In contrast to utility value estimates data on costs, resource use and clinical effect size were mostly estimated using high-ranked evidence. For instance, more than 50% of the studies used cost information from sources where quality was ranked as 1+, 1, 2+, 2. Costs calculations were mainly based on reliable administrative database or data sources conducted for the specific study, and on recently published cost calculations based on reliable databases. Similarly, in 58% of the studies clinical effect sizes were estimated from meta-analysis or RCTs.

FIGURE 6

Figure 6. Quality of the sources of evidence used in the full health economic evaluation (n = 197), Brazil, 1980–2013.

Discussion

To the best of our knowledge, this the first systematic review describing the number, characteristics, and quality of reporting of HEE studies in Brazil.

The absolute number of all HEE published in Brazil between 1980 and 2013 (n = 535) is substantial. The volume of full HEE (n = 197) directly relevant to Brazilian settings is considerably higher than the number of HEE identified in other countries such as: Italy (n = 92), Colombia (n = 48), South Africa (n = 45), Thailand (n = 39), Korea (n = 33), Iran (n = 30), Vietnam (n = 26), China (n = 26), Saudi Arabia (n = 10), Bangladesh (n = 12), Nigeria (n = 10), and Zimbabwe (n = 3) (14, 15, 17, 19, 21, 24–28, 40). Conversely, Brazil still produces a lower number of HEE compared with high-income countries such as United Kingdom (n = 675), Canada (n = 300), German (n = 283), and Australia (n = 245), where economic evaluation is well established and formally used for regulating reimbursement policies for the adoption of new technologies (16, 20, 41).

BRICS (Brazil, Russia, India, and China) share in global wealth grew tremendously over past decade. In parallel to that, healthcare spending, health economic productivity, and research funding for health economics shifted toward low- and middle-income countries, especially in top emerging BRICS. These circumstances created drivers for further development of health economics in these countries (8, 42–44).

In recent years in Brazil, as in other LAC countries, there has been increased interest in incorporating HEEs as a formal tool to inform decision-making processes (12). This systematic review reveals a steady growing trend in the number of HEE studies being published in the last 6 years (2008–2013). Interestingly, this phenomenon is also observed in Colombia (26).

The steady growth in the economic evaluation literature relevant at national level in Brazil—the focus of this analysis—reached a peak in the year 2007. The reason for this may be multi-fold; firstly, from as early as 1980 Brazilian researchers, funders, and public health system users have successfully promoted the conduct and use of HEE studies. Secondly, internationally there was been an increased interest in HEE studies demonstrated by a continuous growth of published articles and books, as well as the creation of several new specialist journals in this field (45). A third crucially important factor is some recent initiatives of the Brazilian Ministry of Health such as CITEC and CONITEC establishments. The development of this formal structure for regulating reimbursement policies for the adoption of new technologies, provides a strong incentive to promote the implementation and publication of a growing number of HEE studies.

Findings from systematics reviews suggest that the increase in the number of HEE studies published internationally may be related to requirements to use information from these studies to inform reimbursement decisions (41).

As reported in earlier systematic reviews of HEE studies in Latin America (17, 26, 27), our findings indicated that CEA was the most prevalent (39.1%, 77/197) study type for full HEE conducted in Brazilian settings between 1980 and 2013. This might be explained by the relative simplicity of the CEA approach compared with CUA that requires developing robust methodology to value health state preferences.

Despite this, the current review found considerable growth in CUA for the Brazilian setting from 2005 onward (P = 0.028), in line with a growth in CUA observed in the international literature (41). The absolute number of CUA in Brazil (n = 40), however, is still small. This may, in part, be explained by the fact that CUA is more labor and resource intensive than CEA. In addition to this, the 2009 Brazilian HTA guideline gave equal weight to CEA and CUA (46). While CONITEC came into force in 2011 it did not update its methods guidelines to recommend the use of CUA. This is in contrast with current recommendations in a number of countries worldwide (e.g., Australia, Canada, Ireland, New Zealand, Scotland, Sweden, and United Kingdom) (47). CUA was indicated as the preferred type of study, only in the update of the Brazilian HEE guideline published in 2014 (48).

National Incorporation of Technologies in SUS has a formal requirement for BIA alongside a full HEE. In spite of this, the number of published BIA is still limited (n = 15), and most of the so-called BIA submitted to CONITEC are cost studies with 2–5-year annual costs for a specific cohort instead of a real estimate of the financial impact of a new intervention for the Brazilian healthcare system. These findings are consistent with a review of BIA studies (49).

Like other HEE in Latin America, Africa, and South/West Asia most studies focused on infectious disease (14, 21, 23, 36). The majority of these studies focused on vaccines, driven largely by increased investment on CEA and related activities by major global health players such as Bill & Melinda Gates Foundation, GAVI alliance, and the World Health Organization (48).

We found that 75.5% (297/535) of the studies were published in medical and public heath journals. In reviews of Spanish, Iran, German, and South African, HEE studies covering earlier years of publication, the majority of studies (77, 77, 79, and 88%, respectively) were also published in medical journals (13, 21, 24, 49). This high numbers may be related to lower publication standards in medical and public health journals when compared with specialized health economic journals.

Even though researchers prefer to publish their research in international journals, with higher impact factors, and a wider audience. Most of the studies (72.5%) were published in Brazilian journals. Among the 12% published in health economics specialized journals, 8% were published in a national health economic journal, which was not even indexed until 2013.

This may be related to publication requirements as editors of Brazilian journals have less stringent requirements to make use of international methodological guidelines as part of their peer reviewing processes. Consequently, articles with an inferior quality of reporting and equivocal methodological quality are published. Our data also suggest a need for improvements in the peer review process, especially among journals with limited experience publishing economic evaluations (50).

The majority (75%, 148/197) of identified full HEE studies were published between 2008 and 2013, and the reporting quality increased progressively during the study period (P < 0.001). Although overall quality of reporting was considered satisfactory, the review highlighted a number of issues associated with the reporting and methodological quality of the included HEE studies.

Two issues on quality of reporting deserve further attention. Firstly, reporting of funding source, 56%, 110 of the 197 full HEE studies identified here did not state their source of funding. This is in line with findings from reviews on the state of HEE studies conducted in South Africa (45%, 49/108) (24), Nigeria (55%, 24/44) (21), and Zimbabwe (62%, 16/26) (19). Secondly, and directly related to reporting source of funding is declaring potential for any conflicts of interest. Only 36% of the articles reported conflict of interest. As highlighted by Valachis et al. (34), conflicts of interest may be directly related to sources of funding. This is a phenomenon that has been researched by several authors (51–53), if studies are funded by the healthcare industry this could have a direct impact on the conclusion drawn. Industry-sponsored HEE are believed to be more likely to report incremental cost-effectiveness ratios that favor products manufactured by the sponsor. Missing details on these two crucial pieces of information described above may impact on the credibility and transparency of results from HEE studies.

Two final points are identified as requiring further consideration and recommendations for improvement, these are (1) methods for the estimation of resources quantities and unit costs and (2) methods for the estimation of utilities parameters. While the majority of studies provided a source for resource utilization and costs, they omitted details on the identification and quantification of categories of resources and estimation of unit costs. This is in contrast to HEE guidelines (54–56), which indicate that all the relevant quantities of resources should be measured in a correct and transparent manner, and reported separately from the prices (unit costs) of those resources. This lack of detailed reporting on the quantification of healthcare resources and methods used for their valuation limits the ability to replicate costing processes in future studies.

The final critical issue identified for discussion was the poor quality of information used for estimating utilities parameters. Authors’ often reported their main sources of information for the measurement and valuation of preference-based outcomes, these sources of evidence, however, were studies ranked as low quality. Many studies reported having used international data from previously published studies or expert opinion. This is in contrast with international guidelines on HEE which state that utility values obtained from other countries are, in general, not transferable because of cultural differences (57). Recently, in 2011, two of the most widely used generic preference-based utility instruments—EQ-5D and SF-6D—were cross-cultural adapted and validated, in addition societal preferences weights were estimated for the Brazilian population in 2013 (58–61). We expect that current efforts to estimate Brazilian utility weight “tariffs” will increase consistency in quality-adjusted life year calculations in future HEE studies.

One limitation of our study was that we critiqued the reporting, and not necessarily the actual manner that authors conducted their studies. However, this review was useful to assess the practice of HEE in the Brazilian setting.

Conclusion

This review identified that an increasing number of HEE studies are being conducted and published in Brazil. Their reporting quality has increased progressively during the study period. Overall, the quality of these HEE studies is satisfactory, but we identified key areas where significant improvements could be made such as: reporting of funding source, conflict of interest, methods for the estimation of resources quantities and unit costs, methods and source of evidence to estimate utility parameters utilities parameters. Our findings can contribute to improve the way HEE studies are designed, implemented, and reported in Brazil.

Consent for Publication

There are no any individual person’s data.

Ethics Statement

There are no human participants involved.

Author Contributions

All authors drafted the systematic review protocol. TD, RL, and PS conducted the search, selection of records, and data extraction. Quality appraisal was conducted by TD, LR, and PS. All authors have read and approved the final manuscript.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Funding

Supported by the Fundação de Amparo à Pesquisa do Estado de São Paulo (FAPESP, São Paulo Research Foundation), as part of a larger project entitled “Systematic review of health economic evaluations conducted in Brazil, 1980–2013,” carried out under the leadership of Professor Patricia Coelho de Soárez (FAPESP Research Grant no. 2012/22126-3). The research design, analysis, and interpretation were conducted independently of the funders.

Abbreviations

HEEs, health economic evaluations; SUS, Unified Health System; GDP, gross domestic product; DECIT, Department of Science and Technology; CITEC, Committee on Incorporation of Technologies of the Ministry of Health; CONITEC, National Incorporation of Technologies in SUS; NICE, National Institute for Health and Care Excellence; CCA, cost-consequences analysis; CMA, cost-minimization analysis; CEA, cost-effectiveness analysis; CUA, cost-utility analysis; CBA, cost-benefit analysis; CHEERS, Consolidated Health Economic Evaluation Reporting Standards; HTA, health technology assessment; BJHE, Brazilian Journal of Health Economics; SISREBRATS, Sistema de Informação da Rede Brasileira de Avaliação de Tecnologia em Saúde; QALY, quality-adjusted life year.

References

1. Marten R, McIntyre D, Travassos C, Shishkin S, Longde W, Reddy S, et al. An assessment of progress towards universal health coverage in Brazil, Russia, India, China, and South Africa (BRICS). Lancet (2014) 384(9960):2164–71. doi:10.1016/S0140-6736(14)60075-1

PubMed Abstract | CrossRef Full Text | Google Scholar

2. OECD. OECD Health Statistics 2014. How Does Brazil Compare? The Organisation for Economic Co-operation and Development (OECD) (2014).

Google Scholar

3. Rajkumar AS, Cavagnero E, Class D, Ferl K. Health Financing Profile—Brazil. Washington: World Bank Group (2014).

Google Scholar

4. Health technology assessment: institutionalization of actions in the Brazilian Ministry of Health. Rev Saude Publica (2006) 40(4):743–7. doi:10.1590/S0034-89102006000500029

CrossRef Full Text | Google Scholar

5. Brasil. Ministério da Saúde. Lei n° 12.401, de 28 de abril de 2011. Altera a Lei n° 8.080, de 19 de setembro de 1990 para dispor sobre a assistência terapêutica e a incorporação de tecnologia em saúde no âmbito do Sistema Único de Saúde. (2011).

Google Scholar

6. Brasil. Ministério da Saúde. Decreto n° 7.646, de 21 de dezembro de 2011. Regulamenta a Lei n° 12.401, de 28 de abril de 2011 e Dispõe sobre a Comissão Nacional de Incorporação de Tecnologias no Sistema Único de Saúde e sobre o processo administrativo para incorporação, exclusão e alteração de tecnologias em saúde pelo Sistema Único de Saúde – SUS. (2011).

Google Scholar

7. Brasil. Ministério da Saúde. Secretaria de Ciência, Tecnologia e Insumos Estratégicos. Departamento de Ciência e Tecnologia. Diretrizes metodológicas: diretriz de Avaliação Econômica/Ministério da Saúde, Secretaria de Ciência, Tecnologia e Insumos Estratégicos. 2nd ed. In: da Saúde M, editor. Secretaria de Ciência, Tecnologia e Insumos Estratégicos. Brasília: Ministério da Saúde (2014). 132 p.

Google Scholar

8. Jakovljevic MM, Pejcic AV. Growth of global publishing output of health economics in the twenty-first century: a bibliographic insight. Front Public Health (2017) 5:211. doi:10.3389/fpubh.2017.00211

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Vianna SM. Evolução e perspectivas da pesquisa em economia da saúde no Brasil. Brasília: IPEA (1998).

Google Scholar

10. SG S. Estudo bibliométrico das publicações em economia da saúde, no Brasil, 1989–1998/Bibliometric Study of Publications in Health Economics, Brazil, 1989 to 1998. Brazil: Universidade de São Paulo (2000).

Google Scholar

11. Andrade EIG, Acúrcio FA, Cherchiglia ML, Belisário SA, Júnior AAG, Szuster DAC, et al. Pesquisa e produção científica em economia da saúde no Brasil. Rev Admin Pública (2007) 41(2):211–35. doi:10.1590/S0034-76122007000200003

CrossRef Full Text | Google Scholar

12. Iglesias CP, Drummond MF, Rovira J, Group NP. Health-care decision-making processes in Latin America: problems and prospects for the use of economic evaluation. Int J Technol Assess Health Care (2005) 21(1):1–14. doi:10.1017/S0266462305050014

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Augustovski F, Iglesias C, Manca A, Drummond M, Rubinstein A, Martí SG. Barriers to generalizability of health economic evaluations in Latin America and the Caribbean region. Pharmacoeconomics (2009) 27(11):919–29. doi:10.2165/11313670-000000000-00000

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Lee KS, Brouwer WB, Lee SI, Koo HW. Introducing economic evaluation as a policy tool in Korea: will decision makers get quality information? A critical review of published Korean economic evaluations. Pharmacoeconomics (2005) 23(7):709–21. doi:10.2165/00019053-200523070-00005

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Cornago D, Li Bassi L, De Compadri P, Garattini L. Pharmacoeconomic studies in Italy: a critical review of the literature. Eur J Health Econ (2007) 8(2):89–95. doi:10.1007/s10198-006-0011-8

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Schwappach DL, Boluarte TA. HEE-GER: a systematic review of German economic evaluations of health care published 1990–2004. BMC Health Serv Res (2007) 7:7. doi:10.1186/1472-6963-7-7

CrossRef Full Text | Google Scholar

17. Teerawattananon Y, Russell S, Mugford M. A systematic review of economic evaluation literature in Thailand: are the data good enough to be used by policy-makers? Pharmacoeconomics (2007) 25(6):467–79. doi:10.2165/00019053-200725060-00003

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Drummond MF, Iglesias CP, Cooper NJ. Systematic reviews and economic evaluations conducted for the National Institute for Health and Clinical Excellence in the United Kingdom: a game of two halves? Int J Technol Assess Health Care (2008) 24(2):146–50. doi:10.1017/S0266462308080203

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Gavaza P, Rascati K, Brown C, Lawson K, Mann T. The state of health economic and pharmacoeconomic evaluation research in Zimbabwe: a review. Curr Ther Res Clin Exp (2008) 69(3):268–85. doi:10.1016/j.curtheres.2008.06.005

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Dalziel K, Segal L, Mortimer D. Review of Australian health economic evaluation—245 interventions: what can we say about cost effectiveness? Cost Eff Resour Alloc (2008) 6:9. doi:10.1186/1478-7547-6-9

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Gavaza P, Rascati KL, Oladapo AO, Khoza S. The state of health economic evaluation research in Nigeria: a systematic review. Pharmacoeconomics (2010) 28(7):539–53. doi:10.2165/11536170-000000000-00000

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Hoque ME, Khan JA, Hossain SS, Gazi R, Rashid HA, Koehlmoos TP, et al. A systematic review of economic evaluations of health and health-related interventions in Bangladesh. Cost Eff Resour Alloc (2011) 9:12. doi:10.1186/1478-7547-9-12

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Hoomans T, Severens JL, van der Roer N, Delwel GO. Methodological quality of economic evaluations of new pharmaceuticals in the Netherlands. Pharmacoeconomics (2012) 30(3):219–27. doi:10.2165/11539850-000000000-00000

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Gavaza P, Rascati KL, Oladapo AO, Khoza S. The state of health economic research in South Africa: a systematic review. Pharmacoeconomics (2012) 30(10):925–40. doi:10.2165/11589450-000000000-00000

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Al-Aqeel SA. State of health economic evaluation research in Saudi Arabia: a review. Clinicoecon Outcomes Res (2012) 4:177–84. doi:10.2147/CEOR.S31087

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Atehortúa S, Ceballos M, Gaviria CF, Mejía A. Quality assessment of economic evaluations in health care in Colombia: a systematic review. Biomedica (2013) 33(4):615–30. doi:10.7705/biomedica.v33i4.1464

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Haghparast-Bidgoli H, Kiadaliri AA, Skordis-Worrall J. Do economic evaluation studies inform effective healthcare resource allocation in Iran? A critical review of the literature. Cost Eff Resour Alloc (2014) 12:15. doi:10.1186/1478-7547-12-15

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Tran BX, Nong VM, Maher RM, Nguyen PK, Luu HN. A systematic review of scope and quality of health economic evaluation studies in Vietnam. PLoS One (2014) 9(8):e103825. doi:10.1371/journal.pone.0103825

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Chapter 5: Systematic Reviews of Economic Evaluations. CRD’ Guidance for Undertaking Reviews in Health Care. Centre for Reviews and Disseminations, University of York (2009).

Google Scholar

30. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. J Clin Epidemiol (2009) 62(10):1006–12. doi:10.1016/j.jclinepi.2009.06.005

CrossRef Full Text | Google Scholar

31. Drummond M, Sculpher M, Torrance G, O’Brien B, Stoddart G. Methods for the Economic Evaluation of Health Care Programmes. 3rd ed. New York: Oxford Medical Publications (2005). 379 p.

Google Scholar

32. Craig D, Rice S. NHS Economic Evaluation Database (NHS EED) Handbook 2007. 3rd ed. York, UK: Centre for Reviews and Dissemination, University of York (2007). 110 p. Available from: https://pdfs.semanticscholar.org/9700/ac214b5b4a635819d1bf26b2d6e8dd41036d.pdf

Google Scholar

33. Organization WH. International Classifications of Diseases. ICD-10 Version. (2016). Available from: http://apps.who.int/classifications/icd10/browse/2016/en

Google Scholar

34. Valachis A, Polyzos NP, Nearchou A, Lind P, Mauri D. Financial relationships in economic analyses of targeted therapies in oncology. J Clin Oncol (2012) 30(12):1316–20. doi:10.1200/JCO.2011.38.6078

CrossRef Full Text | Google Scholar

35. Walker DG, Wilson RF, Sharma R, Bridges J, Niessen L, Bass EB, et al. Best Practices for Conducting Economic Evaluations in Health Care: A Systematic Review of Quality Assessment Tools. Methods Research Report. Rockville, MD: Agency for Healthcare Research and Quality (2012). Prepared by Johns Hopkins University Evidence-based Practice Center under Contract No. 290-2007-10061-I. AHRQ Publication No.12(13)-EHC132-EF. Available from: www.effectivehealthcare.ahrq.gov/reports/final.cfm

Google Scholar

36. Husereau D, Drummond M, Petrou S, Carswell C, Moher D, Greenberg D, et al. Consolidated health economic evaluation reporting standards (CHEERS) – explanation and elaboration: a report of the ISPOR Health Economic Evaluation Publication Guidelines Good Reporting Practices task force. Value Health (2013) 16(2):231–50. doi:10.1016/j.jval.2013.02.002

CrossRef Full Text | Google Scholar

37. Coyle D, Lee KM. Evidence-based economic evaluation: how the use of different data sources can impact results. In: Donaldson C, Mugford M, Vale L, editors. Evidence-Based Health Economics: From Effectiveness to Efficiency in Systematic Review. London: BMJ Publishing Group (2002). p. 55–66.

Google Scholar

38. Cooper N, Coyle D, Abrams K, Mugford M, Sutton A. Use of evidence in decision models: an appraisal of health technology assessments in the UK since 1997. J Health Serv Res Policy (2005) 10(4):245–50. doi:10.1258/135581905774414187

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Kaltenthaler E, Tappenden P, Paisley S, Squires H. Nice DSU Technical Support Document 13: Identifying and Reviewing Evidence to Inform the Conceptualisation and Population of Cost-Effectiveness Models. Sheffield: Decision Support Unit, ScHARR, University of Sheffield (2011).

Google Scholar

40. Thorat T, Lin PJ, PJ N. The state of cost-utility analyses in Asia: a systematic review. Value Health Reg Issues (2015) 6C:7–13. doi:10.1016/j.vhri.2015.02.001

CrossRef Full Text | Google Scholar

41. Neumann PJ, Thorat T, Shi J, Saret CJ, Cohen JT. The changing face of the cost-utility literature, 1990–2012. Value Health (2015) 18(2):271–7. doi:10.1016/j.jval.2014.12.002

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Jakovljevic MM, Ogura S. Health economics at the crossroads of centuries—from the past to the future. Front Public Health (2016) 4:115. doi:10.3389/fpubh.2016.00115

CrossRef Full Text | Google Scholar

43. Jakovljevic M, Potapchik E, Popovich L, Barik D, Getzen TE. Evolving health expenditure landscape of the BRICS nations and projections to 2025. Health Econ (2017) 26(7):844–52. doi:10.1002/hec.3406

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Jakovljevic MB. BRIC’s growing share of global health spending and their diverging pathways. Front Public Health (2015) 3:135. doi:10.3389/fpubh.2015.00135

CrossRef Full Text | Google Scholar

45. Hutton J. ’Health economics’ and the evolution of economic evaluation of health technologies. Health Econ (2012) 21(1):13–8. doi:10.1002/hec.1818

CrossRef Full Text | Google Scholar

46. Brasil. Ministério da Saúde. Portaria n° 3.323, de 27 de dezembro de 2006. Institui a comissão para incorporação de tecnologias no âmbito do Sistema Único de Saúde e da Saúde Suplementar. (2006).

Google Scholar

47. Brasil. Ministério da Saúde. Portaria n° 2.587, de 30 de outubro de 2008. Dispõe sobre a Comissão de Incorporação de Tecnologias no Ministério da Saúde e vincula sua gestão à Secretaria de Ciência, Tecnologia e Insumos Estratégicos. (2008).

Google Scholar

48. Novaes H, Elias F. Uso da avaliação de tecnologias em saúde em processos de análise para incorporação de tecnologias no Sistema Único de Saúde no Ministério da Saúde. Cad Saúde Pública (2013) 29(Sup):S7–16. doi:10.1590/0102-311X00008413

CrossRef Full Text | Google Scholar

49. Brasil. Ministério da Saúde. Secretaria de Ciência, Tecnologia e Insumos Estratégicos. Departamento de Ciência e Tecnologia. Diretrizes Metodológicas: estudos de avaliação econômica de tecnologias em saúde. Brasília (2009). Availablefrom:http://www.ispor.org/PEguidelines/source/Economic-Evaluation-Guidelines-in-Brazil-Final-Version-2009.pdf

Google Scholar

50. Barnieh L, Manns B, Harris A, Blom M, Donaldson C, Klarenbach S, et al. A synthesis of drug reimbursement decision-making processes in organisation for economic co-operation and development countries. Value Health (2014) 17(1):98–108. doi:10.1016/j.jval.2013.10.008

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Barbieri M, Drummond MF. Conflict of interest in industry-sponsored economic evaluations: real or imagined? Curr Oncol Rep (2001) 3(5):410–3. doi:10.1007/s11912-001-0027-2

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Jang S, Chae YK, Haddad T, Majhail NS. Conflict of interest in economic analyses of aromatase inhibitors in breast cancer: a systematic review. Breast Cancer Res Treat (2010) 121(2):273–9. doi:10.1007/s10549-010-0870-7

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Neumann PJ, Fang CH, Cohen JT. 30 years of pharmaceutical cost-utility analyses: growth, diversity and methodological improvement. Pharmacoeconomics (2009) 27(10):861–72. doi:10.2165/11312720-000000000-00000

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Williams A. The cost-benefit approach. Br Med Bull (1974) 30(3):252–6. doi:10.1093/oxfordjournals.bmb.a071211

CrossRef Full Text | Google Scholar

55. Weinstein MC, Stason WB. Foundations of cost-effectiveness analysis for health and medical practices. N Engl J Med (1977) 296(13):716–21. doi:10.1056/NEJM197703312961304

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Drummond M, Brandt A, Luce B, Rovira J. Standardizing methodologies for economic evaluation in health care. Practice, problems, and potential. Int J Technol Assess Health Care (1993) 9(1):26–36. doi:10.1017/S0266462300003007

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Barbieri M, Drummond M, Rutten F, Cook J, Glick HA, Lis J, et al. What do international pharmacoeconomic guidelines say about economic data transferability? Value Health (2010) 13(8):1028–37. doi:10.1111/j.1524-4733.2010.00771.x

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Campolina AG, Bortoluzzo AB, Ferraz MB, Ciconelli RM. Validation of the Brazilian version of the generic six-dimensional short form quality of life questionnaire (SF-6D Brazil). Cien Saude Colet (2011) 16(7):3103–10. doi:10.1590/S1413-81232011000800010

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Cruz LN, Camey SA, Hoffmann JF, Rowen D, Brazier JE, Fleck MP, et al. Estimating the SF-6D value set for a population-based sample of Brazilians. Value Health (2011) 14(5 Suppl 1):S108–14. doi:10.1016/j.jval.2011.05.012

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Andrade MV, Noronha KV, Maia AC, Kind P. What matters most? Evidence-based findings of health dimensions affecting the societal preferences for EQ-5D health states. Cad Saude Publica (2013) 29(Suppl 1):S59–72. doi:10.1590/0102-311X00021113

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Andrade MV, Noronha K, Kind P, Maia AC, Menezes RM, Reis CB, et al. Societal prefrences for EQ-5D health states from a Brazilian population survey. Value Health Reg Issues (2013) 2:405–12. doi:10.1016/j.vhri.2013.01.009

CrossRef Full Text | Google Scholar