Knowledge Mapping of Multicriteria Decision Analysis in Healthcare: A Bibliometric Analysis

Objective: Multicriteria decision analysis (MCDA) is a useful tool in complex decision-making situations, and has been used in medical fields to evaluate treatment options and drug selection. This study aims to provide valuable insights into MCDA in healthcare through examining the research focus of existing studies, major fields, major applications, most productive authors and countries, and most common journals in the domain.

Methods: A bibliometric analysis was conducted on the publication related to MCDA in healthcare from the Web of Science Core Collection (WoSCC) database on 14 July 2021. Three bibliometric software (VOSviewer, R-bibliometrix, and CiteSpace) were used to conduct the analysis including years, countries, institutes, authors, journals, co-citation references, and keywords.

Results: A total of 410 publications were identified with an average yearly growth rate of 32% (1999–2021), from 196 academic journals with 23,637 co-citation references by 871 institutions from 70 countries/regions. The United States was the most productive country (n = 80). Universiti Pendidikan Sultan Idris (n = 16), Université de Montréal (n = 13), and Syreon Research Institute (n = 12) were the top productive institutions. A A Zaidan, Mireille Goetghebeur and Zoltan Kalo were the biggest nodes in every cluster of authors' networks. The top journals in terms of the number of articles (n = 17) and citations (n = 1,673) were Value in Health and Journal of Medical Systems, respectively. The extant literature has focused on four aspects, including the analytic hierarchy process (AHP), decision-making, health technology assessment, and healthcare waste management. COVID-19 and fuzzy TOPSIS received careful attention from MCDA applications recently. MCDA in big data, telemedicine, TOPSIS, and fuzzy AHP is well-developed and an important theme, which may be the trend in future research.

Conclusion: This study uncovers a holistic picture of the performance of MCDA-related literature published in healthcare. MCDA has a broad application on different topics and would be helpful for practitioners, researchers, and decision-makers working in healthcare to advance the wheel of medical complex decision-making. It can be argued that the door is still open for improving the role of MCDA in healthcare, whether in its methodology (e.g., fuzzy TOPSIS) or application (e.g., telemedicine).

Introduction

Since the emergence of multicriteria decision analysis (MCDA) in the 1970s (1), it has been widely used in many non-health sectors (2–6), such as energy, environment, military, management, architecture, and local and central government. MCDA is a sub-discipline of operations research with foundations in economics, management, mathematics, and psychology. MCDA is a process that integrates objective measurement with value judgment while also attempting to manage subjectivity (7). When facing complex decision-making situations in which multiple and conflicting criteria, objectives, or attributes are combined or aggregated, MCDA is a decision aid that helps stakeholders summarize complex value trade-offs in a way that is consistent and transparent, thus leading to fairer decision-making (7, 8). In the last decade, MCDA has been increasingly used as a transparent, participatory framework for supporting decision-making and policy setting in healthcare (9, 10). Decision-making in healthcare is inherently complex, as it usually involves confronting trade-offs between multiple objectives, requires the involvement of many stakeholders, and is often a constrained resource. This is particularly common in the field of health technology assessment (HTA). Decision-makers use explicit methods to determine the value of health technology at different points in its lifecycle (11). Evidence and Value:Impact on DEcisionMaking (EVIDEM) framework was developed by combining standardized HTA report with MCDA to promote transparent and efficient healthcare decision-making through systematic assessment and dissemination of the evidence and values on which decisions are based (12, 13).

MCDA can quantify benefits, risks, and uncertainties arising in decision-making, by considering an explicit set of criteria and their relative importance under a fully transparent process, while incorporating a wide range of stakeholder views to express a more societal perspective (14). MCDA process broadly involves problem structuring (i.e., selection of participants, alternatives, and criteria); modeling (i.e., weighting, scoring, and aggregation); and decision-making (i.e., interpretation of results and decision-making) (7). MCDA has been used in healthcare in different contexts, ranging from the marketing authorization stage, coverage decisions, the decision while prescribing, to other types of policy questions (14).

There have been several literature reviews that assessed different MCDA methods applied in healthcare decision-making under specific healthcare contexts. For instance, one systematic review in 2020 analyzed the use of multicriteria software in health priority settings and found that only a few studies used MCDA software in healthcare decision-making (15). One systematic review in 2015 showed that MCDA has been applied to a broad range of areas in healthcare, with the use of a variety of methodological approaches (16). Another systematic review in 2015 reported the applications of MCDA methods in decisions addressing the trade-off between costs and benefits at specific phases of medical innovation (17). One scoping review in 2019 assessed 70 case studies about the application of MCDA from three aspects: type of health services, type of interventions, and healthcare area (18). One review in 2014 assessed the value of healthcare interventions using MCDA, including pharmaceuticals, public health interventions, screening, surgical interventions, and devices (19). Recently, an extensive narrative review described the performance of fuzzy MCDA in the emergency system in the coronavirus disease 2019 (COVID-19) pandemic (20). It highlighted the importance of the fuzzy MCDA method as a beneficial tool for healthcare workers and first responders' emergency professionals to face this pandemic. Almost all of these reviews concluded and highlighted the need for guidance on the application of MCDA. Thus, it is imperative to provide an overall quantitative and longitudinal perspective of the works on this topic over time.

Bibliometrics refers to the quantitative analysis of all the knowledge carriers of a certain discipline using statistical and mathematical methods (21). It can be used not only to examine the history of scientific research in a specific field but also to identify potential future research directions and collaboration opportunities. In this study, scientometric visualization softwares—VOSviewer, R—bibliometrix, and CiteSpace—were used as a text mining and visualization tool for bibliometric analysis (22–24). Two previous bibliometric literature analyses of MCDA were performed covering the period from 1960 to 2011 (9) and 1980 to 2013 (16), respectively. Therefore, to help realize the current evidence landscape of global MCDA studies, we conducted a new bibliometric analysis of MCDA research to characterize global collaboration patterns, and map the developmental trends of MCDA over the past years.

The purpose of this study is to (1) analyze the distribution of publication outputs, countries, institutes, authors, journals, keywords, and references on MCDA research; (2) identify the cooperation of countries and institutes; (3) and explore the research trend and existing hotspots, which will help readers learn more about MCDA in healthcare, and the justification and significance of this study's analysis are obtained from two research topics: issues in the literature review on MCDA and the future trends. The following two suggested research questions (RQ) will help the study accomplish its aims.

RQ1.What changes have occurred in the literature on MCDA (e.g., the most influential research, important references with the most impact on the studies, the journals on this topic, and the change in the number of publications over time)?

RQ2.What are the most important topics and problems discussed in the scholarly literature on MCDA?

Methods

Data Source

In view of the special requirements of bibliometric software for data structure and content, the data used for analysis were collected from the following Web of Science Core Collection (WoSCC) indexes: the “Science Citation Index-Expanded (SCI-EXPANDED) (1985–2021)”, “Social Sciences Citation Index (SSCI) (1985–2021)”, “Emerging Sources Citation Index (ESCI) (2015–2021)”, and “Arts & Humanities Citation Index (A&HCI) (1985–2021)”, performed from 1 January 1985 to 14 July 2021. The searching terms were as follows: TS = (“multi-criteria decision analysis” OR “multiple criteria decision analysis” OR “multicriteria decision analysis” OR “multi-criteria decision making” OR “multicriteria decision making” OR “multiple criteria decision making” OR “multi-attribute decision analysis” OR “multiple attribute decision analysis” OR “multiattribute decision making” OR MCDA OR MCDM) AND TS = (“health care” OR healthcare OR health-care). We restricted our search results to papers published in English only. And the literature types are limited to Article and Review.

Finally, literature records, including titles, abstracts, and cited references, downloaded as plain text, formed the local database for subsequent analysis with VOSviewer, CiteSpace, and R-bibliometrix. The records were preprocessed with the deduplication function of CiteSpace.

Study Selection

To avoid the bias caused by frequent database updates, all literature was retrieved and downloaded on the same day (14 July 2021). The title and abstract were read by two authors independently. All the research marked as potentially relevant by either author was advanced to the full-text review stage. Two authors read each selected full-text article separately to determine which research would be included based on the pre-defined selection criteria. Any discrepancies were resolved by the third author.

Bibliometric Analysis and Visualization

In this study, Microsoft Office Excel 2016 was used to manage the data, and analyze the publication trend with linear regression. Publication data including publication counts, countries, institutes, authors, journals, references, and keywords were extracted from the WoSCC search results. The impact factor (IF) of the academic journals was collected from the 2020 Journal Citation Reports (JCR) (Clarivate Analytics, Philadelphia, United States). VOSviewer (1.6.17), a network analysis software tool used to analyze bibliometric data, was used to identify productive countries/regions, institutions, authors, and co-occurrence keywords. In VOSviewer network maps, different nodes indicate components, while the size of the nodes reflects the number of publications or co-occurrence frequencies. The links between nodes represent the co-occurrence relationships, and the size of the links indicates the co-occurrence frequencies between nodes. The VOSviewer settings were as follows: counting method (full counting), while thresholds (T) of items (countries/regions, institutions, authors, and keywords) with more than five publications were adopted. The bibliometrix package in R (4.1.1) was utilized to analyze the research hotspots and trends with word dynamics, trend topic, and thematic map. CiteSpace software (5.7.R2) was utilized for visual exploration of the distribution of co-citation references with the timezone diagram. The visual network created consists of nodes and lines. The nodes in the timezone diagram indicate the co-citation references. The size of the nodes reflects the number of citation frequencies. The colors of the node and line represent different years. The X-axis represents the time. The parameters of CiteSpace were as follows: time (1999–2021), years per slice (1), links (strength: cosine, scope: within slices), and selection criteria (Top N Per slice = 10).

Results

Search Results

A total of 461 articles in the WoSCC database were retrieved on 14 July 2021 (see Supplementary File 1). They were contained in four indexes, including SCI-Expanded (n = 353), SSCI (n = 253), ESCI (n = 58), and A&HCI (n = 1) (see Supplementary File 2). In total, the majority of the publications (n = 203) are indexed in both SCI-Expanded and SSCI Indexes, followed by SCI-Expanded alone (n = 150). Finally, 410 met the selection criteria and were used for the bibliometric analysis (Figure 1). The result of distribution by time indicates that the literature in this field was published between 1999 and 2021, including 360 articles and 50 reviews. They were published in 196 academic journals with 23,637 co-citation references by 871 institutions from 70 countries/regions around the world.

FIGURE 1

Figure 1. Flowchart of the study selection process.

Publication Outputs

The annual growth trends of publications regarding MCDA are shown in Figure 2. There is a steady increase in publications by year, with a notably higher output in the last 5 years (2016–2021, 76.83%). The earliest published literature comes from 1999 and the publications increased at an average rate of 9.6 articles per year from 2016 to 2021. According to the characteristics of the publication outputs, a research period was divided into two periods (1999–2013, and 2014–2021) (Figure 3). The period from 1999 to 2013 was a slow growth period, with a small number of 41 articles published; 2014–2020 was a period of rapid growth, and the number of TP showed a significant ascending curve, which accounted for 90% of all the articles.

FIGURE 2

Figure 2. Annual trend chart of publications.

FIGURE 3

Figure 3. The curve fitting results of annual publications.

Regions, Institutes, and Authors' Analysis

A total of 70 countries contributed to the publications on MCDA in healthcare, and 35 countries with more than five publications were analyzed (Figure 4). The top five ranked countries by publication count were United States (80), the United Kingdom (25), Turkey (26), China (27), and Canada (28).

FIGURE 4

Figure 4. Collaboration network of countries/regions. The size of a single node represents the number of publications of the countries/regions. The color of the nodes represents the publication in different countries with the change of time (yellow: earlier, orange: later).

A total of 871 institutions published articles related to MCDA, out of which 34 institutions published more than five publications (Figure 5). The top leading institution by publication count was Universiti Pendidikan Sultan Idris (n = 16) in Malaysia, mainly published by the Department of Computing. Besides, Université de Montréal (n = 13) in Canada was the second, followed by Syreon Research Institute (n = 12) in Hungary, University of Twente (n = 11) in the Netherlands, and Eötvös Loránd University (n = 11) in Hungary. In terms of average citations (Citations/Documents) of articles, Office Health Economics (492/6), the United Kingdom; University of Washington (536/8), the United States; Tongji University (460/7), China; Shanghai University (382/6), China; and University of Twente (685/11), the Netherlands, have higher average citations.

FIGURE 5

Figure 5. Collaboration network of institutions. The size of a single node represents the number of publications of the institutions. The color of the nodes represents the average citation in different institutions.

A total of 1,475 authors published articles related to MCDA, and 25 authors with more than five publications were analyzed (Figure 6). The top 10 leading authors by publication count published 124 articles in total. A A Zaidan (n = 17) ranked first, followed by B B Zaidan (n = 16), O S Albahri (n = 15), A S Albahri (n = 15), M A Alsalem (n = 13), Zoltan Kalo (n = 12), Mireille Goetghebeur (n = 10), Kevin Marsh (n = 10), Rob Baltussen (n = 9), and M Hashim (n = 7). Figure 6 shows the network visualization between the coauthors of MCDA research. Three teams contributed to most publications. The biggest cluster mainly consisted of A A Zaidan, M A Alsalem, and M Hashim who are from Universiti Pendidikan Sultan Idris. The next cluster is mainly composed of Zoltan Kalo and Kevin Marsh, followed by the third cluster with Mireille Goetghebeur and Monika Wagner.

FIGURE 6

Figure 6. Collaboration network of authors. The size of a single node represents the number of publications of the author. Each color represents a different cluster.

Journal Analysis

A total of 410 publications were published in 196 academic journals and were cited 23,637 times. The 15 most productive journals and 15 most co-citation journals in the MCDA research are presented in Table 1. The most productive journal was the Value in Health (n = 17, IF2020 = 5.725, Q1), followed by BMC Medical Informatics and Decision Making (n = 13, IF2020 = 2.796, Q3), Expert Review of Pharmacoeconomics & Outcomes Research (n = 12, IF2020 = 2.217, Q3/Q4), Journal of Medical Systems (n = 11, IF2020 = 4.460, Q1/Q2), and Cost Effectiveness and Resource Allocation (n = 9, IF2020 = 2.532, Q3). The 15 most co-citation journals had 8,967 citations. Journal of Medical Systems had the most citations (n = 1673), followed by Value in Health (n = 942), BMC Medical Informatics and Decision Making (n = 746), Expert Systems with Applications (n = 632), and Expert Review of Pharmacoeconomics & Outcomes Research (n = 563).

TABLE 1

Table 1. The top 15 most productive journals and most co-citation journals in MCDA research.

Co-citation Analysis

A timezone graph is a view of representing knowledge evolution in time. All the nodes in the timezone graph are located in a two-dimensional coordinate with the horizontal axis of time. According to the time of the first cited, the nodes are set in different timezones, and their positions are upward with the time axis. The co-citation timezone graph can clearly show the number and relationship of co-citation references in different periods. The top 10 cited references were as follows, Thokala P 2016 (29) (n = 69), Marsh K 2016 (30) (n = 45), Adunlin G 2015 (16) (n = 27), Marsh K 2014 (19) (n = 26), Thokala P 2012 (31) (n = 24), Diaby V 2013 (9) (n = 23), Muhlbacher AC 2016 (32) (n = 18), Dolan JG 2010 (33) (n = 16), Sussex J 2013 (34) (n = 14), and Goetghebeur M 2012 (35) (n = 14) (see Figure 7).

FIGURE 7

Figure 7. References' timezone network. The nodes indicate the co-citation references. The size of the nodes reflects the number of citation frequencies. The colors of the node and line represent different years. The X-axis represents the time.

Keywords Analysis

An analysis of keywords used in articles on MCDA is shown in Figure 8. After merging the synonyms of keywords, such as “multiple criteria decision analysis” and “multi-criteria decision analysis” into “MCDA,” a total of 1,091 keywords were found. We analyzed 42 keywords that were identified as having occurred more than five times (Figure 8). The results showed that the top 10 keywords are “MCDA” (n = 224), “analytic hierarchy process (AHP)” (n = 46), “decision-making” (n = 35), “healthcare” (n = 34), “HTA” (n = 31), “healthcare waste management (HWM)” (n = 20), “priority setting” (n = 19), “technique for order preference by similarity to an ideal solution (TOPSIS)” (n = 18), “fuzzy AHP” (n = 16), and “Vise Kriterijumska Optimizacija I Kompromisno Resenje (VIKOR)” (n = 16). The colors in the overlay visualization indicated the average publication year of the identified keywords. The majority of keywords were published after 2016, with yellow or orange color.

FIGURE 8

Figure 8. Co-occurrence network of keywords according to the average publication year. The size of a single node represents the number of publications of each keyword. The color of the nodes represents the emergence time of different keywords (yellow: earlier, orange: later).

To further study the research hotspot tendency, we conducted the word dynamics, trend topic, and thematic map analysis by R-bibliometrix.

Analyzing the annual growth of keywords, the top 20 keywords, such as “MCDA”, “AHP”, “decision-making”, “HTA”, “healthcare”, and “priority setting”, are shown in Figure 9. Among them, “MCDA” showed the J-shaped curves, with sustained growth from a continuous growth trend from 2001 to 2021. The growth trend of four keywords (“AHP”, “decision-making”, “HTA”, and “healthcare”) is relatively stable compared with “MCDA”. The most recent “COVID-19” grew rapidly.

FIGURE 9

Figure 9. The cumulative growth keywords (top 20).

The trend topic analysis is an important mapping tool that helps to portray the seed of trend integration rooted in the previous stream. The buzz topics in MCDA research are shown in Figure 10. The following topics have been identified when examining the authors' keywords and maintaining a minimum five-word frequency and maximum of ten words per year. Except for the words “MCDA” and “healthcare” that we searched for, keywords such as “medical decision-making”, “decision-making”, “HTA”, “HWM”, and “equity” had the longest duration of 6 and 5 years, respectively. Here, the notable topics are “HTA” and “HWM”, which have both a higher frequency and longer duration. Very recently, in 2020 and 2021, “COVID-19” and “TOPSIS” have also been examined.

FIGURE 10

Figure 10. Trend topics. The X-axis represents the year, while the Y-axis is the cumulate occurrences of the keywords.

Finally, we conducted the keywords thematic map, which allows visualization of four different typologies of themes based on two dimensions namely- density- that is the strength of internal ties among all the keywords that are used to describe the research theme and centrality–that is the strength of external ties to other themes by exploiting the authors' keyword field as shown in Figure 11. For the creation of a thematic map, a total of 300 keywords were examined where a minimum cluster frequency is 5 and the number of labels for each cluster is 2. The motor theme, the upper right quadrant, which is characterized by a high density and centrality shows probably the well-developed and important themes for the structuring of the MCDA research field, classifying them into two clusters. Cluster one includes “healthcare services”, “big data”, and “telemedicine”, and the second one includes “TOPSIS” and “fuzzy AHP”. In the upper-left quadrant (niche themes), it is possible to find the themes “healthcare service quality”, “public procurement”, and “healthcare facility” as major keywords. The cluster in the third quadrant (emerging or declining themes) was characterized by low centrality and density, which means that it was weakly developed and marginal, including “performance measurement” and “pharmaceutical policy” as the principal. The fourth quadrant (basic themes) contained “decision-making”, “cost-effectiveness”, “HWM”, “VIKOR”, “MCDA”, and “AHP” as the most common keywords. They concern general topics that are transversal to different research areas of the field.

FIGURE 11

Figure 11. Thematic map showing clusters and the keywords from 1999 to 2021 identified by the co-occurrence network. The bubble size is proportional to the occurrences of the words in the cluster. The X-axis represents the centrality (i.e., the degree of interaction of a network cluster in comparison with other clusters) and gives information about the importance of a theme. The Y-axis symbolizes the density (i.e., measures the internal strength of a cluster network, and it can be assumed as a measure of the theme's development).

Discussion

Within the aim of the study, the current bibliometric analysis provided information about the structure of MCDA publications in the field of healthcare in various categories, which helped researchers recognize publication activities in relation to citations, journals, authors, etc. Since 2014, the number of scholarly MCDA publications has increased steadily. The earliest record was published in 1999 (36) and reported the process of health technology procurement in a University Hospital in Rio de Janeiro in Brazil.

The countries where the research was conducted can be summarized by bibliometric analysis. Several bibliometric analysis studies in different fields (37–39) showed that developed countries have more publications, and low-income countries need not only to promote their research but also to establish research collaboration with developed countries. In our study, North America (e.g., the United States, Canada), Europe (e.g., the United Kingdom, the Netherlands, Spain, Germany, Italy), and Asia (e.g., Turkey, China) have published a significant number of MCDA publications, which is similar to the findings of a systematic review (40). Among them, the United States which is the most productive country, has conducted MCDA research since 2001, followed by Turkey since 2006 and Canada since 2008. In the past decade, the United Kingdom, China, the Netherlands, India, and Spain have also started MCDA research. Both developed and developing countries have made certain contributions to MCDA research. Although Malaysia, Hungary, and other countries did not rank in the top 10 in terms of the publication number, however, for institutions, Universiti Pendidikan Sultan Idris in Malaysia and Syreon Research Institute in Hungary ranked first and third separately.

In terms of authors, there are three large cooperation teams in the current research field. The first is the core team formed by A A Zaidan, B B Zaidan, O S Albahri, A S Albahri, and M A Alsalem. These authors are mainly from Universiti Pendidikan Sultan Idris (UPSI), and have published 13 articles, following the research of MCDA and telemedicine closely, such as real-time remote-health monitoring systems (41), the new m-health hospital selection framework (42). However, this collaborative author network is separate from other networks, indicating that it rarely cooperates with other authors or teams. Second, Zoltan Kalo, Kevin Marsh, and Rob Baltussen cooperated closely and participated in two reports on MCDA used in healthcare decision-making by the task forces of the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) (29, 30). Task forces have developed ISPOR's Good Practices Reports, which are highly cited expert consensus guidance recommendations that set international standards for outcomes research and its use in healthcare decision-making (43). Furthermore, three authors also cooperated with other authors extensively. Zoltan Kalo focused on the research of health economics, such as the weighting methods used in MCDA frameworks in healthcare, potential criteria for frameworks to support the evaluation of innovative medicines in the upper-middle-income countries, and the potential impact of MCDA on pricing and reimbursement process of orphan drugs in Poland. Rob Baltussen has concerns for HIV/AIDS and HTA, such as priority setting of HIV/AIDS interventions, MCDA to support HTA agencies, and value assessment frameworks for HTA agencies. Kevin Marsh emphasizes more on preference information and decision analysis, such as lessons from MCDA for quantifying patient preferences and prioritizing investments in new vaccines against epidemic infectious diseases. Third, Mireille Goetghebeur, Monika Wagner, and Hanane Khoury from LASER Analytica, also worked closely and published four articles together (28, 35, 44, 45) on EVIDEM and HTA. First, they piloted a proof-of-concept evaluation using the EVIDEM framework to appraise 10 medicines covering six therapeutic areas. Then they used tramadol for chronic non-cancer pain as a case study, and the application of rare diseases has also been studied in recent years. In terms of the journal, both the most co-cited journals and productive journals played important roles in the MCDA research. Top co-cited journals could be used as reference sources while writing manuscripts and the top productive journals could be taken into consideration while submitting manuscripts (46).

A Co-citation network represents how frequently two publications are cited together by other publications; therefore, it can be regarded as a knowledge base in a special field (27). In this study, the top 10 co-citation references were selected to identify the knowledge base of MCDA. The two most-cited studies, with 69 and 45 citations separately, are the reports of ISPOR task forces on MCDA for healthcare decision-making (29, 30). Report 1 introduces MCDA, including definition, type, key steps, and application. Report 2 provides more in-depth practice guidance on the implementation of MCDA, including a checklist to guide the design and reporting. These two reports give fundamental advice on how to use MCDA best to support healthcare decision-making. The 3rd to 8th co-citation references are six reviews, involving literature review, systematic review, and bibliometrics analysis, of which five studies reviewed the research of MCDA in healthcare (9, 16, 19, 32, 33), and one study reviewed the application of MCDA in the field of HTA (31). The last two studies are both the application of MCDA. The 9th most-cited study piloted the use of MCDA to establish and apply a framework of weighted attributes to value orphan medicinal products (34), and found that the MCDA approach for rare disease treatment value assessment has the merit of ensuring shared understanding of the elements of value as well as a clear articulation of trade-offs between those elements. The 10th most-cited study (35) combined HTA with MCDA, and a pan-Canadian group of policy and clinical decision-makers and researchers appraising 10 medicines covering six therapeutic areas. This proof-of-concept study demonstrated the usefulness of incorporating MCDA in HTA to support a transparent and systematic appraisal of healthcare interventions. Generally, we found that the top 10 co-citations mainly discussed the definition, steps, approaches, application, and impact, all of which were the foundations of MCDA research.

Based on keywords co-occurrence, word dynamics, trend topic, and thematic map, the current potential research hotspots and trends were obtained. In the past decades, researchers of MCDA in healthcare have rapidly increased their attention to applied categories such as “HTA”, “HWM”, and “AHP”. At present, keywords such as “COVID-19” and “TOPSIS” have also attracted the attention of researchers, which may represent a trend in this field.

Many healthcare policy decision-makers have been studying the application of MCDA to support HTA decisions (32). HTA is a multidisciplinary process that uses explicit methods to determine the value of health technology at different points in its lifecycle (47). MCDA has the potential to enhance and complement the HTA decision-making process (26), and its application in HTA has received widespread attention. For instance, the workshop in Canada discusses opportunities and concerns with reference to the implementation of MCDA in Canada (26). Some researchers have also explored the application of MCDA in the assessment and appraisal of orphan drugs for HTA (48). The National Institute for Health and Clinical Excellence (NICE) is also considering MCDA as one of the ways to implement its new value-based pricing scheme (31).

HWM is also one of the potential hotspots of this research. Healthcare waste today poses grave challenges to hospitals and medical institutions, especially in developing countries where medical waste is very often combined with municipal waste, threatening the health and safety of the handling staff, the general public, and the environment (49). When planning a country's HWM system, decision-makers should consider a variety of parameters, including technical, economic, social, and political factors (50). At this point, MCDA will help decision-makers to solve this complex problem. For example, HWM research has been conducted in Myanmar, India, Istanbul, and other countries. Some researchers used VIKOR to find out the priority of healthcare waste disposal plans during COVID-19 (51).

Understanding the MCDA technologies is crucial. AHP as one of the hot MCDA technologies appears in the research. AHP aims to solve multifactorial and multidimensional problems. It decomposes a complex decision-making problem into different levels. And then, in the pairwise comparison, the weight of each criterion and alternative is judged, and the Eigenvector method is used to calculate the priority (52). AHP was developed by Saaty in the late 1970s (52, 53). Since 1989, it has been accepted slowly as a method of MCDA in healthcare (52). TOPSIS, as one of the classical MCDA technologies, had high frequency in the recent 2 years. TOPSIS is based on the idea that the chosen alternative should have the shortest distance from the positive ideal solution, and on the other side, the farthest distance from the negative ideal solution (54), ranking its advantages and disadvantages based on idealized goals after determining the closeness of its limited evaluation objects. Besides, VIKOR, Decision-making Trial and Evaluation Laboratory (DEMATEL), Evidence and Value: Impact on DEcision-Making (EVIDEM), and MULTIMOORA are also widely used in MCDA (25, 55–57). According to the different research questions, there are preferences in the choice of technologies (51).

It is universally known that COVID-19 broke out at the end of 2019 and remains a threat to global health. As one of the most recent research hotspots, the application of MCDA to COVID-19 mainly involves the following aspects: healthcare waste management (51, 58), prioritization of hospital admission of patients (59), the assessment of intervention and care (60, 61), selection of isolation hospital location (62), and evaluation of a national healthcare system (63). These studies exert impact on both decision-making and management of COVID-19, and this impact will likely continue in the next few years. This information will provide directions for research on other pandemics.

A thematic map can analyze which themes are likely to have long-term development in the future. The keywords, which appear in the motor themes, are both important and well-developed. It is evident that the motor theme now discusses two main perspectives. The cluster one included “healthcare services”, “big data”, and “telemedicine” as the main keywords, followed by “triage”, “hospital management”, “mobile health (m-health)”, and “prioritization”. Big data refer to data that are so large, fast, or complex that they is difficult or impossible to process using traditional methods. Big data are receiving increasing attention in biomedicine and healthcare. In the era of big data, telemedicine and m-health are increasingly used in the modern healthcare systems. Patients, providers, and payers can benefit from the emergence of telemedicine, in terms of the geographical location, costs, emergency treatment, child care/elder care challenges, and so on (64). During COVID-19, telemedicine can attempt to reduce the spread between patients, families, and clinicians (65). However, telemedicine faces challenges in triage and prioritization. MCDA may be one of the solutions for dealing with these challenges. After incorporating the MCDA, people may consider the multiple criteria simultaneously and assign appropriate weight to each criterion, and score patients according to their urgency (66), which optimizes the process of triage and prioritization. The second cluster in motor themes mainly includes “TOPSIS” and “fuzzy AHP”, as well as “internet of things (IOT)”, “medical services”, and other keywords. Based on this result and the previous analysis, MCDA technology has always been one of the research hotspots and the key technologies supporting MCDA research.

Our study analyzed the global trends and application status of MCDA research over 23 years in healthcare from the WoSCC database. It provided insights into scientific research, which will assist in generating evidence-based descriptions, comparisons, and visualizations of research output in MCDA by employing three bibliometric softwares. Nevertheless, our analyses have some limitations. First, the searches were only conducted in the WoSCC database and only in the English language. This can lead to selection bias due to the omission of some studies. Second, as this is an emerging and developing research field and the time we searched was up to July 2021, we might have underestimated the contribution to different analyses of the recently published studies. Third, we only analyzed the information such as authors, keywords, and citations of the included studies from the perspective of bibliometrics, and did not analyze the specific content of the studies. Therefore, the analysis may not provide a better overview of the MCDA-published literature.

Conclusion

The total number of publications shows exponential growth in the past decades. The author teams are mainly from UPSI, LASER Analytica, Syreon Research Institute, and Evidera. The hotspots mainly included AHP, HTA, and HWM, and concentrated on COVID-19 and fuzzy TOPSIS, recently. Furthermore, big data, telemedicine, TOPSIS, and fuzzy AHP, which are well-developed and important themes, might be the trends in future research. The wider use of MCDA, in either methodology or application, will help promote complex decision-making.

Author Contributions

ZD: conceptualization, methodology, software, investigation, formal analysis, visualization, and writing—original draft. SX: investigation, visualization, and writing—original draft. XW, RH, and HL: investigation. HH: formal analysis and visualization. JH: writing—review and editing. XL: conceptualization, supervision, and writing—review and editing. All authors contributed to the article and approved the submitted version.

Funding

This study was supported by the National Natural Science Foundation of China (No. 82174239), CACMS Innovation Fund (No. CI2021A00701-3), the Fundamental Research Funds for the Central Public Welfare Research Institutes (No. ZZ13-YQ-075), China Center for Evidence Based Traditional Chinese Medicine, CCEBTCM (No. 2020YJSZX-2), and the 14th Basic Scientific Research Project of CACMS (Z0718, Z0724, Z0754).

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2022.895552/full#supplementary-material

References

1. Keeney RL, Raiffa H. Decisions With Multiple Objectives: Preferences and Value Tradeoffs. Cambridge, UK: Cambridge University Press. (1993). doi: 10.1017/CBO9781139174084

CrossRef Full Text | Google Scholar

2. Linkov I, Bates ME, Canis LJ, Seager TP, Keisler JM. A decision-directed approach for prioritizing research into the impact of nanomaterials on the environment and human health. Nat Nanotechnol. (2011) 6:784–7. doi: 10.1038/nnano.2011.163

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Bystrzanowska M, Pena-Pereira F, Marcinkowski L, Tobiszewski M. How green are ionic liquids? - A multicriteria decision analysis approach. Ecotoxicol Environ Saf. (2019) 174:455–58. doi: 10.1016/j.ecoenv.2019.03.014

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Rolles S, Schlag AK, Measham F, Phillips L, Nutt D, Bergsvik D, et al. A multi criteria decision analysis (MCDA) for evaluating and appraising government policy responses to non medical heroin use. Int J Drug Policy. (2021) 91:103180. doi: 10.1016/j.drugpo.2021.103180

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Karvetski CW, Lambert JH, Linkov I. Scenario and multiple criteria decision analysis for energy and environmental security of military and industrial installations. Integr Environ Assess Manag. (2011) 7:228–36. doi: 10.1002/ieam.137

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Morkunaite Ž, Kalibatas D, Kalibatiene D. A bibliometric data analysis of multi-criteria decision making methods in heritage buildings. J Civ Eng Manag. (2019) 25:76–99. doi: 10.3846/jcem.2019.8315

CrossRef Full Text | Google Scholar

7. Belton V, Stewart TJ. Multiple criteria decision analysis: an integrated approach. Springer, Boston, MA. (2002). doi: 10.1007/978-1-4615-1495-4

CrossRef Full Text | Google Scholar

8. Peacock S, Mitton C, Bate A, McCoy B, Donaldson C. Overcoming barriers to priority setting using interdisciplinary methods. Health Policy. (2009) 92:124–32. doi: 10.1016/j.healthpol.2009.02.006

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Diaby V, Campbell K, Goeree R. Multi-criteria decision analysis (MCDA) in health care: A bibliometric analysis. Oper Res Health Care. (2013) 2:20–4. doi: 10.1016/j.orhc.2013.03.001

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Marsh K, Goetghebeur M, Thokala P, Baltussen R. Multi-Criteria Decision Analysis to Support Healthcare Decisions. Cham: Springer. (2017). doi: 10.1007/978-3-319-47540-0

PubMed Abstract | CrossRef Full Text | Google Scholar

11. O'Rourke B, Oortwijn W, Schuller T. The new definition of health technology assessment: A milestone in international collaboration. Int J Technol Assess Health Care. (2020) 36:187–90. doi: 10.1017/S0266462320000215

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Dai ZQ, Xu SM, Wu X, Jia LL, Geng JS, Zhao H, et al. An Introduction of EVIDEM framework and its application in healthcare decision making (in Chinese). Chin J Exp Tradit Med Form. (2022) 28:212–18. doi: 10.13422/j.cnki.syfjx.20220154

CrossRef Full Text

13. Goetghebeur MM, Wagner M, Khoury H, Levitt RJ, Erickson LJ, Rindress D. Evidence and value: impact on DEcisionMaking–the EVIDEM framework and potential applications. BMC Health Serv Res. (2008) 8:270. doi: 10.1186/1472-6963-8-270

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Angelis A, Kanavos P. Multiple criteria decision analysis (MCDA) for evaluating new medicines in health technology assessment and beyond: The advance value framework. Soc Sci Med. (2017) 188:137–56. doi: 10.1016/j.socscimed.2017.06.024

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Moreno-Calderon A, Tong TS, Thokala P. Multi-criteria decision analysis software in healthcare priority setting: a systematic review. Pharmacoeconomics. (2020) 38:269–83. doi: 10.1007/s40273-019-00863-9

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Adunlin G, Diaby V, Xiao H. Application of multicriteria decision analysis in health care: a systematic review and bibliometric analysis. Health Expect. (2015) 18:1894–905. doi: 10.1111/hex.12287

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Wahlster P, Goetghebeur M, Kriza C, Niederlander C, Kolominsky-Rabas P. National Leading-Edge Cluster Medical Technologies ‘Medical Valley EMN'. Balancing costs and benefits at different stages of medical innovation: a systematic review of multi-criteria decision analysis (MCDA). BMC Health Serv Res. (2015) 15:12. doi: 10.1186/s12913-015-0930-0

PubMed Abstract | CrossRef Full Text

18. Glaize A, Duenas A, Di Martinelly C, Fagnot I. Healthcare decision-making applications using multicriteria decision analysis: a scoping review. J Multi-Criteria Decis Anal. (2019) 26:62–83. doi: 10.1002/mcda.1659

CrossRef Full Text | Google Scholar

19. Marsh K, Lanitis T, Neasham D, Orfanos P, Caro J. Assessing the value of healthcare interventions using multi-criteria decision analysis: a review of the literature. Pharmacoeconomics. (2014) 32:345–65. doi: 10.1007/s40273-014-0135-0

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Clemente-Suarez VJ, Navarro-Jimenez E, Ruisoto P, Dalamitros AA, Beltran-Velasco AI, Hormeno-Holgado A, et al. Performance of fuzzy multi-criteria decision analysis of emergency system in COVID-19 pandemic. An extensive narrative review. Int J Environ Res Public Health. (2021) 18:5208. doi: 10.3390/ijerph18105208

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Ma CQ, Su H, Li HJ. Global research trends on prostate diseases and erectile dysfunction: a bibliometric and visualized study. Front Oncol. (2020) 10:627891. doi: 10.3389/fonc.2020.627891

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Aria M, Cuccurullo C. Bibliometrix: An R-tool for comprehensive science mapping analysis. J Informetr. (2017) 11:959–75. doi: 10.1016/j.joi.2017.08.007

PubMed Abstract | CrossRef Full Text | Google Scholar

23. van Eck NJ, Waltman L. Software survey: VOSviewer, a computer program for bibliometric mapping. Scientometrics. (2010) 84:523–38. doi: 10.1007/s11192-009-0146-3

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Chen CM, Chen Y, Horowitz M, Hou HY, Liu ZY, Pellegrino D. Towards an explanatory and computational theory of scientific discovery. J Informetr. (2009) 3:191–209. doi: 10.1016/j.joi.2009.03.004

CrossRef Full Text | Google Scholar

25. Falatoonitoosi E, Leman Z, Sorooshian S, Salimi M. Decision-making trial and evaluation laboratory. Res J Appl Sci, Eng Technol. (2013) 5:3476–80. doi: 10.19026/rjaset.5.4475

CrossRef Full Text

26. Diaby V, Goeree R, Hoch J, Siebert U. Multi- criteria decision analysis for health technology assessment in Canada: insights from an expert panel discussion. Expert Rev Pharmacoecon Outcomes Res. (2015) 15:13–9. doi: 10.1586/14737167.2015.965155

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Chen CM. CiteSpace II: Detecting and visualizing emerging trends and transient patterns in scientific literature. J Am Soc Inf Sci Tec. (2006) 57:359–77. doi: 10.1002/asi.20317

CrossRef Full Text | Google Scholar

28. Wagner M, Khoury H, Bennetts L, Berto P, Ehreth J, Badia X, et al. Appraising the holistic value of Lenvatinib for radio-iodine refractory differentiated thyroid cancer: A multi-country study applying pragmatic MCDA. BMC Cancer. (2017) 17:272. doi: 10.1186/s12885-017-3258-9

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Thokala P, Devlin N, Marsh K, Baltussen R, Boysen M, Kalo Z, et al. Multiple criteria decision analysis for health care decision making-an introduction: report 1 of the ISPOR MCDA Emerging Good Practices Task Force. Value Health. (2016) 19:1–13. doi: 10.1016/j.jval.2015.12.003

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Marsh K, Ijzerman M, Thokala P, Baltussen R, Boysen M, Kalo Z, et al. Multiple criteria decision analysis for health care decision making-emerging good practices: report 2 of the ISPOR MCDA emerging good practices task force. Value Health. (2016) 19:125–37. doi: 10.1016/j.jval.2015.12.016

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Thokala P, Duenas A. Multiple criteria decision analysis for health technology assessment. Value Health. (2012) 15:1172–81. doi: 10.1016/j.jval.2012.06.015

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Muhlbacher AC, Kaczynski A. Making good decisions in healthcare with multi-criteria decision analysis: the use, current research and future development of MCDA. Appl Health Econ Health Policy. (2016) 14:29–40. doi: 10.1007/s40258-015-0203-4

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Dolan JG. Multi-criteria clinical decision support: A primer on the use of multiple criteria decision making methods to promote evidence-based, patient-centered healthcare. Patient. (2010) 3:229–48. doi: 10.2165/11539470-000000000-00000

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Sussex J, Rollet P, Garau M, Schmitt C, Kent A, Hutchings A, et al. pilot study of multicriteria decision analysis for valuing orphan medicines. Value Health. (2013) 16:1163–9. doi: 10.1016/j.jval.2013.10.002

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Goetghebeur MM, Wagner M, Khoury H, Levitt RJ, Erickson LJ, Rindress D. Bridging health technology assessment (HTA) and efficient health care decision making with multicriteria decision analysis (MCDA): Applying the EVIDEM framework to medicines appraisal. Med Decis Mak. (2012) 32:376–88. doi: 10.1177/0272989X11416870

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Nobre FF, Trotta LTF, Gomes L. Multi-criteria decision making - An approach to setting priorities in health care. Stat Med. (1999) 18:3345–54. doi: 10.1002/(SICI)1097-0258(19991215)18:23<3345::AIDSIM321>3.0.CO;2-7

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Ahmad T, Imran M, Ahmad K, Khan M, Baig M, Al-Rifai RH, et al. A bibliometric analysis and global trends in fascioliasis research: a neglected tropical disease. Animals (Basel). (2021) 11:3385. doi: 10.3390/ani11123385

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Ahmad T, Nasir S, Musa TH, AlRyalat SAS, Khan M, Hui J. Epidemiology, diagnosis, vaccines, and bibliometric analysis of the 100 top-cited studies on Hepatitis E virus. Hum Vaccin Immunother. (2021) 17:857–71. doi: 10.1080/21645515.2020.1795458

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Rashid A, Li K, Feng Y, Ahmad T, Getaneh Y, Yu YY, et al. HIV-1 genetic diversity a challenge for AIDS vaccine development: a retrospective bibliometric analysis. Human Vaccines Immunother. (2022) 18:2014733. doi: 10.1080/21645515.2021.2014733

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Frazao TDC, Camilo DGG, Cabral ELS, Souza RP. Multicriteria decision analysis (MCDA) in health care: a systematic review of the main characteristics and methodological steps. BMC Med Inform Decis Mak. (2018) 18:90. doi: 10.1186/s12911-018-0663-1

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Mohammed KI, Zaidan AA, Zaidan BB, Albahri OS, Alsalem MA, Albahri AS, et al. Real-time remote-health monitoring systems: a review on patients prioritisation for multiple-chronic diseases, taxonomy analysis, concerns and solution procedure. J Med Syst. (2019) 43:223. doi: 10.1007/s10916-019-1362-x

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Albahri OS, Zaidan AA, Zaidan BB, Albahri AS, Mohsin AH, Mohammed KI, et al. New mHealth hospital selection framework supporting decentralised telemedicine architecture for outpatient cardiovascular disease-based integrated techniques: Haversine-GPS and AHP-VIKOR. J Ambient Intell Humaniz Comput. (2021) 13:219–39. doi: 10.1007/s12652-021-02897-4

CrossRef Full Text | Google Scholar

43. ISPOR-Task Forces. Available online at: https://www.ispor.org/member-groups/task-forces. (accessed July 24, 2021).

44. Wagner M, Khoury H, Willet J, Rindress D, Goetghebeur M. Can the EVIDEM framework tackle issues raised by evaluating treatments for rare diseases: Analysis of issues and policies, and context-specific adaptation. Pharmacoeconomics. (2016) 34:285–301. doi: 10.1007/s40273-015-0340-5

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Tony M, Wagner M, Khoury H, Rindress D, Papastavros T, Oh P, et al. Bridging health technology assessment (HTA) with multicriteria decision analyses (MCDA): field testing of the EVIDEM framework for coverage decisions by a public payer in Canada. BMC Health Serv Res. (2011) 11:329. doi: 10.1186/1472-6963-11-329

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Ke LX, Lu CC, Shen R, Lu TT, Ma B, Hua YP. Knowledge mapping of drug-induced liver injury: a scientometric investigation (2010-2019). Front Pharmacol. (2020) 11:842. doi: 10.3389/fphar.2020.00842

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Health Technology Assessment Glossary. Health Technology Assessment (HTA) (2021). Available online at: http://htaglossary.net/health-technology-assessment (accessed May 8, 2021).

Google Scholar

48. Iskrov G, Miteva-Katrandzhieva T, Stefanov R. Multi-criteria decision analysis for assessment and appraisal of orphan drugs. Front Public Health. (2016) 4:13. doi: 10.3389/fpubh.2016.00214

PubMed Abstract | CrossRef Full Text

49. Thakur V, Ramesh A. Healthcare waste management research: A structured analysis and review (2005-2014). Waste Manag Res. (2015) 33:855–70. doi: 10.1177/0734242X15594248

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Ozkan A. Evaluation of healthcare waste treatment/disposal alternatives by using multi-criteria decision-making techniques. Waste Manage Res. (2013) 31:141–49. doi: 10.1177/0734242X12471578

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Manupati VK, Ramkumar M, Baba V, Agarwal A. Selection of the best healthcare waste disposal techniques during and post COVID-19 pandemic era. J Clean Prod. (2021) 281:10. doi: 10.1016/j.jclepro.2020.125175

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Schmidt K, Aumann I, Hollander I, Damm K, von der Schulenburg JM. Applying the analytic hierarchy process in healthcare research: a systematic literature review and evaluation of reporting. BMC Med Inform Decis Mak. (2015) 15:27. doi: 10.1186/s12911-015-0234-7

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Pecchia L, Bath PA, Pendleton N, Bracale M. Analytic hierarchy process (AHP) for examining healthcare professionals' assessments of risk factors. The relative importance of risk factors for falls in community-dwelling older people. Methods Inf Med. (2011) 50:435–44. doi: 10.3414/ME10-01-0028

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Wei GW. Extension of TOPSIS method for 2-tuple linguistic multiple attribute group decision making with incomplete weight information. Knowl and Inf Syst. (2009) 25:623–34. doi: 10.1007/s10115-009-0258-3

CrossRef Full Text | Google Scholar

55. Hafezalkotob A, Hafezalkotob A, Liao HC, Herrera F. An overview of MULTIMOORA for multi-criteria decision-making: theory, developments, applications, and challenges. Inf Fusion. (2019) 51:145–77. doi: 10.1016/j.inffus.2018.12.002

CrossRef Full Text | Google Scholar

56. Goetghebeur MM, Cellier MS. Can reflective multicriteria be the new paradigm for healthcare decision-making? The EVIDEM journey. Cost Effect Resour Alloc. (2018) 16:11. doi: 10.1186/s12962-018-0116-9

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Mardani A, Zavadskas E, Govindan K, Amat Senin A, Jusoh A, VIKOR. technique: a systematic review of the state of the art literature on methodologies and applications. Sustainability. (2016) 8:37. doi: 10.3390/su8010037

CrossRef Full Text | Google Scholar

58. Hugo AD, Lima RD. Healthcare waste management assessment: Challenges for hospitals in COVID-19 pandemic times. Waste Manage Res. (2021) 39:56–63. doi: 10.1177/0734242X211010362

PubMed Abstract | CrossRef Full Text | Google Scholar

59. De Nardo P, Gentilotti E, Mazzaferri F, Cremonini E, Hansen P, Goossens H, et al. Multi-criteria decision analysis to prioritize hospital admission of patients affected by COVID-19 in low-resource settings with hospital-bed shortage. Int J Infect Dis. (2020) 98:494–500. doi: 10.1016/j.ijid.2020.06.082

PubMed Abstract | CrossRef Full Text | Google Scholar

60. Khan A, Abosuliman SS, Ashraf S, Abdullah S. Hospital admission and care of COVID-19 patients problem based on spherical hesitant fuzzy decision support system. Int J Intell Syst. (2021) 36:4167–209. doi: 10.1002/int.22455

CrossRef Full Text | Google Scholar

61. Mardani A, Saraji MK, Mishra AR, Rani P. A novel extended approach under hesitant fuzzy sets to design a framework for assessing the key challenges of digital health interventions adoption during the COVID-19 outbreak. Appl Soft Comput. (2020) 96:14. doi: 10.1016/j.asoc.2020.106613

PubMed Abstract | CrossRef Full Text | Google Scholar

62. Aydin N, Seker S. Determining the location of isolation hospitals for COVID-19 via Delphi-based MCDM method. Int J Intell Syst. (2021) 36:3011–34. doi: 10.1002/int.22410

CrossRef Full Text | Google Scholar

63. Huang SW, Liou JJH, Chuang HH, Tzeng GH. Using a modified VIKOR technique for evaluating and improving the national healthcare system quality. Mathematics. (2021) 9:1349. doi: 10.3390/math9121349

CrossRef Full Text | Google Scholar

64. Wootton R. Recent advances: telemedicine. BMJ. (2001) 323:557–60. doi: 10.1136/bmj.323.7312.557

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Calton B, Abedini N, Fratkin M. Telemedicine in the time of coronavirus. J Pain Symptom Manage. (2020) 60:e12–4. doi: 10.1016/j.jpainsymman.2020.03.019

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Kalid N, Zaidan AA, Zaidan BB, Salman OH, Hashim M, Muzammil H. Based real time remote health monitoring systems: a review on patients prioritization and related “big data” using body sensors information and communication technology. J Med Syst. (2018) 42:30. doi: 10.1007/s10916-017-0883-4

PubMed Abstract | CrossRef Full Text