Rapid risk assessment tool (RRAT) to prioritize emerging and re-emerging livestock diseases for risk management

Increasing globalization and international trade contribute to rapid expansion of animal and human diseases. Hence, preparedness is warranted to prevent outbreaks of emerging and re-emerging diseases or detect outbreaks in an early stage. We developed a rapid risk assessment tool (RRAT) to inform risk managers on the incursion risk of multiple livestock diseases, about the main sources for incursion and the change of risk over time. RRAT was built as a relational database to link data on disease outbreaks worldwide, on introduction routes and on disease-specific parameters. The tool was parameterized to assess the incursion risk of 10 livestock diseases for the Netherlands by three introduction routes: legal trade in live animals, legal trade of animal products, and animal products illegally carried by air travelers. RRAT calculates a semi-quantitative risk score for the incursion risk of each disease, the results of which allow for prioritization. Results based on the years 2016-2018 indicated that the legal introduction routes had the highest incursion risk for bovine tuberculosis, whereas the illegal route posed the highest risk for classical swine fever. The overall incursion risk via the illegal route was lower than via the legal routes. The incursion risk of African swine fever increased over the period considered, whereas the risk of equine infectious anemia decreased. The variation in the incursion risk over time illustrates the need to update the risk estimates on a regular basis. RRAT has been designed such that the risk assessment can be automatically updated when new data becomes available. For diseases with high-risk scores, model results can be analyzed in more detail to see which countries and trade flows contribute most to the risk, the results of which can be used to design risk-based surveillance. RRAT thus provides a multitude of information to evaluate the incursion risk of livestock diseases at different levels of detail. To give risk managers access to all results of RRAT, an online visualization tool was built.

Introduction

Increasing globalization and international trade contribute to rapid expansion of animal and human diseases. Introduction of animal diseases into naive livestock populations can result in large-scale epidemics with serious economic and socio-ethical impact. Illustrative examples include the introduction of foot and mouth disease (FMD) in the United Kingdom in 2001 (1) and subsequent spread to the Netherlands (2), the introduction of bluetongue (BT) in the Netherlands in 2006 with subsequent spread to neighboring countries (3, 4), and the introduction of African swine fever (ASF) into Georgia in 2007 (5). ASF subsequently spread into Europe and Asia (6), and in 2021 the disease was also introduced into the Americas (7). Recent incursions of diseases that had not been reported before in Europe, such as lumpy skin disease (LSD) in South-Eastern Europe in 2014-2017 and peste des petits ruminants (PPR) in Bulgaria in 2018 highlight the continuous threat of emerging and re-emerging disease outbreaks (7–9). Preparedness is thus warranted to prevent outbreaks of livestock diseases in new territories or to detect outbreaks in an early stage. Risk assessment is a useful tool to inform risk managers on the incursion risk of livestock diseases that can not only provide information on the magnitude of the risk, but also on the main sources of risk and the change of risk over time.

Most introduction risk assessments performed over the last decades focused on a single disease and a single introduction route [e.g., (10–15)] and were initiated to address specific risk questions. Those risk questions often arise in response to new disease events to evaluate the increased incursion risk from such an event. In recent years, several generic risk assessment tools were developed that accommodate multiple diseases and/or introduction routes (16). An important asset of these generic risk assessment tools is the ability to prioritize diseases or risk regions for their incursion risk, enabling the targeted use of limited resources for prevention and surveillance. The repeated use of these tools to inform risk managers is, however, limited, either because the tools do not have an underlying database and need to be populated with data before each use, or because expert opinion is needed to update results. One of the earliest prioritization tools was developed by Roberts et al. (17) for the United Kingdom. This tool integrates expert opinion with data on disease outbreaks and trade into a semi-quantitative risk score for each disease. Updates are performed manually, and expert opinion is key to keep the tool up and running. An automatic update of data and calculations when new data becomes available would facilitate repeated use of generic risk assessment tools.

The incursion risk of a livestock disease is largely determined by the distribution of the pathogen in the world and the connections of a disease-free territory with these regions. These connections are the so-called introduction routes and can either be trade in livestock or their products, trade in exotic animals, migrating wildlife, movements of people if the disease is zoonotic, or introduction of vectors if the disease is vector-borne. Data on the worldwide distribution of animal diseases and on the volume of introduction routes is largely available from global databases such as WAHIS (World Animal Health Information System) (7), EMPRES-i (Global Animal Disease Information System) (18), Comtrade (19) and Comext (20). Integration of this data is mostly done by disease experts leaving it a labor-intensive and subjective exercise to evaluate the incursion risk. The increased accessibility and interoperability of most of these global databases has opened the door to a more automated risk assessment approach. To fully exploit the available data, we developed a rapid risk assessment tool (RRAT) that combines the data from global databases into an automated estimate of the incursion risk for multiple livestock diseases. The main objective of this tool is to support risk managers in prioritizing diseases for risk management. Furthermore, RRAT can indicate high risk trade flows and source countries, the results of which can be used for risk-based surveillance. In this paper, RRAT is described and results for the incursion risk for the Netherlands are presented and discussed.

Materials and methods

RRAT has been built as a relational database in R (21) and SQLite (22) with the main tables in the tool describing: the worldwide occurrence of animal diseases; the volumes of the introduction routes; and disease-specific parameters to assess the risk of each introduction route. RRAT is a semi-quantitative risk assessment tool that provides the user with a risk score for the probability that a specific disease enters a new region or country (“target area”) and will result in a first infection of local livestock animals. Introduction routes considered in RRAT comprise the legal trade in live animals (“animal route”), the legal trade of animal products (“product route”), and animal products illegally carried by air travelers (“traveler route”). The introduction routes are all subdivided into multiple pathways to account for diversity in animal species and animal products. Up till now, RRAT has been parameterized for 10 diseases that are considered a potential threat to the Netherlands, viz. African horse sickness (AHS), ASF, Aujeszky's disease (Auj), BT, bovine tuberculosis (bTB), classical swine fever (CSF), equine infectious anemia (EIA), foot and mouth disease (FMD), LSD, and PPR (Table 1). Calculations have been performed for the years 2016, 2017 and 2018 with the Netherlands as target area.

TABLE 1

Table 1. Overview of causing pathogens, reservoir livestock hosts and main transmission routes of ten diseases in RRAT.

Calculations in RRAT are based on the Binomial process considering (1) the number of animals or products entering the target area, (2) the probability that an individual animal or product is infected, and (3) the probability that entry of an infected animal or product results in a first infection of local animals. An overview of the model parameters in RRAT is given in Table 2.

TABLE 2

Table 2. Overview of model parameters in RRAT.

The overall risk score R_P for a target area by a single introduction route i is calculated as:

$\begin{array}{ll}{R_P}_i=1-{\displaystyle \prod _{C=1}^c}{\displaystyle \prod _{P=1}^p}{\displaystyle \prod _{D=1}^d}{(1-P_{entr{y}_{CPD}}\times P_{es{t}_{PD}})}^{N_{CP}}& (1)\end{array}$

where N_CP is the number of pathway units (animals for livestock, pets and exotic mammals; batches for poultry, exotic birds and germplasm; kg for animal products) of pathway P that enters the target area from source country C in the time period considered, P_entryCPD is the probability of entry of disease D from source country C by pathway P, and P_estPD is the probability that entry of disease D by pathway P results in a first local infection (establishment) in the target area. Although the overall risk score of RRAT is calculated as the probability of a successful introduction of any disease in the tool, it cannot be interpreted as such, because input into the tool is partly based on proxy values that were assigned to risk classes, rather than strictly quantitative data derived from e.g., scientific literature or animal experiments. Proxy values were defined as approximate values that represent the—sometimes unknown, and mostly uncertain – actual values of input parameters. The risk score is thus a semi-quantitative score, that can be calculated at different levels, e.g., per disease, source country or pathway, and can as such be used to rank diseases, source countries and pathways for their incursion risk.

Due to its asymptotic nature, the overall risk score R_P is not discriminating if its value approaches one for multiple diseases, source countries or pathways. Therefore, a second risk score indicating the number of successful introductions, R_N, is calculated as:

$\begin{array}{ll}{R_N}_i={\displaystyle \sum _{C=1}^c}{\displaystyle \sum _{P=1}^p}{\displaystyle \sum _{D=1}^d}{N}_{CP}\times P_{entr{y}_{CPD}}\times P_{es{t}_{PD}}& (2)\end{array}$

Again, although being calculated as the number of successful introductions of any disease in the tool, this risk score cannot be interpreted as such given its semi-quantitative nature.

Trade of live animals

Data on the numbers and batches of animals traded to the Netherlands from each source country was derived from TRACES (23) (Supplementary Table S1.1). RRAT not only considers the trade in livestock, but also trade in equines, dogs, cats, and exotic mammals and birds. Animals were grouped based on species and destination (for life or for slaughter). This resulted in a total of 38 animal species groups (pathways) considered for this introduction route.

For each disease in RRAT, the animal species groups were assigned a susceptibility class based on information derived from factsheets and scientific literature (24–30). Five susceptibility classes were used: (1) reservoir host, (2) spill-over host possibly contributing to transmission (3) host in which only experimental infections have been described, (4) dead-end host, and (5) not susceptible (Supplementary Tables S2.1, S3.1).

To estimate the probability of entry (P_entryCPD), two main parameters were used: (1) the probability of an individual animal being infected (P_infCPD), and (2) the probability of an infected animal not being detected before transport (1−P_detCPD) (Figure 1). P_entryCPD was calculated as:

$\begin{array}{ll}{P}_{entr{y}_{CPD}}=P_{in{f}_{CPD}}\times (1-P_{de{t}_{CPD}})& (3)\end{array}$

P_infCPD was estimated using data on disease incidence in the source countries and disease-specific parameters. P_infCPD was calculated as:

$\begin{array}{ll}{P}_{in{f}_{CPD}}=In{c}_{CD}\times T_{in{f}_D}\times P_{infsu{s}_P}& (4)\end{array}$

where Inc_CD is the incidence of disease D in source country C, P_infsusP is a proxy value to account for the probability of infection of animal species P with disease D dependent on its susceptibility class, and T_infD is a proxy value to account for the average infectious period of disease D in reservoir hosts. The calculation of Inc_CD was based on all cases reported to the OIE in a one-year period (see Section “Disease incidence” for more details). However, for most diseases, animals are only infectious for a relatively short period. Therefore, T_infD was used to correct for the average infectious period of infected animals, using four classes and accompanying proxy values (T_infD = 0.05 if infectious period <2 weeks; T_infD = 0.1 if infectious period > 2 weeks and <1 month; T_infD= 0.25 if infectious period > 1 month and <1 year; T_infD= 1 if infectious period > 1 year) (Supplementary Table S3.2). As Inc_CD was based on reported cases in reservoir livestock hosts only, P_infsusP was used to correct for the expected incidence of disease in non-reservoir hosts with the value of P_infsusP dependent on the animal's susceptibility class (Supplementary Table S3.1).

FIGURE 1

Figure 1. Scenario tree outlining the steps to assess the probability of entry and first infection for the legal trade in live animals (“animal route”).

P_detCPD was estimated using data on European legislation regarding both intracommunity trade (between European Union (EU) member states) and importation of animals from non-EU countries (31–38). Legal requirements such as clinical inspection, quarantine and testing, or importations from disease-free regions only were listed per disease, pathway (animal species) and source country. Individual measures were rated with a score between 0 and 1 for their effectiveness using information on e.g., length of the incubation period, severity of clinical signs, and test sensitivity. If > 1 measure was in force, P_detCPD was set equal to the effectiveness of the most effective measure.

The probability of a first infection (P_estPD) was estimated considering the infectiousness and the destination of the imported animals. P_estPD was calculated as:

$\begin{array}{ll}{P}_{es{t}_{PD}}=P_{contac{t}_P}\times P_{tran{s}_{PD}}& (5)\end{array}$

where P_contactP is a proxy value to account for the probability that the imported infected animal comes into contact with susceptible livestock in the target area, and P_transPD is a proxy value to account for the probability that the infected animal will transmit the disease if in contact with susceptible livestock. The value of P_contactP depends on the destination of the imported animal (Supplementary Table S3.3). The value of P_transPD depends on the susceptibility class of the imported animal (Supplementry Table S3.1). Although the infection can be carried by dead-end hosts, they do not contribute to transmission of the disease and will as such not result in a successful introduction.

Trade of animal products

Data on the import of live animal products (germplasm and hatching eggs) and manure was derived from TRACES (23), because TRACES provided most detail on the animal species from which these products were derived. Imports of germplasm (semen, embryos and ova) were, however, only available at batch level. Data on the import of other animal products was derived from Comext (20) (Supplementary Table S1.2). Animal products were assigned to animal product groups considering product type (meat, casings, milk and dairy products, eggs and egg products, hides, feathers and down, etc.), treatment (fresh, frozen, dried, salted, heated, etc.) and the animal species from which the product was derived (bovines, pigs, sheep, goats, equines, poultry, etc.). This resulted in a total of 139 pathways considered for this introduction route. For presentation purposes, results were aggregated in 16 summarizing product groups (Supplementary Table S2.2).

To estimate the probability of entry (P_entryCPD), two main parameters were used: (1) the probability that the product is derived from an infected animal (P_infanCPD), and (2) the probability that the product itself is contaminated (P_contPD) (Figure 2). P_entryCPD was calculated as:

$\begin{array}{ll}{P}_{entr{y}_{CPD}}=P_{infa{n}_{CPD}}\times P_{con{t}_{PD}}& (6)\end{array}$

P_infanCPD was estimated taking into account the infection probability of the animal from which the product was derived in the source country (P_infCPD) and the probability of detection of the infection in the animal at ante-mortem or post-mortem inspection in the slaughterhouse (P_detslD, Supplementary Table S3.4). The latter only applied to products derived from slaughtered animals such as meat and hides, but not to products derived from live animals such as milk, eggs, germplasm and manure (Supplementray Table S2.2). To retrieve the value of P_infanCPD, the animal species from which the products were derived were linked to the 38 animal species groups used for the introduction route of live animal imports. A worst-case approach was applied here, i.e., if a product could have been derived from > 1 animal species, it was linked to all relevant animal species groups and given the value of P_infCPD of the animal species group that was most susceptible to the disease. P_infanCPD was calculated as:

$\begin{array}{ll}{P}_{infa{n}_{CPD}}=P_{in{f}_{CPD}}\times (1-P_{dets{l}_D})& (7)\end{array}$

P_contPD was based on data derived from factsheets and scientific literature and entered into RRAT as an absence/presence score (Supplementary Figure S3.1, Supplementary Table S3.5).

FIGURE 2

Figure 2. Scenario tree outlining the steps to assess the probability of entry and first infection for the legal trade of animal products including germplasm (“product route”).

To estimate the probability of first infection (P_estPD), three parameters were used: (1) the probability that a local animal is exposed to the product in the target area (P_exP), the probability that the product still contains viable pathogen when local animals are exposed to it (P_contexPD), and (3) the probability that such exposure results in infection (P_infexPD) (Figure 2). P_estPD was calculated as:

$\begin{array}{ll}{P}_{es{t}_{PD}}=P_{e{x}_P}\times P_{conte{x}_{PD}}\times P_{infe{x}_{PD}}& (8)\end{array}$

P_exP is given by a proxy value accounting for the probability that the imported product will end up with local livestock animals and was made dependent on the intended use of the product (Supplementary Table S3.6). P_contexPD depends on the survival time of the pathogen in the product and the average time it will take for the product to reach local animals, which is dependent on e.g., shipping time and shelf life. The latter is difficult to estimate and will probably be quite long for most products. P_contexPD was therefore based on risk classes accounting for survival time and products were assigned to a risk class based on reported survival time in factsheets and literature (Supplementary Table S3.7, Supplementary Figure S3.1). The risk class was reduced by one level if import of the product from infected territories was subjected to import restrictions according to OIE (25) or EU legislation (39). Each risk class was given a probability score on a log₁₀ scale to obtain proxy values for P_contexPD (Supplementary Table S3.8). P_infexPD was also given a proxy value based on risk classes, using the same log₁₀ scale as for P_contexPD. The risk classes for this parameter were assigned considering the most likely exposure route to the pathogen dependent on its intended use (Supplementary Table S3.6). The probability that exposure results in infection depends on the exposure route and is disease specific, with some diseases more readily transmitted by e.g., aerosols whereas others are more readily transmitted by oral ingestion. The risk class assigned to each exposure route was therefore made disease-dependent (Supplementary Table S3.9).

Animal products carried by air travelers

No database was available to directly input the amount of animal products carried by air travelers into RRAT. To estimate the volume of this introduction route, data on air passenger transport between the main airports of the Netherlands and their main partner airports (40) was combined with data from Great Britain on seizures of animal products (both meat and dairy products) (41–50) and input from scientific literature. Carrying animal products for own use into the Netherlands is illegal only if imported from non-EU countries. Therefore, customs do not search luggage of people traveling within the EU and hence no data was available to estimate the flow of products coming from EU member states. The incursion risk by this introduction route was thus evaluated for non-EU countries only.

Seizures of animal products were classified according to the type of product (meat; dairy; eggs), the animal species of which the product was derived (bovines; pigs; sheep; goats; poultry; buffalo; bushmeat), and the treatment of meat (fresh and frozen; dried and salted; heated). This resulted in a total of 21 pathways considered for this introduction route (Supplementary Table S2.3). No equine products were considered for the traveler route, and hence the incursion risk of the equine diseases AHS and EIA was not estimated for this introduction route. Data on seizures of animal products was not available at country level, but for 14 geographical regions comprising multiple countries (41) (Supplementary Figure S3.2). In RRAT, each source country was assigned to one of those 14 regions to extract the corresponding values from the database and calculations were performed at country level. Results of this introduction route are, however, presented at regional level, matching the lowest spatial resolution in the data.

The amount of animal products carried by people traveling to the Netherlands (N_CP) was calculated as:

$\begin{array}{ll}{N}_{CP}=N{t}_C\times P{t}_C\times R{P}_{CP}\times W_{CP}& (9)\end{array}$

where Nt_C is the number of air travelers arriving in the Netherlands from source country C during a one-year period, Pt_C is the fraction of travelers carrying products of animal origin when arriving from source country C, RP_CP is the fraction of products carried by travelers from source country C that is of product type P, and W_CP is the average weight (kg) of product type P carried per traveler arriving from source country C. Note that values for Pt_C, RP_CP and W_CP were only available at regional level (Supplementary Table S2.3). The calculated amounts are given in Supplementary Table S1.3.

The number of air travelers (Nt_C) was derived from the Eurostat database table avia_par_nl, where the transport measurement (tra_meas) was passengers carried – arrivals (PAS_CRD_ARR) (40). This table reports on all passengers on a specific flight (with a single flight number) that terminate their journey at the reporting airport. Therefore, it was assumed that all passengers would have the Netherlands as destination (no transit passengers included). For journeys including multiple flights, the airport of embarkation was not known, resulting in an underestimate of the number of travelers arriving from non-EU countries.

Very little information was available to estimate the fraction of air travelers carrying products of animal origin (Pt_C) and most estimates from literature were biased, i.e., passenger checks were risk-based, likely resulting in an overestimate of the probability that travelers will carry animal products. In RRAT, Pt_C was set to 15.5% based on estimates from Great Britain that 63.8% of the travelers that carry products of animal origin bring meat (43–50), and that 9.9% of all travelers bring meat (42). The value of 15.5% was used for all source regions. The RRAT is, however, flexible to include source region-specific values for this parameter. The probability that an animal product carried by a traveler arriving from source country C is of product type P (RP_CP) was based on the proportion of seizures per product type from travelers arriving from the 14 different regions. Estimates for the proportions of meat from bovines, pigs, small ruminants, buffaloes and bushmeat were derived from VLA (42). These were complemented with data from Defra (43–50) to estimate the ratio of meat to dairy products. This ratio varied widely between regions with dairy constituting only 5% of seizures from Southern Africa and as much as 60% of seizures from Southern Asia (Supplementary Table S2.3). Estimates for the proportion of poultry in total meat were derived from scientific literature, with several publications reporting proportions of approximately 40% (51, 52). Eggs and egg products were estimated to be only 1% of animal products carried by travelers (51–55). Reported average weights per seizure (W_CP) are mostly between 2 and 4 kg (42, 51, 53, 54, 56, 57), although seizures of bushmeat tend to have a higher weight than seizures of livestock meat (52, 57). In RRAT, region-based weights to estimate W_CP were derived from a study from Switzerland (52). No detail was available to estimate the weights for the different product types. The RRAT is, however, flexible to include product type-specific values for this parameter if new data would become available.

The calculations to estimate the probability of entry (P_entryCPD) and establishment (P_estPD) of pathogens via animal products carried by travelers were analogous to the calculations for the product route. Input parameters were derived by connecting each of the 21 pathways of the traveler route to one of the 139 pathways of the product route that had similar characteristics with respect to product type, treatment and animal species. Because it was assumed that part of the products carried by travelers were derived from animals slaughtered at home, an additional parameter (P_hmP) was introduced to account for the fact that detection of infected animals at the slaughterhouse was less likely for products carried by travelers than for legally imported products. P_hmP is the proportion of products carried by travelers that is “homemade” and its value was set to 0.29 for all product types derived from dead livestock animals (53), and to 1 for bushmeat. The RRAT is, however, flexible to include product type-specific values for this parameter. The probability that the product carried by a traveler was derived from an infected animal (P_infanCPD) was therefore calculated as:

$\begin{array}{ll}{P}_{infa{n}_{CPD}}=P_{in{f}_{CPD}}\times (1-P_{dets{l}_D}\times (1-P_{h{m}_P}))& (10)\end{array}$

Products carried by travelers were assumed to have the same intended use as legally imported products, i.e., the probability of exposure to local animals is equal for both introduction routes. However, products carried by travelers escape import controls and therefore cannot be assessed for compliance with OIE standards or EU legislation. Therefore, the input values for the probability that the product still contains viable pathogen when local animals are exposed to it (P_contexPD) were separately estimated for this introduction route (Supplementary Table S3.10, Supplementary Figure S3.1).

Disease incidence

Data on disease presence in the world was based on annual reports of individual countries in WAHIS (58). These reports were obtained using web scraping, because the data was not downloadable at the time RRAT was built. A decision tree was used to assign each country to a risk class considering the information the country had provided in the annual report to the OIE (Figure 3), and the reported disease incidence by countries in the same UN subregion (59). The decision tree distinguishes three main groups of countries: those that reported presence of disease in either wildlife or domestic animals (upper branch), those that reported absence of disease (middle branch), and those that had not provided any information on the disease for the year considered (lower branch). Only for countries that had reported cases to the OIE (risk class 4), the incidence of disease D in source country C (Inc_CD) could be calculated by dividing the number of cases of disease D in livestock reservoir hosts in source country C (58) by the population of affected livestock reservoir hosts for disease D present in source country C (58, 60). For EU member states, the OIE data was complemented with data from the Animal Disease Information System (ADIS) (61) and reports from the European Commission (EC) on diseases in bovines and swine (62–64). These sources only provided the number of outbreaks, not the number of cases. The number of outbreaks was therefore multiplied with the median number of cases per outbreak as reported by WAHIS (58) (Supplementary Table S3.2) to arrive at an estimate of the number of cases in order to calculate Inc_CD. If no data was available on the number of cases at all, a proxy value was used for Inc_CD based on the assigned risk class for disease presence. If a country had reported absence of disease, the year of last occurrence was considered for the risk classification, where we assumed that real absence was more likely if the disease had not been reported for a longer period (X years, where X was disease-dependent, Supplementary Table S3.2) and if the disease was not present in the UN subregion either. If no information was available on the disease status of a country, information on disease occurrence in the UN subregion was used to assign this country to a risk class. Only for countries assigned to risk class 0, Inc_CD was set to 0 as we deemed presence of the disease in those countries extremely unlikely. For the countries assigned to other risk classes, proxy values were used for Inc_CD that were either based on Inc_absD or Inc_unkD. Inc_absD equaled an incidence 100 times lower than the minimum incidence for disease D calculated for countries in risk class 4 (disease present and cases reported), whereas Inc_unkD equaled the maximum incidence calculated for disease D for countries in risk class 4 (Supplementary Table S3.2).

FIGURE 3

Figure 3. Decision tree to assign countries to one out of seven risk classes regarding disease incidence, considering the information available in the OIE annual reports (7).

Baseline scenario

In the baseline scenario, model input as described above was used to estimate the incursion risk of 10 livestock diseases (AHS, ASF, Auj, BT, bTB, CSF, EIA, FMD, LSD, PPR) for the Netherlands for the years 2016, 2017, and 2018. Model calculations in RRAT are deterministic resulting in a point estimate for each output parameter. Main output parameters considered were the risk scores for individual diseases (_{RPi, D}) for each introduction route per year, and the contribution of source countries and pathways to these estimated risk scores.

Sensitivity analysis

Sensitivity analysis was performed to explore the impact of assumptions and input parameters on the results of RRAT. Three main areas of input uncertainty were investigated: (1) the incidence of disease in source countries, (2) the use of proxy values to estimate probabilities, and (3) the databases used to derive the volume of animals and animal products for the animal and traveler pathway, respectively (Table 3). A total of 12 alternative scenarios was run and results were compared to the baseline scenario for the overall risk score _RNi (Eq. 2) per introduction route. Considering that the main objective of RRAT is to prioritize diseases for risk management, changes in the overall risk score might be of less concern than changes in ranking of diseases, source countries, or pathways with respect to their incursion risk. To analyze the impact of uncertainty on ranking, the risk scores for individual diseases (_{RNi, D}) and individual source countries (_{RNi, C}) were ranked for both the baseline scenario and the alternative scenarios and compared using Spearman's rank correlation coefficient. The number-based risk score R_N was used for the sensitivity analysis rather than the probability-based risk score R_P, because differences between scenarios cannot be observed for high probability-based scores due to the asymptotic nature of R_P.

TABLE 3

Table 3. Alternative scenarios explored in the sensitivity analysis to evaluate the impact of uncertain input parameters on the results of RRAT.

Results

Baseline scenario

Model calculations returned a risk score (_{RPi, D}) for each disease and each introduction route in RRAT for the Netherlands for the years 2016-2018 (Figure 4). The overall risk was highest for bTB with risk scores approaching 1 for both the animal route and the product route. The incursion risk of AHS, LSD and PPR, on the other hand, was very low for all introduction routes. Trade in live animals also posed a risk for EIA incursion, although the risk decreased over the years considered, and – to a much lesser extent – for BT incursion (Figure 4A). Despite the threatening ASF situation in Europe in the period 2016-2018, the probability of ASF incursion by the animal route was very low, because no live pigs were imported from infected countries. The relatively low incursion risk of most diseases for the animal route is explained by the fact that livestock animals were almost exclusively imported from European countries in which most of the diseases considered were reported absent. Trade of animal products entailed an incursion risk for a larger number of diseases than trade in live animals, since products were imported from a much wider geographical range including sometimes infected areas. The highest incursion risks for the product route were observed for bTB, Auj, BT and FMD. The incursion risk for ASF had increased tremendously in 2018 if compared to previous years which is explained by the expansion of ASF-infected territories in 2018, both in Europe and South-East Asia (6). Calculated risk scores for the traveler route were much lower than for the product route. Diseases most likely introduced via the traveler route were CSF, ASF, FMD and bTB. Although travelers are not allowed to carry animal products from outside the EU, products were carried from all over the world including regions from which legal import of products is restricted. This resulted in a different ranking of diseases for the traveler route than the product route.

FIGURE 4

Figure 4. Probability-based risk score for the incursion risk of 10 diseases for the Netherlands in 2016, 2017, and 2018 for the animal route (A), the product route (B), and the traveler route (C). The incursion risk of AHS and EIA were not considered for the traveler route. Diseases: AHS, African horse sickness; ASF, African swine fever; Auj, Aujeszky's disease; BT, bluetongue; bTB, bovine tuberculosis; CSF, classical swine fever; EIA, equine infectious anemia; FMD, foot-and-mouth disease; LSD, lumpy skin disease; PPR, peste des petits ruminants.

As the RRAT calculates individual risk scores for each disease, pathway and source country, results can be explored in more detail to elucidate the countries and/or pathways contributing most to the incursion risk for a specific disease. Figure 5A shows the incursion risk of bovine tuberculosis per source country for the animal route and indicates that the incursion risk mainly originated from Ireland, Poland, Belgium, the United Kingdom and Spain. This was either related to a high incidence of bTB in those countries, high numbers of bovines imported from those countries (mainly veal calves), or both. Remarkably, there was also a risk of introducing bTB by trade of animals originating from Chile. This was related to the importation of camelids (lamas and vicunas). Horses entering the Netherlands more frequently originated from countries outside Europe than livestock animals. This is reflected by the countries contributing most to the incursion risk of EIA, not only being Bulgaria and Italy, but also the United States of America (Figure 5B).

FIGURE 5

Figure 5. Probability-based risk score for the incursion risk of bovine tuberculosis (A) and equine infectious anemia (B) for the Netherlands in 2016, 2017, and 2018 for the animal route per source country (only source countries included with a risk score > 0.01 in any year).

The incursion risk due to legal trade of animal products was explored in more detail for bTB, Auj, BT, FMD, and ASF. Whereas the incursion risk of bTB via the product route was high for multiple countries, with Ireland, United Kingdom, Spain, Belgium, Italy and China contributing most to the risk, the incursion risk of the other diseases was largely due to importations from a few countries only (Figure 6). The countries contributing most to the incursion risk for Auj were Italy, Bulgaria and Romania; for BT it was the USA; for FMD it were Pakistan, Thailand and South Korea; and for ASF it was Romania. Importations of meat products (fresh and frozen) contributed most to the incursion risk for Auj and ASF, whereas for bTB and FMD importations of milk and dairy products constituted a high incursion risk. The incursion risk of BT was mainly related to the import of products for pharmaceutical use (containing blood-derived products) (Figure 7A). It is noteworthy that import of litter and manure contributed considerably to the incursion risk of bTB and that import of hides contributed considerably to the incursion risk of FMD. Litter and manure were imported mainly from Belgium and Germany in large quantities (>2 × 10³ tons annually). This combined with bTB reported in Belgium resulted in a non-negligible incursion risk, albeit the probability of bTB infection of local animals upon entry of manure (P_estPD) was low. Hides were imported from all over the world with FMD-infected countries such as Thailand, India, Pakistan and South Korea being main suppliers from outside the EU. Although this contributed to the incursion risk of FMD, the overall incursion risk by hides was scored as low.

FIGURE 6

Figure 6. Probability-based risk score for the incursion risk of bovine tuberculosis (A), Aujeszky's disease (B), bluetongue (C), foot-and-mouth disease (D) and African swine fever (E) for the Netherlands in 2016, 2017, and 2018 for the product route per source country (only source countries included with a risk score > 0.01 in any year).

FIGURE 7

Figure 7. Contribution of different pathways to the incursion risk of selected diseases for the Netherlands in 2016, 2017, and 2018 for the product route (A) and the traveler route (B). Diseases: bTB, bovine tuberculosis; Auj, Aujeszky's disease; BT, bluetongue; FMD, foot-and-mouth disease; ASF, African swine fever; CSF, classical swine fever. Products: FF, fresh and frozen meat; DS, dried and salted meat.

Figure 8 shows the incursion risk due to travelers per source region for CSF, ASF, FMD, and bTB. CSF was most likely introduced from the Caribbean and Eastern Asia, although a steep increase in the incursion risk from Eastern Europe was observed in 2018. A similar risk profile was observed for ASF, although Western Africa was also a risk region for incursion of ASF via the traveler route. Overall risk scores for ASF were, however, lower than for CSF. For FMD, Eastern Asia and the Near and Middle East were the most likely source regions. bTB was most likely to be introduced from Northern Africa and the Near and Middle East. The incursion risk of CSF and ASF was completely related to carriage of pig meat, with fresh and frozen meat contributing approximately 90% to the risk and dried and salted meat approximately 10%. The incursion risk of FMD and bTB was related to both meat and dairy products, with dairy contributing approximately 50–60% of the risk (Figure 7B).

FIGURE 8

Figure 8. Probability-based risk score for the incursion risk of classical swine fever (A), African swine fever (B), foot-and-mouth disease (C) and bovine tuberculosis (D) for the Netherlands in 2016, 2017, and 2018 for the traveler route per source region.

Sensitivity analysis

Results of the alternative scenarios (Table 3) were compared to the baseline scenario for the overall risk score _RNi, which indicates the incursion risk of any of the diseases in RRAT to the Netherlands for each of the introduction routes i for the years 2016, 2017 and 2018. The number-based risk score was used rather than the probability-based risk score, as the probability-based risk has an asymptote at 1, making a comparison of results useless. The results of the animal route were most sensitive to the database used for the trade figures (Scenario 3A) with the use of Comext data (20) resulting in a 10-fold higher overall risk score (Figure 9). The other introduction routes were not affected by this scenario, since only the database for trade in live animals was changed. Scenario 1C affected the overall risk score most (Figure 9). In this scenario the value for Inc_unkD was increased 10- to 100-fold, resulting in a similar increase for the overall risk score of the product and traveler routes. The impact on the animal route was less pronounced, because imports of live animals mostly originated from source countries for which disease was absent or the incidence was known (i.e., Inc_unkD was not needed to estimate disease incidence for these countries). Scenario 1E also affected the overall risk score of all three introduction routes, although to a lesser extent. In this scenario, an underreporting factor was included to estimate disease incidence for countries that had reported cases to the OIE, resulting in higher incidence estimates for these countries. Scenarios 2D (proxy values for probability of contamination of a product at exposure) and 2E (proxy values for probability of infection upon exposure to a contaminated product) resulted in an increased overall risk score for the product and traveler routes. This was not unexpected as higher proxy values were used in the alternative scenarios. These scenarios did not affect the animal route. All other scenarios had limited effect on the calculated overall risk scores.

FIGURE 9

Figure 9. Number-based risk scores for the incursion risk of any disease for the Netherlands in 2016, 2017, and 2018 for the animal route (A), the product route (B), and the traveler route (C) for the baseline scenario and each alternative scenario. Risk scores are given on a log₁₀ scale.

Changes in ranking of diseases and source countries (source regions for the traveler route) when running the alternative scenarios were evaluated using Spearman's rank correlation coefficient. Figure 10 shows the correlation coefficients between the baseline and the alternative scenarios for the source countries/regions (x-axis) and the diseases (y-axis). Correlation coefficients for the product route were all > 0.9, indicating that changes in ranking were limited. For the animal route, only scenario 3A (trade figures based on Comext database) resulted in considerable changes of the ranking of both source countries and diseases for all 3 years evaluated. For the traveler route, results were slightly less stable than for the other two routes, but only scenario 1C (higher value for Inc_unkD) resulted in considerable changes of the ranking of both source countries and diseases for all 3 years evaluated. The relative sensitivity of this route to the value of Inc_unkD is explained from the fact that travelers could come from any country in the world, including those countries with an unknown disease status, whereas imports of live animals and animal products were mostly limited to countries with a known disease status, although not exclusively.

FIGURE 10

Figure 10. Spearman's rank correlation coefficients indicating the agreement in ranking of risk scores for individual source countries^a (x-axis, “geographical”) and individual diseases (y-axis, “disease”) between the baseline scenario and each alternative scenario. ^a Ranking for the traveler route was based on source regions.

Discussion

Interpretation of results

RRAT is a useful tool to assess the incursion risk of multiple diseases and results can be used to prioritize diseases for risk management and early warning. RRAT provides a multitude of information to evaluate the incursion risk of livestock diseases at different levels of detail. Results of the tool can be queried to indicate the pathways (animal species or product types) and source countries/regions contributing most to the risk (as shown in Figures 5–8). This information is useful input for the design of risk-based surveillance. To give risk managers access to all results of RRAT, an online visualization tool was built (https://shiny.wur.nl/content/941b9565-64d1-490c-b11b-d5f2cc45c44e/).

RRAT was built such that it can be automatically updated when new data becomes available. Input data from WAHIS (58), TRACES (23) and Comext (20) are automatically processed into risk scores for the diseases included in the tool. Adding a new disease to the tool is relatively easy as it only requires an update of disease-specific parameters. Adapting the tool to assess the risk for other target areas (countries) is also relatively easy, since this only requires an update of the volumes of trade and travel. The tool does, however, not provide the full remit of the incursion risk as introduction routes related to e.g., vector and wildlife ecology are not considered. This might explain the relatively low risk calculated for ASF despite the presence of ASF virus in Europe. Inclusion of additional introduction routes will increase the accuracy of the estimated incursion risk. Counotte et al. (65) designed a complementary module for RRAT using a generic approach to assess the animal disease incursion risk via wildlife migration. They showed that also the incursion risk of ASF via migration of wild boar is very low for the Netherlands given the relatively large geographical distance between reported outbreaks in wild boar and the Dutch border for the years analyzed. Results of the wildlife module can, however, not be directly compared to the results obtained for the introduction routes in RRAT, because the wildlife module only estimates the probability of entry of infected wild boar into the Netherlands and not the subsequent exposure of local livestock. It must be noted that also the results of the introduction routes in RRAT itself cannot be compared directly, as the risk estimates for the animal route are based on individual animals, whereas the risk estimates for the product and traveler route are based on kilograms of product. This might have resulted in a slight overestimate of the incursion risk by the product and traveler route, since from a single slaughtered animal more than one kilogram of product is harvested. There is no easy way to account for this in the risk assessment tool, as it is not known whether imported animal products are mostly derived from the same animals or from different animals, i.e., 10 kg of pork could have been derived from the same pig or from 10 pigs or even more. The ratio between slaughtered animals and the weight of imported products will probably also differ for different animal product types and source regions. To guide policy makers in interpreting the results of RRAT, we translated the semi-quantitative risk scores into qualitative risk levels. When doing so, we accounted for the fact that animal products will mostly present a lower incursion risk than live animals (Supplementary Table S5.1,S5.2). The qualitative risk levels were used to present results in the online visualization tool.

In contrast to the animal route, results of the product and traveler route can to some extent be compared, as both are based on kilograms of products. From Figure 4 it is clear that calculated risk scores for the traveler route are much lower than for the product route. This is mainly explained from the volumes for both introduction routes, with the quantities of products carried by travelers being approximately 10³ times less than the quantities imported legally (Supplementary Table S1.2,S1.3). On the other hand, products carried by travelers have a more diverse geographic origin and are not subjected to import controls, resulting in a potentially higher incursion risk per kilogram of product. The incursion risk via the product route might have been slightly overestimated by RRAT, as we had quite some uncertainty on the animal origin of products not intended for human consumption (e.g., casings, hides, products for pharmaceutical use). Most CN codes (combined nomenclature) (66) for these products represent composite groups and a worst-case approach was used considering all products a risk when these were derived from at least one susceptible domestic livestock species. Although products not intended for human consumption only made up about 10% of the total legal import flows, they had a very high contribution to the incursion risk of BT (Figure 7). In contrast to the product route, the incursion risk of the traveler route was based only on animal products for human consumption. The incursion risk via this route has definitely been underestimated by RRAT. We only included products carried by air passengers from outside the EU, since no data was available on products carried by travelers within the EU, because bringing products of animal origin from other EU member states is not illegal and thus not checked at customs. In addition, the incursion risk via animal products carried by travelers over land (train, bus, car) is not considered in RRAT.

Validation of results

Validation of the results of RRAT is difficult as the tool estimates the incursion risk of diseases that are not introduced into the Netherlands regularly. The only exception is bTB for which 23 introductions occurred in the period 1999-2013 by trade in live animals (67). RRAT indeed indicated that trade in live animals entails a high risk of bTB introduction, indicating to a large extent the same source countries as high risk as the study of De Vos et al. (67). The estimated EIA incursion risk by legal trade in live animals was very high for 2016 and decreased in the years after. In 2017, the first (and until now only) case of EIA in the Netherlands was detected by serology, the moment of introduction of the infection being unknown (68). The estimated ASF incursion risk was relatively low, although a steep increase of the incursion risk by the product route was seen for 2018. Despite the increasing threat of ASF in Europe in recent years, at the time of writing (June 2022), ASF was absent from the Netherlands. The most likely introduction route for ASF, based on results of RRAT, is via legal trade of animal products. Although it cannot be excluded that contaminated pork products have been imported in recent years, this has not resulted in disease outbreaks. The probability that contaminated pork products end up with pigs is expected to be very low, as swill feeding is not allowed in the EU (69). The results of RRAT can also be partly validated by comparing the results of RRAT to those of bespoke RA models, although one should keep in mind that the risk estimates given by RRAT are semi-quantitative risk scores rather than absolute numbers. The incursion risk of AHS was, e.g., estimated to be very low by RRAT (Figure 4), which is in agreement with a quantitative risk assessment for movements of live equines by De Vos et al. (11).

RRAT was cross-validated against other generic risk assessment tools that recently were developed in Europe by applying all tools to the same case study on ASF (16). Results indicated that the generic tools largely agreed on the relative risks across countries and scenarios, resulting in the same ranking. RRAT was primarily designed for prioritization purposes, the ranking of diseases, source countries and pathways being thus the most important output of the tool. Therefore, the cross-validation contributed to the credibility of results obtained with RRAT. In addition, results for the years 2016-2018 were face validated by risk assessors and risk managers and any unexpected results were queried by investigating the underlying data in the tool. For instance, contrary to our expectations, China did not contribute much to the ASF incursion risk in 2018, despite presence of ASF in China since August 2018 (6, 70) and large volumes of pork products being imported from China (20). The huge pig population (4.3 × 10⁸ heads) (60) in China resulted, however, in a low estimate for the incidence of ASF and consequently also for the incursion risk posed by pig products imported from China.

Robustness of results

RRAT can be classified as a semi-quantitative risk tool. The output of RRAT is presented as risk scores between 0 and 1. Although the risk score is calculated as if it were the probability of at least one introduction per year, the absolute value of the risk score cannot be interpreted as such, because input values for probabilities in RRAT are to a large extent based on risk classes rather than quantitative data derived from literature or experiments. These risk classes have been translated into proxy values to allow for the calculation of risk scores. Results of RRAT thus give an indication of relative risks rather than absolute risks and are therefore most useful for prioritization.

The impact of the proxy values was evaluated in the sensitivity analysis and appeared to be limited. In most scenarios, the change in proxy values did not affect the estimated risk scores. However, higher values for the probability of contamination of products at exposure, P_contexPD, and the probability of infection upon exposure, P_infexPD (Figure 9; scenarios 2D and 2E), resulted in higher risk scores for the product and traveler route. The ranking of diseases, pathways and source countries/regions was, however, only slightly affected in these scenarios (Figure 10 and Supplementary Figure S4.2). Changing of the proxy values used to estimate the incidence of disease if countries had an unknown disease status (Inc_unkD) had a large impact on the estimated risk scores (Figure 9; scenario 1C). For the traveler route, the change of Inc_unkD also had an impact on the ranking of diseases, pathways and source countries/regions (Figure 10 and Supplementary Figure S4.2).

Even though data from global databases is inputted into RRAT as purely quantitative data, these also contain uncertainty. Numbers of livestock imported, e.g., differ considerably between TRACES and Comext. The effect of using data from Comext (20) rather than TRACES was explored in scenario 3A. Results indicated that risk estimates based on Comext were much higher than based on TRACES (Figure 9). Ranking of diseases, pathways and source countries was also highly affected by the global database used (Figure 10 and Supplementary Figure S4.2). Similarly, data from WAHIS on disease occurrence worldwide is biased due to underreporting or non-reporting. Scenario 1E of the sensitivity analysis indicated that risk estimates were higher, especially for the animal route, when correcting for underreporting (Figure 9). Ranking of diseases, pathways and source countries/regions was, however, not affected (Figure 10 and Supplementary Figure S4.2). In this scenario we assumed equal underreporting for all geographic regions, whereas in reality there might be differences depending on, e.g., surveillance in place. Disease incidence could only be calculated for a subset of countries in which disease was present. Therefore, a decision tree was used in RRAT to classify countries for their disease risk based on quantitative and qualitative data available from WAHIS (Figure 3). If countries did not report at all (neither absence nor presence), they were classified as high risk, unless we had evidence that disease was likely to be absent based on information from other countries in the same region. For the EU, data on disease outbreaks from WAHIS was complemented with data from the Animal Disease Information System (ADIS) (61) and EC reports (62–64) if available. For countries in other regions in the world, the data in RRAT was solely derived from WAHIS. To account for the fact that disease might be present unnoticed, we also considered the disease status of neighboring countries (based on UN subregions) (59) to assign a disease status to countries that reported absence of disease. This sometimes resulted in a likely overestimate of the incursion risk, e.g., when considering the ASF incursion risk from Denmark, that is clustered with the Baltic states in which ASF has been present since 2014 (71).

Based on the results of the what-if analysis, we conclude that risk estimates given by RRAT are more sensitive to uncertainties in data reported by global databases than uncertainty introduced by expert opinion when using proxy values to assign quantitative probabilities to risk classes. Uncertainties in global databases can directly be traced to reporting issues, both when considering disease outbreaks and trade of animals and animal products. Where TRACES was built to track and trace animal movements within the EU from the perspective of animal and public health, the data in Comext is primarily obtained from import and export flows as declared by customs from an economic perspective.

We also calculated Spearman's rank correlation coefficients to compare the ranking of diseases, pathways and countries/regions among different years (Supplementary Figure S4.3). Strikingly, the differences between years were in general bigger than the differences observed between scenarios in the sensitivity analysis. This emphasizes that historical data cannot directly be used to predict future incursion risks. When we conceptualized RRAT, we aimed at regular updates of the risk assessments in an automated fashion to ensure that the estimated incursion risks reflect the current conditions. Therefore, RRAT has been designed such that updates of the assessment can be easily made when new data becomes available. RRAT is, however, dependent on data from global databases on disease outbreaks, and trade and travel, making the tool vulnerable to changes in these databases. In 2021, the OIE has launched a new WAHIS interface (7), making the R scripts that we prepared to scrape the annual reports off their website useless. This, and the delay in the launch and realization of the new WAHIS interface, has hampered timely updates of RRAT with 2019 and 2020 data. The next step in the development of RRAT is to adapt the R scripts such that we can easily import data on disease outbreaks from the new WAHIS website. The availability of application programming interfaces (api) to import data facilitates the use of global databases in estimating disease incursion risks. The development of generic risk assessment tools such as RRAT also illustrates the importance of building and maintaining global databases using the FAIR principles (findable, accessible, interoperable and reusable). Disease-specific parameters in RRAT have been entered once and are considered not to be subjected to change at short notice. The only exception is the legislation for import of live animals. EU requirements for importations of live animals have been regularly updated in recent years, especially for equines. Most changes had, however, little effect on the estimated incursion risks as they concerned source countries and animal species with low-volume trade flows. However, with the implementation of the Animal Health Law (72) in 2021, an update of the legislation tables in RRAT is needed. Unfortunately, we have not been able to design an automated procedure for this task.

Comparison with other generic risk assessment tools

Several other generic risk assessment tools have been developed in recent years [e.g., (16, 17, 73–79)]. Each of these tools were developed with different objectives, and different approaches were used (16). Some of these tools can be used for rapid risk assessment in response to disease events and have expert opinion as input [e.g., (74, 75, 80)]. However, only few of these tools have, like RRAT, the data needed for the risk estimates available in the tool [e.g., (17, 76)], allowing for a rapid response without the need to bring disease experts together. The main asset of these tools is that risk assessments can be updated relatively easy, making the tools suitable for horizon scanning. Another difference is that some of the tools only address the probability of entry into a new area [e.g., (76)], whereas others also include epidemiological [e.g., (77)] or economic consequences [e.g., (75, 78, 79)]. RRAT has an in-between position by including the exposure assessment and the probability of a new infection, but not estimating subsequent spread of disease, or impact on animal health and economics. We deemed the inclusion of a first infection in local livestock a minimal requirement to make results of the tool meaningful, as import of contaminated products does not by definition result in disease outbreaks, nor does import of animals for slaughter or import of exotic animals in case of subclinical infections and no contacts with livestock farms. A shared challenge for these generic risk assessment tools is to keep them up and running and to have added value to policy makers in setting priorities for preventive measures and surveillance. Bianchini et al. (81) did a survey on the use of animal health information systems and risk analysis tools among professionals in animal and public health around the world. They concluded that the main areas of interest from these systems and tools are information on where diseases are present, pathways of introduction, and spread assessment. RRAT provides insight into the first two areas of interest. Results of RRAT are easily accessible via the online visualization tool, allowing for independent use by policy makers.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary material, further inquiries can be directed to the corresponding author.

Author contributions

CV and MS initiated and conceptualized the research and drafted the equations and algorithms for RRAT. CV and RP designed the outline of RRAT. CV and RP collected data from global databases. RP cleaned and analyzed all data inputted into RRAT. CV, MS, and EK collected data on disease-specific parameters, legislation for animal trade and travelers. RP built the online visualization tool. CV analyzed the results of RRAT and drafted the manuscript. All authors contributed to the article and approved the submitted version.

Funding

The development of RRAT was funded by the Dutch Ministry of Agriculture, Nature and Food Quality (KB-21-006-028, KB-37-003-033, WOT-01-003-078, and WOT-01-003-094) and Wageningen University & Research (KB-33-001-008-WBVR).

Acknowledgments

Data from TRACES was made available by the Netherlands Food and Consumer Product Safety Authority (NVWA). We thank Marcel Spierenburg, Patrick van Dijk and Ruud Oberndorff for their support in retrieving these data. The R script used to obtain data on disease occurrence was developed by Dr. Robin Simons (APHA, United kingdom) with funding agreed through the Animal Health and Welfare ERA-NET consortium (https://www.anihwa.eu/) under the SPARE project (‘Spatial risk assessment framework for assessing exotic disease incursion and spread through Europe'). It was originally described in a conference contribution (Simons et al. Mighty models from little data grow: Estimating animal disease prevalence. Proceedings of the 2017 SVEPM conference, Inverness, Scotland. Pp. 166-175). We thank Michel Counotte (WBVR, Netherlands) for critically reading an earlier version of the manuscript.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fvets.2022.963758/full#supplementary-material

References

1. Davies G. The foot and mouth disease (FMD) epidemic in the United Kingdom 2001. Comp Immunol Microbiol Infect Dis. (2002) 25:331–43. doi: 10.1016/S0147-9571(02)00030-9

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Pluimers FH, Akkerman AM, Van der Wal P, Dekker A, Bianchi A. Lessons from the foot and mouth disease outbreak in the Netherlands in 2001. Rev Sci Tech. (2002) 21:711–21. doi: 10.20506/rst.21.3.1371

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Elbers ARW, Backx A, Meroc E, Gerbier G, Staubach C, Hendrickx G, et al. Field observations during the bluetongue serotype 8 epidemic in 2006 I. Detection of first outbreaks and clinical signs in sheep and cattle in Belgium, France and the Netherlands. Prev Vet Med. (2008) 87:21–30. doi: 10.1016/j.prevetmed.2008.06.004

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Saegerman C, Berkvens D, Mellor PS. Bluetongue epidemiology in the European Union. Emerg Inf Dis. (2008) 14:539–44. doi: 10.3201/eid1404.071441

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Sánchez-Vizcaíno JM, Mur L, Martínez-López B. African swine fever (ASF): five years around Europe. Vet Microbiol. (2013) 165:45–50. doi: 10.1016/j.vetmic.2012.11.030

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Dixon LK, Sun H, Roberts H. African swine fever. Antiviral Res. (2019) 165:34–41. doi: 10.1016/j.antiviral.2019.02.018

PubMed Abstract | CrossRef Full Text | Google Scholar

7. OIE. World Animal Health Information System. World Organisation for Animal Health. Available online at: https://wahis.oie.int/#/home (2022). (accessed March 16, 2022).

Google Scholar

8. EFSA (European Food Safety Authority). Scientific report on lumpy skin disease: i. Data collection and analysis. EFSA J. (2017) 15:4773. doi: 10.2903/j.efsa.2017.4773

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Tuppurainen E, Alexandrov, T, Beltrán-Alcrudo, D,. Lumpy skin disease. A field manual for veterinarians. FAO Animal Production Health Manual No. 20. Rome, Italy: Food Agriculture Organization of the United Nations (FAO) Available online at: https://www.fao.org/3/i7330en/I7330EN.pdf (2017). (accessed May 30, 2022).

Google Scholar

10. Martínez-López B, Perez AM. De la Torre A, Sánchez-Vizcaíno Rodriguez JM. Quantitative risk assessment of foot-and-mouth disease introduction into Spain via importation of live animals. Prev Vet Med. (2008) 86:43–56. doi: 10.1016/j.prevetmed.2008.03.003

PubMed Abstract | CrossRef Full Text | Google Scholar

11. De Vos CJ, Hoek CA, Nodelijk G. Risk of introducing African horse sickness virus into the Netherlands by international equine movements. Prev Vet Med. (2012) 106:108–22. doi: 10.1016/j.prevetmed.2012.01.019

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Mur L, Martinez-Lopez B, Martinez-Aviles M, Costard S, Wieland B, Pfeiffer DU, et al. Quantitative risk assessment for the introduction of African swine fever virus into the European Union by legal import of live pigs. Transbound Emerg Dis. (2012) 59:134–44. doi: 10.1111/j.1865-1682.2011.01253.x

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Costard S, Jones BA, Martínez-López B, Mur L., de la Torre A, Martínez M, et al. Introduction of African swine fever into the European Union through illegal importation of pork and pork products. PLoS ONE. (2013) 8:e61104. doi: 10.1371/journal.pone.0061104

PubMed Abstract | CrossRef Full Text | Google Scholar

14. De la Torre A, Bosch J, Iglesias I, Mu noz MJ, Mur L, Martínez-López B, et al. Assessing the risk of African swine fever introduction into the European Union (EU) by wild boar. Transbound Emerg Dis. (2015) 62:272–9. doi: 10.1111/tbed.12129

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Gierak A, Smietanka K, De Vos CJ. Quantitative risk assessment of the introduction of low pathogenic avian influenza H5 and H7 strains into Poland via legal import of live poultry. Prev Vet Med. (2021) 189:105289. doi: 10.1016/j.prevetmed.2021.105289

PubMed Abstract | CrossRef Full Text | Google Scholar

16. De Vos CJ, Taylor RA, Simons RRL, Roberts H, Hultén C, De Koeijer AA, et al. Cross-validation of generic risk assessment tools for animal disease incursion based on a case study for African swine fever. Front Vet Sci. (2020) 7:56. doi: 10.3389/fvets.2020.00056

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Roberts H, Carbon M, Hartley M, Sabirovic M. Assessing the risk of disease introduction in imports. Vet Rec. (2011) 168:447–8. doi: 10.1136/vr.d1784

PubMed Abstract | CrossRef Full Text | Google Scholar

18. FAO. EMPRES Global Animal Disease Information System (EMPRES-i). Available online at: https://empres-i.apps.fao.org/ (2022). (accessed March 16, 2022).

Google Scholar

19. UN. UN Comtrade Database. United Nations. Available online at: https://comtrade.un.org/ (2022). (accessed March 16, 2022).

Google Scholar

20. Eurostat. Comext Bulk Download. Available online at: https://ec.europa.eu/eurostat/estat-navtree-portlet-prod/BulkDownloadListing?sort=1&dir=comext (2022). (accessed March 16, 2022).

Google Scholar

21. R Core Team. The R Project for Statistical Computing. Available online at: https://www.r-project.org/ (2022). (accessed March 16, 2022).

Google Scholar

22. SQLite. DB Browser for SQLite. Available online at: https://sqlitebrowser.org/ (2022). (accessed March 16, 2022).

Google Scholar

23. EC (European Commission). TRACES: TRAde Control Expert System. Available online at: https://ec.europa.eu/food/animals/traces_en (2022). (accessed March 16, 2022).

Google Scholar

24. Dórea FC, Swanenburg M, Van Roermund H, Horigan V, De Vos C, Gale P, et al. Data collection for risk assessments on animal health. EFSA Support Publ. (2017) EN-1171. doi: 10.2903/sp.efsa.2017.EN-1171

CrossRef Full Text | Google Scholar

25. OIE. Terrestrial Animal Health Code. World Organisation for Animal Health. Available online at: https://www.oie.int/en/what-we-do/standards/codes-and-manuals/terrestrial-code-online-access/ (2021). (accessed March 16, 2022).

Google Scholar

26. OIE. Manual of Diagnostic Tests and Vaccines for Terrestrial Animals. World Organisation for Animal Health. Available online at: https://www.oie.int/en/what-we-do/standards/codes-and-manuals/terrestrial-manual-online-access/ (2021). (accessed March 16, 2022).

Google Scholar

27. OIE. Animal Diseases. World Organisation for Animal Health. Available online at: https://www.oie.int/en/what-we-do/animal-health-and-welfare/animal-diseases/ (2022). (accessed March 16, 2022).

Google Scholar

28. CFSPH. The Center for Food Security and Animal Health, Iowa State University. Available online at: https://www.cfsph.iastate.edu/diseaseinfo/ (2022). (accessed March 16, 2022).

Google Scholar

29. Discontools. Research Gaps for Improving Infectious Disease Control in Animals. Available online at: https://www.discontools.eu/database.html (2022). (accessed March 16, 2022).

Google Scholar

30. EFSA. EFSA Disease Profiles. Available online at: https://animal-diseases.efsa.europa.eu/ (2022). (accessed March 16, 2022).

Google Scholar

31. EU (European Union). Council Directive 64/432/EEC of 26 June 1964 on animal health problems affecting intra-Community trade in bovine animals and swine. Off J Eur Union. (1964) 121:1977–2012.

Google Scholar

32. EU (European Union). Council Directive 91/68/EEC of 28 January 1991 on animal health conditions governing intra-Community trade in ovine and caprine animals. Off J Eur Union. (1991) L46:19–36.

Google Scholar

33. EU (European Union). Commission Decision 92/260/EEC of 10 April 1992 on animal health conditions and veterinary certification for temporary admission of registered horses. Off J Eur Union. (1992) L130: 67–83.

Google Scholar

34. EU (European Union). Council Directive 92/65/EEC of 13 July 1992 laying down animal health requirements governing trade in and imports into the Community of animals, semen, ova and embryos not subject to animal health requirements laid down in specific Community rules referred to in Annex A (I) to Directive 90/425/EEC. Off J Eur Union. (1992) L268:54–72.

Google Scholar

35. EU (European Union). Commission Decision 93/195/EEC of 2 February 1993 on animal health conditions and veterinary certification for the re-entry of registered horses for racing, competition and cultural events after temporary export. Off J Eur Union. (1993) L86:1-6.

Google Scholar

36. EU (European Union). Commission Decision 93/196/EEC of 5 February 1993 on animal health conditions and veterinary certification for imports of equidae for slaughter. Off J Eur Union. (1993) L86:7–15.

Google Scholar

37. EU (European Union). Commission Decision 93/197/EEC of 5 February 1993 on animal health conditions and veterinary certification for imports of registered equidae and equidae for breeding and production. Off J Eur Union. (1993) L86:16–33.

Google Scholar

38. EU (European Union). Commission Regulation (EU) No 206/2010 of 12 March 2010 laying down lists of third countries, territories or parts thereof authorised for the introduction into the European Union of certain animals and fresh meat and the veterinary certification requirements. Off J Eur Union. (2010) L073:1–171.

Google Scholar

39. EU (European Union). COMMISSION REGULATION (EU) No 142/2011 of 25 February 2011 implementing regulation (EC) No 1069/2009 of the European Parliament and of the Council laying down health rules as regards animal by-products and derived products not intended for human consumption and implementing Council Directive 97/78/EC as regards certain samples and items exempt from veterinary checks at the border under that Directive. Off J Eur Union. (2011) L54:1–254.

Google Scholar

40. Eurostat. European Statistics on Transport. https://ec.europa.eu/eurostat/web/transport/data/database (2022). (accessed April 11, 2022).

Google Scholar

41. VLA. Risk assessment for the illegal import of meat and meat products contaminated with Foot and Mouth Disease (FMD), March 2003. UK: Centre for Epidemiology and Risk Analysis, Veterinary Laboratories Agency. Available online at: https://webarchive.nationalarchives.gov.uk/ukgwa/20060303120000/http:/www.defra.gov.uk/animalh/illegali/reports/index.htm (2003). (accessed March 16, 2022).

Google Scholar

42. VLA. Risk assessment for the illegal import of meat and meat products contaminated with Foot and Mouth Disease (FMD), 2004. UK: Centre for Epidemiology and Risk Analysis, Veterinary Laboratories Agency. Available online at: https://webarchive.nationalarchives.gov.uk/ukgwa/20060303120000/http:/www.defra.gov.uk/animalh/illegali/reports/index.htm (2004). (accessed March 16, 2022).

Google Scholar

43. Defra. Annual Review of Controls on Imports of Animal Products. April 2002 – March 2003. UK: Department for Environment Food and Rural Affairs. Available online at: https://webarchive.nationalarchives.gov.uk/ukgwa/20060303120000/http:/www.defra.gov.uk/animalh/illegali/news.htm (2003). (accessed December 16, 2019).

Google Scholar

44. Defra. Annual Review of Controls on Imports of Animal Products. April 2003 – March 2004. UK: Department for Environment Food and Rural Affairs. Available online at: https://webarchive.nationalarchives.gov.uk/ukgwa/20060303120000/http://www.defra.gov.uk/animalh/illegali/reports/index.htm (2004). (accessed December 16, 2019).

Google Scholar

45. Defra. Annual Review of Controls on Imports of Animal Products. April 2004 – March 2005. UK: Department for Environment Food and Rural Affairs. Available online at: https://webarchive.nationalarchives.gov.uk/ukgwa/20060303120000/http://www.defra.gov.uk/animalh/illegali/reports/index.htm (2005). (accessed December 16, 2019).

Google Scholar

46. Defra. Annual Review of Controls on Imports of Animal Products. April 2005 – March 2006. UK: Department for Environment Food and Rural Affairs. Available online at: https://webarchive.nationalarchives.gov.uk/20090903183114/http://www.official-documents.gov.uk/document/cm68/6897/6897.asp (2006). (accessed December 16, 2019).

Google Scholar

47. Defra. Annual Review of Controls on Imports of Animal Products. April 2006 – March 2007. UK: Department for Environment Food and Rural Affairs. Available online at: https://webarchive.nationalarchives.gov.uk/20090903175233/http://www.official-documents.gov.uk/document/cm71/7179/7179.asp (2007). (accessed December 16, 2019).

Google Scholar

48. Defra. Annual Review of Controls on Imports of Animal Products. April 2007 – March 2008. UK: Department for Environment Food and Rural Affairs. Available online at: https://webarchive.nationalarchives.gov.uk/20090903174059/http://www.official-documents.gov.uk/document/cm74/7444/7444.asp (2008). (accessed December 16, 2019).

Google Scholar

49. Defra. Annual Review of Controls on Imports of Animal Products. April 2008 – March 2009. UK: Department for Environment Food and Rural Affairs. Available online at: https://webarchive.nationalarchives.gov.uk/ukgwa/20121205170130/http:/www.official-documents.gov.uk/document/other/9780101764827/9780101764827.asp (2009). (accessed December 16, 2019).

Google Scholar

50. Defra. Annual Review of Controls on Imports of Animal Products. April 2009 – March 2010. UK: Department for Environment Food and Rural Affairs. Available online at: https://webarchive.nationalarchives.gov.uk/20121205161459/http://www.official-documents.gov.uk/document/other/9780102968576/9780102968576.asp (2010). (accessed December 16, 2019).

Google Scholar

51. Schoder D, Strauss A, Szakmary-Brändle K, Stessl B, Schlager S. Wagner, M. Prevalence of major foodborne pathogens in food confiscated from air passenger luggage. Internat J Food Microbiol. (2015) 209:3–12. doi: 10.1016/j.ijfoodmicro.2014.08.010

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Falk H, Dürr S, Hauser R, Wood K, Tenger B, Lörtscher M, et al. Illegal import of bushmeat and other meat products into Switzerland on commercial passenger flights. Rev Sci Tech. (2013) 32:727–39. doi: 10.20506/rst.32.2.2221

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Beutlich J, Hammerl JA, Appel B, Nöckler K, Helmuth R, Jöst K, et al. Characterization of illegal food items and identification of foodborne pathogens brought into the European Union via two major German airports. Internat J Food Microbiol. (2015) 209:13–9. doi: 10.1016/j.ijfoodmicro.2014.10.017

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Nagy B, Szmolka A, Smole MoŽina S, Kovač J, Strauss A, Schlager S, et al. Virulence and antimicrobial resistance determinants of verotoxigenic Escherichia coli (VTEC) and of multidrug-resistant E. coli from foods of animal origin illegally imported to the EU by flight passengers Internat J Food Microbiol. (2015) 209:52–9. doi: 10.1016/j.ijfoodmicro.2015.06.026

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Rodríguez-Lázaro D, Ariza-Miguel J. Diez-Valcarce M, Fernández-Natal I, Hernández M, Rovira J. Foods confiscated from non-EU flights as a neglected route of potential methicillin-resistant Staphylococcus aureus transmission. Internat J Food Microbiol. (2015) 209:29–33. doi: 10.1016/j.ijfoodmicro.2014.08.016

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Jansen W, Merkle M, Daun A, Flor M, Grabowski NT, Klein G. The quantity and quality of illegally imported products of animal origin in personal consignments into the European Union seized at two German airports between 2010 and 2014. PLoS ONE. (2016) 11:e0150023. doi: 10.1371/journal.pone.0150023

PubMed Abstract | CrossRef Full Text | Google Scholar

57. Chaber A-L, Allebone-Webb S, Lignereux Y, Cunningham AA, Rowcliffe JM. The scale of illegal meat importation from Africa to Europe via Paris. Conserv Lett. (2010) 3:317–23. doi: 10.1111/j.1755-263X.2010.00121.x

CrossRef Full Text | Google Scholar

58. OIE. World Animal Health Information System. World Organisation for Animal Health. Available online: http://www.oie.int/wahis_2/public/wahid.php/Wahidhome/Home/ (2020). (accessed June 3, 2020). Note that this website is no longer available after OIE launched the new WAHIS interface (see reference 7).

Google Scholar

59. UN. UN Statistics Division. Standard country or area codes for statistical use (M49). United Nations. Available online at: https://unstats.un.org/unsd/methodology/m49 (2022). (accessed March 15, 2022).

Google Scholar

60. FAO. FAOSTAT statistical database. Food and Agriculture Organization of the United Nations. Available online at: https://www.fao.org/faostat/en/#data (2022). (accessed March 16, 2022).

Google Scholar

61. EC (European Commission). Animal Disease Information System (ADIS). Available online at: https://ec.europa.eu/food/animals/animal-diseases/animal-disease-information-system-adis_en (2022). (accessed March 16, 2022).

Google Scholar

62. EC (European Commission). Bovine and Swine Diseases. Situation 2016. Available online at: https://www.visavet.es/bovinetuberculosis/data/reports/European-Commission-Bovine-swine-diseases-2016-Situation.pdf (2018). (accessed March 16, 2022).

Google Scholar

63. EC (European Commission). Bovine and Swine Diseases. Situation 2017. Available online at: https://www.visavet.es/bovinetuberculosis/data/reports/European-Commission-Bovine-swine-diseases-2017-Situation.pdf (2019). (accessed March 16, 2022).

Google Scholar

64. EC (European Commission). Bovine and Swine Diseases. Situation 2018. Available online at: https://www.visavet.es/bovinetuberculosis/data/reports/European-Commission-Bovine-swine-diseases-2018-Situation.pdf (2020). (accessed March 16, 2022).

Google Scholar

65. Counotte MJ, Petie R, Van Klink EGM, De Vos CJ A. generic risk assessment model for animal disease incursion through wildlife. BioRxiv. (2022). doi: 10.1101/2022.04.25.489353 [preprint].

PubMed Abstract | CrossRef Full Text | Google Scholar

66. EU (European Union). Council Regulation (EEC) No 2658/87 of 23 July 1987 on the tariff and statistical nomenclature and on the Common Customs Tariff. Off J Eur Union. (1987) L256:1–1069.

Google Scholar

67. De Vos CJ, Van der Goot JA, Van Zijderveld FG, Swanenburg M, Elbers ARW. Risk-based testing of imported animals: A case study for bovine tuberculosis in The Netherlands. Prev Vet Med. (2015) 121:8–20. doi: 10.1016/j.prevetmed.2015.04.017

PubMed Abstract | CrossRef Full Text | Google Scholar

68. Sloet van Oldruitenborgh-Oosterbaan M, Lommers H, Spierenburg M, Weesendorp E, Van Maanen K. Equine infectieuze anemie (EIA) nu ook in Nederland. Dier en Arts. 8/9:206-215. Available online at: https://www.wur.nl/upload_mm/a/7/3/890003e9-b20b-4064-8038-8b2fad4ebf96_2017%20EIA%20Sloet%20et%20al%20Dier-en-Arts.pdf (2017). (accessed March 15, 2022). (in Dutch).

Google Scholar

69. EU (European Union). Regulation (EC) No 1069/2009 of the European Parliament and of the Council of 21 October 2009 laying down health rules as regards animal by-products and derived products not intended for human consumption and repealing Regulation (EC) No 1774/2002 (Animal by-products Regulation). Off J Eur Union. (2009) L300:1–33.

Google Scholar

70. Gao L, Sun X, Yang H, Xu Q, Li J, Kang J, et al. Epidemic situation and control measures of African swine fever outbreaks in China 2018–2020. Transbound Emerg Dis. (2021) 68:2676–86. doi: 10.1111/tbed.13968

PubMed Abstract | CrossRef Full Text | Google Scholar

71. EFSA AHAW. Panel (EFSA Panel on Animal Health and Welfare). Scientific opinion on African swine fever EFSA J. (2015) 13:4163. doi: 10.2903/j.efsa.2015.4163

CrossRef Full Text | Google Scholar

72. European Union. Regulation (EU) 2016/429 of the European Parliament and of the Council of 9 March 2016 on transmissible animal diseases and amending and repealing certain acts in the area of animal health (‘Animal Health Law'). Off J Eur Union. (2016) L084:1–236. Available online at: http://data.europa.eu/eli/reg/2016/429/oj

Google Scholar

73. ANSES. Hiérarchisation de 103 maladies animales présentes dans les filiéres ruminants, équidés, porcs, volailles et lapins en France métropolitaine. Avis de l'Anses. Rapport d'expertise collective. Maisons-Alfort (2012). Available online at: https://www.anses.fr/fr/system/files/SANT2010sa0280Ra.pdf (accessed March 16, 2022).

Google Scholar

74. Roelandt S, Van der Stede Y, D'hondt B, Koenen F. The assessment of African swine fever risk to Belgium early 2014 using the quick and semi quantitative pandora screening tool. Transbound Emerg Dis. (2017) 64:237–49. doi: 10.1111/tbed.12365

PubMed Abstract | CrossRef Full Text | Google Scholar

75. Kyyrö J, Sahlström L, Lyytikäinen T. Assessment of the risk of African swine fever introduction into Finland using NORA - a rapid tool for semi quantitative assessment of the risk. Transbound Emerg Dis. (2017) 64:2113–25. doi: 10.1111/tbed.12633

PubMed Abstract | CrossRef Full Text | Google Scholar

76. Simons RRL, Horigan V, Ip S, Taylor RA, Crescio MI, Maurella C, et al. A spatial risk assessment model framework for incursion of exotic animal disease into the European Union Member States. Microbial Risk Anal. (2019) 13:100075. doi: 10.1016/j.mran.2019.05.001

CrossRef Full Text | Google Scholar

77. Taylor RA, Berriman ADC, Gale P, Kelly LA, Snary EL. A generic framework for spatial quantitative risk assessments of infectious diseases: lumpy skin disease case study. Transbound Emerg Dis. (2019) 66:131–43. doi: 10.1111/tbed.12993

PubMed Abstract | CrossRef Full Text | Google Scholar

78. Bessell PR, Auty HK, Roberts H, McKendrick IJ, Bronsvoort BMC, Boden LA. A tool for prioritizing livestock disease threats to Scotland. Front Vet Sci. (2020) 7:223. doi: 10.3389/fvets.2020.00223

PubMed Abstract | CrossRef Full Text | Google Scholar

79. De Vos CJ, Hennen WHGJ, Van Roermund HJW, Dhollander S, Fischer EAJ, De Koeijer AA. Assessing the introduction risk of vector-borne animal diseases for the Netherlands using MINTRISK: A Model for INTegrated RISK assessment. PLoS ONE. (2021) 16:e0259466. doi: 10.1371/journal.pone.0259466

PubMed Abstract | CrossRef Full Text | Google Scholar

80. De Vos CJ, Taylor RA, Simons RRL, Roberts H, Hultén C, De Koeijer AA, et al. Generic Approaches for Risk Assessment of Infectious Animal Disease Introduction (G-RAID). EFSA Supporting Publication. (2019). doi: 10.2903/sp.efsa.2019.EN-1743

CrossRef Full Text | Google Scholar

81. Bianchini J, Simons X, Faes C, Nicolas G, Vilain A, Hendrickx G, et al. Assessing the use of animal health platforms: User's needs, preferences and constraints. Transbound Emerg Dis. (2021) 00:1–15. doi: 10.1111/tbed.14008

PubMed Abstract | CrossRef Full Text | Google Scholar