Field Evaluation of Diagnostic Test Sensitivity and Specificity for Salmonid Alphavirus (SAV) Infection and Pancreas Disease (PD) in Farmed Atlantic salmon (Salmo salar L.) in Norway Using Bayesian Latent Class Analysis

Salmonid alphavirus (SAV) is the OIE-listed, viral cause of pancreas disease (PD) in farmed Atlantic salmon. SAV is routinely detected by PCR–methods while typical histopathological lesions are additionally used to confirm the diagnosis. Field evaluation of diagnostic test performance is essential to ensure confidence in a test's ability to predict the infection or disease status of a target animal. For most tests used in aquaculture, characteristics like sensitivity (Se) and specificity (Sp) at the analytical level may be known. Few tests are, however, evaluated at the diagnostic level according to the OIE standard. In the present work, we estimated diagnostic test sensitivity (DSe) and diagnostic test specificity (DSp) for five laboratory tests used for SAV detection. As there is no gold standard, the study was designed using Bayesian latent class analysis. Real-time RT-PCR, cell culture, histopathology, virus neutralization test, and immunohistochemistry were compared using samples taken from three different farmed Atlantic salmon populations with different infection status; one population regarded negative, one in an early stage of infection, and one in a later stage of infection. The average fish weight in the three populations was 2.0, 1.6, and 1.5 kg, respectively. The DSe and DSp of real-time RT-PCR is of particular interest due to its common use as a screening tool. The method showed high DSe (≥0.977) and moderate DSp (0.831) in all 3-populations models. The results further suggest that a follow-up test of serum samples in real-time RT-PCR negative populations may be prudent in cases where epidemiological information suggest a high risk of infection and where a false negative result is of high consequence. This study underlines the need to choose a test appropriate for the purpose of the testing. In the case of a weak positive PCR-result, a follow-up test should be conducted to verify the presence of SAV. Cell culture showed high DSe and DSp and may be used to verify viral presence.

Introduction

The ability to reliably detect the presence of important infectious disease agents in aquatic animal populations is essential for such diverse reasons as disease surveillance and international trade. All member countries of the World Organization for Animal Health (OIE) are obliged to provide timely and transparent information on its situation on listed animal diseases, information that plays an important part in relation to international trade and trade partner trust. The evaluation of a diagnostic test for its performance is essential to ensure confidence in the test's ability to accurately predict the infection or exposure status of the target animal (1). An important stage in OIE's test development and validation pathway is the validation of a test's diagnostic sensitivity (DSe) and diagnostic specificity (DSp) under field conditions (2), using a reference or gold standard test i.e., a diagnostic test which correctly identifies all true positive animals as positive and all true negative animals as negative.

The DSe in a veterinary context is the conditional probability that an infected animal will be test-positive, while DSp is the conditional probability that an uninfected animal will be test-negative (1). A gold standard test does not always exist or might not be available for field conditions. One alternative analytical tool for estimation of DSe and DSp is Bayesian latent class analysis (3, 4). The Bayesian approach for estimating diagnostic test accuracy does not require knowledge of true infection status and allows for the incorporation of prior scientific knowledge, such as disease prevalence in study populations, when specifying model parameters. Both prior information and new, collected data can then be used to assess diagnostic test performance. The models provide posterior distributions of the parameters of interest e.g., the median and 95% probability interval for test DSe and DSp.

The test performance (DSe and DSp), which are the properties of the diagnostic tool, are not subject to change with prevalence. However, DSe and DSp can be influenced by individual characteristics, and so the estimates can differ when applied to populations under different stages of infection e.g., infection during a long latent period or chronic infection. Bayesian latent class analysis is a robust method which has been widely used for the estimation of diagnostic test performance when a gold standard test does not exist. It is also the only method for combining existing knowledge and expert opinions into equations. However, there have been very few uses of the method in the field of aquatic animal diseases (5–8).

The Norwegian salmonid aquaculture industry is largely relying on foreign markets for its products and according to the Norwegian Seafood Council, one million metric tons of salmon was exported from Norway in 2017 (9). One of the major disease challenges affecting the Norwegian production is the OIE-listed viral agent salmonid alphavirus (SAV), with around 140–160 new grow-out populations being infected every year (10). There are six known genotypes of SAV (11), and the geographical distribution of the two endemic SAV-genotypes present in Norway, SAV3 and marine SAV2, are partially overlapping (12–14). As reviewed by Jansen et al. (14) infected fish commonly present with reduced appetite and abnormal swimming behavior prior to the onset of mortality. Post-mortem inspection commonly reveals empty intestines, fecal casts and signs of circulatory disturbance. On histological examination, major lesions include loss of exocrine pancreatic tissue, cardiac myocytic necrosis, and inflammation and degeneration of skeletal muscle. While mortality levels have been reported to be significantly higher in populations affected by SAV3 than marine SAV2 both during experimental- and field studies (15, 16), the histopathological changes appear similar regardless of underlying SAV genotype (16, 17). A proportion of surviving fish may develop severe fibrosis of the peri-acinar tissue and subsequently fail to grow (17). Clinical signs and macroscopic changes are not in themselves pathognomonic, and thus a diagnosis depends on the application of laboratory diagnostic tests (17). Subclinical infections of SAV occur (18, 19), and variation in prevalence due to SAV genotype has been reported. In Norway, infections with marine SAV2 are reported to generate a higher proportion of subclinical cases than SAV3 infections (15).

The OIE disease card for SAV lists a number of laboratory tests that have been deemed appropriate for diagnostic purposes for detection of SAV or histopathological changes consistent with SAV infection (20). These include real-time RT-PCR, conventional RT-PCR (with sequencing for genotyping), immunohistochemistry, histopathology, serum neutralization assay, and isolation of virus in cell culture. A serum-based selective precipitation reaction (SPR) has also been recently described (21), however this is currently not in use in Norway and is not listed on the OIE disease card.

There has been a lack of scientific field-validation of the diagnostic tests used in conjunction with SAV detection and histopathological changes consistent with SAV infection in Norway. In the autumn of 2017 Norway introduced a new screening protocol where all salmonid seawater grow-out sites that are stocking fish are sampled monthly for the detection of SAV by real-time RT-PCR until SAV is eventually detected or the population is slaughtered. As a result, an evaluation of the DSe and DSp of the applied diagnostic tests in general, and the real-time RT-PCR in particular, is of interest to a range of stakeholders including the government, the competent authority, and the industry. The aim of this study was therefore to evaluate the field performance of the available diagnostic tests for SAV in Norway assuming there was no gold standard.

Materials and Methods

Source Populations and Sampling Procedures

Three Atlantic salmon seawater grow-out sites were included in this study. The sites were purposely selected on the basis of the assumed infection status of their populations; one non-infected population and two SAV-infected populations at different stages of infection.

Site 1 was located in the eastern part of the northernmost county of Norway (Finnmark), a county with no SAV detections since 2013 and without any previous SAV detections in its eastern part. This site was therefore assumed to stock a non-infected population, and the site was sampled in May 2017. At the time of sampling, the site stocked ~1.5 million fish at an average weight of ~2 kg.

Site 2 was located in south-western Norway (Hordaland county), an area that has a high prevalence of SAV3-infected sites. Following routine disease investigation procedures, the site was diagnosed with SAV3 PD (SAV3 detection and histopathological changes indicative of SAV infection in the same individual) in May 2017 and was sampled for this study in the same month. The fish population had been vaccinated with a commercially available SAV vaccine (based on SAV1). The fish were transferred to sea in September 2016, and at time of sampling the site stocked ~1 million fish at an average weight of ~1.6 kg. There was no major clinical disease outbreak at the time of sampling.

Site 3 was located in mid-Norway (Møre and Romsdal county), an area that has a high prevalence of SAV2-infected sites. The site was diagnosed with SAV2 PD (SAV2 detection and histopathological changes indicative of SAV infection in the same individual) in February 2017, and sample collection was performed in June 2017. The fish population was not vaccinated against SAV. The fish were transferred to sea in September 2016, and at time of sampling, the site stocked ~630,000 fish at an average weight of ~1.5 kg. There was no major clinical disease outbreak after the diagnosis date until the time of sampling.

Sampling procedures were similar at all three sites. One cage was selected at each site; a randomly selected cage at Site 1 and a cage where PD had been diagnosed at Site 2 and Site 3. One hundred fish were convenience sampled from each cage using a seine net. All fish were sedated and subsequently humanely euthanized by a blow to the head. Based on the sampling procedures outlined by the Norwegian Veterinary Institute (22), the following set of samples were aseptically collected from each fish: tissue samples in RNAlater^TM (head kidney, heart ventricle), in 10% buffered formalin (NVI, product number 30.054.08) (gill, head kidney, heart ventricle, liver, pancreas, red and white muscle, spleen) and in viral transport medium (NVI, product number 50.008.00) (head kidney, heart ventricle), and blood samples on standard tubes for serological investigation (BD Plastipak^TM 2 ml syringes with 21G needles). Samples were stored on ice until arrival at the laboratory storage facilities.

Diagnostic Tests

Samples from each fish were tested at the Norwegian Veterinary Institute laboratory for the presence of SAV, SAV antibodies or for typical histopathological lesions consistent with SAV infection by five different diagnostic tests, namely real-time RT-PCR, isolation in cell culture, virus neutralization test, histopathology, and immunohistochemistry. See Table 1 for details on the test purpose, target tissue, and classification criteria of test-positive and test-negative samples.

TABLE 1

Table 1. Test purpose, target tissue and criteria for positive and negative results for the five evaluated diagnostic tests.

Real-Time RT-PCR (PCR)

For the detection of SAV-RNA in heart and kidney tissues, a real-time RT-PCR assay was used to test for the presence of the conserved SAV Qnsp1 gene as described by Hodneland and Endresen (23), with some modifications. Briefly, nucleic acids were extracted using the NucliSens^® easyMAG™ (bioMérieux) system according to the manufacturer's instructions. The Brilliant III Ultra-Fast QRT-PCR (Agilent Technologies) master mix was used according to the manufacturer's instructions and amplification was performed using a Stratagene Mx3005P system (Agilent Technologies) over 40 cycles. Reactions with a cycle threshold (Ct) < 40 were considered positive. The Qnsp1 assay is capable of detecting all currently known SAV genotypes and, as a result, all SAV-positive populations will have the SAV genotype determined by subsequent sequencing.

Isolation in Cell Culture (CELL)

SAV isolation from tissue samples was performed as previously described by Jansen et al. (24) and inoculated in 1:10 and 1:80 dilutions onto Chinook salmon embryo culture (CHSE-214) plates which had been grown at 20°C. After 2 weeks of incubation at 15°C, the plates were freeze-thawed and the cell lysate were inoculated on new cell cultures and incubated for a further 2 weeks. As the Norwegian field isolates of SAV2 and SAV3 rarely induce CPE in CHSE-214 cells, indirect immunofluorescence antibody test (IFAT) was used to visualize SAV-infected cells. Briefly, a 96-well CHSE plate was inoculated with cell culture supernatants and incubated at 15°C for 10 days. After fixation in 80% acetone, 50 μl of diluted SAV-specific mouse monoclonal antibody 17H23 directed against the E2 glycoprotein (25) was added per well and incubated for 1 h, followed by subsequent incubation for 1 h with diluted secondary biotinylated goat anti-mouse IgG antibody (DAKO) before the final incubation with streptavidin-fluorescein isothiocyanate (FITC) conjugate (eBioscience). Stained cell cultures were examined on an inverted fluorescence microscope. Positive samples were those with two or more fluorescent cells in at least two parallel wells and where the negative controls did not show any fluorescent cells.

Virus Neutralization Test (NT)

The presence of SAV neutralizing antibodies in fish serum samples was tested for using a virus neutralization test as described by Graham et al. (26) with modifications as described by Taksdal et al. (27). Briefly, plasma samples diluted to 1:20 and 1:80 were mixed with equal parts of a SAV1 reference strain (F93-125, Ireland) and incubated for 2 h at room temperature before inoculation onto 96-well CHSE plates. A 1:20 serum dilution without virus was used as the control. Plates were incubated for 12 days before IFAT staining, as described in section Isolation in Cell Culture (CELL). Virus neutralization at antibody titres ≥1:20 were considered positive.

Histopathological Examination (HIST)

Tissue samples were examined for histopathological changes consistent with SAV infection as described by Taksdal et al. (16). Samples were fixed with formaldehyde, and 4–6 μm hemotoxylin and eosin stained sections were examined under a light microscope. Samples were classified as either showing (1) changes consistent with SAV infection, (2) changes that might indicate SAV infection, or (3) no changes consistent with SAV infection. Samples with changes consistent with SAV infection or changes that might indicate SAV infection were considered positive.

Immunohistochemistry (IHC)

Detection of SAV in tissues by immunohistochemistry was performed as described by Taksdal et al. (27). Briefly, after blocking non-specific antibody binding sites, primary anti-SAV mAb was diluted to 1:3,000 and added to the tissue samples before overnight incubation at room temperature. Tissues were incubated with a 1:300 dilution of secondary biotinylated rabbit anti-mouse IgG and IgM before further incubation with a 1:500 dilution of streptavidin-alkaline phosphatase complex at room temperature. Tissue sections were stained with fast red salt in fast red substrate solution and examined under a light microscope for antibody-antigen binding. Samples with positive staining of necrotic exocrine pancreatic cells were considered positive.

Estimation of Test Diagnostic Sensitivity and Specificity

Bayesian Latent Class Analysis

Bayesian latent class analysis was used to estimate the test performance characteristics (DSe and DSp) of the five tests, assuming that none of these tests could be considered as a gold standard. The model was modified from the study of bacterial kidney disease test accuracy (6) and the code was provided by Jaramillo et al. (6). The study followed the guidelines for diagnostic test evaluation as described by Laurin et al. (28) and consulted the standards for the reporting of diagnostic accuracy studies that use Bayesian latent class models as described by Kostoulas et al. (29).

In this study, non-informative priors were assumed for the sensitivity and specificity of all five tests considering that the test characteristics have not previously been established and was implemented using a flat beta distribution: beta (1,1). The existing knowledge of SAV infection status was incorporated as informative priors for Site 1 and Site 2 prevalence. The fish health veterinarians responsible for health monitoring on the study sites were asked to provide the most likely value, and lower or upper limit of the site prevalence, before the samples were collected. The prior for Site 1 SAV-prevalence reflects the assumption that the site was likely to be uninfected, and the most likely number of positive fish would be 1 in 100,000 fish, with an upper limit of 10 in 100,000 fish. The prior of site 2 prevalence was chosen to reflect the knowledge that the site was infected. Site 2 was sampled in the same month as the diagnosis and could therefore either be heavily infected, if infection had been undetected for a while, or have an increasing prevalence. The most likely prevalence was assumed to be 60% with a large uncertainty ranging between 5 and 95%. Site 3 was sampled 4 months after diagnosis and could therefore either be moderately infected or have a low infection prevalence due to recovery. Due to this uncertainty about the most likely prevalence value for Site 3, a non-informative prior was assumed. The priors of Site 1, Site 2, and Site 3 prevalence were modeled using a beta distribution of beta (2, 100,000), beta (6,4), and beta (1,1), respectively (Table 2).

TABLE 2

Table 2. Prior information used in the models for site prevalence and diagnostic sensitivity and specificity of all five tests.

In this study, seven models with different number of tests and number of populations were evaluated. The main model (model 1) was a model with 3-populations and 5-tests where the samples from all sites and all tests were included. Models 2 and 3 were models with 3-populations and 4-tests, while model 4 was a model with 3-populations and 3-tests. Models 5, 6, 7 were models with 2-populations and 5-tests. Details of which populations and tests that were included in each model are presented in Table 3. For biological reasons, potential conditional dependencies between PCR and CELL, and between HIST and IHC, were investigated by estimation of covariance of DSe and DSp for each pair of the tests when possible.

TABLE 3

Table 3. List of the seven evaluated models with their prior distributions, and the changes to the model priors for sensitivity analysis.

Model convergence was assessed using graphical diagnostics by running the models with multiple chains of different initial values and checking the trace plots for each parameter. Convergence was further tested for all parameters by Gelman-Rubin statistics (R-hat) < 1.1 (30) using the CODA package in R (https://cran.r-project.org/web/packages/coda/coda.pdf) and Bayesian p-value was estimated to assess the model goodness-of-fit. The models were coded in OpenBUGS, v3.2.3 rev 1012 (31). Each model was run for 100,000 iterations of which the first 10,000 iterations were discarded as burn-in. Thinning by 10 was applied to eliminate autocorrelation. Model outputs are presented as the posterior median and 95% probability intervals (2.5th and 97.5th percentiles) and as boxplots of the posterior estimates of site prevalence, DSe and DSp.

Difference of DSe and DSp Between Tests

Differences beyond chance of the DSe and DSp between tests were assessed using a built-in step function in OpenBUGS to estimate the probability of the differences between tests for which a probability >0.05 was considered significant. The process was performed using default values of model 1, and focused on selected pairs of tests for which the difference was not determined by visualization alone.

Sensitivity Analysis

The effect of changing priors on the estimated DSe and DSp of all tests was evaluated by replacing prior of Site 2 with non-informative prior, and shifting the prior distribution of Site 1 prevalence by 30% from the median default value (Table 3).

Ethical Approval

This study was performed in accordance with the ethical guidelines of the Norwegian Veterinary Institute. All fish were sedated and humanely euthanized prior to sample collection in accordance with standard procedures.

Results

Summary of Test Results

In total, 268 of the 300 collected samples were analyzed and included in the study, while the remaining 32 samples were excluded due to a lack of relevant tissue type or inadequate quality of preservation.

An overview of the sample results for each site, and all sites combined, is presented in Table 4. Test results varied across the three sites and between the five tests. For Site 1, no samples were test-positive by any of the tests. For Site 2, 89% of samples were positive by PCR, followed by CELL (81%), histopathology (56%), NT (6%) and IHC (4%). For Site 3, 53% of samples tested positive by NT, followed by PCR (30%), HIST (5%), and CELL (1%). No samples from Site 3 tested positive by IHC.

TABLE 4

Table 4. Summary of samples tested using PCR, CELL, NT, HIST and IHC in three Norwegian fish farms, Sites 1–3, investigated for PD in Salmo salar L.

The combination of all obtained test results in the study is shown in Table 5. For samples from Site 2, which was assumed to have the highest prevalence of infected individuals in the study, 90% of samples were classified as positive by at least one test. In comparison, 55% of samples from Site 3 were found to test positive by at least one test.

TABLE 5

Table 5. Diagnostic test result combinations and associated results for PCR, CELL, NT, HIST, and IHC across three Norwegian fish farms, Sites 1–3, investigated for PD in Salmo salar L.

Bayesian Estimation of Diagnostic Sensitivity and Specificity

All models converged and showed no sign of autocorrelation with the exception of model 7. Model 7 showed bimodal posterior estimates; however, the model converged after restricting all DSp priors to be at least 0.5. Gelman-Rubin statistics of all model parameters were around 1. The posteriors estimates (median and 95% probability interval) of DSe, DSp and site prevalence based on the 3-populations models are presented in Table 6, and the results based on 2-populations models are presented in Table 7. Outputs from OpenBUGS including node statistics, density plots, plots of Gelman-Rubin diagnostic and plots of autocorrelation of all models are provided in the Supplementary Information.

TABLE 6

Table 6. Posterior estimates for 3-populations models with 5-tests (Model 1), with 4-tests (Models 2 and 3), and with 3-tests (Model 4).

TABLE 7

Table 7. Posterior estimates for 2-populations models with 5-tests: Model 5 for Site 1 and Site 2, Model 6 for Site 2 and Site 3, Model 7 for Site 1 and Site 3.

The posterior estimates of site prevalence, DSe and DSp were very similar across all 3-populations models (Table 6). The posteriors estimates showed a very low probability of infection for Site 1, a high prevalence with a narrow probability interval for Site 2, and a low prevalence with a moderate probability interval for Site 3. In model 1, PCR had the highest DSe (0.978) followed by CELL (0.950) and HIST (0.637). Two tests, NT (0.085) and IHC (0.051), were not considered functional as their DSe estimates were very low. The DSe estimates of PCR and CELL had narrow probability intervals (0.912–0.999 and 0.856–0.996, respectively) giving a high certainty in the obtained estimates. In contrast, a larger uncertainty was observed for the DSe estimate of HIST (0.517–0.747). As for DSp, IHC (0.996) and CELL (0.993) yielded high DSp estimates with narrow probability intervals (0.981–0.999 and 0.972–0.999, respectively). HIST also had a high DSp estimate (0.967) with a slightly wider probability interval (0.936–0.986). PCR had a moderate DSp estimate (0.831) and NT (0.744) the lowest DSp estimate. DSe and DSp based on the models 2, 3, and 4 yielded consistent results with those obtained in model 1. Figure 1 shows boxplots of posterior estimates of all DSe and DSp based on model 1.

FIGURE 1

Figure 1. Boxplots of the posterior estimates based on model 1. se[1] = DSe of PCR, se[2] = DSe of CELL, se[3] = DSe of NT, se[4] = DSe of HIST, se[5] = DSe of IHC, sp[1] = DSp of PCR, sp[2] = DSp of CELL, sp[3] = DSe of NT, sp[4] = DSp of HIST, sp[5] = DSp of IHC.

The covariance estimates between PCR and CELL and between HIST and IHC showed a very low test dependency between the pairs, where the highest covariance of the DSe estimate was 0.002 for the pair IHC and HIST and the highest covariance of the DSp estimate was 0.008 for the pair IHC and HIST.

For posterior estimates based on 2-populations models (Table 7), the model without site 3 (model 5) and the model without site 1 (model 6) had similar results compared to the 3-populations models (Table 6) regarding to site prevalence and DSp estimates. The model without site 2 (model 7), on the other hand, showed a considerable difference in DSe estimates. As for DSp, the model with 2-populations yielded very high DSp estimates, except for model 6 which had low estimates of DSp for PCR (0.686) and DSp for NT (0.546).

Difference in DSe and DSp Between Tests

For DSe, all pairs of the tests showed significant differences in their DSe estimates; except the pair of PCR and CELL and the pair of NT and IHC. The probability that PCR had a higher DSe estimate than CELL was 0.7913. The probability that NT had a higher DSe estimate than IHC was 0.7916. For DSp, the only non-significant difference was the pair of IHC and CELL. The probability that IHC had a higher DSp estimate than CELL was 0.649. A table containing the outputs from the Step function is available in the Supplementary Information.

Sensitivity Analysis

No site prevalence, DSe or DSp estimates were sensitive to changes in the priors of the models with 3-populations. Figure 2 shows caterpillar plots comparing the posterior estimates based on model 1, while the caterpillar plots based on models 2, 3, and 4 are available in the Supplementary Information.

FIGURE 2

Figure 2. Caterpillar plots of the posterior estimates based on model 1. pi[1] = Site 1 prevalence, pi[2] = Site 2 prevalence, pi[3] = Site 3 prevalence, se[1] = DSe of PCR, se[2] = DSe of CELL, se[3] = DSe of NT, se[4] = DSe of HIST, se[5] = DSe of IHC, sp[1] = DSp of PCR, sp[2] = DSp of CELL, sp[3] = DSe of NT, sp[4] = DSp of HIST, sp[5] = DSp of IHC.

Discussion

This study evaluated the performance of five different diagnostic tests (real-time RT-PCR, CELL, histopathology, NT, and IHC) for detection of SAV infection and histopathological changes consistent with SAV infection in farmed Atlantic salmon, as listed in the chapter on infection with SAV in the OIE Manual of Diagnostic Tests for Aquatic Animals (20). All tests are direct, independent tests for the presence of SAV, with the exception of histopathology which detects tissue lesions as a sequela of SAV infection. In the current study the tests were applied to three different salmon farming sites, of which Site 1 was known to be free from SAV infection. Sites 2 and 3 were known to be infected by two different SAV genotypes and were assumed to be at different stages of infection. For the tests used, the SAV-genotype should not directly influence the test DSe and DSp although the infections themselves may behave differently in the infected populations, which may affect the test outcome.

When the tests were applied to a population considered free from SAV infection (Site 1) all tests performed well, and no sample was suggested SAV-positive by any of the test methods. When tests were applied to the infected populations, as expected, the test results from Site 2 and Site 3 showed that the two sites differed in their infection stage allowing us to investigate the diagnostic test performance using a Bayesian method. In our main model (model 1), all samples from the three different populations were compared to each other within the five-test panel. In this model, real-time RT-PCR was found to have the highest DSe. Real-time RT-PCR is commonly used as a screening test to determine the infection status of a population at on-growing seawater sites in salmonid aquaculture, and our results from this model support this as the method of choice for this purpose. Furthermore, isolation in cell culture was also found to have high DSe and DSp. While being more time-consuming and less suitable for screening purposes, an isolation in cell culture detects the presence of viable SAV-particles and not sections of genetic material only. A positive result indicates that the population may be active shedders of viral particles and thereby constitute a risk to naïve populations in the area. Models 2, 3, 4, 5, and 6 showed a close similarity in DSe to Model 1, with real-time RT-PCR having the highest DSe followed by isolation in cell culture and histopathology, respectively. SAV prevalence and DSp estimates are similar across the 3-populations models.

The consistent results for the posterior estimates across all the 3-populations models support the idea that these 3-populations models are more robust than the 2-populations models. Removing one population reduced the data by about 33%, which should significantly affect the estimates if the removed population provided a substantial contribution to the parameter estimates. When Site 1 was removed (model 6), the model became non-identifiable and required an informative prior for DSp. Similarly, when removing site 2 the model required informative priors for DSe. Our observation is in agreement with Berkvens et al. (32) and Dendukuri et al. (33) that adding constraints or using informative priors to some model parameters could provide a solution for a non-identifiable model.

The only previously published study on the DSe and DSp of real-time RT-PCR (head kidney) and cell culture (heart ventricle and heart kidney) for SAV were performed on samples from a Scottish outbreak of PD caused by SAV 1 (34). While the DSp estimates obtained were above 99% for both tests, the estimates for DSe were 50% (95% CI: 41–62%) for cell culture and 39% (95% CI: 31–49%) for real-time RT-PCR, respectively. This DSe are lower than those observed in the current study and could be due to the previous study not including heart tissue in the sampled material, an organ which should always be included when sampling for SAV detection according to the OIE manual (20). While not yet in common use in Norway, portable field kits for screening purposes are becoming available for a rapidly increasing range of infections agents in aquatic animals. A recent publication evaluated the DSe and DSp of a commercially available, portable field kit for SAV detection, which was found to have acceptable DSe and DSp although the reproducibility was somewhat reduced compared to the accredited in-laboratory method (35).

While none of the sites had a clinical outbreak of PD detectable by observation at the time of sampling, or prior to the time of sampling for Site 3, there were some important differences between the sites. Site 2 was sampled in the same month as the PD diagnosis was obtained, however as this area did not have compulsory screening for SAV the actual month of infection remains unknown. Site 3 was sampled 4 months after the diagnosis, however as this site was located in an area where compulsory screening was in place it is likely that the site was only infected shortly before the diagnosis was confirmed. While both experimental (16) and field (15) studies have shown differences between the two Norwegian SAV genotypes in the severity of the resultant clinical outbreaks, other biological factors influencing diagnostic test functionality appears to be relatively genotype-unspecific. A Norwegian experimental study found 57% of SAV2 cohabitant fish and 84% of SAV3 cohabitant fish to test positive by real-time RT-PCR at 2 weeks post-infection, which increased to 98% (regardless of SAV genotype) by week 3 and remained close to 100% until the end of the experiment (week 12) (16). Previous studies suggest that a prolonged positive PCR signal can be found in infected populations (36) and that populations remain positive until slaughter even when infected early during the seawater phase (24, 37). However, the prevalence of positive individuals decreased in some cases over time following the outbreak (24, 37). A recent Norwegian study sampling three Atlantic salmon populations at 2 weeks, 5, and 14 months after SAV2 detection, respectively, identified some individuals that tested positive for SAV2-RNA in red muscle tissue while the heart tissue tested negative (38). While scientific findings suggest that a PCR-positive population should test SAV-positive by real-time RT-PCR analyses at a population level at all times between infection and slaughter, there may be a reduced sensitivity at the individual fish level if not screening additional tissues later in the infection, perhaps particularly for SAV2 with its milder clinical course. It may be that a slower spread of SAV2 through the population, or the lack of tested muscle tissue for Site 3, accounts for the lower prevalence estimate obtained for this population despite it having been infected for several months.

The Norwegian experimental study of SAV2 and SAV3 found the onset of histopathological lesions from week 4 post-infection, with more than half of the fish still having SAV-related tissue changes as detected by histopathological examination at 12 weeks post-infection (16). Histopathological changes appear to be similar regardless of underlying SAV genotype (16, 17) and genotype should therefore be unlikely to affect the DSe and DSp of this test. The inherent biological variation in tissue lesions and the human factor of interpreting the range of possible changes likely account for the larger probability intervals obtained for the DSe and DSp estimates for this test method.

The NT is aimed at detecting antibodies/neutralizing activity against SAV in serum. High seroprevalences of virus neutralizing antibodies have been reported from studies of natural PD outbreaks (24, 37, 39), with the majority of populations remaining seropositive until slaughter (24, 37). No differences in virus titres or antibody levels were detected when comparing SAV2 and SAV3 experimental infections (16), suggesting that SAV-genotype differences are not the underlying reason for the observed result. The NT test performed particularly well for Site 3 (SAV2) which was sampled several months after diagnosis, and the failure to detect infectious virus particles by isolation in cell culture supports the interpretation that the population at this site was in a later stage of infection. It furthermore supports the assumption that Site 2 (SAV3) was infected relatively shortly before sampling, as research indicates that blood serum or plasma is suitable for a virus neutralization test that identifies neutralizing antibodies against SAV from ~3 weeks after SAV infection (26). It also highlights NT as a potential method for non-lethal screening of SAV exposure, which may be of particular relevance in the screening of larger, high-value salmon. However, the specificity of the NT test can be an issue, as cross-reactivity between PRV-1 and SAV has been demonstrated (40) In addition, vaccination against SAV, which is common in the endemic SAV3 area, will affect the results of the NT test. Successful vaccination against SAV will produce neutralizing antibodies of varying levels dependent on the vaccine type used, the efficacy of the vaccination and the time lag between vaccination and testing. In vaccinated fish, this test therefore has to be used with caution, and a positive result can only be interpreted as an indication of infection and not used as a confirmatory diagnostic test.

To our knowledge this is the first scientific publication of a field validation of the five major diagnostic tests recommended for SAV detection in accordance with the OIE Manual of Diagnostic Tests for Aquatic Animals (9). The OIE Manual lists recommended diagnostic methods for presumptive diagnosis, confirmatory diagnosis and targeted surveillance for SAV, and categorize the different methods based on availability, utility, and diagnostic sensitivity and specificity (20). While stating that these categorisations are somewhat subjective, the OIE concludes that the diagnostic tests classified as recommended methods and standard methods are acceptable due to their routine nature and the fact that they have been widely used without dubious results (20). For presumptive diagnosis histopathology and serum neutralization assay are listed as recommended methods while immunohistochemistry, real-time RT-PCR and RT-PCR with sequencing are listed as standard methods. For confirmatory diagnosis histopathology and RT-PCR with sequencing are recommended methods and immunohistochemistry, serum neutralization assay and real-time RT-PCR standard methods. For targeted surveillance real-time RT-PCR is listed as a standard method across fish life stages (fry, juveniles, and adults) (20). Field validation of diagnostic tests assesses the entire procedure from fish selection and sample collection to laboratory procedures, with each step potentially contributing to a reduction of the DSe and/or DSp. In our study, tissue samples included heart ventricle and mid-kidney, the recommended organs for the detection of SAV (2). The assessments of the real-time RT-PCR method for SAV were in accordance with that stipulated in the OIE manual (20), where it is listed as a standard method and is the method with highest ranking as a universal diagnostic test. Assessment of the analytical sensitivity (the limit of detection) and analytical specificity (the ability to detect the target sequence only) of the QnsP1 assay has shown it to be highly sensitive and specific with an ability to detect <0.1 TCID50 of virus stocks, with reproducible results at different RNA concentration levels (23), with a more recently developed assay reported to be 100 times more sensitive (41). Due to its application as a screening tool, and the potential consequences of false positive and false negative results, the DSe and DSp of real-time RT-PCR is of particular interest. A positive result (whether true or false) may be disputed by a range of stakeholders, in particular where the SAV-detection results in implementation of mitigation measures leading to economic losses and business interruption. While it was shown to perform well in the main model and in a recently infected population it showed a lower DSe in a population at a later stage of infection. A false negative result will be of particular importance in two scenarios: where the test-negative population sheds virus, or where the population is destined for export and subjected to import tests where SAV-RNA detection results in goods rejection and possibly subsequent trade sanctions.

In the case of a weak positive result, a follow-up test should be conducted in order to verify the presence of the infectious agent. In the case of SAV, isolation in cell culture, which was found to have high DSe and DSp in more recently infected populations, may be used to verify viral presence. Similarly, follow-up tests may be appropriate in cases where a negative PCR result is obtained from a population where the available epidemiological information suggest a high probability of infection. In this case, follow-up tests of serum samples may be appropriate in order to evaluate and monitor infection status. This is of particular relevance in cases where the consequences of false negative results are high, such as in cases of potential spread to previously non-infected areas or in relation to trade. Histopathological examination is an important test for identifying lesions associated with the agent of interest but also as a test for important differential diagnoses and passive surveillance. Based on our samples, IHC demonstrated both a limited DSe and DSp, which is as expected as the test is only recommended for samples from fish with acute necrosis of exocrine pancreatic tissue in the OIE manual (9).

Our results confirms that the evaluated tests ability correctly to detect SAV or the resultant disease is dependent on the infectious status of the population in question. To achieve the most reliable results for different situations such as early detection, monitoring prevalence and documentation of disease freedom, a thorough consideration of the appropriate method(s) has to be done at the beginning of the testing program.

Data Availability Statement

The data sets supporting the conclusions of this study will be made available upon request to any qualified researcher.

Ethics Statement

Ethical review and approval was not required for the animal study because this study was performed in accordance with the ethical guidelines of the Norwegian Veterinary Institute. All fish were sedated and humanely euthanized prior to sample collection in accordance with standard procedures. Written informed consent was obtained from the owners for the participation of their animals in this study.

Author Contributions

All authors contributed to ideas for analysis, discussion of methodology and results, writing and editing of the manuscript.

Funding

This study is part of the project Biosecurity in fish farming funded by the Norwegian Ministry of Trade, Industry and Fisheries.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

The authors would like to thank the farms for their participation and facilitation during sample collection and additional laboratory staff at the Norwegian Veterinary Institute for their assistance during sample processing and testing. Special thanks to Katharine Rose Dean at the Norwegian Veterinary Institute for assistance with the modeling.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fvets.2019.00419/full#supplementary-material

References

1. Greiner M, Gardner IA. Epidemiologic issues in the validation of veterinary diagnostic tests. Prev Vet Med. (2000) 45:3–22. doi: 10.1016/S0167-5877(00)00114-8

PubMed Abstract | CrossRef Full Text | Google Scholar

2. OIE. Chapter 1.1.2. Principles and methods of validation of dianostic assays for infectious diseases. In: Manual of Diagnostic Tests for Aquatic Animals. (2019) Available online at: http://www.oie.int/index.php?id=2439&L=0&htmfile=chapitre_validation_diagnostics_assays.htm (accessed August 15, 2019).

PubMed Abstract | Google Scholar

3. Hui SL, Walter SD. Estimating the Error Rates of Diagnostic Tests. Biometrics. (1980) 36:167–71.

PubMed Abstract | Google Scholar

4. Branscum AJ, Gardner IA, Johnson WO. Estimation of diagnostic test sensitivity and specificity through Bayesian modeling. Prev Vet Med. (2005) 68:145–63. doi: 10.1016/j.prevetmed.2004.12.005

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Assis GBN, de Oliveira TF, Gardner IA, Figueiredo HCP, Leal CAG. Sensitivity and specificity of real-time PCR and bacteriological culture for francisellosis in farm-raised Nile tilapia (Oreochromis niloticus L.). J Fish Dis. (2017) 40:785–95. doi: 10.1111/jfd.12559

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Jaramillo D, Gardner IA, Stryhn H, Burnley H, Hammell KL. Bayesian latent class analysis of diagnostic sensitivity and specificity of tests for surveillance for bacterial kidney disease in Atlantic Salmon Salmo Salar. Aquaculture. (2017) 476:86–93. doi: 10.1016/j.aquaculture.2017.03.034

CrossRef Full Text | Google Scholar

7. Buss JJ, Wiltshire KH, Prowse TAA, Harris JO, Deveney MR. Bonamia in Ostrea angasi: diagnostic performance, field prevalence and intensity. J Fish Dis. (2019) 42:63–74. doi: 10.1111/jfd.12906

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Laurin E, Morrison D, Gardner IA, Siah A, Powell JFF, Kamaitis M. Bayesian latent class analysis of ELISA and RT-rPCR diagnostic accuracy for subclinical Renibacterium salmoninarum infection in Atlantic salmon (Salmo salar) broodstock. J Fish Dis. (2019) 42:303–13. doi: 10.1111/jfd.12933

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Anonymous. En Million Tonn Laks for 64.7 Milliarder i 2017. Norges Sjømatråd. (2018). Available online at https://seafood.no/Aktuelt/Nyheter/En-Million-Tonn-Laks-for-647-Milliarder-i-2017/ (accessed June 04, 2019).

Google Scholar

10. Hjeltnes B, Bang-Jensen B, Bornø G, Haukaas H, Walde CS. Fiskehelserapporten 2018. (2019). Available online at: https://www.vetinst.no/rapporter-og-publikasjoner/rapporter/2019/fiskehelserapporten-2018 (accessed Febrauary 14, 2019).

Google Scholar

11. Fringuelli E, Rowley HM, Wilson JC, Hunter R, Rodger H, Graham DA. Phylogenetic analyses and molecular epidemiology of European salmonid alphaviruses (SAV) based on partial E2 and NsP3 gene nucleotide sequences. J Fish Dis. (2008) 31:811–23. doi: 10.1111/j.1365-2761.2008.00944.x

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Hodneland K, Bratland A, Christie KE, Endresen C, Nylund A. New subtype of salmonid alphavirus (SAV), Togaviridae, from Atlantic salmon Salmo salar and rainbow trout Oncorhynchus Mykiss in Norway. Dis Aquat Organ. (2005) 66:113–20. doi: 10.3354/dao066113

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Hjortaas MJ, Skjelstad HR, Taksdal T, Olsen AB, Johansen R, Bang-Jensen B, et al. The first detections of subtype 2-related salmonid alphavirus (SAV2) in Atlantic salmon, Salmo Salar L., in Norway. J Fish Dis. (2013) 36:71–4. doi: 10.1111/j.1365-2761.2012.01445.x

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Jansen MD, Bang Jensen B, McLoughlin MF, Rodger HD, Taksdal T, Sindre H, et al. The epidemiology of pancreas disease in salmonid aquaculture: a summary of the current state of knowledge. J Fish Dis. (2017) 40:141–55. doi: 10.1111/jfd.12478

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Jansen MD, Bang Jensen B, Brun E. Clinical manifestations of pancreas disease outbreaks in Norwegian marine salmon farming - variations due to salmonid alphavirus subtype. J Fish Dis. (2015) 38:343–53. doi: 10.1111/jfd.12238

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Taksdal T, Bang Jensen B, Böckerman I, Mcloughlin MF, Hjortaas MJ, Ramstad A, et al. Mortality and weight loss of Atlantic salmon, Salmo Salar L., experimentally infected with salmonid alphavirus subtype 2 and subtype 3 isolates from Norway. J Fish Dis. (2015) 38:1047–61. doi: 10.1111/jfd.12312

CrossRef Full Text | Google Scholar

17. McLoughlin MF, Graham DA. Alphavirus infections in salmonids - a review. J Fish Dis. (2007) 30:511–31. doi: 10.1111/j.1365-2761.2007.00848.x

PubMed Abstract | CrossRef Full Text | Google Scholar

18. McLoughlin MF, Rowley HM, Doherty CE. A serological survey of salmon pancreas disease virus (SPDV) antibodies in farmed Atlantic salmon, Salmo salar L. J Fish Dis. (1998) 21:305–7. doi: 10.1046/j.1365-2761.1998.00109.x

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Graham DA, Jewhurst H, McLoughlin MF, Sourd P, Rowley HM, Taylor C, et al. Sub-clinical infection of farmed Atlantic salmon Salmo salar with salmonid alphavirus – a prospective longitudinal study. Dis Aquat Organ. (2006) 72:193–9. doi: 10.3354/dao072193

PubMed Abstract | CrossRef Full Text | Google Scholar

20. OIE. Chapter 2.3.6, Infection with salmonid alphavirus. In: Manual of Diagnostic Tests for Aquatic Animals. (2019). Available online at: http://www.oie.int/fileadmin/Home/eng/Health_standards/aahm/current/chapitre_salmonid_alphavirus.pdf (accessed August 15, 2019).

Google Scholar

21. Braceland M, Tinsley J, Cockerill D, Bickerdike R, McLoughlin MF, Eckersall PD. Selective precipitation reaction: a novel diagnostic test for tissue pathology in atlantic salmon, Salmo Salar, infected with salmonid alphavirus (SAV3). J Fish Dis. (2017) 40:1077–87. doi: 10.1111/jfd.12587

PubMed Abstract | CrossRef Full Text | Google Scholar

22. NVI (2019). Prøvetaking av Fisk. Available online at: https://www.vetinst.no/provetaking-og-diagnostikk/hvordan-ta-ut-prover/provetaking-av-fisk (accessed August 15, 2019).

Google Scholar

23. Hodneland K, Endresen C. Sensitive and specific detection of salmonid alphavirus using real-time PCR (TaqMan^®). J Virol Methods. (2006) 131:184–92. doi: 10.1016/j.jviromet.2005.08.012

CrossRef Full Text | Google Scholar

24. Jansen MD, Wasmuth MA, Olsen AB, Gjerset B, Modahl I, Breck O, et al. Pancreas disease (PD) in sea-reared Atlantic salmon, Salmo salar L., in Norway; a prospective, longitudinal study of disease development and agreement between diagnostic test results. J Fish Dis. (2010) 33:723–36. doi: 10.1111/j.1365-2761.2010.01176.x

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Merour E, Lamoureux A, Biacchesi S, Bremont M. Fine mapping of a salmonid E2 alphavirus neutralizing epitope. J Gen Virol. (2016) 97:893–900. doi: 10.1099/jgv.0.000411

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Graham DA, Rowley HM, Walker IW, Weston JH, Branson EJ, Todd D. First isolation of sleeping disease virus from rainbow trout, Oncorhynchus Mykiss (Walbaum), in the United Kingdom. J Fish Dis. (2003) 26:691–4. doi: 10.1046/j.1365-2761.2003.00505.x

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Taksdal T, Olsen AB, Bjerkas I, Hjortaas MJ, Dannevig BH, Graham DA, et al. Pancreas disease in farmed Atlantic salmon, Salmo Salar L., and rainbow trout, Oncorhynchus Mykiss (Walbaum), in Norway. J Fish Dis. (2007) 30:545–58. doi: 10.1111/j.1365-2761.2007.00845.x

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Laurin E, Thakur KK, Gardner IA, Hick P, Moody NJG, Crane MSJ, et al. Design standards for experimental and field studies to evaluate diagnostic accuracy of tests for infectious diseases in aquatic animals. J Fish Dis. (2018) 41:729–49. doi: 10.1111/jfd.12792

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Kostoulas P, Nielsen SS, Branscum AJ, Johnson WO, Dendukuri N, Dhand NK, et al. STARD-BLCM: Stardards for the reporting of diagnostic accuracy studies that use Bayesian latent class models. Prev Vet Med. (2017) 138:37–47. doi: 10.1016/j.prevetmed.2017.01.006

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Gelman A, Rubin DB. Inference from iterative simulation using multiple sequences. Stat Sci. (1992) 7:457–72. doi: 10.1214/ss/1177011136

CrossRef Full Text | Google Scholar

31. Lunn DJ, Thomas A, Best N, Spiegelhalter D. WinBUGS - a Bayesian modelling framework: concepts, structure, and extensibility. Stat Comput. (2000) 10:325–37. doi: 10.1023/A:1008929526011

CrossRef Full Text | Google Scholar

32. Berkvens D, Speybroeck N, Praet N, Adel A, Lesaffre E. Estimating disease prevalence in a Bayesian framework using probabilistic constraints. Epidemiology. (2006) 17:145–53. doi: 10.1097/01.ede.0000198422.64801.8d

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Dendukuri N, Hadgu A, Wang L. Modeling conditional dependence between diagnostic tests: a multiple latent variable model. Stat Med. (2009) 28:441–61. doi: 10.1002/sim.3470

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Hall LM, Munro LA, Wallace IS, McIntosh R, MacNeish K, Murray AG. An approach to evaluating the reliability of diagnostic tests on pooled groups of infected individuals. Prev Vet Med. (2014) 116:305–12. doi: 10.1016/j.prevetmed.2014.01.021

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Stagg HEB, Merino R, Oladimeji P, Steiropoulos N, Munro ES. Preliminary validation of a portable real-time RT-PCR assay for the detection of salmonid alphavirus. Bull Euro Assoc Fish Pathol. (2018) 38:121–8.

Google Scholar

36. Graham DA, Fringuelli E, Wilson C, Rowley HM, Brown A, Rodger H, et al. Prospective longitudinal studies of salmonid alphavirus infections on two Atlantic salmon farms in Ireland; evidence for viral persistence. J Fish Dis. (2010) 33:123–35. doi: 10.1111/j.1365-2761.2009.01096.x

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Jansen MD, Taksdal T, Wasmuth MA, Gjerset B, Brun E, Olsen AB, et al. Salmonid alphavirus (SAV) and pancreas disease (PD) in Atlantic salmon, Salmo Salar L., in freshwater and seawater sites in Norway from 2006 to 2008. J Fish Dis. (2010) 33:391–402. doi: 10.1111/j.1365-2761.2009.01131.x

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Stene A, Jansen MD, Gansel LC. Potential Strategies to Control Pancreas Disease (PD) in Salmonid Aquaculture. Aquaculture Europe 2017. (2017). Available online at: https://www.was.org/EasOnline/Mobile/Paper.aspx?i=8738 (accessed Febrauary 15, 2019).

Google Scholar

39. Graham DA, Jewhurst VA, Rowley HM, McLoughlin MF, Rodger H, Todd D. Longitudinal serological surveys of Atlantic salmon (Salmo salar L.) using a rapid immunoperoxidase-based neutralization assay for salmonid alphavirus. J Fish Dis. (2005) 28:373–9. doi: 10.1111/j.1365-2761.2005.00638.x

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Røsæg MV, Lund M, Nyman IB, Markussen T, Aspehaug V, Sindre H, et al. Immunologival interactions between Piscine orthoreovirus and Salmonid aplhavirus in Atlantic salmon. Fish Shellfish Immunol. (2017) 64:308–19. doi: 10.1016/j.fsi.2017.03.036

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Shi W, Song A, Gao S, Wang Y, Tang L, Xu Y, et al. Rapid and sensitive detection of salmonid alphavirus using TaqMan real-time PCR. Mol Cell Probes. (2017) 34:13–20. doi: 10.1016/j.mcp.2017.04.003

PubMed Abstract | CrossRef Full Text | Google Scholar