Using an experiment among clinical experts to determine the cost and clinical impact of rapid whole exome sequencing in acute pediatric settings

Objective: Evaluate the cost and clinical impacts of rapid whole-exome sequencing (rWES) for managing pediatric patients with unknown etiologies of critical illnesses through an expert elicitation experiment.

Method: Physicians in the intervention group (n = 10) could order rWES to complete three real-world case studies, while physicians in the control group (n = 8) could not. Costs and health outcomes between and within groups were compared.

Results: The cost incurred in the intervention group was consistently higher than the control by 60,000–70,000 THB. Fewer other investigation costs were incurred when rWES could provide a diagnosis. Less cost was incurred when an rWES that could lead to a change in management was ordered earlier. Diagnostic accuracy and the quality of non-pharmaceutical interventions were superior when rWES was available.

Conclusion: In acute pediatric settings, rWES offered clinical benefits at the average cost of 60,000–70,000 THB. Whether this test is cost-effective warrants further investigations. Several challenges, including cost and ethical concerns for assessing high-cost technology for rare diseases in resource-limited settings, were potentially overcome by our study design using expert elicitation methods.

Introduction

Genetic disorders in children contribute to acute hospitalizations and mortality (1–3). Early diagnosis is critical for better outcomes but difficult to achieve due to atypical presentations for thousands of potential genetic diseases (4). Next-generation sequencing (NGS) is a precision medicine that can scan a large amount of DNA rapidly for a diagnosis, which has demonstrated reduced morbidity, mortality, length of stay, and unnecessary procedures in hospitalized children (5–7). Unlike traditional genetic sequencing, NGS is more advantageous due to its shorter turnaround time, which is critical in acute settings (8). In certain instances, an earlier diagnosis also helps to make palliative care decisions (5, 6). In a Thai study, rapid whole exome sequencing (rWES), a type of NGS, led to a diagnosis in 46% of patients with unknown etiologies of critical illnesses, which resulted in a change in clinical management and improved outcomes (9). While NGS may have tremendous benefits and is becoming increasingly available in higher-income countries due to declined costs, it has not yet gained wide-scale implementation in lower-income countries (10). To our knowledge, there are only a couple of economic studies (11, 12) on NGS in Thailand and none about using rWES in pediatric patients with unknown etiologies of critical illnesses. Therefore, this study aims to generate cost data and assess clinical outcomes of pediatric patients with unknown etiologies of critical illnesses when rWES is available to Thai physicians.

Collecting clinical trial data on pediatric patients with unknown etiologies of critical illnesses for an economic study is challenging – getting an adequate sample size is near impossible for this rare condition, and withholding a potentially life-saving intervention is highly unethical. As such, the research team employed an expert elicitation design to acquire the necessary data bypassing the aforementioned challenges. With the results of this study, the research team aims to conduct a follow-up cost-effectiveness study to inform population health decisions, including reimbursing rWES in Thailand's universal health benefits package.

Methods

Study design

The experiment randomly assigned clinical experts to the intervention group or the control group to undergo expert elicitation on three real-world patient case studies. Experts assigned to the intervention group could order rWES tests for all three case studies when performing the elicitation task, while experts in the control group could not. For each case study, experts were asked to investigate and provide treatment at each of the three visits. At the third visit, experts were asked to diagnose the patient, predict the prognosis at one-year post-discharge, and write a discharge plan, including non-pharmaceutical interventions. A detailed illustration of the expert elicitation experiment, the case studies, order and treatment forms, and laboratory results are available in the Supplementary Material. The development of the study design closely followed recommendations from the Reference Case Methods for Expert Elicitation in Health Care Decision Making with appropriate adaptations (13).

Sample selection and sampling

A survey distributed to the Royal College of Physicians in Thailand was used to recruit as many experts as possible for the full range of expert opinions. Inclusion criteria were: (1) currently practicing in pediatric critical care medicine or related critical care, (2) willingness to participate in the in-person experiment in Bangkok. Exclusion criteria were: (1) physicians without formal training in pediatrics and (2) physicians who treated the patients in the three real-world case studies.

Ten and eight experts were assigned to the intervention and control groups through stratified random sampling. The stratifications were gender, years of experience (under or over ten years), location of work (intra- or extra-Bangkok), and work setting (private or public). The intervention group had two more experts than the control to generate additional variations in response in the intervention group. The sample size in this study was adequate as per the Reference Case Methods for Expert Elicitation in Health Care Decision Making, which recommended at least five experts to balance different viewpoints in conducting expert elicitations (13).

Case selection

Cases were selected from a 2021 study that reported on 54 patients admitted to ICUs or inpatient wards with seriously ill conditions without obvious causes recruited from 11 tertiary hospitals in Thailand between 2018 and 2020 (9). Adults (n = 7) were excluded from the case selection because they constituted a smaller proportion of the cases and were treated by physicians without pediatric training. Thus, focusing the study on physicians treating children was more feasible. The remaining 47 pediatric cases were assigned into three strata based on their real-world rWES results and outcomes: (1) rWES provided a diagnosis and caused a change in clinical management (n = 20), (2) rWES provided a diagnosis but did not result in a change in clinical management (n = 1), (3) rWES did not provide a diagnosis and did not change the clinical management (n = 26). One sample was randomly selected for Stratum 1 and 3.

Due to jurisdictional restrictions in accessing the clinical data of the patient in Strata 2, the research team chose a patient from Strata 1 that fits more closely with the definition of Strata 2. For this chosen case, an rWES test provided a diagnosis that could inform the chronic management but not an immediate change in management. A detailed summary of the three cases is available in the Supplementary Material.

Materials and data collection

The materials for data collection were adapted from forms used within hospitals to enhance the simulation experience and minimize procedure-based errors. The materials are available upon request.

This experiment was piloted once with experts not recruited for the study to inform the final protocol. For example, additional time was allotted for the experiment to allow for more meaningful investigations.

Two research coordinators with extensive experience managing pediatric patients in acute settings were recruited to oversee the elicitation task for each group. They were responsible for providing instructions, timing each case study, and responding to emerging problems. Pharmacists, researchers, and research assistants were recruited as facilitators to help individual experts by providing the materials required for the task and answering procedural-related questions - see the procedures illustrated in Supplementary Material. Both research coordinators and facilitators received prior training in their respective roles. Lastly, experts were assigned a code to de-identify elicitation outcomes. All experts were briefed on this study's objectives, procedures and expected benefits and provided written consent to participate before the experiment.

Outcomes

The primary outcome was the cost incurred per physician, calculated from the laboratory tests and treatments ordered. At each visit, experts selected options from a pre-determined list of investigatory laboratory tests and wrote a treatment plan. Laboratory test results, including rWES results, were provided at the next visit. On the order form, physicians were informed that the turnaround time for rWES was three days. Ordering tests not on the pre-determined list was permitted. However, the ordering physicians would be told later that the results were unavailable. These steps were repeated on the second and final visit. At the last visit, physicians were also prompted to write about discharge medications. Two weeks' worth of medication was used if the physicians did not specify the duration of the discharge medication. The cost was calculated by the quantity of material used, deduced by two clinical experts, multiplied by the unit cost retrieved from three sources: the Drug and Medical Supply Information Centre, Ministry of Public Health (drug cost), Chulalongkorn University (lab and procedure cost), and Health Intervention Technology Assessment Program (HITAP) for drug and procedure costs, adjusted to the year 2022. Furthermore, the average costs incurred by physicians that ordered rWES in visit 1 compared to visits 2 and 3 combined were also calculated to investigate the relationships between costs and the temporality of rWES orders.

Secondary outcomes were the clinical impact represented by the number of physicians in each group that made a correct diagnosis and prognosis and ordered complete and accurate non-pharmaceutical interventions. At the final visit, experts were asked to diagnose and predict the one-year post-discharge prognosis by selecting one option: (1) alive with controlled symptoms, (2) alive with uncontrolled symptoms, (3) deceased, and (4) unsure. Clinical experts, as part of the research team, determined a correct diagnosis based on specificity and accuracy, i.e., a definite diagnosis was deemed correct, but a diagnosis of the disease group may be incorrect. See Supplemental Material for examples of diagnostic correctness based on specificity. The prognosis is evaluated against the real-world case outcomes: Cases 1 and 3 were alive without complication, and Case 2 was deceased. Lastly, non-pharmaceutical interventions were measured through physicians' discharge plans written on a blank page at the last visit. Based on the response, the research team determined whether the physicians had completely and correctly planned for (1) providing family planning advice, (2) providing behavioural modification advice, and (3) issuing a medical ID letter. A medical ID letter is a piece of paper with the patient's medical information that should be carried by the patient to help the healthcare team (paramedics or the emergency department) respond to an emergency.

Data analysis

Physician gender, affiliated hospital, and region were compared using χ² tests, and years of experience were compared using the Mann–Whitney U test. The direct medical cost between groups for each case study was compared using the Independent-Samples Mann–Whitney U test. The average costs incurred by physicians that ordered rWES were compared using the Independent-Sample Mann–Whitney U test. The difference between groups for correct diagnosis, prognosis, and NPI were compared using Fisher's exact test. Statistical significance was set at p-value <0.05. All statistical analyses were conducted using SPSS Statistical Software.

Ethics review

Ethics approval was acquired through Silpakorn University's Human Research Ethics Review Board which determined compliance with ethical principles.

Results

Eighteen participants were recruited for the study. Most were female (n = 11) working in university hospitals (n = 7) in the Bangkok region (n = 12). The years of experience ranged from two to twenty years. The characteristics of physicians in the two groups were not significantly different at p-value <0.05. Detailed information on the physicians' characteristics is listed in Table 1.

TABLE 1

Table 1. Physician characteristics between the control and intervention groups undergoing the expert elicitation task (Thailand, 2022).

The direct medical cost incurred per physician between the two groups was summarized in Table 2 by the cost of investigation, cost of treatment, and total cost. For Case 1, although the cost was higher in the intervention group, it was not significantly different from the control group, tested by the Mann–Whitney U test. For Cases 2 and 3, the cost of investigation and the total cost were higher in the intervention group, with statistical significance using the Mann–Whitney test. The absolute cost difference between the control and intervention groups was summarized in Table 3.

TABLE 2

Table 2. Total cost, investigation cost, and treatment cost, in Thai baht, calculated with and without the cost of rapid exome sequencing, incurred between the control and the intervention groups (Thailand, 2022).

TABLE 3

Table 3. The absolute cost differences between groups with and without the cost of rapid whole-exome sequencing, in thousands of Thai baht (Thailand, 2022).

The comparison between the average investigation, treatment, and total cost incurred by physicians who ordered rWES in visit 1 and visits 2 and 3 combined was summarized in Table 4. For Case 1, the cost of investigation incurred by physicians that ordered rWES in visit 1 was lower than when physicians ordered rWES in visits 2 and 3 combined, and the difference was significant.

TABLE 4

Table 4. The cost comparison between physicians that ordered rapid-whole exome sequencing in visit 1 vs. visits 2 and 3 combined, in thousands of Thai Baht (Thailand, 2022).

Results for diagnostic correctness, prognosis accuracy, and NPI were summarized in Table 5. More physicians in the intervention group made a correct diagnosis for Cases 1 and 2 with statistical significance confirmed by Fisher's exact test. The prognosis accuracy was not significantly different between groups. Only physicians in the intervention group provided NPI in discharge planning for Cases 1 and 2. However, the difference is only significant in Case 1 for providing behavioral modification advice (p = 0.036) and issuing medical ID letters (p = 0.013).

TABLE 5

Table 5. Comparison of diagnostic correctness, prognostic accuracy, and non-pharmaceutical interventions represented by advice for family planning, advice for behavioural modification, and issuing medical identification letters (Thailand, 2022).

Discussion

The cost of rWES (trio-rWES) in Thailand at the time of writing was 80,000 THB (approximately 2,250 USD based on the exchange rate in 2022), comparable to the cost in high-income settings and relatively high in proportion to the costs incurred in the intervention arm: rWES constituted 38%, 51%, 50% of the total cost in cases 1, 2, and 3. The high cost of rWES may explain why the investigation and total cost in the intervention group were consistently higher than the control, as seen in Table 2. Should rWES cost reduce in the future, we believe the availability of this technology in Thailand will not become an economic burden to the healthcare system. Currently, the Thai government is investigating personalized medicines for inclusion in universal health coverage (UHC), i.e., a parallel study is undertaken to assess the cost of genetic testing for intractable epilepsy. As genetic testing, such as rWES, becomes increasingly available, the cost will reduce through two mechanisms. First, through economies of scale, where a larger test quantity will reduce the significance of its fixed costs, reducing the cost per test (14, 15). Economies of scale can also be seen when payers purchase larger quantities of the material at a discounted price (14). Second, when universally covered indications for exome sequencing and other genetic tests expand, we may observe economies of scope that reduce the cost per test by synergistically sharing input, such as facilities and human resources to process tests unrelated to idiopathic acute symptoms. Economies of scale and scope in the health service context have been documented extensively in the literature (14, 15).

When dividing the total cost into investigation and treatment costs, we also noted that physicians in the intervention group incurred fewer costs from other investigations in Cases 1 and 2, although only significant in Case 1, as seen in Table 2. Given that rWES can provide a definite diagnosis in Cases 1 and 2, physicians in the intervention group likely did not need to order as many other investigations. Not surprisingly, the cost of other investigations was not lower in the intervention group for Case 3 since rWES could not provide a definite diagnosis.

When comparing the cost incurred by physicians that ordered rWES, it was noted that those in visit 1, compared to those in visits 2 and 3 combined, incurred less investigation and treatment cost for Case 1, although only significant for the cost of investigation (Table 4). We hypothesize that knowing the rWES results earlier allowed physicians to avoid subsequent unnecessary investigation and treatment expenditures. Although rWES should not offer immediate clinical benefits for patients in Strata 2 since its results cannot cause a change in management, we hypothesize that rWES may reduce physicians' stress when treating critically ill patients as it can determine a diagnosis. A future study is underway to understand the association between physician stress and access to genetic test options.

Although the availability of the rWES test led to higher spending, it also offered clinical benefits, especially for patients that fall under the Case 1 stratum, where an accurate diagnosis could lead to a change in clinical management. This may be the most common type of patient - of the 47 idiopathic severe illness cases used in case selection, 20 (43%) fall into this stratum (9). An accurate diagnosis may have led to physicians providing helpful NPI in their discharge plans, such as family planning and behavioural modification advice and issuing medical ID letters. These NPIs offer benefits beyond the length of stay during hospitalization (16), as seen in the management of other genetic diseases (17–20). Ultimately, the clinical benefits of rWES are realized at approximately 60,000–70,000 THB. Whether this cost is cost-effective warrants a follow-up study.

Our study revealed clinical benefits such as more accurate diagnosis and NPIs in discharge planning when the rWES test was available, with lower economic consequences the earlier it was ordered. This finding can inform the clinical practice guidelines in that ordering rWES in idiopathic critical illness as a first-line investigation may be warranted for both clinical and economic benefits. Next-generation sequencing has also been recommended as a first-line investigation in patients suspected of genetic diseases to prevent a diagnostic odyssey, where the patient undergoes years of serial testing, incurring costly medical expenses in the hopes of finding a diagnosis (21–23).

Interestingly, physicians in the intervention group used rWES rather appropriately. Cases 1 and 2 were medically complicated to diagnose. As such, 80% and 90% of the physicians in Cases 1 and 2 resorted to ordering rWES. In contrast, Case 3 was less medically complicated and could be diagnosed with immunology profile tests, likely explaining that only 60% of the physicians ordered rWES. Experts in this study were physicians with real-world experience in pediatric critical care medicine. It is reasonable to assume that the cost and clinical outcomes would look very different if physicians without this expertise were recruited for this study.

Our study result is invaluable, conducted in a low-middle-income setting, to inform health policy decisions, such as whether to fund rWES through universal health coverage based on anticipated cost and clinical impact. We consider the research design an ex-ante approach, defined by evaluating the clinical and economic impact of rWES through an expert elicitation using case studies before providing universal coverage for this technology. The use of ex-ante and ex-post evaluations for public health products and policies has been documented in the literature (24–26). The expert elicitation approach is advantageous by avoiding the cost of conducting a trial-based analysis, where follow-ups to patient outcomes over a sufficient time horizon are resource intensive (10), especially in rare conditions and for low-resource settings. Therefore, we recommend that lower-middle-income countries consider using expert elicitations when clinical trials are challenging to conduct. In higher-income settings where rWES is used routinely, an ex-post approach can be used to understand the real-world economic and clinical impact. We encourage researchers to compare ex-ante and ex-post health economic study designs to evaluate the validity of expert elicitations.

The most common expert elicitation approach in economic evaluation is the unblind self-control method. In this approach, experts are provided with the outcome of a test or treatment and predict the cost and outcomes as if they did not know the test results or treatment. However, knowing the outcome in advance can influence expert judgment. Therefore, our study used the blinded self-control method instead. We believe our approach provided a more unbiased result.

There are several limitations to this study. Physicians in real-world settings may have more time, resources, investigation, and treatment options than presented in the experiment, which could affect the accuracy of cost and health outcomes. Although we assumed all physicians put in their best effort to complete the case studies, we could not control the quality of their input. Recruitment of physicians was non-random, and participation was volunteered, which could have added selection bias for physicians wishing to advance access to rWES. The generalizability of the expert opinion result has limits as most physicians in the study practiced in Bangkok, and medical training varies significantly between countries, especially in Southeast Asia. All cases were selected from a cohort of patients in Thailand, which may not represent patient profiles from other countries. In addition, we could not account for how rWES would affect the patient's length of stay in this experiment, the benefits of an earlier discharge resulting from an accurate diagnosis, downstream treatment cost because of a diagnosis, and costs unrelated to health services (10). Lastly, a scenario where a negative rWES test led to a change in management was not accounted for in the study. A negative rWES result could be beneficial in ruling out genetic diseases. Thus, the benefit of rWES could be even greater than that captured by the study.

In conclusion, the availability of rWES did not reduce direct medical costs by avoiding unnecessary investigation and treatment procedures. Our study found that the average cost incurred per physician for managing pediatric patients with unknown etiologies of critical illness was higher by 60,000–70,000 THB when rWES was available. However, when rWES was available, physicians were more likely to diagnose patients with unknown etiologies of critical illness accurately, offer behavioural modification advice, and issue medical ID letters, which could lead to tremendous and positive clinical implications. Moreover, physicians incurred less investigation cost when rWES was ordered earlier in patients that could be diagnosed through an rWES test. Whether the benefits are cost-effective warrants further study. This study adds to the existing knowledge by investigating the implication of rWES on Thai physicians' management of patients and the clinical and cost consequences, using an expert elicitation design that can be replicated in resource-limited settings to inform policymakers' decisions to fund this technology in the universal health package.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by Silpakorn University's Human Research Ethics Review Board. The patients/participants provided their written informed consent to participate in this study.

Author contributions

Conceptualization: YT; Methodology design: YT, NK, NSr, SL-A, VS; Case study design: WK, PB, LK; Workshop administration: NSa, TD, PJ, CS; Performing the experiment: NK, WK, LK, NSr, SL-A, EH-K; Data interpretation and analysis: NK, WK, LK, NSa; Drafting the manuscript: EH-K; Review and edit the manuscript: YT, NK, WK, VS All authors contributed to the article and approved the submitted version.

Funding

Health Systems Research Institute (HSRI), Thailand. (Grant no. HSRI 64-160).

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fped.2023.1204853/full#supplementary-material

References

1. Gjorgioski S, Halliday J, Riley M, Amor DJ, Delatycki MB, Bankier A. Genetics and pediatric hospital admissions, 1985–2017. Genet Med. (2020) 22(11):1777–85. doi: 10.1038/s41436-020-0871-9

PubMed Abstract | CrossRef Full Text | Google Scholar

2. O’Malley M, Hutcheon RG. Genetic disorders and congenital malformations in pediatric long-term care. J Am Med Dir Assoc. (2007) 8(5):332–4. doi: 10.1016/j.jamda.2007.02.008

CrossRef Full Text | Google Scholar

3. Yoon PW, Olney RS, Khoury MJ, Sappenfield WM, Chavez GF, Taylor D. Contribution of birth defects and genetic diseases to pediatric hospitalizations: a population-based study. Arch Pediatr Adolesc Med. (1997) 151(11):1096–103. doi: 10.1001/archpedi.1997.02170480026004

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Petrikin JE, Willig LK, Smith LD, Kingsmore SF. Rapid whole genome sequencing and precision neonatology. Semin Perinatol. (2015) 39(8):623–31. doi: 10.1053/j.semperi.2015.09.009

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Farnaes L, Hildreth A, Sweeney NM, Clark MM, Chowdhury S, Nahas S, et al. Rapid whole-genome sequencing decreases infant morbidity and cost of hospitalization. NPJ Genom Med. (2018) 3(1):10–8. doi: 10.1038/s41525-018-0049-4

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Willig LK, Petrikin JE, Smith LD, Saunders CJ, Thiffault I, Miller NA, et al. Whole-genome sequencing for identification of mendelian disorders in critically ill infants: a retrospective analysis of diagnostic and clinical findings. Lancet Respir Med. (2015) 3(5):377–87. doi: 10.1016/S2213-2600(15)00139-3

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Śmigiel R, Biela M, Szmyd K, Błoch M, Szmida E, Skiba P, et al. Rapid whole-exome sequencing as a diagnostic tool in a neonatal/pediatric intensive care unit. J Clin Med. (2020) 9(7):2220. doi: 10.3390/jcm9072220

CrossRef Full Text | Google Scholar

8. Powis Z, Farwell Hagman KD, Blanco K, Au M, Graham JM, Singh K, et al. When moments matter: finding answers with rapid exome sequencing. Mol Genet Genomic Med. (2020) 8(2):e1027. -n/a. doi: 10.1002/mgg3.1027

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Kamolvisit W, Phowthongkum P, Boonsimma P, Kuptanon C, Rojnueangnit K, Wattanasirichaigoon D, et al. Rapid exome sequencing as the first-tier investigation for diagnosis of acutely and severely ill children and adults in Thailand. Clin Genet. (2021) 100(1):100–5. doi: 10.1111/cge.13963

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Payne K, Gavan SP, Wright SJ, Thompson AJ. Cost-effectiveness analyses of genetic and genomic diagnostic tests. Nat Rev Genet. (2018) 19(4):235–46. doi: 10.1038/nrg.2017.108

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Jittikoon J, Sangroongruangsri S, Thavorncharoensap M, Chitpim N, Chaikledkaew U. Economic impact of medical genetic testing on clinical applications in Thailand. PLoS One. (2020) 15(12):e0243934. doi: 10.1371/journal.pone.0243934

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Pipitprapat W, Pattanaprateep O, Iemwimangsa N, Sensorn I, Panthan B, Jiaranai P, et al. Cost-minimization analysis of sequential genetic testing versus targeted next-generation sequencing gene panels in patients with pheochromocytoma and paraganglioma. Ann Med. (2021) 53(1):1244–56. doi: 10.1080/07853890.2021.1956687

CrossRef Full Text | Google Scholar

13. Bojke L, Soares MO, Claxton K, Colson A, Fox A, Jackson C, et al. Reference case methods for expert elicitation in health care decision making. Med Decis Making. (2022) 42(2):182–93. doi: 10.1177/0272989X211028236

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Turner HC, Toor J, Hollingsworth TD, Anderson RM. Economic evaluations of mass drug administration: the importance of economies of scale and scope. Clin Infect Dis. (2018) 66(8):1298–303. doi: 10.1093/cid/cix1001

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Freeman M, Savva N, Scholtes S. Economies of scale and scope in hospitals: an empirical study of volume spillovers. Manage Sci. (2021) 67(2):673–97. doi: 10.1287/mnsc.2019.3572

CrossRef Full Text | Google Scholar

16. ACMG Board of Directors. Clinical utility of genetic and genomic services: a position statement of the American college of medical genetics and genomics. Genet Med. (2015) 17(6):505–7. doi: 10.1038/gim.2015.41

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Meissen GJ, Mastromauro CA, Kiely DK, McNamara DS, Myers RH. Understanding the decision to take the predictive test for Huntington disease. Am J Med Genet. (1991) 39(4):404–10. doi: 10.1002/ajmg.1320390408

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Levy HL. Newborn screening: the genomic challenge. Mol Genet Genomic Med. (2014) 2(2):81–4. doi: 10.1002/mgg3.74

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Burke W, Tarini B, Press NA, Evans JP. Genetic screening. Epidemiol Rev. (2011) 33(1):148–64. doi: 10.1093/epirev/mxr008

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Khoury MJ. Genetics and genomics in practice: the continuum from genetic disease to genetic information in health and disease. Genet Med. (2003) 5(4):261–8. doi: 10.1097/01.GIM.0000076977.90682.A5

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Scocchia A, Wigby KM, Masser-Frye D, Del Campo M, Galarreta CI, Thorpe E, et al. Clinical whole genome sequencing as a first-tier test at a resource-limited dysmorphology clinic in Mexico. NPJ Genom Med. (2019) 4(1):5. doi: 10.1038/s41525-018-0076-1

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Lindstrand A, Ek M, Kvarnung M, Anderlid B, Björck E, Carlsten J, et al. Genome sequencing is a sensitive first-line test to diagnose individuals with intellectual disability. Genet Med. (2022) 24(11):2296–307. doi: 10.1016/j.gim.2022.07.022

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Møller RS, Hammer TB, Rubboli G, Lemke JR, Johannesen KM. From next-generation sequencing to targeted treatment of non-acquired epilepsies. Expert Rev Mol Diagn. (2019) 19(3):217–28. doi: 10.1080/14737159.2019.1573144

CrossRef Full Text | Google Scholar

24. Persson U, Willis M, Ödegaard K. A case study of ex ante, value-based price and reimbursement decision-making: TLV and rimonabant in Sweden. Eur J Health Econ. (2010) 11(2):195–203. doi: 10.1007/s10198-009-0166-1

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Uzochukwu BS, Onwujekwe OE, Uguru NP, Ughasoro MD, Ezeoke OP. Willingness to pay for rapid diagnostic tests for the diagnosis and treatment of malaria in southeast Nigeria: ex post and ex ante. Int J Equity Health. (2010) 9(1):1. doi: 10.1186/1475-9276-9-1

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Grosse SD, Waitzman NJ, Romano PS, Mulinare J. Reevaluating the benefits of folic acid fortification in the United States: economic analysis, regulation, and public health. Am J Public Health. (2005) 95(11):1917–22.16195513

PubMed Abstract | Google Scholar