Access to care through telehealth among U.S. Medicare beneficiaries in the wake of the COVID-19 pandemic

Background: The coronavirus disease 2019 (COVID-19) public health emergency has amplified the potential value of deploying telehealth solutions. Less is known about how trends in access to care through telehealth changed over time.

Objectives: To investigate trends in forgone care and telehealth coverage among Medicare beneficiaries during the COVID-19 pandemic.

Methods: A cross-sectional study design was used to analyze the outcomes of 31,907 Medicare beneficiaries using data from three waves of survey data from the Medicare Current Beneficiary Survey COVID-19 Supplement (Summer 2020, Fall 2020, and Winter 2021). We identified informative variables through a multivariate classification analysis utilizing Random Forest machine learning techniques.

Findings: The rate of reported forgone medical care because of COVID-19 decreased largely (22.89–3.31%) with a small increase in telehealth coverage (56.24–61.84%) from the week of June 7, 2020, to the week of April 4 to 25, 2021. Overall, there were 21.97% of respondents did not know whether their primary care providers offered telehealth services; the rates of forgone care and telehealth coverage were 11.68 and 59.52% (11.73 and 81.18% from yes and no responses). Our machine learning model predicted the outcomes accurately utilizing 43 variables. Informative factors included Medicare beneficiaries' age, Medicare-Medicaid dual eligibility, ability to access basic needs, certain mental and physical health conditions, and interview date.

Conclusions: This cross-sectional survey study found proliferation and utilization of telehealth services in certain subgroups during the COVID-19 pandemic, providing important access to care. There is a need to confront traditional barriers to the proliferation of telehealth. Policymakers must continue to identify effective means of maintaining continuity of care and growth of telehealth services.

1. Introduction

Access to medical care is an ongoing problem for vulnerable populations (1, 2), and the COVID-19 pandemic has had a devastating impact on disparities in forgone care (3–5). There were about 40% of U.S. adults reporting forgone medical care during the COVID-19 pandemic, citing fear of infection among the reasons (6). Another reason is that the physician's office may present logistical barriers, such as inconvenient clinic hours and lack of care coordinators (7). In the meanwhile, telehealth virtual visits offer a way to reduce exposure to COVID-19 infections. When Medicare beneficiaries tend to reduce traditional in-person medical visits, telehealth has been widely utilized because of its usability and safety in providing healthcare services (8, 9).

Several studies that contributed to the use of telehealth were published during the initial stage of the COVID-19 pandemic when health systems lacked medical supplies and staff (9–13). However, no telehealth program can be created overnight. There has been limited investigation since telemedical innovations and vaccine administration were implemented (14–20). After health systems regained the capacity to treat patients in person and the diverse contributions of telehealth were made, there is limited understanding of the experiences of Medicare beneficiaries, who are a higher risk population for COVID-19 mortality since most of them are 65 years or older (4).

In this study, we examined trends in patient-reported access to care and telehealth utilization among Medicare beneficiaries in three waves of data collection during the COVID-19 pandemic (Summer 2020, Fall 2020, and Winter 2021). We expected that these two outcomes were correlated, and therefore we conducted a multivariate classification analysis. Reasons for disparities included socio-demographic factors, personal experiences with COVID-19, electronic device usage, economic and mental effects of the pandemic, non-COVID-19 health status and interview date. Since there are many correlated predictors with missing values, multivariate classification analysis utilized Random Forest machine learning techniques (21–23).

2. Methods

2.1. Data

Data sets were downloaded from the Medicare Current Beneficiary Survey (MCBS) COVID-19 Supplement Public Use File, collected via a telephone survey in Summer 2020 (June to July), Fall 2020 (October to November), and Winter 2021 (February to April). These three waves of survey data contained a nationally representative sample of all Medicare beneficiaries, and the survey was conducted in either English or Spanish. The MCBS is sponsored by the Centers for Medicare & Medicaid Services (CMS) in the U.S., and the original MCBS primarily has focused on economic and beneficiary topics, including health care use, access barriers, and expenditures. With the emergence of COVID-19, CMS was uniquely positioned to collect vital information on how the pandemic is impacting the Medicare population by using the MCBS as a vehicle to collect data. Ethics approval and consent to participate in the entire project were obtained by CMS and NORC at the University of Chicago; both organizations uphold provisions established under the Privacy Act of 1974, the NORC Institutional Review Board (IRB), the Office of Management and Budget, and the Federal Information Security Management Act of 2002.

In total, 235 variables were included in all three waves of surveys (see Supplementary Table 1). Among the three variables describing interview characteristics, interview week was included as a predictor since it is more relevant than the other two variables, interview language and interview with proxy. We utilized all the variables recording beneficiaries' demographic information as predictors. Among the 121 variables describing access to care during the pandemic, two variables were chosen as primary outcomes: forgone medical care because of COVID-19 and the status of whether primary care physicians (PCP) offered telehealth appointments by the date of interview. The description of these two outcome variables in the MCBS survey is listed in Supplementary Table 2. We chose five predictors from these 121 variables since they are relevant to telehealth, including owning a computer, owning a smartphone, owning a tablet, access to the Internet, and using video/voice calls; other variables in this group are too sparse to be added in the classification model since most of them are follow-up questions if beneficiaries forwent medical care, such as unable to get care for vision, dental, hearing, etc. Most of the beneficiaries did not have experience with COVID-19. From the 42 variables describing beneficiaries' personal experiences with COVID-19 and 27 variables describing preventive measures and knowledge about COVID-19, we chose two variables, including the results for COVID-19 and COVID-19 antibody tests—they were included in the descriptive summary but not included in the classification model because of missing data since most respondents did not conduct the tests. We included all the variables describing the economic and mental effects of the pandemicӑand beneficiaries' non-COVID-19 health status.

2.2. Statistical analysis

To investigate patterns of access to care and telehealth offerings among Medicare beneficiaries during COVID-19, we first run descriptive analyses for 46 independent variables, including socio-demographic factors, personal experiences with COVID-19, electronic device usage, etc. Next, we conducted a multivariate classification analysis to assess whether these variables were significant predictors of access to care and telehealth utilization. In this step, a variable recording interview date was added, while four variables were excluded because they are only available in the survey conducted in Winter 2021, including two socio-demographic variables (Medicare Advantage and Part D plan) and two variables describing personal experiences with COVID-19 (COVID-19 test and COVID-19 antibody test results).

For all analyses, a complex sample design was used with sampling weights provided by the MCBS to produce nationally representative estimates. All percentages and proportions that appeared in this study were calculated using survey weights. In the descriptive analyses, weighted chi-squared tests were used to test the association between each predictor and the outcome. In the multivariate classification analysis, Random Forest (21) model was applied for the prediction of outcomes, a modern machine learning technique that has been utilized widely to explore a large number of predictors and identify replicable sets of risk factors (24–28). Weighted chi-squared tests and the Random Forest model were implemented in the open-source R software using the weights (29) and randomForestSRC (23, 30) packages correspondingly. From the randomForestSRC package, the functions rfsrc and tune were applied with 1,000 trees. The parameters case.wt and na.action were set for survey weighting and missing data imputation for independent variables. All statistical inferences were based on a significance level of P (two-sided) ≤ 0.05. Model performance was evaluated through out-of-bag misclassification error, where out-of-bag refers to the data proportion that is not used for fitting the model (Classification trees were “grown” from bootstrap samples of the original dataset, leaving an average of 37% of unsampled data referred to as out-of-bag data) but for calculating the cross-validated prediction error and VIMP.

2.2.1. Variable importance (VIMP) and partial plot

From the Random Forest classification model, the estimated VIMP (21, 22) was adopted for ranking variables, which utilizes a prediction-based approach by estimating classification error attributable to the predictor. The VIMP can be interpreted as the increase in the misclassification error when the corresponding predictor is randomly permutated into a noise variable. For example, a VIMP of 4.29% indicates that a variable improves by 4.29% the ability of the model to classify the status of the outcome. Standard errors and P-values were generated by a delete-d-jackknife procedure (22, 31). In addition, partial dependence plots were used to visualize the variables' impact on the outcome through mapping their marginal effects (32, 33), where predicted probability is adjusted by integrating out all variables other than the selected variable. Inferences of VIMP and partial plots were generated using the functions subsample and plot.variable (setting partial = TRUE) from the randomForestSRC R package with default settings.

3. Results

3.1. Description of the sample

The main characteristics of the sample from the three waves of survey data are displayed in Table 1. There are 46 variables in the rows, including ten socio-demographic variables, two variables describing personal experiences with COVID-19, five variables describing electronic device usage, seven variables describing economic and mental effects of the pandemic, and 22 variables recording non-COVID-19 health status. Among these groups of variables, the two variables describing personal experiences with COVID-19, which recorded COVID-19 test and COVID-19 antibody test results, are not significantly associated with any of the two outcomes; all the five variables describing electronic device usage are significantly associated with both outcomes. The variable recording interview date is displayed in Figure 1 plotted against the percentage of forgone care and type of telehealth provided by PCP. From Summer 2020 to Winter 2021, the proportion of forgone care decreased largely from 22.89 to 3.31%. However, the increase in telehealth coverage is not as large (56.24 to 61.84%), as shown in Figure 1. The type of telehealth offered was summarized as “telephone,” “video,” and “both,” whose survey-weighted percentages in June 2020 were 30.46, 8.30, and 61.24%, respectively, and in April 2021 were 22.30, 5.12, and 72.58%, respectively. There was an increase in the usage of both video and telephone for telehealth.

TABLE 1

Table 1. Descriptive analysis of forgone care and telehealth coverage reported by Medicare beneficiaries.

FIGURE 1

Figure 1. Trends of forgone care and telehealth utilization. (A) The percentage of forgone care decreased largely with a small increase in telehealth coverage. (B) The usage of both video and telephone for telehealth increased over time.

In total, there are 31,907 Medicare beneficiaries included in the final sample, among which 11,114 are from Summer 2020, 9,686 from Fall 2020 and 11,107 from Winter 2021. For the two outcomes, 135 and seven beneficiaries reported “don't know” and “refused,” respectively, for answering whether they were unable to get care because of COVID-19; 7,174 and three beneficiaries reported “don't know” and “refused,” respectively, with 1,486 inapplicable/missing data for answering whether PCP offered telehealth appointments. These categories were discarded in the descriptive analysis for both outcomes and independent variables shown in Table 1. There were 21.97% of respondents unknown whether their PCP offered telehealth services; the rates of forgone care and telehealth coverage were 11.68 and 59.52% (11.73 and 81.18% from yes and no responses). Forgone care was negatively correlated telehealth coverage (χ² = 18.40, p < 0.001).

Among the 10 socio-demographic variables, six were significantly associated with both outcomes including age, gender, race/ethnicity, region, income and use of a language other than English at home (shown as non-English in Table 1). Beneficiaries who were metro residents and not eligible for Medicaid benefits (non-dual-eligible) significantly tended to have telehealth coverage. Among the seven variables describing economic and mental effects of the pandemic, four were significantly associated with both outcomes, indicating that beneficiaries who were able to get food and felt less stressed, lonely, or sad and more socially connected were more likely to have access to care; beneficiaries with access to care were those who were able to or need not pay rent/mortgage as well as get home supplies and those who did not feel less financially secure. Most of the 22 variables recording non-COVID-19 health status were significantly associated with either of the outcomes.

3.2. Relationship of outcomes to important variables

We selected variables for predicting both outcomes by machine learning using the Random Forest multivariate classification model shown in Table 2. Only yes and no responses of the outcomes were included in this classification model (n = 22, 138, p = 43). The Random Forest model predicts the outcomes accurately: the out-of-bag misclassification error is 11.63% for predicting forgone care and 21.18% for telehealth coverage. The full version of Table 2 can be found in Supplementary Table 3. For method comparison, we also analyzed the data with logistic regression. The model of regular logistic regression did not converge because of the missing data problem. Utilizing penalized logistic regression (34, 35) with 10-fold cross-validation provides misclassification errors of 11.94% for predicting forgone care and 21.91% for telehealth coverage; the coefficients were listed in Supplementary Table 4.

TABLE 2

Table 2. Informative variables from multivariate classification analysis using Random Forest.

Table 2 presents the estimate, standard error (SE), and P-value of Random Forest VIMP. A large estimate of VIMP indicates a variable that is more informative for predicting the corresponding outcome, while a negative estimate indicates a noise variable. For example, a VIMP value of 4.81% for forgone care indicates that the variable improves by 4.81% the ability of the model to classify the status of forgone care. However, VIMP can not provide the direction of the association, for which we used the odds ratio (OR). To interpret the OR conveniently, we can consider Table 1 as a list of stacked contingency tables of variables, and we used the first two rows of each contingency table for each variable for calculating an OR with survey weights. Odds ratios >1 indicate a positive association between the first category of the variable of interest and the corresponding outcome compared with its second category; odds ratios <1 represent a negative association. For binary variables with yes or no response, odds ratios >1 indicate a positive association since the first category is always for the yes response.

We detected 20 variables that were significantly associated with both forgone care and telehealth coverage, as shown in Table 2. Two variables are not significantly associated with any of the two outcomes, statuses of owning a tablet and having any heart condition (see Supplementary Table 3). However, variables describing specific heart conditions are significantly associated with the outcomes, which possibly masks the effect of having any heart condition as the overall status. The effects of age, census region, and race/ethnicity are shown in Figure 2 and Supplementary Figure 1. As demonstrated in Figure 2D, the probability of forgone care decreased largely across time after adjusting for other variables, indicating a strengthened health system.

FIGURE 2

Figure 2. Random forest estimated probabilities of outcomes plotted against candidate variables after adjusting for other variables. (A) The association between age and forgone care. (B) The association between age and telehealth coverage. (C) The association between region and forgone care. (D) The association between interview date and forgone care.

3.2.1. Forgone care

The most informative variable is interview date for predicting if the beneficiary was unable to get care because of COVID-19, contributing 2.09% prediction accuracy (SE = 0.27, p < 0.001). The status of being able to get home supplies is the second most informative variable, contributing 1.20% prediction accuracy (SE = 0.33, p < 0.001). Medicare-Medicaid dual eligibility and age are also significantly associated with the outcome, indicating that non-dual-eligible beneficiaries (not eligible for Medicaid benefits) with younger age were more likely to forgo care. People who reported using video/voice calls were more likely to forgo care. The groups that reported being unable to pay rent/mortgage or get food or home supplies were more likely to be unable to get care.

In terms of mental effects of the pandemic and non-COVID-19 health status or habit, beneficiaries with forgone care tended to feel less financially secure and more lonely or sad and have depression. Forgone care was associated with e-cigarette usage and health conditions such as angina pectoris/coronary heart disease (CHD), congestive heart failure, other heart cond such as abnormal valve/rhythm, stroke/brain hemorrhage, cancer (non-skin), osteoporosis/soft bones, broken hip, emphysema/asthma/chronic obstructive pulmonary disease (COPD), weak immune system, and diabetes/high blood sugar. Alzheimers/dementia is negatively associated with self-reported forgone care.

3.2.2. Telehealth coverage

Among the 43 variables, 39 were significantly associated with coverage of telehealth. The three most informative factors are Medicare-Medicaid dual eligibility (VIMP = 4.81, SE = 0.59 p < 0.001, OR = 0.58), residing area (metro residence, VIMP = 4.57, SE = 0.33, p < 0.001, OR = 1.87), and race/ethnicity (VIMP = 4.13, SE = 0.39 p < 0.001, OR = 2.00), indicating that non-dual-eligible beneficiaries (not eligible for Medicaid benefits), non-hispanic white and metro residents were more likely to have telehealth coverage provided by PCP.

The age group of 65–74 years old, the female gender group and the midwest/west-region group had higher coverage of telehealth. Beneficiaries using English language at home and those with higher income also had higher coverage. Owning a computer or smartphone with access to the Internet and usage of video/voice calls was positively associated with telehealth coverage. Respondents with telehealth coverage reported being able to pay rent/mortgage and get food, feeling more financially secure but more stressed, more lonely or sad, less socially connected, and having depression. Being able to get home supplies is negatively associated with telehealth coverage. Most variables describing non-COVID-19 health conditions and smoking status are negatively associated with the outcome, except cancer (non-skin), osteoporosis/soft bones/broken hip, and emphysema/asthma/COPD.

3.2.3. Variable interactions

We detected three pairs of variables that intensified the disparity in both outcomes when different statuses were combined. Figures 3A,B demonstrate the interaction between statuses of Internet access and whether respondents felt financially secure during the pandemic. The group with Internet access and felt less financially secure had higher probabilities of forgone care (20.47%) than the group without Internet access and felt more financially secure (7.92%). The group with Internet access and felt more financially secure had higher probabilities of telehealth coverage (86.68%) than the group without Internet access and felt less financially secure (67.32%). Medicare-Medicaid dual eligibility interacted with the variable income, as shown in Supplementary Figures 2A,B. The higher-income group with full eligibility had higher probabilities of forgone care and telehealth coverage (17.88 and 84.98%) than the lower-income group not eligible for Medicaid (non-dual, 9.6 and 74.28%). The female group with the status of metro residence had higher probabilities of telehealth coverage (83.62%) than the male group with the status of non-metro residence (71.29%), as shown in Supplementary Figure 2D. However, such a difference is small for forgone care (12.26 vs. 10.08%), as shown in Supplementary Figure 2C, indicating that forgone care is not caused by telehealth coverage for this subgroup.

FIGURE 3

Figure 3. Interaction of access to the internet and the status of whether beneficiaries felt financially secure during the pandemic. The survey weighted proportions of positive outcomes are listed in parentheses. (A) The interaction for predicting forgone care. (B) The interaction for predicting telehealth coverage.

4. Discussion

Utilizing three waves of nationally representative survey data for Medicare beneficiaries, we analyzed trends in and reasons for forgone care and telehealth coverage during the COVID-19 pandemic. Although the percentage of forgone care decreased largely during the COVID-19 pandemic, telehealth coverage increased only on a small scale. Although CMS temporarily provided reimbursement for telehealth regardless of patient location (36), patient-based barriers to access to telehealth may still exist.

We found that the disparity in access to care through telehealth was associated with age, Medicare-Medicaid dual eligibility, electronic device usage, ability to access basic needs, and certain mental and physical health conditions. Among these factors, some socio-demographic factors had a similar influence in some prior studies reporting pre-COVID disparities in access to care (8, 37, 38); other factors may reflect an increased risk of spillover effects of the COVID-19 on non-COVID patients, including provision of essential chronic care (39) and changes in mental health (40). People with underlying chronic conditions are more susceptible to the infection due to weakened immunity, and therefore more likely to forgo needed treatments (41).

After adjusting for other factors, residence (metro or non-metro), region (northeast, midwest, south, or west) and income were not significantly associated with forgone care because of COVID-19, but they were significantly associated with telehealth coverage. As (42) stated, stated, health care partners should be informed about the collaborative use of telehealth-centered strategies to improve facility outcomes during the COVID-19 outbreaks. The disparity in telehealth coverage will eventually be reflected in poor access to care unless rapidly technological solutions are deployed and components of equity are examined. Further challenges replicating in-person care using telehealth formats include comorbidities. Take heart conditions, for example. Our data showed that Medicare beneficiaries with heart conditions were more likely to forgo care and less likely to have telehealth coverage. Radhakrishnan et al. (43) reported that for patients with heart failure on telehealth, comorbidity characteristics of renal failure, cancer, and depression comorbidities were significantly associated with withdrawal from telehealth services. However, a more recent study showed that for older persons living with HIV, the number of comorbidities was positively related to telehealth use via telehealth apps such as the MyChart App (44). Because of risk factors for severe COVID-19, the role of telehealth use will become more and more critical for the early identification of patients who need their care, care coordination, and the assessment of daily facility needs.

Medicare beneficiaries with depression had higher coverage of telehealth, indicating the absence of inequities between mental health coverage and coverage for other medical conditions. However, depression is positively associated with forgone care. This may reflect the fact that the share of adults with common mental disorders (primarily anxiety and depression), post-traumatic stress disorder, substance use disorders, behavioral disorders and suicidal behavior increased during the pandemic (45–47). Social isolation and lack of access to medical or behavioral health care may be associated with negative mental health outcomes (48). Therefore, it is important to consider how these factors are associated and explore ways to foster health system resilience to support vulnerable patients.

In contrast to previous studies (3, 18, 49), we analyzed related factors in a more inclusive fashion for ranking variables and identifying complex interactions. After adjustment for different factors, the discoveries could be more reproducible. For survey data with a large set of correlated variables, flexible statistical assumptions of the prediction model are often required. We are able to show that Random Forest provides a set of useful prediction tools when applied to a standard national survey data set. For this dataset, classical logistic regression and lasso penalized logistic regression suffered from multicollinearity and missing data problems, while Random Forest could provide prediction accuracy as high as almost 90%. Further, Random Forest provides an interpretable nonparametric variable important index that is useful for variable ranking (22, 50, 51). Although regular logistic regression suffers the problem of missing data, penalized logistic regression provides similar prediction performance. Overall, we saw potentially significant returns to statistical and machine learning methods.

4.1. Limitations

Because of the nature of survey data, this study is subject to recall and social desirability biases. Its results are not generalizable to non-medicare beneficiaries. In addition, we do not yet have data recording beneficiaries' education level or reasons for accessing telehealth. The variables we used were defined in wide categories with few details. For example, age was coded on three levels, income was recorded on only two levels, and measures of mental wellbeing were not sufficiently defined for different aspects. Finally, our findings should be interpreted cautiously because they were based on analyses addressing prediction or association, not causality.

5. Conclusions

In conclusion, existing barriers to telehealth may influence patients' forgone care during the COVID-19 pandemic. There is a need to develop telehealth services, enhance patients' awareness of telehealth, and ensure equal access and utilization of telehealth. Identifying the associations among forgone care, telehealth coverage and patients' socio-demographic and clinical characteristics is essential for policymakers, patients and clinics in making informed health care decisions.

Data availability statement

The data are publicly available on GitHub: https://github.com/luminwin/MCBS_2020_2021.

Author contributions

ML: conceptualization, methodology, software, formal analysis, investigation, writing—original draft preparation, writing—review and editing, supervision, and funding acquisition. XL: project administration and writing—review and editing. All authors have read and agreed to the published version of the manuscript.

Funding

This study was funded by the Department of Public Health Sciences 2022 Copeland Foundation Award and the Relief Funding Award from the Office of the Vice Provost for Research and Scholarship and the Office of Faculty Affairs, University of Miami.

Acknowledgments

Special acknowledgment goes to the editor for superb feedback on this manuscript and to the reviewers for helpful discussions.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpubh.2022.946944/full#supplementary-material

References

1. Hanna TP, King WD, Thibodeau S, Jalink M, Paulin GA, Harvey-Jones E, et al. Mortality due to cancer treatment delay: systematic review and meta-analysis. BMJ. (2020) 371:m4087. doi: 10.1136/bmj.m4087

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Clark CR, Ommerborn MJ, Coull B, Pham DQ, Haas JS. Income inequities and Medicaid expansion are related to racial and ethnic disparities in delayed or forgone care due to cost. Med Care. (2016) 54:555. doi: 10.1097/MLR.0000000000000525

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Park S, Stimpson JP. Trends in self-reported forgone medical care among medicare beneficiaries during the COVID-19 pandemic. JAMA Health Forum. (2021) 2:e214299. doi: 10.1001/jamahealthforum.2021.4299

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Park C, Ng BP, Kim K. Inability to access health care due to COVID-19 among medicare beneficiaries. Am J Manag Care. (2022) 28:75–80. doi: 10.37765/ajmc.2022.88823

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Busch AB, Huskamp HA, Raja P, Rose S, Mehrotra A. Disruptions in care for medicare beneficiaries with severe mental illness during the COVID-19 pandemic. JAMA Netw Open. (2022) 5:e2145677. doi: 10.1001/jamanetworkopen.2021.45677

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Whaley CM, Pera MF, Cantor J, Chang J, Velasco J, Hagg HK, et al. Changes in health services use among commercially insured US populations during the COVID-19 pandemic. JAMA Netw Open. (2020) 3:e2024984. doi: 10.1001/jamanetworkopen.2020.24984

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Reece JC, Neal EF, Nguyen P, McIntosh JG, Emery JD. Delayed or failure to follow-up abnormal breast cancer screening mammograms in primary care: a systematic review. BMC Cancer. (2021) 21:1–14. doi: 10.1186/s12885-021-08100-3

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Cantor JH, McBain RK, Pera MF, Bravata DM, Whaley CM. Who is (and is not) receiving telemedicine care during the COVID-19 pandemic. Am J Prevent Med. (2021) 61:434–8. doi: 10.1016/j.amepre.2021.01.030

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Garfan S, Alamoodi AH, Zaidan BB, Al-Zobbi M, Hamid RA, Alwan JK, et al. Telehealth utilization during the COVID-19 pandemic: a systematic review. Comput Biol Med. (2021) 138:104878. doi: 10.1016/j.compbiomed.2021.104878

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Hoffman DA. Increasing access to care: telehealth during COVID-19. J Law Biosci. (2020) 7:Lsaa043. doi: 10.1093/jlb/lsaa043

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Martinez-Martin N, Dasgupta I, Carter A, Chandler JA, Kellmeyer P, Kreitmair K, et al. Ethics of digital mental health during COVID-19: crisis and opportunities. JMIR Ment Health. (2020) 7:e23776. doi: 10.2196/23776

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Koonin LM, Hoots B, Tsang CA, Leroy Z, Farris K, Jolly B, et al. Trends in the use of telehealth during the emergence of the COVID-19 pandemic-United States, January-March 2020. Morbid Mortal Wkly Rep. (2020) 69:1595. doi: 10.15585/mmwr.mm6943a3

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Bose S, Dun C, Zhang GQ, Walsh C, Makary MA, Hicks CW. Medicare beneficiaries in disadvantaged neighborhoods increased telemedicine use during the COVID-19 pandemic. Health Affairs. (2022) 41:635–42. doi: 10.1377/hlthaff.2021.01706

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Giacalone A, Marin L, Febbi M, Franchi T, Tovani-Palone MR. eHealth, telehealth, and telemedicine in the management of the COVID-19 pandemic and beyond: lessons learned and future perspectives. World J Clin Cases. (2022) 10:2363. doi: 10.12998/wjcc.v10.i8.2363

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Haroz EE, Kemp CG, O'Keefe VM, Pocock K, Wilson DR, Christensen L, et al. Nurturing innovation at the roots: the success of COVID-19 vaccination in American Indian and Alaska native communities. Am J Publ Health. (2022) 112:383–7. doi: 10.2105/AJPH.2021.306635

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Harris R, Rosecrans A, Zoltick M, Willman C, Saxton R, Cotterell M, et al. Utilizing telemedicine during COVID-19 pandemic for a low-threshold, street-based buprenorphine program. Drug Alcohol Depend. (2022) 230:109187. doi: 10.1016/j.drugalcdep.2021.109187

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Friedman AB, Gervasi S, Song H, Bond AM, Chen AT, Bergman A, et al. Telemedicine catches on: changes in the utilization of telemedicine services during the COVID-19 pandemic. Am J Manag Care. (2022) 28:e1–6. doi: 10.37765/ajmc.2022.88771

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Ng BP, Park C, Silverman CL, Eckhoff DO, Guest JC, Díaz DA. Accessibility and utilisation of telehealth services among older adults during COVID-19 pandemic in the United States. Health Soc Care Commun. (2022). doi: 10.1111/hsc.13709

PubMed Abstract | CrossRef Full Text | Google Scholar

19. Lu M, Ishwaran H. Cure and death play a role in understanding dynamics for COVID-19: data-driven competing risk compartmental models, with and without vaccination. PLoS ONE. (2021) 16:e254397. doi: 10.1371/journal.pone.0254397

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Lu M. Dynamic modeling COVID-19 for comparing containment strategies in a pandemic scenario. Ann Biostat Biometr Appl. (2020) 4:1–4. doi: 10.33552/ABBA.2020.04.000579

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Breiman L. Random forests. Mach Learn. (2001) 45:5–32. doi: 10.1023/A:1010933404324

CrossRef Full Text | Google Scholar

22. Ishwaran H, Lu M. Standard errors and confidence intervals for variable importance in random forest regression, classification, and survival. Stat Med. (2019) 38:558–82. doi: 10.1002/sim.7803

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Ishwaran H, Tang F, Lu M, Kogalur UB. RandomForestSRC: Multivariate Splitting Rule Vignette. (2021). Available online at: http://randomforestsrc.org/articles/mvsplit.html (accessed May 16, 2022).

Google Scholar

24. Lu M, Parel JM, Miller D. Interactions between staphylococcal enterotoxins A and D and superantigen-like proteins 1 and 5 for predicting methicillin and multidrug resistance profiles among Staphylococcus aureus ocular isolates. PLoS ONE. (2021) 16:e254519. doi: 10.1371/journal.pone.0254519

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Fang X, Liu W, Ai J, He M, Wu Y, Shi Y, et al. Forecasting incidence of infectious diarrhea using random forest in Jiangsu Province, China. BMC Infect Dis. (2020) 20:222. doi: 10.1186/s12879-020-4930-2

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Ong J, Liu X, Rajarethinam J, Kok SY, Liang S, Tang CS, et al. Mapping dengue risk in Singapore using random forest. PLoS Negl Trop Dis. (2018) 12:e0006587. doi: 10.1371/journal.pntd.0006587

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Coates-Brown R, Moran JC, Pongchaikul P, Darby AC, Horsburgh MJ. Comparative genomics of Staphylococcus reveals determinants of speciation and diversification of antimicrobial defense. Front Microbiol. (2018) 9:2753. doi: 10.3389/fmicb.2018.02753

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Lu M, Sha Y, Silva T, Colaprico A, Sun X, Ban Y, et al. LR hunting: a random forest based cell-cell interaction discovery method for single-cell gene expression data. Front Genet. (2021) 12:1431. doi: 10.3389/fgene.2021.708835

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Pasek J. With Some Assistance From Alex Tahk, Some Code Modified From R-Core; Additional Contributions by Gene Culter, Schwemmle M. Weights: Weighting Weighted Statistics. R package version 1.0.4. (2021). Available online at: https://CRAN.R-project.org/package=weights (accessed May 16, 2022).

30. Ishwaran H, Kogalur UB. Fast Unified Random Forests for Survival, Regression, Classification (RF-SRC). R package version 3.1.0 (2022). Available online at: https://cran.r-project.org/package=randomForestSRC (accessed May 16, 2022).

31. Ishwaran H, Lu M, Kogalur UB. randomForestSRC: Variable Importance (VIMP) With Subsampling Inference Vignette (2021). Available online at: http://randomforestsrc.org/articles/vimp.html (accessed May 16, 2022).

Google Scholar

32. Hastie T, Tibshirani R, Friedman J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. New York, NY: Springer Science & Business Media (2009). doi: 10.1007/978-0-387-84858-7

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Ishwaran H, Lu M, Kogalur UB. RandomForestSRC: Partial Plots Vignette (2021). Available online at: http://randomforestsrc.org/articles/partial.html (accessed May 16, 2022).

Google Scholar

34. Tibshirani R, Bien J, Friedman J, Hastie T, Simon N, Taylor J, et al. Strong rules for discarding predictors in lasso-type problems. J R Stat Soc Ser B. (2012) 74:245–66. doi: 10.1111/j.1467-9868.2011.01004.x

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Friedman J, Hastie T, Tibshirani R. Regularization paths for generalized linear models via coordinate descent. J Stat Softw. (2010) 33:1. doi: 10.18637/jss.v033.i01

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Hamadi HY, Zhao M, Haley DR, Dunn A, Paryani S, Spaulding A. Medicare and telehealth: the impact of COVID-19 pandemic. J Eval Clin Pract. (2022) 28:43–8. doi: 10.1111/jep.13634

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Smith KT, Monti D, Mir N, Peters E, Tipirneni R, Politi MC. Access is necessary but not sufficient: factors influencing delay and avoidance of health care services. MDM Policy Pract. (2018) 3:2381468318760298. doi: 10.1177/2381468318760298

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Fung V, Graetz I, Galbraith A, Hamity C, Huang J, Vollmer WM, et al. Financial barriers to care among low-income children with asthma: health care reform implications. JAMA Pediatr. (2014) 168:649–56. doi: 10.1001/jamapediatrics.2014.79

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Yoon S, Goh H, Chan A, Malhotra R, Visaria A, Matchar D, et al. Spillover effects of COVID-19 on essential chronic care and ways to foster health system resilience to support vulnerable non-COVID patients: a multistakeholder study. J Am Med Direc Assoc. (2022) 23:7–14. doi: 10.1016/j.jamda.2021.11.004

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Choi KR, Heilemann MV, Fauer A, Mead M. A second pandemic: mental health spillover from the novel coronavirus (COVID-19). J Am Psychiatr Nurses Assoc. (2020) 26:340–3. doi: 10.1177/1078390320919803

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Sanyaolu A, Okorie C, Marinkovic A, Patidar R, Younis K, Desai P, et al. Comorbidity and its impact on patients with COVID-19. SN Comprehens Clin Med. (2020) 2:1069–76. doi: 10.1007/s42399-020-00363-4

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Harris DA, Archbald-Pannone L, Kaur J, Cattell-Gordon D, Rheuban KS, Ombres RL, et al. Rapid telehealth-centered response to COVID-19 outbreaks in postacute and long-term care facilities. Telemed e-Health. (2021) 27:102–6. doi: 10.1089/tmj.2020.0236

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Radhakrishnan K, Jacelon CS, Bigelow C, Roche JP, Marquard JL, Bowles KH. Association of comorbidities with home care service utilization of patients with heart failure while receiving telehealth. J Cardiovasc Nurs. (2013) 28:216–27. doi: 10.1097/JCN.0b013e3182512331

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Baim-Lance A, Angulo M, Chiasson MA, Lekas HM, Schenkel R, Villarreal J, et al. Challenges and opportunities of telehealth digital equity to manage HIV and comorbidities for older persons living with HIV in New York State. BMC Health Services Res. (2022) 22:5. doi: 10.1186/s12913-022-08010-5

PubMed Abstract | CrossRef Full Text | Google Scholar

45. Pfefferbaum B, North CS. Mental Health and the Covid-19 Pandemic. N Engl J Med. (2020) 383:510–2. doi: 10.1056/NEJMp2008017

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Usher K, Durkin J, Bhullar N. The COVID-19 pandemic and mental health impacts. Int J Ment Health Nurs. (2020) 29:315–8. doi: 10.1111/inm.12726

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Kumar A, Nayar KR. COVID 19 and its mental health consequences. J Mental Health. (2021) 30:1–2. doi: 10.1080/09638237.2020.1757052

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Moreno C, Wykes T, Galderisi S, Nordentoft M, Crossley N, Jones N, et al. How mental health care should change as a consequence of the COVID-19 pandemic. Lancet Psychiatry. (2020) 7:813–24. doi: 10.1016/S2215-0366(20)30307-2

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Hsiao V, Chandereng T, Lankton RL, Huebner JA, Baltus JJ, Flood GE, et al. Disparities in telemedicine access: a cross-sectional study of a newly established infrastructure during the COVID-19 pandemic. Appl Clin Inform. (2021) 12:445–58. doi: 10.1055/s-0041-1730026

PubMed Abstract | CrossRef Full Text | Google Scholar

50. Lu M, Ishwaran H. A prediction-based alternative to P values in regression models. J Thorac Cardiovasc Surg. (2018) 155:1130. doi: 10.1016/j.jtcvs.2017.08.056

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Lu M, Ishwaran H. Discussion on “nonparametric variable importance assessment using machine learning techniques” by Brian D. Williamson, Peter B. Gilbert, Marco Carone, and Noah Simon. Biometrics. (2021) 77:23–7. doi: 10.1111/biom.13391

PubMed Abstract | CrossRef Full Text | Google Scholar