“Lock to Live” for firearm and medication safety: Feasibility and acceptability of a suicide prevention tool in a learning healthcare system

Objective: Few patients with suicide risk are counseled on lethal means safety by health providers. This study tested the feasibility of different delivery methods for Lock to Live (L2L), a web-based decision aid of safe storage options for firearms and medications.

Methods: Patients reporting suicide ideation on the PHQ9 depression screener during outpatient health visits were included. Invitation messages to visit L2L were sent via combinations of email, text, Electronic Health Record (EHR) message, mailed letter, or provider referral, followed by a survey about storage behavior and acceptability. Provider interviews evaluated logistical considerations and acceptability.

Results: The population-based method reached 2,729 patients and the best method (EHR message plus 2 email reminders) had 11% uptake (L2L visitation rate). Provider referral had small reach (14 patients) and 100% uptake (all visited). Provider interviews identified several strategies to promote uptake including: EHR reminders, provider training, quality metrics with accountability, a clearly communicated lethal means screening/counseling policy, and strong organizational leadership support.

Conclusion: Despite the low uptake for population-based (11%), far more patients with suicide risk were engaged in the L2L tool through population-based outreach than provider-referral over the same time frame.

Introduction

Reducing access to lethal means for suicide is highly effective for suicide prevention (1–3). In the United States, this involves targeting the most common lethal means—securing firearms and mitigating risk for overdose (e.g., disposing of unused medications or asking a support person to dispense prescription medications during high-risk times) (2). While many countries rely on legal interventions to address firearm safety broadly (e.g., for safe storage, training for owners, and mental health related restrictions on access) (1), there are considerable political roadblocks to firearm legislation in the United States. Therefore, the health care setting has become an important avenue to deliver public health interventions to increase firearm safety for patients at risk of suicide (4). Lethal means counseling is recommended (5) and involves reducing access to common lethal means during suicidal crises. However, many patients with suicide risk are not counseled on lethal means safety across health settings (6), indicating the need for new approaches to address this gap.

There are several factors contributing to low rates of lethal means counseling particularly within emergency medicine and behavioral health, which include: time-constraints, providers’ lack of perceived benefit for preventing suicide (among emergency medicine providers), lack of training or knowledge of safe storage options, and fear that asking about firearms will damage the patient/clinician relationship (7, 8). Population-based interventions that don't rely on providers can address these challenges and be broadly disseminated because they are typically lower-cost, particularly at scale. Caring letters are an example of a low-intensity population-based intervention resulting in reduction in suicidal behavior (9).

One challenge to expanding firearm safety interventions delivered by clinicians is whether patients disclose access to firearms. There is disagreement in the literature on this point. Some surveys of firearm owners suggest that most are receptive and understand the importance within the context of suicide prevention (10, 11). Other studies show that firearm owners tend to have increased concerns around privacy, both in general and specifically as it relates to recording firearm ownership in medical records (12–16). Education on safe storage delivered anonymously may be more acceptable to firearm owners.

To address these challenges, we tested a population-based approach using an anonymous, web-based, self-administered decision aid for safe firearm and medication storage in patients with suicide risk called Lock to Live (http://lock2live.org/) (17). Lock to Live (L2L) was designed with input from over 60 stakeholders, including patients with lived experience (18), medical and mental health providers and firearm owners. Decision aids have been shown to effectively facilitate health decisions that are complicated by first providing education, then options to patients that incorporate their values and preferences into the decision-making process. L2L is based on the Ottawa's Decision Support Framework (19), which posits that the quality of decisions is higher when people are knowledgeable, understand the conflict or uncertainty points, and have identified and incorporated their own values. The behavioral economics concept of “nudges” is also incorporated by presenting effort-laden storage options alongside easy ones to subtly persuade patients to choose the easy options (18). L2L is anonymous (no patient identifiers are entered into the tool) and focuses on voluntary, temporary reductions in access; it provides information on costs for both in-home and out-of-home storage options, whether a background check is required to return firearms, and medication management options that address the need for access to daily medications. L2L has information on what types of storage options require background checks to encourage compliance with state background check laws for transfers between non-family members.

Initial testing of L2L with a tablet in an emergency department setting showed that it was highly acceptable across a broad range of provider roles (physicians, nurses, social workers). L2L addresses the lack of provider knowledge for different firearm and medication storage options (20). A survey showed that clinicians thought L2L would help them counsel suicidal adult patients on safe firearm storage options (21). A small pilot trial of 49 adult patients, evaluated in the emergency setting for a suicide-related concern, showed high satisfaction with L2L and non-significant increases in safe storage behavior (22). Enrollment was low—likely due to privacy concerns about firearm access alongside a high level of suicide risk and vulnerability in the emergency setting. These pilot findings support the need to pilot test new implementation strategies that address concerns about privacy and reach greater volumes of patients. Importantly, we recognize the need for more effectiveness data on the Lock to Live tool. However, the implementation strategy needs to be tested first because it would be paired with the tool when tested in a future larger trial.

The purpose of this study was to test the feasibility (reach and uptake) and acceptability (patient survey and provider interviews) of delivering the L2L web-based decision aid for safe firearm storage using population-based and provider-referral outreach in the outpatient setting. Reach was defined as the number of patients outreached and uptake was the number of patients who visited L2L. An anonymous follow-up survey evaluated impacts on storage behavior and patient acceptability. Provider interviews evaluated logistical considerations and acceptability.

Methods

Setting

The sample included adult members of a large integrated health care system (Kaiser Permanente Colorado) who reported any frequency of suicidal thoughts on the ninth question of the Patient Health Questionnaire (PHQ-9) (23) from October 2019 to April 2020. The ninth question asked whether patients have had “thoughts that you would be better off dead, or thoughts of hurting yourself in some way” in the past 2 weeks. Response options include: not at all (0), several days (1), more than half the days (2), or nearly every day (3). The PHQ-9 was administered on a Tablet in the waiting room prior to all mental health visits and on paper for selected medical visits where patients are routinely screened or treated for depression and/or anxiety.

Study design

We used a mixed-methods (QUAN qual) embedded design (24) to assess the feasibility and acceptability of different outreach methods for L2L via different combinations of population-based outreach methods [email, text, Electronic Health Record (EHR) patient message, or mailed letter] and direct provider referral. Provider interviews helped characterize the context of current lethal means screening and counseling practices within the health system, logistical considerations, and acceptability of L2L.

Population-based invitation messages to visit L2L emphasized anonymity and were co-designed with input from clinicians, public health researchers, and members of the Colorado Firearm Safety Coalition. Six combinations of initial + reminder messages (e.g., email + text; email + mail) were tested. Letter, email or EHR message were used for initial invites, but never text—due to length restrictions. Combinations of different methods were tested on the hypothesis that people will have varying response propensities to different methods.

For the provider referral pilot, nine mental health providers agreed to test L2L with patients for 6 weeks. Patients were selected for L2L referral from clinical judgment of the patient's suicide risk (not just the PHQ-9). Providers had a 1-h orientation to L2L with discussion and agreement on the best referral process. They were provided with scripting templates for introducing L2L. Collaboratively, a goal was set of each provider to refer 10 patients over a 6-week period (providers felt this was easily achievable at the outset). A follow-up discussion with providers identified barriers and facilitators to referral. All study procedures were approved by the Kaiser Permanente Institutional Review Board.

Measures

Web analytics (Uptake)

Website visit rates assessed uptake of L2L for each outreach method using separate URLs (e.g., EHR + text, mail + text). This allowed tracking of visits associated with different outreach approaches but maintained patient anonymity. Text messages were only used as reminders, but never as initial messages due to length restrictions.

Patient survey (Storage behavior and acceptability – see Appendix A)

All patients received an anonymous survey to evaluate acceptability with L2L and safe storage behavior 4 weeks after their first invitation to visit L2L. Items assessed uptake (visits to website) and satisfaction with L2L, lethal means storage behavior (i.e., firearms and medications), and whether patients received lethal means counseling from medical or mental health providers. Stage of Change theory (25) informed survey questions for storage behavior by assessing pre-contemplative beliefs about storage, whether the patient was thinking about changing storage behavior, planning changes, or had taken action to change storage behavior. Survey development was undertaken in phases with an initial testing phase among non-study research colleagues and community members of the Colorado Firearm Safety Coalition (26). Part way through we amended some of the branching logic to capture whether L2L was influential towards patients considering or planning storage changes.

Provider interviews

The goal of the interviews was to characterize the context of current lethal means screening and counseling practices within the health system, logistical considerations and acceptability of L2L. Eligible providers were mental health therapists, psychiatrists, and primary care providers who responded to email invitations. An invitation email to participate in a 30–45-min interview was sent to a convenience sample of 13 mental health and 10 primary care providers. Interviews were held via video conference (Microsoft Teams) and followed an interview guide that assessed current lethal means safety counseling practices, barriers and facilitators for delivering lethal means counseling, and whether and how L2L could be a useful tool for integration either during visits or separately using population-based outreach methods. All interviewees provided informed consent and there was no incentive.

Analysis

Reach

Defined as all eligible adult patients who endorsed suicide ideation on the PHQ-9 for the population-based approach. Patients were outreached in batches to evaluate the impact of different outreach methods. For the provider-referral approach, reach included all patients who were referred by providers.

Uptake

Uptake was defined by the number of unique IP addresses that visited each unique URL divided by the number of patients sent a message for that batch (outreach method). Email, mail, or EHR messages that bounced back were not included in the denominator. A logistic regression model determined differences in response rates between outreach methods.

Patient survey

Descriptive frequencies were calculated for each survey question based on the number of patients who answered that question. Patients who skipped questions or answered “prefer not to answer” were included in the denominator, but those who were never presented with the question because of branching logic or early discontinuation were not counted.

Provider interviews

Interviews were audio recorded. Two coders (who conducted the interviews) independently listened to the recordings and noted themes within the apriori topics from the interview guide. The coders then met and shared their ideas and resolved any discrepancies in interpretation. Themes were summarized in a table by the apriori topics and sent to each interviewee for member checking (24). Interviewees provided minor edits to the interpretations via email response.

Results

Table 1 details patient and clinical characteristics of the sample. The patient sample (N = 2,931) was predominantly female (65%) with moderate to severe depression reported on the PHQ-9 (mean total score = 15.6).

TABLE 1

Table 1. Patient characteristics (N = 2,931).

Website visits

Population-based outreach messages were sent to 2,729 of 2,931 adults identified via the EHR; 202 patients (6.8%) were excluded because they were unreachable (i.e., no email address, no mobile number, returned mail). Table 2 illustrates the response rates for each outreach method tested. EHR messages plus two email reminders resulted in the highest L2L visitation rates of 11% (52/491) (OR = 13.7 (4.9–38.2) compared to mailed letter plus text reminder (REF) with the lowest L2L visitation rate (4/480; <1%).

TABLE 2

Table 2. Lock to Live visit and survey completion rates by outreach method.

In the 6-week provider referral pilot, three of the nine outpatient providers referred a total of 14 patients, with a 100% L2L visit rate. One patient (out of 14) had previously received an invitation via population-based outreach. In follow-up discussions, providers identified barriers to referral including perceived need for real-time EHR reminders about L2L availability, lack of visit time to bring up L2L, patients declining to learn more about the program, and perception that L2L is better suited for patients with moderate to severe suicide-risk.

Patient survey

Of the 326 patients who started the survey, 44 (13.5%) didn't answer any questions, 114 (35.5%) answered some questions, and 168 (51%) completed all questions. Survey completion varied by outreach method (Table 2). Table 3 reports the satisfaction with L2L among survey respondents who reported visiting. Most indicated they would prefer to receive L2L through EHR messages (78%), followed by email (41%), provider (40%), poster in clinic (32%), text (26%), and mail (15%). Of the 286 patients who completed part/full survey, 46 (16%) indicated that a medical or mental health provider had discussed firearm safety and 34 (12%) discussed medication safety at a recent visit.

TABLE 3

Table 3. Lock to Live satisfaction (N = 73).

Patient survey: Firearm storage section

Table 4 describes firearm storage beliefs and behaviors indicating 35% (n = 82/231) had a firearm. Of those with a firearm (N = 82), half (N = 41, 50%) did not endorse any secure storage behaviors such as locking devices or firearm safes. Of those who endorsed at least 1 storage behavior, one person indicated that L2L was influential (N = 41). Most said a friend, family member, or firearms safety class influenced their current storage behavior. Of those with access to a firearm who did not endorse any storage behaviors, 44% (n = 18/41) said they were thinking about firearm storage changes and 31% (n = 14/41) said they were planning changes. A later version of the survey indicated that 50% of those who had access to a firearm and were thinking about storage changes, thought L2L was influential to their considerations (n = 5/10, 50%).

TABLE 4

Table 4. Patient reported firearm and medication storage beliefs and behaviors.

Patient survey: Medication storage

Table 4 describes medication storage behaviors. Of the 248 who answered questions in this section, 181 (73%) indicated that all medications are accessible. Of those with medications accessible, 36 (27%) were thinking about changes and of those, 57% (20) were planning changes. Disposing of old medications (N = 36, 73%) and medication lockbox (N = 23, 47%) were cited as the most common planned behaviors compared to family/friend managing meds (N = 12, 24%) and receiving smaller prescription amounts (N = 1, 2%). Of those who had medications stored inaccessibly, L2L was cited as influential in 4 cases (6%). In later batches, where we assessed influences on decision making for those considering changes, L2L was cited as the most influential factor, more so than providers, friends/family or other influences. Of the 268 patients who completed, 84 (31%) provided qualitative responses to the open-ended question about barriers to medication storage (Table 4).

Provider interviews

Of the 23 providers invited, 16 responded and were interviewed: 5 primary care physicians, 3 psychiatry physicians, 3 behavioral health crisis clinicians, 3 behavioral health outpatient therapists, and 2 primary-care based psychotherapists.

Current lethal means counseling policies and practices

Providers were split evenly on whether they felt that patients are forthcoming when discussing access to firearms or medications. Most felt that they could handle resistance to firearm discussions by emphasizing their interest in the patient's safety and providing reassurance that firearm removal was temporary until suicidal thoughts have passed. There was disagreement on whether there was a standard operating procedure within the health system to ask all patients with suicide risk about firearm and medication access. Some mentioned that they ask all patients at intake visits but may not ask again if the initial assessment for access is negative. Providers noted that there are quality metrics for depression and suicide screening within the health system, but none for lethal means assessment.

“We don't necessarily ask at every visit, but hopefully we ask in every first visit. If it's documented previously that they don't have access, we don't ask in every follow-up visit.”

Providers said they often problem solve about how to limit access and ask if there are family or friends who can hold the lethal means temporarily. For the highest risk patients, this may involve reaching out to the patient's spouse, other family member or a friend. If the patient has a history of suicide risk or attempts, but isn't currently reporting ideation, safe storage options are recommended that might be necessary during higher-risk times.

“We will problem-solve around how to put space between the patient and whatever lethal means are identified as accessible. We talk about giving firearms to the police, neighbor, etc. It seems like patients are honest about restricting access to lethal means; if they're not willing to restrict access, they're pretty quick to say so. For patients who don't endorse suicide ideation, we still recommend that they keep firearms secure, especially while going through that episode of depression.”

Strategies to inform Lock to Live uptake among patients and providers

Providers said that it was important to highlight the privacy and anonymity of L2L. However, it was emphasized that it would be better not to mention the reason for outreach was their endorsement of suicidal thoughts on the PHQ-9 screening tool. Providers described how patients may be more suspicious of firearm safety information sent via email that was connected to their responses to a suicide screener unless it came directly from a trusted clinical provider (even though it came from the integrated health system).

“I’m not sure it's feasible, but this would be good to send to everyone. There are people out there who aren't getting help; this program might be helpful to them. It would be beneficial to visit the site in-person with the provider. That would eliminate the concern about someone else having accessed their data. A provider can encourage them to participate, even just by giving them a card with the website link.”

Provider perceptions of Lock to Live usefulness and patient acceptability

Nearly all the providers interviewed felt that L2L was a great resource that provided storage options they didn't know about (such as disassembling a firearm). In general, providers felt that patients would react positively to L2L, that it could promote behavior change and prompt conversations between patients and their care providers. An important consideration was that patients will have to be receptive to lethal means safety steps (beyond pre-contemplative stage of change) before accepting the tool. Many providers felt comfortable addressing patient resistance to the lethal means conversation but were less knowledgeable of different types of storage options.

“I think L2L could promote behavior change and prompt discussion with providers; it takes away some stigma. Some of my colleagues are uncomfortable having these conversations, and L2L could be helpful in that regard.”

Discussion

We found that population-based outreach using the Lock to Live (L2L) web-based decision aid for safe storage of lethal means in those with suicide risk reached many patients who had not previously discussed lethal means safety with providers. While uptake to population-based approaches was low (11%) in comparison to provider-referral (100%), most of the low-risk patients outreached using population-based methods had not been previously counseled on lethal means safety per survey report. The absolute number who visited L2L in a 6-week time was 52 for population-based with the best method versus 14 for provider referral. Every patient referred by providers visited L2L, but without more organizational supports (e.g., reminders, quality metrics, more intensive training), reach will remain low. We learned a great deal from patients and providers about ways to improve uptake of L2L for both self-directed population-based methods and provider referral as part of lethal means counseling.

We saw that sending invitations securely through the electronic medical record with two email reminders had the highest L2L visit rate of 11%. EHR messages were the most preferred outreach method based on patient survey feedback and website-usage data. Low-response may be due to stigma often associated with seeking care for mental health conditions (27). The response rate was comparable to other population-based outreach efforts for sensitive issues, such as chlamydia screening (28), that have between 15% and 20% response.

Follow-up discussion with providers from the referral pilot illustrated that referral only felt appropriate for patients they assessed as moderate to high risk (e.g., via clinical assessment). Past studies have shown that providers are concerned that lethal means counseling could have a negative impact on the therapeutic relationship (10–12, 29)—although we did not hear that from providers in our study. Providers identified barriers including a lack of reminders, time restrictions, unclear protocols for lethal means access assessment and counseling, and no quality metrics to bring about accountability.

Our findings highlighted the need for future lethal means safety approaches to incorporate stages of change to promote secure lethal means storage. We observed that 18% reported access to an unsecured firearm, with 44% of these thinking about changing storage (contemplative) and 50% not considering changes (pre-contemplative). Behavior change interventions that don't incorporate patients readiness for change may be less likely to succeed (25). Provider interviews were consistent with this idea indicating that it may take more than one “nudge” to get patients to consider safe storage. Population-based outreach may be less effective to engage patients in a lethal means safety tool than provider-delivered intervention, but still valuable to move someone closer to behavior change over time. Combining these two methods could bring the most impact, particularly if the low-rate of provider-delivered counseling were improved through the suggestions offered here which were: EHR reminder, provider training, quality metrics with accountability, a clearly communicated lethal means screening/counseling policy, and strong organizational leadership support.

Future studies should not only measure whether someone with suicide risk has access to a firearm or medication, but whether the current level of suicide risk is consistent with the current storage behavior. Patient survey and provider interview feedback indicated that it may be important to approach changes in firearm storage using a risk-stratification—especially in individuals with low or intermittent suicide risk. Instead of static goals about safe storage, different storage options from less to more inaccessible may be considered based on current risk. Patients with protection weapons (36% of firearm owner respondents) may be more amenable to the idea of temporary off-site storage only during high-risk times, a strategy used by the Gun Shop project (30). For medication storage, 70% indicated that they don't need medication safety at this time, further illustrating the need to consider risk and timing with lethal means safety.

It is important to note limitations to our study that may impact generalizability to other populations. There are many patients who likely didn't respond to the survey or visit L2L due to uneasiness about privacy, even though both were anonymous to address this concern. The invitation message came from the health system and many are skeptical of sharing firearm ownership with medical establishments (10–12, 29). Response rates would likely be improved from community-based surveys or from organizations within the firearm community. However, it would be harder to identify those with suicide risk outside of an integrated health system that does routine screening. Another reason for non-response may be severity of depression symptoms (e.g., lack of motivation) since reported symptom severity was high (56% had total score PHQ9 > 15). One option would be to survey patients later after they have completed a course of treatment and experienced symptom improvement, but this may not be representative of patients’ receptiveness during times of suicide risk when lethal means safety is important. Since our sample was limited by those seeking mental health care services within an integrated healthcare system who reported suicide ideation, which were majority female, we may have missed men who are much higher risk of firearm suicide (31). Future studies of firearm safety should consider using alternative ways to identify patients at risk for firearm suicide such as predictive models (32). Finally, while we tested the text and email messages with common providers including AT&T and Verizon for text message and Gmail (google) for emails, there is a possibility that spam-blocking software prevented patients from receiving text or email messages. However, this would not impact the EHR messages. COVID-19 contextual factors must be considered for all research that was ongoing during this time. The pandemic may have detrimentally impacted providers’ ability to incorporate a new process that included referral to L2L. Conversely, increased number of new firearm owners during this time (33) may have fueled increased interest in firearm safety among patients and providers. Finally, our tracking methods did not allow us to measure how long patients spent on the L2L tool to understand whether they spent sufficient time to comprehend the content.

The goal of the current project was to assess feasibility (reach and uptake) and acceptability (patient survey and provider interviews) of outreach methods to the L2L web-based decision aid. We found that population-based approaches have higher reach and smaller uptake compared to provider referral, but the absolute number reached through population-based approaches was higher. Expanding provider-delivered lethal means safety interventions to all patients reporting suicide risk will require an EHR reminder, provider training, quality metrics with accountability, a clearly communicated lethal means screening/counseling policy, and strong organizational leadership support. Population-based outreach of lethal means safety is a viable low-cost option and may be particularly important tool as a primer to discussion of firearm access with providers.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving human participants were reviewed and approved by Kaiser Permanente Institutional Review Board. Written informed consent for participation was not required for this study in accordance with the national legislation and the institutional requirements.

Author contributions

JB obtained funding, served as the principle investigator on the study, conducted the data analysis and drafted the manuscript. LQ was the project manager, helped developed, test and administer the survey, conducted qualitative interviews and analysis, and critically reviewed the manuscript. AB served as a mentor to JB for the study and consulted on the management of the study, and critically reviewed the manuscript. SC, LR, and AC were clinical advisors to the study. They helped develop and update the invitation language and outreach methods, engage with system leaders for the provider-delivered pilot, and critically reviewed the manuscript. JR was an Co-Investigator and expert consultant on firearm safety messaging and helped develop the invitation language and critically reviewed the manuscript. MB was a Co-Investigator, creator of the Lock to Live tool, expert on firearm safety messaging, consulted on study decisions, reviewed the survey data with JB, and critically reviewed the manuscript. All authors contributed to the article and approved the submitted version.

Funding

This research study was funded by Kaiser Permanente’s Office of Community Health as part of its Firearm Injury Prevention Program.

Acknowledgements

We would like to thank Carolyn Joyce, BS and Megan Rogers, MS, LCSW, for conducting provider interviews and analyzing the interview data. We'd like to thank Megan A. Baldwin, BA for her support developing the REDCap project database to implement timely and targeted outreach to eligible subjects. We'd like to thank Chris Boyd who aided the study team in developing study-specific Lock to Live website domains and helped implement website analytics used to track visits to Lock to Live. We also want to recognize Angela Comer, MS who was instrumental in developing programming code to identify patients from the electronic health record for outreach. Finally, we want to extend our gratitude to James T. Hardee, MD, a primary care provider and leader who gave feedback on patient and provider communications and identified potential provider interviewees.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Author disclaimer

The views and opinions expressed in this article are the responsibility of the authors and do not necessarily represent the official views of Kaiser Permanente.

References

1. Mann J, Apter A, Bertolote J, Beautrais A, Currier D, Haas A, et al. Suicide prevention strategies: a systematic review. J Am Med Assoc. (2005) 294(16):2064–74. doi: 10.1001/jama.294.16.2064

CrossRef Full Text | Google Scholar

2. Barber CW, Miller MJ. Reducing a suicidal person’s access to lethal means of suicide: a research agenda. Am J Prev Med. (2014) 47(3):S264–72. doi: 10.1016/j.amepre.2014.05.028

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Barnett BS, Kudler H, Swanson J. To prevent suicide among veterans, their physicians should discuss gun safety. JAMA Health Forum. (2020) 1(11):e201407. doi: 10.1001/jamahealthforum.2020.1407

CrossRef Full Text | Google Scholar

4. Runyan CW, Brooks-Russell A, Betz ME. Points of influence for lethal means counseling and safe gun storage practices. J Public Health Manag Pract. (2019) 25(1):86. doi: 10.1097/PHH.0000000000000801

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Suicide Prevention Resource Center. CALM: counseling on access to lethal means (2018). Available at: https://www.sprc.org/resources-programs/calm-counseling-access-lethal-means

6. Boggs JM, Quintana LM, Powers JD, Hochberg S, Beck A. Frequency of clinicians’ assessments for access to lethal means in persons at risk for suicide. Arch Suicide Res. (2020) 26(1):1–10. doi: 10.1080/13811118.2020.176917

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Betz ME, Kautzman M, Segal DL, Miller I, Camargo CA, Boudreaux ED, et al. Frequency of lethal means assessment among emergency department patients with a positive suicide risk screen. Psychiatry Res. (2018) 260:30–5. doi: 10.1016/j.psychres.2017.11.038

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Diurba S, Johnson RL, Siry BJ, Knoepke CE, Suresh K, Simpson SA. Lethal means assessment and counseling in the emergency department: differences by provider type and personal home firearms. Suicide Life Threat Behav. (2020) 50(5):1054–64. doi: 10.1111/sltb.12649

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Luxton DD, June JD, Comtois KA. Can postdischarge follow-up contacts prevent suicide and suicidal behavior? Crisis. (2013) 34(1):32–41. doi: 10.1027/0227-5910/a000158

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Betz ME, Barber CW, Miller M. Firearm restriction as suicide prevention: variation in belief and practice among providers in an urban emergency department. Inj Prev. (2010) 16(4):278–81. doi: 10.1136/ip.2009.025296

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Roszko PJ, Ameli J, Carter PM, Cunningham RM, Ranney ML. Clinician attitudes, screening practices, and interventions to reduce firearm-related injury. Epidemiol Rev. (2016) 38(1):87–110. doi: 10.1093/epirev/mxv005

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Valenstein M, Walters H, Pfeiffer PN, Ganoczy D, Miller M, Fiorillo M, et al. Acceptability of potential interventions to increase firearm safety among patients in VA mental health treatment. Gen Hosp Psychiatry. (2018) 55:77–83. doi: 10.1016/j.genhosppsych.2018.10.010

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Betz ME, Azrael D, Barber C, Miller M. Public opinion regarding whether speaking with patients about firearms is appropriate: results of a national survey. Ann Intern Med. (2016) 165(8):543–50. doi: 10.7326/M16-0739

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Richards JE, Hohl SD, Segal CD, Grossman DC, Lee AK, Whiteside U, et al. What will happen if i say yes? Perspectives on a standardized firearm access question among adults with depressive symptoms. Psychiatr Serv. (2021) 72(8):898–904. doi: 10.1176/appi.ps.202000187

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Knoepke CE, Allen A, Ranney ML, Wintemute GJ, Matlock DD, Betz ME. Loaded questions: internet commenters’ opinions on physician-patient firearm safety conversations. West J Emerg Med. (2017) 18(5):903–12. doi: 10.5811/westjem.2017.6.34849

PubMed Abstract | CrossRef Full Text | Google Scholar

16. Monuteaux MC, Lee LK, Hemenway D, Mannix R, Fleegler EW. Firearm ownership and violent crime in the US: an ecologic study. Am J Prev Med. (2015) 49(2):207–14. doi: 10.1016/j.amepre.2015.02.008

PubMed Abstract | CrossRef Full Text | Google Scholar

17. Betz ME, Knoepke CE, Siry B, Clement A, Azrael D, Ernestus S, et al. ‘Lock to Live’: development of a firearm storage decision aid to enhance lethal means counselling and prevent suicide. Inj Prev. (2019) 25(Suppl 1):i18–24. doi: 10.1136/injuryprev-2018-042944

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Thaler RH, Sunstein CR. Nudge: Improving decisions about health, wealth, and happiness. New York: Penguin Books (2009).

19. Stacey D, Légaré F, Boland L, Lewis KB, Loiselle MC, Hoefel L, et al. 20th Anniversary Ottawa decision support framework: part 3 overview of systematic reviews and updated framework. Med Decis Making. (2020) 40(3):379–98. doi: 10.1177/0272989X20911870

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Siry BJ, Knoepke CE, Ernestus SM, Matlock DD, Betz ME. Lethal means counseling for suicidal adults in the emergency department: a qualitative study. West J Emerg Med. (2021) 22(3):471–7. doi: 10.5811/westjem.2021.8.49485

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Siry BJ, Polzer E, Omeragic F, Knoepke CE, Matlock DD, Betz ME. Lethal means counseling for suicide prevention: views of emergency department clinicians. Gen Hosp Psychiatry. (2021) 71:95–101. doi: 10.1016/j.genhosppsych.2021.04.011

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Betz ME, Knoepke CE, Simpson S, Siry BJ, Clement A, Saunders T, et al. An interactive web-based lethal means safety decision aid for suicidal adults (lock to live): pilot randomized controlled trial. J Med Internet Res. (2020) 22(1):e16253. doi: 10.2196/16253

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med. (2001) 16(9):606–13. doi: 10.1046/j.1525-1497.2001.016009606.x

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Creswell JW, Clark VLP. Designing and conducting mixed methods research. 3rd ed. Sage Publications, Newbury Park, California (2006).

25. Prochaska JO, Norcross JC. Stages of change. Psychother Theory Res Pract Train. (2001) 38(4):443. doi: 10.1037/0033-3204.38.4.443

CrossRef Full Text | Google Scholar

26. Colorado Firearm Safety Coalition. Available at: https://coloradofirearmsafetycoalition.org/

27. Jagdeo A, Cox BJ, Stein MB, Sareen J. Negative attitudes toward help seeking for mental illness in 2 population—based surveys from the United States and Canada. Can J Psychiatry. (2009) 54(11):757–66. doi: 10.1177/070674370905401106

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Andersen B, Olesen F, Møller JK, Østergaard L. Population-based strategies for outreach screening of urogenital chlamydia trachomatis infections: a randomized, controlled trial. J Infect Dis. (2002) 185(2):252–8. doi: 10.1086/338268

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Brent DA, Bridge J. Firearms availability and suicide evidence, interventions, and future directions. Am Behav Sci. (2003) 46(9):1192–210. doi: 10.1177/0002764202250662

CrossRef Full Text | Google Scholar

30. Gun Safety and Suicide. Colorado department of public health and environment. Available at: https://cdphe.colorado.gov/suicide-prevention/gun-safety-and-suicide

31. Centers for Disease Control and Prevention, National Center for Health Statistics. National vital statistics system, mortality 1999-2020 on CDC WONDER online database, released in 2021. Data are from the multiple cause of death files, 1999-2020, as compiled from data provided by the 57 vital statistics jurisdictions through the vital statistics cooperative program. Available at: http://wonder.cdc.gov/ucd-icd10.html (Accessed August 4, 2022).

32. Simon GE, Johnson E, Lawrence JM, Rossom RC, Ahmedani B, Lynch FL. Predicting suicide attempts and suicide deaths following outpatient visits using electronic health records. Am J Psychiatry. (2018) 175(10):951–60. doi: 10.1176/appi.ajp.2018.17101167

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Lyons VH, Haviland MJ, Azrael D, Adhia A, Bellenger MA, Ellyson A, et al. Firearm purchasing and storage during the COVID-19 pandemic. Inj Prev. (2021) 27:87–92. doi: 10.1136/injuryprev-2020-043872

PubMed Abstract | CrossRef Full Text | Google Scholar

Appendix A

Sample Survey Questions

Definitions (Appear at the top of each page)

Medications: refers to any medications in the home, including prescription and over-the-counter medications, regardless of who owns or uses them.

Firearm: refers to any firearms in the home, including hand guns and long guns (such as rifles, shotguns, submachine guns, etc.), regardless of who owns the gun or its working condition.

Provider: generally refers to any health or mental health professional. This includes medical doctors (MD) or doctors of osteopathy (DO) in any medical specialty including primary care, endocrinology, cardiology, etc. A provider may also be a psychiatrist, psychologist, mental health therapist, social worker, nurse practitioner (NP) or physician's assistant (PA).

Survey introduction

The survey should take less than 10 min to complete. At the end of the survey, you will have the chance to enter a drawing for a $100 Amazon gift card. This survey is anonymous, which means we cannot link your answers back to you. We appreciate your time! There are no right or wrong answers.

Before beginning, please tell us a few things to help us understand the entire group of people responding to this survey.

Which category below includes your age?

- 18–24 years old

- 25–44 years old

- 45–64 years old

- 65–89 years old

- Prefer not to answer

What gender do you identify with? Choose all that apply.

- Female

- Male

- Transgender man

- Transgender woman

- Other (please specify)

- Prefer not to answer

What is your race? Choose all that apply.

- American Indian

- Asian

- Black

- Native Hawaiian

- White

- Other

- Unknown

- Prefer not to answer

Are you of Hispanic, Latino, or Spanish origin (ethnicity)?

- Yes

- No

- Unknown

- Prefer not to answer

What is the highest level of school you have completed?

- Some high school

- High school graduate or GED equivalent Some college

- Trade/technical training

- Bachelor’s degree or higher

- Prefer not to answer

Instructions

This next set of questions is about how medications and firearms are currently stored in your home.

How much do you agree or disagree with the following statements? (Precontemplation)

Strongly agree: Strongly Disagree (4-point likert scale)

- It is important to securely store medications when I or someone in my household has had recent thoughts of suicide.

- It is important to securely store other potentially dangerous household items (such as knives, scissors, razor blades, ropes, or chemicals) when I or someone in my household has had recent thoughts of suicide.

- It is important to securely store firearms when I or someone in my household has had recent thoughts of suicide.

- It is always important to store firearms securely in a locked space, or with a cable/trigger lock, regardless of suicide risk for someone in the home.

- It is always important to securely store firearms unloaded regardless of suicide risk for someone in the home.

B6.) Are there any restrictions to medication access in your home for you or other members of your household? Choose the best option. B6

• Yes

• No

• I’m not sure

• Prefer Not to Answer

B6a.) Please choose the options that best describes medication storage in your home. (Check all that apply)

• Medications are locked up or secured and cannot be easily accessed

• Medications have been removed from the home

• A doctor has reduced the number of pills available for pick up at one time

• A family member or friend is now managing medications

• Unused medications have been thrown away

• Prefer not to answer

B6d.) Have you started thinking about ways to store medications differently (even if you haven't made any changes yet)? (Yes, No, Prefer not to answer)

B6e.) Are you planning to change how medications are stored in your home, such as finding or ordering a secure storage device, setting aside time to collect all medications, setting up time to talk with a trusted friend or family member about helping with medication management (even if you haven't made any changes yet)? (Yes, No, Prefer not to answer)

If Yes….

B6e1.) Have you thought about any of the following changes? Check all that apply.

○ Gathering medications and putting them in a locked cabinet, safe or lockbox.

○ Throwing away (or getting rid) of old or unused medications

○ Asking a trusted family member or friend to help manage medications

○ Asking for smaller amounts of prescription medications from medical providers

○ Other (please describe)

○ Prefer Not to Answer

REMINDER: Information about your participation on the website and responses to the survey are completely anonymous. We cannot link this survey back to you in any way.

B7) Are you able to access a firearm(Yes, No, I’m Not Sure, Prefer Not to Answer). Choose the best option.

B7a.) Please choose the options that best describe your firearm access (Check all that apply)

- There are no firearms in my home or anywhere else where I regularly spend time (e.g., friend or family's house, workplace).

- Firearms are locked-up (gun safe, cabinet, etc.) and cannot be accessed.

- Firearms have a locking device and cannot be accessed.

- Firearms have been moved to a different location outside the home that cannot be accessed.

- Firearms are unloaded and ammunition cannot be accessed.

- I am not sure where the firearms are located in my home, so I do not know how they are stored.

- Prefer Not to Answer

B7c.) Have you started to think about ways to store firearms differently (even if you haven't made any changes yet)? (Yes, No, Prefer not to answer)

B7d.) Have you started planning changes to how firearms are stored in your home, such as: talking with a friend/family about storing your firearms, finding or ordering a new locking device or setting aside time to unload firearms (even if you haven't made any changes yet)? (Yes, No, Prefer not to answer)

If Yes….

B7e1.) Have you thought about any of the following changes? Check all that apply.

○ I have thought about purchasing locking devices

○ I have thought about a safe or lockbox

○ I have thought about temporarily giving my firearm to a trusted family member or friend for safekeeping

○ I have thought about temporarily storing my firearms at a gun store, range, or other private business

○ I have thought about temporarily storing my firearms at a law enforcement agency

○ Other (please describe)

○ Prefer Not to Answer

B7e2.) Did any of the following help you to think about different ways firearms could be stored in the home? (Check all that apply)

○ Information from the Lock to Live website

○ Talking with a Kaiser Permanente provider

○ Talking with a provider outside of Kaiser Permanente

○ Talking with a friend or family member

○ Other (please describe)

○ None of the above

○ Prefer not to answer