Development and user testing of gene pilot: an electronic health decision support tool for Black cancer patients about tumor genomic profile testing

Introduction: Tumor genomic profiling (TGP) is used to optimize cancer treatment but is underutilized by Black patients, despite having disproportionately higher cancer morbidity and mortality. No interventions using electronic health decision support tools (eHealth DSTs) have been developed to assist patients with understanding this test or address barriers to uptake and communicating preferences with a doctor.

Methods: Using the Ottawa Decision Support Framework, we systematically developed the Gene Pilot eHealth DST with Black cancer patients. We conducted qualitative focus groups (five groups, N = 33) and surveys (N = 121), elicited community advisory board feedback (N = 10) to devise DST content and communication strategies, and then conducted user testing (N = 10). Content was informed by commercial marketing techniques - segmentation, perceptual mapping, vector message modeling – to elucidate how medical mistrust was an important construct to address in Gene Pilot.

Results: User testing (1–7 scale) indicated Gene Pilot was highly accepted, including ease of use (M = 6.67, SD = 0.50), that it addressed important barriers such as medical mistrust and genetic literacy (6.63, SD = 0.52), and allowed patients to prepare for the decision (M = 6.44, SD = 0.73) and to communicate with their doctor (M = 6.33, SD = 0.73). Suggestions for improved navigability were addressed.

Conclusion: Overall, Gene Pilot was found to be acceptable, suggesting its readiness for efficacy testing.

1 Introduction

For the past 30 years, precision medicine through the use of genetic testing has enabled improved diagnosis, treatment, and outcomes for cancer patients (Beaubier et al., 2019; Cobain et al., 2021; Lee et al., 2019) by uncovering somatic pathogenic variants or mutations that can be targeted through available therapies (Chung et al., 2019; Pishvaian et al., 2020). As a result, genetic tests like tumor genomic profiling (TGP) are now standard of care across many cancer types (Akhoundova and Rubin, 2022; Tjota et al., 2024).

However, accessing tests like TGP remains a challenge, with testing rates far below the current testing guidelines (Lynch et al., 2018). Though access issues are partly caused by institutional barriers, including whether the location has ongoing clinical trials and specialists who are confident in ordering, interpreting, and explaining what the test means to their patients (Anderson et al., 2021; de Moor et al., 2020; Levit et al., 2019), there is also a significant gap in which patients are offered or agree to the test. Living in a low-income household, having Medicaid insurance, and having lower educational attainment are all correlated with lower likelihood of receiving genetic testing as part of cancer care (Lynch et al., 2018; Palazzo et al., 2019; Zhang et al., 2020). But race and ethnicity are also important. Use of TGP has occurred mostly in non-Hispanic, White cancer patients (Palazzo et al., 2019; Punekar et al., 2021). Black cancer patients undergo TGP at significantly lower rates, even when controlling for sociodemographic and other factors (Lynch et al., 2018).

General barriers to uptake of TGP include worries about the test, such as cost or potential discomfort, or other psychosocial barriers, such as worries about results and potential secondary hereditary findings like susceptibility to other cancers (Hann et al., 2017; Yushak et al., 2016). Many genetic tests, including TGP, are complex, requiring physicians to be able to adequately interpret and explain results (Anderson et al., 2021; de Moor et al., 2020), but formal training in how to communicate the complexities of genetic testing results with patients often does not occur (Hall et al., 2021). As a result, physicians may be less likely to discuss TGP testing with their patients, thereby reducing opportunity for patients to understand genetics and how inheritable risk may shape their cancer treatment options (Best et al., 2019; Hall et al., 2024). This is especially problematic if the patient has low genetic literacy. Lower levels of genetic literacy are associated with health disparities and have been shown to be more prevalent in racial and ethnic minority patients (Dumenci et al., 2014; Kanu et al., 2021). Though limited research has been done on barriers to TGP in racial and ethnic minority cancer patients, genetic literacy has been suggested to be a significant barrier because interpreting TGP requires at least some knowledge of genetics (Blanchette et al., 2014; Hann et al., 2017).

Medical mistrust, which stems from a history of past and ongoing systematic injustices such as the Tuskegee Syphilis Experiment and the Henrietta Lacks incident (Hall et al., 2022; Rogers et al., 2018), is another important potential barrier to uptake of TGP. Emergent research has found a relationship between medical mistrust and negative perceptions of and likely use of TGP in Black cancer patients (Hoadley et al., 2022), as it relates to concerns about privacy, stigmatization, and discrimination based on results of genetic tests (Hann et al., 2017). Similarly, specific experiences within the healthcare system, such as race-based discrimination, have been shown to be associated with Black patients avoiding seeking care (Hall et al., 2022), which also may be important to understanding the disparity in TGP testing.

Maintaining patient autonomy in navigating genetic testing decisions and how results are used in treatment and care is important to potentially mitigate genetic literacy and medical mistrust among Black patients and increase uptake of an important diagnostic tool like TGP. To help increase self-efficacy and facilitate informed decision making around genetic testing, culturally-sensitive decision support tools (DSTs) would be important to use to improve uptake of genetic testing as part of care (Kanu et al., 2021; Muvuka et al., 2020). While current systems do exist as clinical support tools to assist in the interpretation of TGP results (Perakis et al., 2020), no DSTs aimed specifically at patients have been tested, nor any that are aimed to address barriers that may exist for Black cancer patients. DSTs, including electronic health (eHealth) tools with interactive features (e.g., video, embedded questions, voice over) are attractive because of broad access to the Internet via smart phones and other devices and their effectiveness in vulnerable patient groups (Kruse and Ehrbar, 2020). Based on the International Patient Decision Aid Standards (IPDAS) checklist, DSTs should go through a systematic development process, provide information on options, help clarify values, use patient stories, and use plain language (among others) (Elwyn et al., 2009). DSTs that follow these standards have been shown to improve decision-making outcomes by increasing knowledge, improving accuracy of risk perceptions, enhancing values clarification and reducing decisional conflict (Kruse and Ehrbar, 2020). They can also assist with communicating preferences or focusing questions to the patients’ needs.

Building on Kreps’ best practices approach to health communication (Kreps and Thornton, 1992), we used an iterative strategy to develop a theory-informed eHealth DST called Gene Pilot that involved Black cancer patients at each stage of development. We used the Ottawa Decision Support Framework (ODSF) (Stacey et al., 2020) to guide development. The ODSF recognizes that poorly informed decisions result when knowledge is low and preparation is incomplete. According to ODSF, there are several determinants of sub-optimal health decisions that may be modified by decision support, including: (1) perceptions of the decision (e.g., low knowledge, unrealistic expectations, unclear values, decisional conflict); (2) perceptions of others (e.g., biased/limited perceptions in others’ opinions/practices); and (3) personal and external resources to make the decision (e.g., limited decision making skills, personal risk perceptions) (Stacey et al., 2020). Many of these are consistent with the barriers to adequately communicate about TGP to Black cancer patients.

The primary aim of this study was to understand perceptions of TGP and barriers to uptake among Black cancer patients and integrate findings into the development of Gene Pilot. We used commercial marketing techniques, including segmentation, perceptual mapping, and vector message modeling, to pinpoint persuasive messaging and to create a tool with high salience by providing information that addresses medical mistrust, fears about genetic information sharing, and suggestions for how to discuss these issues with a doctor using the ODSF constructs. The objective was to create a tool that not only increased genetic literacy and addressed concerns of Black cancer patients related to TGP, but could increase self-efficacy, lower decisional conflict, and increase intention to have TGP as part of their cancer treatment plan. The following outlines the formative work and user testing process to create Gene Pilot.

2 Methods

Using a formative evaluation framework, we worked with a Community Advisory Board (CAB) throughout the process (N = 10) who provided input on everything from survey content to messaging and layout of Gene Pilot. The formative work included qualitative focus groups (five groups; N = 33) and surveys (N = 121) with Black cancer patients and then conducting segmentation analysis, perceptual mapping and vector message modeling to create a targeted message strategy. We then developed “mood boards” and concept tested them with the CAB and developed a prototype of the web-based application. We then conducted user testing (N = 10) of Gene Pilot to assess for feasibility and acceptability before finalizing (Figure 1). All research methods were approved by the Fox Chase Cancer Center Institutional Review Board. Following reviews the formative work to develop the tool and the results from the user testing that was completed.

Figure 1

Figure 1. Formative evaluation process for the development of Gene Pilot.

2.1 Formative work

2.1.1 Focus groups

Five in-person focus groups were conducted with Black cancer patients (N = 33) from an NCI-designated cancer center and an affiliated oncology unit in an urban safety-net hospital located in Philadelphia. Eligibility criteria were: (1) self-identified as Black; (2) 18 years or older; (3) diagnosed with a solid tumor cancer. Both patients who had and had not had TGP were eligible to participate. Specific methods and results are published elsewhere (Luck et al., 2024).

Participants were mostly female (81.8%), and one-third had some college education; mean age was 57 years (SD = 11.35); 33.3% reported never having heard of TGP, and 48.5% were not aware of having had TGP as part of their cancer treatment. Qualitative analysis was guided by the principles of applied thematic analysis and yielded five themes: (1) mistrust of medical institutions spurring independent health-information seeking; (2) genetic testing results as both empowering and overwhelming; (3) how provider-patient communication can obviate medical mistrust; (4) how unsupportive patient-family communication undermines interest in secondary-hereditary risk communication; and (5) importance of developing centralized patient support systems outside of treatment decisions. Results of focus groups were used to inform the content of a survey.

2.1.2 Survey

Black cancer patients (N = 121) from the same settings as focus groups were invited to complete surveys. Patients were identified by electronic health records as eligible and invited to participate via a letter to complete an electronic survey or approached in the waiting room or treatment areas and asked to take the survey in-person via an electronic tablet or on paper. Eligibility was the same as the focus groups with one addition; only those who indicated they had not had TGP or were unaware of having had TGP were included, to ensure messaging was relatable to patients as they were making a decision. Survey items queried about sociodemographic characteristics, clinical factors, patient-oncologist relationship quality, medical mistrust, knowledge and attitudes about genetics, and barriers/facilitators to having TGP. Specific methods and information on results is published elsewhere (Hoadley et al., 2022).

Overall, 112 surveys had complete data. Participants were mostly female (77.8%) but were evenly distributed by education level (36.6% high school graduate or less, 35.7% college graduate or more) and annual household income (21.1% $10,000 or less, 21.1% $75,000 or more), with the rest distributed among the other categories. Mean age was 58.7 years (SD 10.3). Analyses included chi-squared tests of independence to examine nominal variables and independent samples t-tests to evaluate continuous variables by demographic characteristics. Results indicated that medical mistrust was an important variable to investigate, which was measured with 10 items informed by both the focus groups and the Group-Based Medical Mistrust Scale (Shelton et al., 2010). Items measured race-based medical and institutional mistrust, including mistrust of science, researchers, pharmaceutical companies, government and regulatory bodies, and insurance companies. Participants indicated their agreement on an 11-point scale (zero = totally disagree to 10 = totally agree).

A k-means cluster analysis was then conducted to discern if differences in perceptions about TGP existed by group. The cluster analysis indicated two distinct clusters—those with low medical mistrust (n = 44, 39.3%) and those with high medical mistrust (n = 68, 60.7%). Figure 2 illustrates differences in means across the 10 items, all of which are statistically significant. Those with higher mistrust reported greater concerns about TGP, including cost, insurance discrimination, sharing of results with others, and physical discomfort of the test. They were also more concerned that their provider could not adequately communicate the results of the test in a way they would understand. Additional in-depth analysis is provided elsewhere (Hoadley et al., 2022).

Figure 2

Figure 2. Cluster segments of black cancer patients by mean levels of medical mistrust.

2.1.2.1 Segmentation

The results of the cluster analysis served as a segmentation of the group to understand potential need for different messaging. Segmentation is an important marketing technique that defines subgroups with similar characteristics to understand nuanced public health perceptions and behavioral decisions (Kubacki et al., 2016) and to support the development of targeted messaging (Donovan and Henley, 2010; Lotenberg et al., 2011). Most uses of segmentation in public health communication involve larger scale social marketing campaigns that segment diverse populations, but segmentation can also be used in a seemingly homogeneous group to identify important attitudinal or behavioral variations (i.e., psychographic differences) that may exist to create potential subgroups for targeted messaging (Gordon et al., 2014).

2.1.2.2 Perceptual mapping and vector message modeling

After using segmentation analysis to characterize the group by medical mistrust, perceptual mapping and vector message modeling were used to understand perceptions of TGP in Black cancer patients. Perceptual mapping and vector message modeling are used in marketing and advertising but are rarely used in public health communication. Based on Woelfel and Fink’s Galileo approach (Woelfel and Fink, 1980), the process uses multi-dimensional scaling analysis to produce three dimensional models. The maps show how Black cancer patients perceive TGP, including barriers and facilitators to getting tested, beliefs about genetic privacy, their relationship with their oncologist, and other important constructs. Our computer software converts scaled judgments into distances, which are derived from the inter-item correlations of the variables. Kruskal’s procedure is used to minimize the cost function called “stress,” a measure of the lack of fit between dissimilarities and distances among variables (Buja et al., 2008). The authors have extensive experience using this method in health communication research (Bass et al., 2013; Bass et al., 2018; Bauerle Bass et al., 2018).

Once perceptual maps are constructed, vector message modeling is used to understand how to move the group in the space toward the desired decision—in this case, intent to have TGP (lines/arrows in Figures 3, 4). This modeling allows us to understand which variables would be most persuasive in a message to pull the group toward the decision. One can then compare maps by segments to see if messaging is different or if one strategy can be employed to represent all groups. For example, in the maps on patient concerns about TGP (Figure 3), those with high medical mistrust would need messages addressing their worries about what the TGP results would be, concerns that the oncologist would not be able to explain the results and worries that the results would be used without them knowing. Importantly, those with low mistrust would also respond to these messages. In the maps that address worries about discrimination and use of genetic information (Figure 4), similar message strategies to move both groups toward intention are indicated, including messages addressing worries that genetic testing is being done just to “get money out of me,” that minorities will be discriminated against with this type of testing, and that genetic information would not be kept private.

Figure 3

Figure 3. Perceptual maps—patient concerns by medical mistrust.

Figure 4

Figure 4. Patient worries about genetic information/testing by medical mistrust.

2.2 Message strategy and tool development

Table 1 outlines the vector message modeling for the two groups by the survey items related to TGP testing. Because messaging was similar across groups, it was determined that tailored messaging to medical mistrust segments was not needed. Six themes emerged from the data and “mood boards” were created (Figure 5) to plan Gene Pilot. These were brought to the CAB for feedback before tool development began. One standout suggestion was that the eHealth DST should be multicultural to avoid the perception that the tool was “targeting” Black patients. This feedback was integrated into the development process as visualizations were developed.

Table 1

Table 1. Message strategies by medical mistrust cluster.

Figure 5

Figure 5. Sample “mood board” for Gene Pilot eHealth decision support tool-"navigate” Section.

After initial input from the CAB, Gene Pilot was developed in collaboration with a private company specializing in health-related technology products (Dog Town Media). It is web-based so it is easily accessible on any type of device connected to the internet and can be accessed both in the clinic and at home on a private device. Voice over narrates the tool, visual cues aid navigation, and animations are multi-cultural though messages address key issues for Black patients identified in the formative work. It also includes video testimonials of real patients and users have the ability to customize their experience by selecting content that is most applicable to them. The tool is organized into four sections (Figure 6):

Figure 6

Figure 6. Screen shots of Gene Pilot eHealth decision support tool.

Fly—provides an overview of TGP testing, secondary hereditary risk, options for testing and knowing results. Presented as short animations to move through.

Glide—provides six patient “personas” from different cultural backgrounds, types of cancers, and attitudes about TGP. Users can pick which they want to review based on a description of the persona and then watch an animated vignette. The vignettes address barriers to TGP, including fears of discovering secondary hereditary information, worries about discrimination, what will happen to genetic information, and concerns about how to talk to family members about results. Users can view as many vignettes as they like, but two must be reviewed before advancement to the next section is allowed.

Soar—provides testimonial videos from real cancer patients talking about TGP and why they did or did not get the test, how to talk to a doctor about TGP, and other issues related to information gathering and medical mistrust about genetic information. Users can pick which patient videos they would like to watch, but must view at least two before advancement to the next section is allowed.

Navigate—provides a list of topics (e.g., how to communicate results to family, worry about what will happen to genetic information) related to TGP that they can highlight as important to them. Once they have chosen three to five topics, they hit a SUBMIT button and a “summary card” is generated that provides sample questions related to each selected topic. This summary care is a tool to help the user talk to their doctor about the topics chosen. The summary card can be printed or emailed to the user directly from the tool. The section ends with an animation that discusses strategies on how to talk with their doctor about their concerns or questions about TGP.

2.3 User testing with Black cancer patients

Once a prototype tool was developed, user testing with Black cancer patients (N = 10) was completed to assess the extent to which the tool was understandable, how easily it could be navigated, and its relevance. Participants were recruited from the two aforementioned recruitment sites via invitation letters and in-person at the clinic or treatment areas. Participants were given the option of completing the user testing in-person using an electronic tablet (N = 3) or online via Zoom (N = 7). Eligibility criteria were the same as noted above. Each encounter lasted approximately 45 min and was audio recorded. All participants received a gift card as a thank-you for their time.

Participants completed a “think aloud” interview with staff while they interacted with the tool. Staff used a standardized moderator’s guide to get input on each section of the tool. The moderator’s guide included questions to be asked following each of the four Gene Pilot sections to gage response to content, graphics, voice over and other items. Staff also documented any navigation issues, comments, and concerns about user experience in an observational note section of the guide. Participants also completed a brief survey (18 items) on the usefulness of the tool and its relevance. Participants were asked to rate each item on a 1–7 scale (strongly disagree to strongly agree). Descriptive statistics, including means, standard deviations, and frequencies for the scaled items were run using SPSS v.28.

3 Results

Overall, user testing participants were 77.8% female, 44.4% had a high school education or less, and the mean age was 56 (range 22–67, SD = 7.5). Survey results showed high acceptability and satisfaction with Gene Pilot. Of the 18 items, means ranged from a high of 6.88 (videos of cancer patients and choosing questions to talk with doctor) to a low of 4.0 (belief that the tool was slanted toward convincing them to get TGP). Most items were in the 6.4–6.6 mean range, indicating participants felt all aspects of the tool were acceptable (Table 2).

Table 2

Table 2. Satisfaction and accessibility of the gene pilot eHealth decision support tool (N = 9).

Qualitative analysis for the 10 sessions was summarized for each part of the tool. Consensus for “Fly” was that it was clear, easily navigable, and provided good information. Others noted that the logo showed a person that appeared “strong and confident” and that the voiceover was appropriately paced. In responding to the animations, most participants enjoyed them and thought the images were clear, although one user felt they moved too quickly. “Now, the animations are nice, they are pleasing to the eye… And I think it’s kind of cool to have the airplane with the cloud. So, you are kind of the pilot and in the pilot seat” (Participant 6). All participants were positive about the content and felt it was easily understandable and addressed concerns. “I appreciate it, being forthcoming with what the possible risks could be, as far as information and what could be revealed and how it might impact the person and how they are feeling” (Participant 1). Another noted how positive they felt it was to see different types of people. “I thought it was cool to show different families, different settings. The slide before this one showed that they were sad. So that was kind of relatable. I think the visuals are, well, you have a very diverse group of people. There’s all different ethnicities. I think that’s pretty cool. And sex, too. And age” (Participant 4). General suggestions for improvement included making instructions larger for those who might not have experience using computers and having animations automatically play after touching or clicking a button, without having to click again on the video. There was also a suggestion to have transcripts of audio provided as a drop-down menu. All of these suggestions were incorporated into the final tool.

Feedback for Glide was also positive, with participants noting that the personas felt “relatable.” “You know what I thought was really cool about this, about James [persona who discusses worries about being discriminated against and distrust of the healthcare system]…I may not think exactly like him, but I understood where James is coming from. And I just appreciate that fact that [you] took the time to take that into consideration” (Participant 2). Another participant noted feeling that the personas covered all the important issues. They said, “You did the costs, the discrimination part, you did about you know, who gets your information. And it’s funny. I remember asking those questions” (Participant 5). Suggestions to improve this section included changing some characters in the animations to appear more open (not having folded arms), ensuring that healthcare team members were represented from a variety of ethnicities, and being explicit on how to close out a window beyond an “X” box. All suggestions were used to modify the final tool.

In the Soar section, participants felt that hearing from patients was valuable and that there is “nothing like firsthand experience.” One participant said, “So having someone that speaks like me, looks like me, you know, or someone I could see a relative resemblance or, you know, just reminds me of somebody that I trust is important” (Participant 1). Suggestions included having the option to have closed captions as well as displaying the information about the patient next to their picture so that the user did not have to click on it to get information. These were integrated into the final version.

Finally, participants felt the Navigate section of the tool was helpful in providing a baseline of questions to talk to a doctor about. It was described as “helpful,” “thorough” and “hitting the right things.” One participant said, “It’s pretty cool. And I think it has like a really positive vibe for someone dealing with a really serious topic” (Participant 2). Another noted the usefulness of the summary card, saying “I love how you did the summary card with the questions up to the sides…and then the answers to questions that you need answered” (Participant 5). Participants also liked the option to have the questions emailed. Suggestions for this section included explicit instructions on how many topics to highlight to improve understanding of how to produce the summary card and to have space at the bottom of the card to add user-generated questions that may not have been on the predetermined list. These suggestions were also integrated into the final version.

Overall, participants felt Gene Pilot provided information in an easy to access way to present information on TGP and genetic testing and included the key areas that patients would want to know about. One participant said, “TGP just sounds like an initial and then when you hear all these people that are going to be involved with you, on top of your doctors, it’s mumbo jumbo. Like you do not know what they are talking about. You do not have a clue. So yeah, this [Gene Pilot]… brings it down to like the basic person level as opposed to the science level” (Participant 4).

4 Discussion

Development of the Gene Pilot eHealth decision support tool included iterative formative evaluation that incorporated the voices of the target audience at every stage of development. The user testing illustrated the utility of our iterative development process and formative work, with overall positive feedback about all aspects of the tool, including content, look/feel, and ability to customize it to the user’s interests. In addition, user testing indicated that it was easy to use and the use of clear communication principles, including voiceover and easy navigation, mitigated most potential problems with health or digital literacy (Lin and Bautista, 2017), a key component in the IPADS guidelines for creating eHealth tools (Elwyn et al., 2009). Importantly, all suggestions for improvement were related to navigability. No participants took issue with any content or messaging, indicating the formative work provided salient messages related to the concerns of Black cancer patients. To date, no other eHealth decision tool has been developed to address TGP in Black cancer patients, so collaborating with these patients to create this well-received tool to meet the decisional needs of the target population is a promising step toward addressing racial disparities in uptake of TGP.

Using the ODSF was an important way to ground the development of the tool. Each section of Gene Pilot addressed the core components of this framework, including providing information to increase knowledge and clarify values, provide balanced information for the patient to make an informed decision that fits with those values, and facilitate communication preferences to healthcare providers (Stacey et al., 2017). Importantly, the user testing supported that these components were welcome and favored. Participants noted that the tool not only provided information in a way they could understand but allowed them to explore different aspects of the decision that could be important to them, such as addressing medical mistrust or how results might impact them and their family. There was also enthusiasm for the section that allowed them to choose relevant topics and to generate example questions they could print or email to have with them to talk to their doctor. Clearly, using ODSF constructs was perceived of as helpful by this patient group. Recent systematic reviews of studies that have used ODSF indicated that decision tools or aids using ODSF were superior to usual care in improving decisional quality and reducing decisional needs (Hoefel et al., 2020a; Hoefel et al., 2020b; Stacey et al., 2020).

Besides using best IPADS practices and following the guidance of the ODSF, a significant innovation was the use of novel analysis methods. By using segmentation, perceptual mapping, and vector message modeling, we were able to explore how a group of demographically similar cancer patients differed in perceptions of and intention to use TGP by a psychographic variable. The role of medical mistrust and its association to feelings about genetic testing was an important finding that informed how to talk about TGP and allowed patients to feel justified in feeling mistrust. Interestingly, despite different levels of mistrust in the sample, vector message modeling results showed similar messaging strategies, indicating the need to address mistrust for all patients. User testing indicated that these message strategies were informational, appropriately worded, and highly relevant. The use of these methods is unique to the development of public health communication interventions and have been shown in other studies by the authors to be effective (Jessop et al., 2020; Ruzek et al., 2016). While user testing indicated the message strategy is promising, the next step of this research will be to test Gene Pilot in a randomized clinical trial with Black cancer patients to assess effect on decisional conflict, genetic literacy, and communication of preferences with a provider.

4.1 Limitations

All formative work and user testing was done with Black cancer patients from two urban clinical locations, although these sites differ in sociodemographic characteristics with one being a suburban cancer center with a primarily privately insured patient population and the other a clinic within a large safety-net hospital with a primarily Medicaid/Medicare patient population. The opinions and attitudes toward TGP may differ than other geographic locations. The population was also primarily female, which may have biased the results. It may be also possible that desirability bias affected the opinions of user testing participants such that there was reluctance to share critical feedback of Gene Pilot. However, we did receive useful suggestions for tool improvement related to usability, minimizing concerns about this potential limitation. The sample size for user testing was small and it is possible that other issues may have been found with more people. However, most guidelines for usability testing indicate that most usability problems are detected with the first three to five subjects (Turner et al., 2006; Sauro and Lewis, 2016). Since this tool was relatively easy and linear, the sample size more than likely uncovered the main navigability issues. Future research should test messages and tool components with a broader population of patients.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by the Fox Chase Cancer Center Institutional Review Board. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

SB: Conceptualization, Formal analysis, Funding acquisition, Methodology, Project administration, Supervision, Writing – original draft. YC: Data curation, Investigation, Supervision, Writing – review & editing. JB: Investigation, Supervision, Writing – review & editing. PK: Investigation, Project administration, Supervision, Writing – review & editing. A-MR: Investigation, Writing – review & editing. PD’A: Formal analysis, Investigation, Writing – review & editing. AH: Formal analysis, Investigation, Writing – review & editing. CL: Formal analysis, Investigation, Writing – review & editing. KS: Investigation, Project administration, Supervision, Writing – review & editing. MH: Conceptualization, Funding acquisition, Methodology, Project administration, Writing – review & editing.

Funding

The author(s) declare that financial support was received for the research and/or publication of this article. This study was funded by the American Cancer Society, Health Equity Grant #131652-18-024-01-CPPB (MH and SB, MPIs).

Acknowledgments

Thank you to all patients who contributed to this research.

Conflict of interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Generative AI statement

The authors declare that no Gen AI was used in the creation of this manuscript.

Publisher’s note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

Akhoundova, D., and Rubin, M. A. (2022). Clinical application of advances multi-omics tumor profiling: shaping precision oncology of the future. Cancer Cell 40, 920–938. doi: 10.1016/j.ccell.2022.08.011

PubMed Abstract | Crossref Full Text | Google Scholar

Anderson, E. C., Hinton, A. C., Lary, C. W., Fenton, A. T. H. R., Antov, A., Edelman, E., et al. (2021). Community oncologists’ perceptions and utilization of large-panel genomic tumor testing. BMC Cancer 21:1273. doi: 10.1186/s12885-021-08985-0

Crossref Full Text | Google Scholar

Bass, S. B., Gordon, T. F., Ruzek, S. B., Wolak, C., Ruggieri, D., Mora, G., et al. (2013). Developing a computer touch-screen interactive colorectal screening decision aid for a low-literacy African American population: lessons learned. Health Promot. Pract. 14, 589–598. doi: 10.1177/1524839912463394

PubMed Abstract | Crossref Full Text | Google Scholar

Bass, S. B., Jessop, A., Maurer, L., Gashat, M., Al Hajji, M., and Gutierrez, M. (2018). Mapping the barriers and facilitators of HCV treatment initiation in methadone maintenance therapy patients: implications for intervention development. J. Health Commun. 23, 117–127. doi: 10.1080/10810730.2017.1414902

PubMed Abstract | Crossref Full Text | Google Scholar

Bauerle Bass, S., Jessop, A., Gashat, M., Maurer, L., Alhajji, M., and Forry, J. (2018). Take charge, get cured: the development and user testing of a culturally targeted mHealth decision tool on HCV treatment initiation for methadone patients. Patient Educ. Couns. 101, 1995–2004. doi: 10.1016/j.pec.2018.07.007

PubMed Abstract | Crossref Full Text | Google Scholar

Best, M. C., Bartley, N., Jacobs, C., Juraskova, I., Goldstein, D., Newson, A. J., et al. (2019). Patient perspectives on molecular tumor profiling: “why wouldn’t you?”. BMC Cancer 19:753. doi: 10.1186/s12885-019-5920-x

Crossref Full Text | Google Scholar

Buja, A., Swayne, D. F., Littman, M. L., Dean, N., Hofmann, H., and Chen, L. (2008). Data Visualization With Multidimensional Scaling. J. Comput. Graph. Stat. 17, 444–472. doi: 10.1198/106186008X318440

Crossref Full Text | Google Scholar

Beaubier, N., Bontrager, M., Huether, R., Igartua, C., Lau, D., Tell, R., et al. (2019). Integrated genomic profiling expands clinical options for patients with cancer. Nat Biotechnol 37, 1351–1360. doi: 10.1038/s41587-019-0259-z

Crossref Full Text | Google Scholar

Blanchette, P. S., Spreafico, A., Miller, F. A., Chan, K., Bytautas, J., Kang, S., et al. (2014). Genomic testing in cancer: patient knowledge, attitudes, and expectations. Cancer 120, 3066–3073. doi: 10.1002/cncr.28807

Crossref Full Text | Google Scholar

Chung, J. H., Dewal, N., Sokol, E., Mathew, P., Whitehead, R., Millis, S. Z., et al. (2019). Prospective comprehensive genomic profiling of primary and metastatic prostate tumors. JCO Precis. Oncol. 3, 1–23. doi: 10.1200/PO.18.00283

PubMed Abstract | Crossref Full Text | Google Scholar

Cobain, E. F., Wu, Y., Vats, P., Chugh, R., Worden, F., Smith, D. C., et al. (2021). Assessment of clinical benefit of integrative genomic profiling in advanced solid tumors. JAMA Oncol 7, 525–533. doi: 10.1001/jamaoncol.2020.7987

Crossref Full Text | Google Scholar

de Moor, J. S., Gray, S. W., Mitchell, S. A., Klabunde, C. N., and Freedman, A. N. (2020). Oncologist confidence in genomic testing and implications for using multimarker tumor panel tests in practice. JCO Precis. Oncol. 4, 620–631. doi: 10.1200/PO.19.00338

PubMed Abstract | Crossref Full Text | Google Scholar

Donovan, R., and Henley, N. (2010). Principles and practice of social marketing: An international perspective. Cambridge: Cambridge University Press.

Google Scholar

Dumenci, L., Matsuyama, R., Riddle, D. L., Cartwright, L. A., Perera, R. A., Chung, H., et al. (2014). Measurement of Cancer health literacy and identification of patients with limited Cancer health literacy. J. Health Commun. 19, 205–224. doi: 10.1080/10810730.2014.943377

PubMed Abstract | Crossref Full Text | Google Scholar

Elwyn, G., O’Connor, A. M., Bennett, C., Newcombe, R. G., Politi, M., Durand, M.-A., et al. (2009). Assessing the quality of decision support technologies using the international patient decision aid standards instrument (IPDASi). Plo S One 4:E4705. doi: 10.1371/journal.pone.0004705

PubMed Abstract | Crossref Full Text | Google Scholar

Gordon, T. F., Bass, S. B., Ruzek, S. B., Wolak, C., Rovito, M. J., Ruggieri, D. G., et al. (2014). Developing a typology of African Americans with limited literacy based on preventive health practice orientation: implications for colorectal Cancer screening strategies. J. Health Commun. 19, 1259–1277. doi: 10.1080/10810730.2013.872725

PubMed Abstract | Crossref Full Text | Google Scholar

Hall, M. J., D’Avanzo, P., Chertock, Y., Brajuha, J., and Bass, S. B. (2021). Oncologists’ perceptions of tumor genomic profiling and the communication of test results and risks. Public Health Genomics 24, 304–309. doi: 10.1159/000517486

PubMed Abstract | Crossref Full Text | Google Scholar

Hall, M. J., D'Avanzo, P., Chertock, Y., Kelly, P. J. A., Brajuha, J., Singley, K., et al. (2024). Oncologists' perceptions of tumor genomic profiling and barriers to communicating secondary hereditary risks to African American patients. BMC Cancer 24:412. doi: 10.1186/s12885-024-12184-y

PubMed Abstract | Crossref Full Text | Google Scholar

Hall, O. T., Jordan, A., Teater, J., Dixon-Shambley, K., McKiever, M. E., Baek, M., et al. (2022). Experiences of racial discrimination in the medical setting and associations with medical mistrust and expectations of care among black patients seeking addiction treatment. J. Subst. Abus. Treat. 133:108551. doi: 10.1016/j.jsat.2021.108551

Crossref Full Text | Google Scholar

Hann, K. E. J., Freeman, M., Fraser, L., Waller, J., Sanderson, S. C., Rahman, B., et al. (2017). Awareness, knowledge, perceptions, and attitudes towards genetic testing for cancer risk among ethnic minority groups: a systematic review. BMC Public Health 17:503. doi: 10.1186/s12889-017-4375-8

Crossref Full Text | Google Scholar

Hoadley, A., Bass, S. B., Chertock, Y., Brajuha, J., D’Avanzo, P., Kelly, P. J., et al. (2022). The role of medical mistrust in concerns about tumor genomic profiling among black and African American Cancer patients. Int. J. Environ. Res. Public Health 19:2598. doi: 10.3390/ijerph19052598

PubMed Abstract | Crossref Full Text | Google Scholar

Hoefel, L., Lewis, K. B., O’Connor, A., and Stacey, D. (2020a). 20th anniversary update of the Ottawa decision support framework: part 2 subanalysis of a systematic review of patient decision aids. Med. Decis. Mak. 40, 522–539. doi: 10.1177/0272989X20924645

PubMed Abstract | Crossref Full Text | Google Scholar

Hoefel, L., O’Connor, A. M., Lewis, K. B., Boland, L., Sikora, L., Hu, J., et al. (2020b). 20th anniversary update of the Ottawa decision support framework part 1: a systematic review of the decisional needs of people making health or social decisions. Med. Decis. Mak. 40, 555–581. doi: 10.1177/0272989X20936209

PubMed Abstract | Crossref Full Text | Google Scholar

Jessop, A. B., Bass, S. B., Brajuha, J., Alhajji, M., Burke, M., Gashat, M. T., et al. (2020). “Take charge, get cured”: pilot testing a targeted mHealth treatment decision support tool for methadone patients with hepatitis C virus for acceptability and promise of efficacy. J. Subst. Abus. Treat. 109, 23–33. doi: 10.1016/j.jsat.2019.11.001

PubMed Abstract | Crossref Full Text | Google Scholar

Kanu, C., Brown, C. M., Rascati, K., Moczygemba, L. R., Mackert, M., and Wilfong, L. (2021). Are health literacy and patient activation related to health outcomes in breast Cancer patients? Health Literacy Res. Prac. 5, E171–E178. doi: 10.3928/24748307-20210524-02

PubMed Abstract | Crossref Full Text | Google Scholar

Kreps, G., and Thornton, B. (1992). Health communication theory and practice 1992. Long Grove, IL: Waveland Press Illinois.

Google Scholar

Kruse, C. S., and Ehrbar, N. (2020). Effects of computerized decision support systems on practitioner performance and patient outcomes: systematic review. JMIR Med. Inform. 8:E17283. doi: 10.2196/17283

PubMed Abstract | Crossref Full Text | Google Scholar

Kubacki, K., Rundle-Thiele, S., Pang, B., Carins, J., Parkinson, J., Fujihira, H., et al. (2016). An umbrella review of the use of segmentation in social marketing interventions. Segmentation Soc. Marketing 2016, 9–23. doi: 10.1007/978-981-10-1835-0_2

Crossref Full Text | Google Scholar

Lee, J., Kim, S. T., Kim, K., Lee, H., Kozarewa, I., Mortimer, P. G. S., et al. (2019). Tumor genomic profiling guides patients with metastatic gastric Cancer to targeted treatment: the VIKTORY umbrella trial. Cancer Discov. 9, 1388–1405. doi: 10.1158/2159-8290.CD-19-0442

PubMed Abstract | Crossref Full Text | Google Scholar

Levit, L. A., Kim, E. S., McAneny, B. L., Nadauld, L. D., Levit, K., Schenkel, C., et al. (2019). Implementing precision medicine in community-based oncology programs: three models. J. Oncol. Pract. 15, 325–329. doi: 10.1200/JOP.18.00661

PubMed Abstract | Crossref Full Text | Google Scholar

Lin, T. T. C., and Bautista, J. R. (2017). Understanding the relationships between mHealth apps’ characteristics, Trialability, and mHealth literacy. J. Health Commun. 22, 346–354. doi: 10.1080/10810730.2017.1296508

PubMed Abstract | Crossref Full Text | Google Scholar

Lotenberg, L. D., Schechter, C., and Strand, J. (2011). Segmentation and targeting. SAGE Handb. Soc. Marketing 2011:125. doi: 10.4135/9781446201008.n9

Crossref Full Text | Google Scholar

Luck, C. C., Bass, S. B., Chertock, Y., Kelly, P. J. A., Singley, K., Hoadley, A., et al. (2024). Understanding perceptions of tumor genomic profile testing in black/African American cancer patients in a qualitative study: the role of medical mistrust, provider communication, and family support. J Community Genet. 15, 281–292. doi: 10.1007/s12687-024-00700-3

PubMed Abstract | Crossref Full Text | Google Scholar

Lynch, J. A., Berse, B., Rabb, M., Mosquin, P., Chew, R., West, S. L., et al. (2018). Underutilization and disparities in access to EGFR testing among Medicare patients with lung cancer from 2010—2013. BMC Cancer 18:306. doi: 10.1186/s12885-018-4190-3

PubMed Abstract | Crossref Full Text | Google Scholar

Muvuka, B., Combs, R. M., Ayangeakaa, S. D., Ali, N. M., Wendel, M. L., and Jackson, T. (2020). Health literacy in African-American communities: barriers and strategies. Health Liter. Res. Prac. 4, E138–E143. doi: 10.3928/24748307-20200617-01

PubMed Abstract | Crossref Full Text | Google Scholar

Palazzo, L. L., Sheehan, D. F., Tramontano, A. C., and Kong, C. Y. (2019). Disparities and trends in genetic testing and Erlotinib treatment among metastatic non-small cell lung Cancer patients. Cancer Epidemiol. Biomarkers Prev. 28, 926–934. doi: 10.1158/1055-9965.EPI-18-0917

PubMed Abstract | Crossref Full Text | Google Scholar

Perakis, S. O., Weber, S., Zhou, Q., Graf, R., Hojass, S., Riedl, J. M., et al. (2020). Comparison of three commercial decision support platforms for matching of next-generation sequencing results with therapies in patients with cancer. ESMO Open 5:e000872. doi: 10.1136/esmoopen-2020-000872

PubMed Abstract | Crossref Full Text | Google Scholar

Pishvaian, M. J., Blais, E. M., Brody, J. R., Lyons, E., DeArbeloa, P., Hendifar, A., et al. (2020). Overall survival in patients with pancreatic cancer receiving matched therapies following molecular profiling: a retrospective analysis of the know your tumor registry trial. Lancet Oncol. 21, 508–518. doi: 10.1016/S1470-2045(20)30074-7

PubMed Abstract | Crossref Full Text | Google Scholar

Punekar, S. R., Griffin, M. M., Masri, L., Roman, S. D., Makarov, D. V., Sherman, S. E., et al. (2021). Socioeconomic determinants of the use of molecular testing in stage IV colorectal Cancer. Am. J. Clin. Oncol. 44, 597–602. doi: 10.1097/COC.0000000000000875

PubMed Abstract | Crossref Full Text | Google Scholar

Rogers, C. R., Rovito, M. J., Hussein, M., Obidike, O. J., Pratt, R., Alexander, M., et al. (2018). Attitudes toward genomic testing and prostate Cancer research among black men. Am. J. Prev. Med. 55, S103–S111. doi: 10.1016/j.amepre.2018.05.028

PubMed Abstract | Crossref Full Text | Google Scholar

Ruzek, S. B., Bass, S. B., Greener, J., Wolak, C., and Gordon, T. F. (2016). Randomized trial of a computerized touch screen decision aid to increase acceptance of colonoscopy screening in an African American population with limited literacy. Health Commun. 31, 1291–1300. doi: 10.1080/10410236.2015.1069165

PubMed Abstract | Crossref Full Text | Google Scholar

Sauro, J., and Lewis, J. R. (2016). Quantifying the user experience: Practical statistics for user research. 2nd Edn. Cambridge, MA: Morgan Kaufmann.

Google Scholar

Shelton, R. C., Winkel, G., Davis, S. N., Roberts, N., Valdimarsdottir, H., Hall, S. J., et al. (2010). Validation of the group-based medical mistrust scale among urban black men. JGIM 25, 549–555. doi: 10.1007/s11606-010-1288-y

PubMed Abstract | Crossref Full Text | Google Scholar

Stacey, D., Légaré, F., Boland, L., Lewis, K. B., Loiselle, M.-C., Hoefel, L., et al. (2020). 20th anniversary Ottawa decision support framework: part 3 overview of systematic reviews and updated framework. Med. Decis. Mak. 40, 379–398. doi: 10.1177/0272989X20911870

PubMed Abstract | Crossref Full Text | Google Scholar

Stacey, D., Légaré, F., Lewis, K., Barry, M. J., Bennett, C. L., Eden, K. B., et al. (2017). Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst. Rev. 2017:CD001431. doi: 10.1002/14651858.CD001431.pub5

PubMed Abstract | Crossref Full Text | Google Scholar

Tjota, M. Y., Segal, J. P., and Wang, P. (2024). Clinical utility and benefits of comprehensive genomic profiling in cancer. J. Appl. Lab. Med. 9, 76–91. doi: 10.1093/jalm/jfad091

Crossref Full Text | Google Scholar

Turner, C. W., Lewis, J. R., and Nielsen, J. (2006). Determining usability test sample size. International encyclopedia of ergonomics and human factors, 2006. London: CRC Press, 1–5.

Google Scholar

Woelfel, J., and Fink, E. L. (1980). The measurement of communication processes: Galileo theory and method. New York: Academic press.

Google Scholar

Yushak, M. L., Han, G., Bouberhan, S., Epstein, L., DiGiovanna, M. P., Mougalian, S. S., et al. (2016). Patient preferences regarding incidental genomic findings discovered during tumor profiling. Cancer 122, 1588–1597. doi: 10.1002/cncr.29951

PubMed Abstract | Crossref Full Text | Google Scholar

Zhang, L., Hsieh, M.-C., Petkov, V., Yu, Q., Chiu, Y., and Wu, X.-C. (2020). Trend and survival benefit of Oncotype DX use among female hormone receptor-positive breast cancer patients in 17 SEER registries, 2004–2015. Breast Cancer Res. Treat. 180, 491–501. doi: 10.1007/s10549-020-05557-x

PubMed Abstract | Crossref Full Text | Google Scholar