Building the foundation for a modern patient-partnered infrastructure to study temporomandibular disorders

Background: Conflicting reports from varying stakeholders related to prognosis and outcomes following placement of temporomandibular joint (TMJ) implants gave rise to the development of the TMJ Patient-Led RoundTable initiative. Following an assessment of the current availability of data, the RoundTable concluded that a strategically Coordinated Registry Network (CRN) is needed to collect and generate accessible data on temporomandibular disorder (TMD) and its care. The aim of this study was therefore to advance the clinical understanding, usage, and adoption of a core minimum dataset for TMD patients as the first foundational step toward building the CRN.

Methods: Candidate data elements were extracted from existing data sources and included in a Delphi survey administered to 92 participants. Data elements receiving less than 75% consensus were dropped. A purposive multi-stakeholder sub-group triangulated the items across patient and clinician-based experience to remove redundancies or duplicate items and reduce the response burden for both patients and clinicians. To reliably collect the identified data elements, the identified core minimum data elements were defined in the context of technical implementation within High-performance Integrated Virtual Environment (HIVE) web-application framework. HIVE was integrated with CHIOS™, an innovative permissioned blockchain platform, to strengthen the provenance of data captured in the registry and drive metadata to record all registry transaction and create a robust consent network.

Results: A total of 59 multi-stakeholder participants responded to the Delphi survey. The completion of the Delphi surveys followed by the application of the required group consensus threshold resulted in the selection of 397 data elements (254 for patient-generated data elements and 143 for clinician generated data elements). The infrastructure development and integration of HIVE and CHIOS™ was completed showing the maintenance of all data transaction information in blockchain, flexible recording of patient consent, data cataloging, and consent validation through smart contracts.

Conclusion: The identified data elements and development of the technological platform establishes a data infrastructure that facilitates the standardization and harmonization of data as well as perform high performance analytics needed to fully leverage the captured patient-generated data, clinical evidence, and other healthcare ecosystem data within the TMJ/TMD-CRN.

Introduction

Temporomandibular disorders (TMD) encompass a wide variety of conditions, (e.g., developmental, genetic, inflammatory, degenerative, neoplastic, traumatic, metabolic and idiopathic); affect the temporomandibular joint (TMJ) as well as the surrounding muscles, bones, connective tissue, nerves, and vasculature; and can refer to a specific and heterogenous group that includes myalgia, myofascial pain with referral, arthralgia and headache attributed to TMD (1).

TMD is common and affects an estimated 31 percent of adults in the worldwide (2). TMD can cause both pain and functional limitations. The intensity and impact of the pain experienced can vary but substantially impacts people's daily lives including their ability to eat, communicate, and sleep (3–7). Further, TMD is associated with substantial morbidity, affecting quality of life and work productivity. As an example, it is estimated that for every 100 million working adults in the US, TMD contributes to 17.8 million lost workdays annually (8). In the United Kingdom, studies demonstrated that the quality and quantity of the work performed by individuals with TMDs is decreased by 12 percent each. This translates to a “hidden” cost to employers of £584 and £1,225 in lost productivity for each 6-month period among individuals living with TMD (9, 10). The natural history and etiology of the disorder are poorly understood and thus, appropriate treatment options are difficult to determine, limited, and complex (11).

The location of the TMJ in the orofacial region has to date resulted in most related research occurring within the clinical area of dentistry. For these reasons, the majority of TMD research has been directed and funded by the National Institute of Dental and Craniofacial Research (NIDCR) in direct contrast to the current understanding of the body as an interdependent system (12). Furthermore, a large proportion of historical and current treatments for TMD have focused (biomechanically) on the jaw joint, teeth, and affiliated musculature. Existing treatments include, but are not limited to: occlusal adjustments; local injections of steroids or botulinum toxin; various surgical procedures such as joint replacement; as well as more conservative, non-surgical treatments including acupuncture, physiotherapy and behavioral modification (13). There is limited evidence regarding the safety and effectiveness of these treatment options (14). As such, there are no formal guidelines for TMD treatment and management formulated by professional groups in the USA. In lieu of formal guidelines, various organizations have put forth scientific statements, parameters of care, and recommendations (15–18).

Conflicting reports from varying stakeholders related to prognosis and outcomes following placement of TMJ implants, in addition to data from the Food and Drug Administration's (FDA) MedWatch system, and patient accounts shared with the TMJ Association (TMJA) raising concerns regarding the safety and effectiveness, gave rise to the development of the TMJ Patient-Led RoundTable initiative. The TMJ Patient-Led RoundTable comprises key stakeholders as partners including patients, the FDA, NIDCR, the Agency for Health Care Research and Quality (AHRQ), the American Association of Oral and Maxillofacial Surgeons (AAOMS), TMJ implant manufacturers, clinicians, scientists, advocacy organizations, and other experts, all under the auspices of Medical Devices Epidemiology Network (MDEpiNet). More importantly, the RoundTable is a vehicle in which patients play critical roles throughout all related activities. Following its first meetings, the members of the RoundTable assessed the current availability of data and the ability of third parties to access, collect, and compile scientifically valid information related to selected aspects of TMD patient therapies. The RoundTable concluded that a strategically Coordinated Registry Network (CRN) is needed to collect and generate accessible data on TMD and its care that is sufficiently relevant and reliable. This is necessary to better understand the disparate treatment pathways and outcomes that patients experience.

While the importance and need of high quality dental records to inform decision-making has been established (19), there remains significant variance in how TMD patient dental and medical records are captured and recorded (20–22). Previous efforts have attempted to develop and validate the content taxonomy for dentistry to standardize and harmonize patient data (23). The development of a CRN complements and builds upon these efforts given its ability to creatively organize relevant data systems, establish and grow the capacity of existing data sources, harmonize various data sources including medical and dental records, and leverage data sources that are created for other purposes such as documenting and billing for care (24). CRNs allow for the efficient capture of evidence needed to evaluate TMJ/TMD throughout the disease course (25). The development and establishment of CRNs for these purposes have been successful for various disorders the neurovascular and gynecological clinical space (26–29).

The first foundational step toward building a CRN within the TMD clinical was to identify a core minimum dataset to inform the TMJ/TMD-CRN. From that point it becomes possible to establish the data infrastructure that facilitates the standardization and harmonization of data as well as perform high performance analytics needed to fully leverage the captured patient-generated data, clinical evidence, and other healthcare ecosystem data within the TMJ/TMD-CRN. The aim of this study was therefore to advance the clinical understanding, usage, and adoption of a core minimum dataset for TMD patients.

Methods

Multistakeholder partnership, data source identification, and data element extraction

The first TMJ Patient-led RoundTable was held on June 16, 2016, at the FDA headquarters in Silver Spring, Maryland. The meeting led to the formation of four working groups that were tasked with addressing specific areas of study and a Steering Committee to oversee the project as a whole. The RoundTable reconvened on May 11, 2018, at the FDA Headquarters to update participants on the results of the working group projects, establish a roadmap to address highlighted gaps in knowledge regarding TMD, and to identify data collection needs for the development of high quality, real-world evidence (RWE).

Findings from the working groups indicated that TMD is a multisystem disorder with complex etiology centered on heightened central nervous system activity that contributes to TMD dysfunction and symptomology (30). For these reasons, many patients experience overlapping comorbidities (8). To comprehensively identify data elements for inclusion in the TMJ/TMD-CRN core minimum dataset for the Delphi process (Figure 1), data elements were extracted from existing data sources that identified conditions overlapping with TMD (31). Additionally, data elements were extracted from published peer-reviewed original research, protocols on clinicaltrials.gov, and existing real-world data sources that aim to evaluate TMD and TMD devices. Finally, data elements in 522 postmarket surveillance (PMS) and premarket studies submitted for 4 different TMJ implants were reviewed and extracted.

FIGURE 1

Figure 1. Delphi process.

The core minimum dataset for TMJ/TMD-CRN was developed with the help of the Delphi method (32). The Delphi method facilitates group decision making with a panel approach by bringing together experts from various backgrounds in a conference or, in this case, an online platform. Delphi can be conducted in a survey format to reach consensus among the identified stakeholders without any group bias generated from group dynamics or face-to-face responses. The Delphi method has been leveraged for consensus building among content taxonomy in patient records within dentistry previously (23).

In total, 531 distinct data elements were captured and extracted from the various sources of published, publicly available, and regulatory data sources. These data sources included 4 premarket studies, 5 postmarket studies (PMS) and 6 real-world data (RWD) sources (33–36), and 1 clinical trial (37). Following the data elements extraction, the informatics and clinical working groups organized the data elements according to their context within the clinical workflow, removed redundancies, and added identified missing data elements to the Delphi process. The informatics and clinical working groups were comprised of patients, patient advocates, clinicians, regulators, researchers, informaticians, as well as representatives of industry and professional societies. The working groups met bi-weekly over the course of 6 months and identified 92 stakeholders for Delphi participation. The identified stakeholders represented various organizations including regulatory bodies, industry, academicians, clinicians, and most importantly patients and patient advocacy groups (Supplementary Appendix Table S1).

Finally, the Delphi survey questionnaire was designed by the MDEpiNet Coordinating Center within Weill Cornell Medicine (WCM) using an online platform, Qualtrics, and distributed to 92 Delphi participants. The survey was analyzed by the WCM team and discussed with the TMD core working group co-chairs via conference calls. The team made a group decision to drop elements that received less than 75% consensus. The consensus percentages were rounded up to the closest number, and all variables achieving more than 75% consensus were finalized as the core minimum dataset for patient-generated and clinical data. Participants were provided with the opportunity to respond with open comments and feedback at the end of the survey. The feedback received was considered by the analysis team to generate recommendations for specific data elements, wherever appropriate. Only one round of the Delphi was conducted as target consensus of 75% was achieved on the first distribution for a majority of the data elements resulting in a finalized core minimum dataset.

Following this process the items reaching 75% were presented to Delphi participants. A purposive multi-stakeholder sub-group was selected from the Delphi participants (n = 5 patients, n = 4 clinicians, n = 4 others including regulators and researchers) to triangulate the items across patient and clinician-based experience to remove redundancies or duplicate items. The group specifically examined the items for any opportunity to reduce response burden for both patient and clinician to maximize the usability of the registry. The process for this triangulation was iterative, qualitative, discursive, and undertaken over 5 one-hour virtual meetings of the sub-group chaired by a clinician from outside of the US (JD). All identified CDE common to existing CRNs, such as demographic variables, were mapped to common standard vocabularies and, where relevant, to the source health information systems standards as specified by the Office of the National Coordinator for Health Information Technology (ONC) United States Core Data for Interoperability (USCDI) (38). CDE that were TMJ-specific and accepted by consensus, will be mapped prior to the implementation of the registry.

Establishing the technological platform to support the TMJ/TMD-CRN

For an established CRN to reliably collect accurate dataset, a robust technological platform such as the High-performance Integrated Virtual Environment (HIVE), was leveraged (39). The HIVE infrastructure promotes the sustainability and applicability of the developed CRN by: (1) ensuring compliance with relation to cybersecurity and privacy of data access (40, 41) and availability (42); (2) providing vertical scalability for large data volumes and horizontal scalability for variety of data types; (3) facilitating interoperability by employing standards supported by standard development organizations; and (4) supporting various different data entry modalities as well as vast spectrum of data analyses.

The identified core minimum data elements (Supplementary Appendix Tables S1, S2) were defined in the context of technical implementation within HIVE web-application framework. More specifically, the variables were assigned types (e.g., strings, numbers), constraints (e.g., choice bound to a dictionary, numerical ranges, mappings to ontologically codified values), hierarchical interrelations, visibility, and the order of appearance within questionnaires, etc. These definitions ensure the validity of data entry process to avoid mistakes, logical consistence, completeness of the forms, and generate visual aids that facilitate data entry by optimizing the appearance and ease of entry through responsive web-design concepts. This variable engineering and form development process, though being separate from the core Delphi processes, required close communication between developers of HIVE web-application platform and Delphi participants. The developers imposed important questions not only about clinical relevance of variables, but also the relevance and validity of values that can be recorded for those variables.

HIVE was integrated with CHIOS™, an innovative permissioned blockchain platform, to strengthen the provenance of data captured in the registry and drive metadata to record all registry transaction and create a robust consent network. This integration would allow recording of all registry transactions and create a robust consent framework. The designed target ecosystem would maintain all existing functionality of HIVE registry with added functionality including maintenance of all data transaction information in blockchain, flexible patient consent recordation, data cataloging, and runtime consent validation through smart contracts. For this technical feasibility project, the engineering team built and implemented several client-side Application Programming Interfaces (API) operations using Python and smart contracts for the consent module. The Python wrapper was developed in the team's GitHub organization.

Results

A total of 59 multi-stakeholder participants responded to the Delphi survey including 30 (51%) participants representing patients or a patient advocacy group and 29 (49%) participants representing a clinician, physician, or researcher (Table 1). The results are presented based on the groupings that were present in the Delphi, patient-generated data elements, clinician-generated data elements and general feedback.

TABLE 1

Table 1. Delphi participants—round 1.

Patient-Generated data elements

The completion of the Delphi surveys followed by the application of the required group consensus threshold resulted in the selection of 397 data elements (254 for patient-generated data elements and 143 for clinician generated data elements). The data elements that were presented to the Delphi participants with the consensus percentage are summarized in Supplementary Appendix Table S2. The final list of identified patient-generated data elements in the minimum core dataset are shown in Table 2. Beyond patient demographics, the identified data elements captured patient-reported provider related information, provider contact information, any additional information about care, and their preference regarding data security. More specifically, data security captures the patient's preference on whether to grant or remove the treating clinicians' access to the data provided in the TMJ/TMD-CRN. To comprehensively capture that patient's current health status, the medications the patient is currently using, any currently implanted devices, any allergies, any experienced symptoms, and the primary reasons for seeking care were among the data elements. For patient's medical history, there was consensus on capturing the presence of major chronic conditions and medical conditions that co-exist with TMD. These conditions include cardiovascular, dental, endocrine, otorhinolaryngological, ophthalmological, gastrointestinal, genitourinary, hematologic, infectious, musculoskeletal, neurological, general or systematic, rheumatologic and immunological conditions. Given that TMD is difficult to distinguish and the tremendous overlap in diagnosing a TMJ problem as a specific diagnosis when the overlap can involve TMJ disease, muscular inflammation, CNS pain, biopsychosocial (BSP) issues, and other soft and hard tissue/genetic disorders. In addition, the lack of understanding of the etiology and the symptoms of the disease, data elements capturing whether TMD has previously been misdiagnosed and the symptoms that mimic TMD that may have led to the diagnosis were included. Additionally, the patient's social history, the family history of TMD, and whether the patient had a previous device implanted were data elements for which consensus was reached. The patient's past surgical history including receipt of any TMJ treatment procedures, dental implants, alternative TMJ treatment, receipt of medication for jaw necrosis or clenching and bruxism was captured. There was consensus on collecting relevant post-operative outcomes including open bite, range of motion, infection, device removal, device failure, complications occurring during removal procedure, any change in disability status after procedure and reoperations. Data elements related to long-term follow-up were captured to assess patients lost to follow-up, the reasons for loss to follow-up, and any hospital readmissions.

TABLE 2

Table 2. Core minimum dataset for patient related information.

Clinician-generated data elements

Data elements collected by clinicians that are included in the minimum core dataset are summarized in Table 3. The clinician-generated data elements that were presented to the Delphi participants with the consensus percentage are summarized in Supplementary Appendix Table S3. These data elements included clinician information, findings from a physical exam, exams related to TMJ (e.g., jaw function/dysfunction, pain onset, pain duration, Wilkes staging classification for internal derangement, Angle's classification), clinical assessment (e.g., malocclusion, deviated opening laterality, maximum interincisal opening, range of motion), and whether these assessments led to the diagnosis of TMD.

TABLE 3

Table 3. Core minimum dataset for clinician collected information.

There was consensus on including specific data elements from any laboratory or imaging obtained prior to a potential TMD diagnosis. These included Antinuclear Antibodies tests, C-Reactive Protein tests, Complete Blood Count tests, Rheumatoid Factor tests, Rheumatological Lab tests (e.g., anti-citrullinated protein antibody test), CT scan with or without contrast, Magnetic Resonance Imaging findings, and Panoramic findings. The participants reached consensus on minimal data elements for all procedures, including the date of the procedure, procedure code, intervention site, procedure status, procedure urgency and length of procedure. Procedure related elements were also captured for arthrocentesis, arthroscopy, arthroplasty, TMJ total joint replacement, coronoidectomy, orthognathic surgery with TMJ total joint replacement, joint replacement for tumor, trauma, others with vascular/bone fibula grafts, and intraoral vertical ramus osteotomy and intermaxillary fixation procedures. If a TMJ implant was implanted then device-related characteristics to be captured were identified such as the unique device identifier (UDI) and the associated data (e.g., the device type, device class, model number, and implant material). Any TMD medications including the dose, dose units, type class, start date, and end date are also to be recorded. Data elements related to post-operative outcomes included the presence of chronic lymphocytic infiltrate and longitudinal follow-up elements included mortality. Finally patient survey tools were identified as crucial elements for the core minimum dataset. The patient survey tools included overall quality of life patient reported outcome measures (PROMs), psychosocial status and TMD symptom specific patient reported outcomes (PROs). This included the EuroQoL five dimension (43), Jaw Function limitation scale (44), Oral Health Impact Profile for Temporomandibular Disorders (OHIP-TMD) (45), Pain Numeric Rating Scale (46), and Short Form-12 (47). Patients will be able to complete the varying PROMs at different timepoints throughout their TMJ/TMD disease course. The TMJ/TMD-CRN will need a flexible data infrastructure to capture any existing or future validated PROMs.

In addition, there were specific concepts that allowed for open comments and/or recommendations for capturing additional elements of clinical information. First, for clinical assessment with exams, 78% participants commented to capture details such as pre-auricular area and external auditory canal (EAC) palpation for pain; pain triggers; pain alleviation; previous facial/head/neck trauma; incisal/occlusal wear of teeth; TMJ palpation; and occlusal contacts in maximum intercuspation. In terms of laboratory findings, although these did not reach the consensus cut-off, it is important to note that 63% participants recommended that there should be additional captures but didn't provide any suggestions. For imaging findings, 44% recommended that there should be additional captures, and at least one person commented that bone scan and simple radiographs should be captured. In terms of clinical assessment with diagnostics tests, 50% participants responded affirmative to capturing diagnostic-related information, and provided the following elements or use of the assessment tools: Beighton score, Research Diagnostic Criteria (RDC)-TMD¹, TMJ ankylosis specific quality of life (QoL) questionnaire, pain on loading, bone scan, clinical exam and imaging, Mahan test, tongue bite test for synovitis, diagnostic criteria for TMD, pain on lateral palpation at the condyle, and response to diagnostic anesthetic block.

General feedback on the survey

Based on the general feedback responses at the end of the survey, the comments generally reiterated the recommendations for additional data elements, identified concerns or suggested limitations of the survey. The recommendations for additional data elements were further reviewed by the working group members from the perspective of achieving optimal balance between the granularity and least burdensome approach.

Triangulation of items by purposively sampled subgroup of delphi participants

The group agreed on the majority (85%) of items in the patient data collection (Table 2). Major points of discussion involved the inclusion of past medical history, past dental history procedures, mimics, and changes in disability status after the procedures. Past medical history includes the determination of history or presentation of symptoms (e.g., Headaches, Clenching and Bruxism), use of medications (e.g., compliance with prescribed medications, receipt of chemotherapy and/or radiotherapy), as well as metal and medication allergies. Past dental history procedures include but are not limited to full mouth restorations, extractions, orthodontics, and their associated outcomes. Discussion ensued around the aforementioned items given their potential for removal following the Delphi process. The group resolved however to retain these items.

Four items, in Table 2, were consolidated due to the redundancy. These were: “currently prescribed medications…”, as it could be covered by another item; “any chemotherapy” as it was merged with another item; two items on metal and medication allergies as it was covered by the broader item on “any known…allergies”. The section (18 items) on mimics for the TMD was simplified to one question thereby removing 17 items and was rephrased carefully—“Were you experiencing symptoms or diagnosed with another condition, which was later determined to be TMD, or vice versa?”—in order to capture the bidirectional nature of a mimic. That is TMD could mimic some other disorder or vice versa and both are important to be captured. Similarly, the section on past dental history was simplified to two items—“Significant dental health issues” and “Significant dental procedures” rather than four due to identified item redundancy.

The group noted and suggested a solution to the lack of a validated variable in the patient data entry relating to the patient's diagnosis of TMD. The suggested solution was to request completion of the 6 item TMD screener validated by Gonzalez et al. as part of the data capture (48). The patients and clinicians in the subgroup emphasized that this screener did not act as a gatekeeper to inputting the remaining data in the registry, but rather served as a validated item to increase utility of analyzing the data produced from the registry.

In the clinician data collection (Table 3) the group also agreed on the majority of items (95%). Two items were identified as being conditionally required in the minimum core dataset: the Wilkes Staging Classification for Internal Derangement and examination of the temporalis tendon. Five items were revised to prevent redundancy and increase utility of the registry. The items included: (1) Medications that may cause jaw necrosis, and clenching and bruxism as they may change over time and are better kept to analysis of the medication list, (2) Utility of clinical assessment tools e.g., Charlson Comorbidity Index, (3) Inclusion of the findings from clinical assessment exams, (4) Laboratory findings that were not specific enough (and underwent additional enumeration with clinical stakeholders during the post Delphi review), and (5) Patient reported outcomes after operation as they may be captured by PROMs and/or in the clinician generated data elements.

Several identified data elements are shared across other CRNs within other clinical spaces. The format, data type, response options and mapping to the current terminology standards specified by ONC's US Core Data for Interoperability (USCDI) and shown in Table 4.

TABLE 4

Table 4. Coordinated registry network shared data Elements and terminology mappings.

Generated technological platform

The developers established permissioned blockchain for storing data to enhance data security scenarios. The integration proof of concept architecture is depicted in Figure 2. The use of a permissioned blockchain for storing transaction hash data allows for enhanced data security scenarios. Every data operation in HIVE was verified for its authenticity by an immutable record in the blockchain along with a set of requisite access control permissions associated with the originator of the operation. To demonstrate the CHIOS™ and HIVE integration, the team built out a simple API client for HIVE since one did not exist. The team was able to successfully implement a handful of client side API operations using Python. The CHIOS Consent Module effort consisted of both an API and a smart contract that were developed by the Softhread team to record HIVE user registration and consent operations on the blockchain. This metadata, as committed to the blockchain, is used to verify whether a user has consented to a set of data operations to the untrusted parties that may be requesting that user's data from HIVE. Both metadata provenance and non-repudiation were of focus. The first time ever integration of HIVE and CHIOS™ within the consent module addressed metadata provenance and non-repudiation, thus, enhancing the fidelity and integrity of the user data consent operations within the HIVE environment. The infrastructure development was completed and demonstrated in early January of 2021 to the TMJ/TMD-CRN showing the maintenance of all data transaction information in blockchain, flexible recording of patient consent, data cataloging, and consent validation through smart contracts. The high-level architecture of this consent module is shown in Figure 3. The developed ecosystem allows patients and clinicians to interact with the web-app to register patient and doctor accounts, enter data fields on a web-app provided pages, and request recorded datasets.

FIGURE 2

Figure 2. Integration of High-performance Integrated Virtual Environment (HIVE) and CHIOS™.

FIGURE 3

Figure 3. Use of smart contact to collect, securely store, and leverage collected data within High-performance Integrated Virtual Environment (HIVE). Steps outlined within the CHIOS™-HIVE Consent Module: (1) Registration: Patients and doctors register on HIVE driven registry web-app portal. (2) Registration: HIVE registers in blockchain on user’s behalf. (3) Data entry: Users enter information in Web-App. (4) Provenance logging: HIVE records the transaction metadata on Blockchain. Information on who, when, which data-type and which fields have been entered will be transmitted to Blockchain via a smart contract. Actual values of the entered fields will not be transmitted. (5) Consent: Patients create and sign a consent form on a web-app allowing particular end users/researchers/doctors access their data. (6) Consent recordation: The signed consents are translated into harmonized constructs and transferred to the blockchain via a smart contract. (7) Data cataloging: Researcher queries on what type of data are available from how many patients in order to understand the landscape of data availability. (8) Consent listing, revocation: patient can list the existing consents they have previously provided; they are given opportunity to revoke consents. (9) Data access permission request: Doctor or researcher requests to see the patients data. (10) Consent validation: HIVE submits request to smart contract on the blockchain to validate consent between list of patients and requestor. Decision is made on allowing the requestor to retrieve data based on the smart contract execution outcome. Transaction history (not shown in diagram): HIVE can request the list of all transaction metadata from blockchain layer for auditing and monitoring purposes.

Discussion

The core minimum dataset, identified through stakeholder and patient engagement, outlines the pre-defined standardized data elements needed to assess TMD treatments and devices. These data elements may be entered by patients, by clinicians, or through a hybrid approach based on available technology. They provide the needed foundation of potential data linkages that will allow for the comprehensive assessment of TMD devices as well as strengthen patients' role in generating real-world evidence including epidemiological surveillance data. The potential data linkages may save time and cost of analyses by leveraging all available information for the different available real-world data sources (e.g., electronic health records, health information systems, registries). The comprehensive assessment of the condition and related treatments using these data sources may be leveraged to inform treatment guidelines, clinical decision making, and regulatory processes such as premarket approvals and postmarket surveillance. The resulting real-world evidence may inform the selection of optimal TMD treatment regimens on an individual basis and the prediction of possible clinical outcomes. It is important to note, however, the presence of equally important data source that informs the final decision-making process— the voices of the patients themselves, who are experiencing symptoms that are not typically captured in clinical studies.

Traditional consensus approaches have chances of bias due to the lack of anonymity and the potential of one person having a large effect on the decision-making process. Delphi is conducted in a series of surveys sent out to stakeholders to collect responses anonymously and individually. The participants are able to provide their honest opinions with suggestions, which are then reviewed and analyzed by the core team. The questionnaires can then be revised and re-distributed to participants for several cycles until target consensus is reached (49, 50). The Delphi process allowed for the patients' voice to be amplified while simultaneously engaging multiple other relevant stakeholders, including clinicians, researchers, manufacturers, and FDA, with varying perspectives and experiences with TMD and their associated treatments. Despite the varying perspectives, consensus was achieved. Of note, patients played a substantial role in leading the initial identification and extraction of data elements. Patients also made up more than half of the respondents within the Delphi process. This allowed for the identification of elements that are vital in understanding the patient journey including symptoms and complementary or alternative treatment options that are not necessarily captured or supported in other clinical evidence. The organization and execution of the Delphi process enabled the enhancement of the clinical workflow model by organizing the data elements in logical groupings that allow for the seamless integration of information into the reported data. These groupings include patient reported symptoms organized by body system, past medical history, clinical evaluation, management, treatment outcomes, as well as PROMS. It is important to note that some identified data elements overlap with the survey items included in the identified relevant PROMs². There was significant concurrence between the working groups with respect to the proposed patient survey and assessment tools identified for TMJ/TMD, and the final recommendations will be issued by the TMD PROMS Working Group.

Several limitations associated with the Delphi process should be noted. Even if consensus was achieved, the results may not represent the priorities of all stakeholders. For example, some respondents noted that the representation of the data elements did not align well with the various stages in the natural progression of the disease. More specifically, some elements are related to the pre-diagnosis phase of TMD while other data elements address complications of surgery in more advanced post-diagnosis phases of TMD. Others emphasized the importance of capturing progression from simple to advanced conditions, temporary complaints vs. chronic symptoms, symptoms prior to diagnosis and following a TMD diagnosis, or receipt of non-surgical treatment vs. surgical treatment. Additional comments expressed by the respondents include that: (1) data elements may not be targeted to the clinicians based on their clinical specialties; (2) the data elements were not always based on current clinical evidence and medical concepts; (3) failed treatment or misdiagnosis was not clearly explained; (4) distinctions need to be made between overlapping conditions with the TMD compared to coexisting conditions not related to the TMD; and (5) refinement of the terminologies (e.g., therapies vs. treatment/management) should be incorporated during the next phase of pilot testing of the core minimum data set capture. The refinement of terminologies will be crucial given that the current core data elements are meant to encompass a wide variety of TMD cases. An additional module identifying data elements, such as complexities with regards to laboratory testing, imaging, and procedure-related characteristics, that are more specific to subpopulations with more severe forms of TMD that require, for example, total joint replacement may be necessary to comprehensively capture the experiences of this population. The data elements are meant to encompass TMD as a whole, however, data elements specific to subpopulations with more severe forms of TMD may be needed. Despite these limitations, it is important to note the strengths of the study. Strengths of our methodology include the use of an iterative approach and the incorporation of perspectives from varying stakeholder groups, especially patients, who played a crucial and significant role throughout the entirety of the study. All participants equally influenced the refinement of the core minimum dataset. The anonymous fielding of the survey decreases the potential bias associated with group dynamics in a face-to-face setting and the final triangulation provided by the sub-group review means the item sets can be considered robust and valid representations of both patient and clinician perspectives. Additional strengths associated with the methodology employed include the meaningful engagement and crucial role of various stakeholders including patients, clinicians, professional societies, researchers, manufacturers, and the FDA. The involvement and collaboration of these stakeholders in the various working groups as well as their participation in the Delphi survey ensured that the identified CDE are comprehensiveness and relevant. The participants' future feedback following the initiation of data collection will help future collaborative work to better align and strategically streamline evidence generation needs for all the stakeholders. Furthermore, reviewing existing data sources and studies ensure that many identified elements are already captured in existing data sources which facilitates data linkage and a greater level of interoperability within the establish CRN. Finally, the efforts through the Delphi and the working groups to reduce the number of CDE, in turn, decreases the burden on respondents, supports continuous engagement for longitudinal data input, and enhances user-centricity while still providing valuable information.

The identification of CDEs is an important step prior to data collection and linkage of existing data sources within a CRN. This data collection and linkage of these real-world data sources (RWD) to generate real world evidence (RWE) will improve data monitoring supporting the quicker identification of clinical concerns, potential avoid poor patient outcomes, and thereby inform clinical- and regulatory- decision-making.

The integration of the technological platform for the TMJ/TMD-CRN provides opportunity for unparalleled data entry transaction traceability and data access auditability. All secure transactions such as logging in, entering data values, sharing data access are tracked in the blockchain system in the form of immutable transaction chains. The use of a blockchain smart contract layer allows for the validation of secure data access attempts not only with relation to those who have access to the various data forms, but also control access within usage context: which data fields can be used with which algorithms in which projects within which timeframe and by whom. Additionally, the API can be used as a foundation for future development tasks that would be needed for more extensive integrations of technological platforms that allow for optimized data integrity and enhanced data provenance which is essential in standardization efforts.

It is important to note that HIVE framework allows continuous evolution of data aggregation subsystems. Longitudinal data collection and analysis may in the future reveal minor issues in the initial vision and variable sets such as lack of a relevant variable or ambiguity of a definition. HIVE can dynamically adapt to such issues: (1) it allows adding new variables to existing registry; (2) novel ontological definitions can be incorporated; (3) customizations can be provided as per institution of entry; and (4) patient categories can be introduced that customize data entry modalities. Powerful semantic mapping rule-based engine in HIVE allows to maintain consistency of datasets in evolving registry network without interruptions of live production ecosystem without backward compatibility issues that usually haunt many other registry systems. This dynamic adaptability is foundational to create long-term, longitudinal data ecosystems that can evolve over time.

In conclusion, the data elements identified through the Delphi process represent items that are currently and routinely captured as part of the clinical record as well as the additional patient-centered items that more comprehensively encompass the patients' experience throughout their journey with TMD.

The next step of this project will be to collect these identified CDE through a national infrastructure that will serve as repository for unbiased and high-quality data on TMJ/TMD devices, treatments, and disease course. The development and advancement of this CRN as a robust source of real-world data that leads to the generation of high-quality real-world evidence addresses current strategic priorities of the FDA and existing legislation (51, 52). The modern, state of the art, national infrastructure for data capture will serve as the needed foundation of the TMJ/TMD-CRN and the accrual of high-quality data on TMD and their treatments in the context of a multi-purpose CRN.

Data availability statement

The original contributions presented in the study are included in the article/Supplementary Material, further inquiries can be directed to the corresponding author.

Ethics statement

This work was approved under IRB #1511016772. Prior to initiation of this study, an MDEpiNet oversight committee was established, and a written protocol was developed which prespecified the study methods. The Delphi process involved feedback from CRN partners and as such no other regulatory requirements were considered relevant.

Author contributions

AS and DM-D initiated the collaborative project. All authors participated in the initial stakeholder meeting to conceptualize the project and design the study protocol. SA designed the Delphi surveys, collated, and analyzed the Delphi survey results, managed the Delphi process, and drafted the first version of the manuscript. TC, JK, AWC, JD, LS, JGF, and MR served as working group co-chairs, provided clinical/epidemiological insights on the Delphi survey results, and co-lead discussions with working group members. All authors served as members of the working group, participated in discussions throughout the Delphi process, reviewed manuscript drafts, and participated in editing and revising the manuscript. LG, MV, YY, IV-F and VH drafted the initial manuscript. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted. All authors contributed to the article and approved the submitted version.

Funding

This work was supported by the Office of the Secretary Patient-Centered Outcomes Research Trust Fund under Interagency Agreement #750119PE060048

Conflict of interest

VS is employed by Embleema. IV-F is employed by SofThread.

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fdgth.2023.1132446/full#supplementary-material.

Footnotes

¹Note that this may be replaced by DC/TMD assessment tool [Eric Schiffman et al. J Oral Facial Pain Headache. 2014; 28(1): 6–27.]

²Additional PROMs related analysis was done by another TMD working group and will be leveraged in the final representation of the data elements.

References

1. de Leeuw R, Klasser GD. Orofacial pain: Guidelines for assessment, diagnosis, and management. Incorporated Hanover Park, IL: Quintessence Publishing Company (2018).

2. Valesan LF, Da-Cas CD, Réus JC, Denardin ACS, Garanhani RR, Bonotto D, et al. Prevalence of temporomandibular joint disorders: a systematic review and meta-analysis. Clin Oral Investig. (2021) 25:441–53. doi: 10.1007/s00784-020-03710-w

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Durham J, Breckons M, Vale L, Shen J. DEEP study: modeling outcomes and costs of persistent orofacial pain. JDR Clin Trans Res. (2021) 8(1): 238008442110638. doi: 10.1177/23800844211063870

CrossRef Full Text | Google Scholar

4. Breckons M, Shen J, Bunga J, Vale L, Durham J. DEEP study: indirect and out-of-pocket costs of persistent orofacial pain. J Dent Res. (2018) 97:1200–6. doi: 10.1177/0022034518773310

PubMed Abstract | CrossRef Full Text | Google Scholar

5. Breckons M, Bissett SM, Exley C, Araujo-Soares V, Durham J. Care pathways in persistent orofacial pain. JDR Clin Trans Res. (2017) 2:48–57. doi: 10.1177/2380084416679648

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Durham J, Shen J, Breckons M, Steele JG, Araujo-Soares V, Exley C, et al. Healthcare cost and impact of persistent orofacial pain. J Dent Res. (2016) 95:1147–54. doi: 10.1177/0022034516648088

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Shueb SS, Nixdorf DR, John MT, Alonso BF, Durham J. What is the impact of acute and chronic orofacial pain on quality of life? J Dent. (2015) 43:1203–10. doi: 10.1016/j.jdent.2015.06.001

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Maixner W, Diatchenko L, Dubner R, Fillingim RB, Greenspan JD, Knott C, et al. Orofacial pain prospective evaluation and risk assessment study–the OPPERA study. J Pain. (2011) 12:T4–11.e1-2. doi: 10.1016/j.jpain.2011.08.002

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Slade G, Durham J. Prevalence, impact, and costs of treatment for temporomandibular disorders. In: National Academies of Sciences E and M, editor. Priorities for research and care. Washington, DC: The National Academies Press (2020). p. 387–401 doi: 10.17226/25652

10. National Academies of Sciences E and M, Health and Medicine Division, Board on Health Care Services, Board on Health Sciences Policy, Committee on Temporomandibular Disorders (TMDs): From Research Discoveries to Clinical Treatment. Temporomandibular disorders: Priorities for research and care. Washington, D.C.: National Academies Press (2020). doi: 10.17226/25652

11. Yokoyama Y, Kakudate N, Sumida F, Matsumoto Y, Gordan VV, Gilbert GH. Dentist’s distress in the management of chronic pain control: the example of TMD pain in a dental practice-based research network. Medicine (Baltimore). (2018) 97:e9553. doi: 10.1097/MD.0000000000009553

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Lee JS, Somerman MJ. The importance of oral health in comprehensive health care. JAMA. (2018) 320:339–40. doi: 10.1001/jama.2017.19777

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Kusiak J, Veasley C, Maixner W, Fillingim R, Mogil J, Diatchenko L, et al. The TMJ Patient-Led RoundTable: A History and Summary of Work. (2018). Available at: http://mdepinet.org/wp-content/uploads/TMJ-Patient-RoundTable-Briefing-Report_9_25_18.pdf. 2018 (Accessed January 31, 2020).

14. Reid KI, Greene CS. Diagnosis and treatment of temporomandibular disorders: an ethical analysis of current practices. J Oral Rehabil. (2013) 40:546–61. doi: 10.1111/joor.12067

PubMed Abstract | CrossRef Full Text | Google Scholar

15. Greene CS, Klasser GD, Epstein JB. Revision of the American association of dental research’s science information statement about temporomandibular disorders. J Can Dent Assoc. (2010) 76:a115. PMID: 20943030

PubMed Abstract | Google Scholar

16. Leeuw K, Gary D. American Academy of Orofacial Pain., R de. Orofacial pain: guidelines for assessment, diagnosis, and management. (2018).

17. American Association of Oral and Maxillofacial Surgeons. Parameters of care for oral and maxillofacial surgery. A guide for practice, monitoring and evaluation. J Oral Maxillofac Surg. (1992) 50:i–xvi. 1–174. PMID: 1365850

PubMed Abstract | Google Scholar

18. Gauer RL, Semidey MJ. Diagnosis and treatment of temporomandibular disorders. Am Fam Physician. (2015) 91:378–86. PMID: 25822556

PubMed Abstract | Google Scholar

19. Oberbreckling PJ. The components of quality dental records. Dent Econ. (1993) 83(5):29–30. PMID: 8243764

PubMed Abstract | Google Scholar

20. Hand JS, Reynolds WE. Dental record documentation in selected ambulatory care facilities. Public Health Rep. (1984) 99:583–90. PMID: 6440203; PMCID: PMC1424640

PubMed Abstract | Google Scholar

21. Osborn JB, Stoltenberg JL, Newell KJ, Osborn SC. Adequacy of dental records in clinical practice: a survey of dentists. J Dent Hyg. (2000) 74:297–306. PMID: 11314481

PubMed Abstract | Google Scholar

22. Morgan RG. Quality evaluation of clinical records of a group of general dental practitioners entering a quality assurance programme. Br Dent J. (2001) 191:436–41. doi: 10.1038/sj.bdj.4801201

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Acharya A, Hernandez P, Thyvalikakath T, Ye H, Song M, Schleyer T. Development and initial validation of a content taxonomy for patient records in general dentistry. Int J Med Inform. (2013) 82:1171–82. doi: 10.1016/j.ijmedinf.2013.06.007

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Krucoff M, Normand S, Edwards F, Lystig T, Ross E, Berliner E, et al. Recommendations for a national medical device evaluation system: strategically coordinated registry networks to bridge clinical care and research. (2015).

25. Network MDE. Medical Device Registry Task Force and Medical Devices Epidemiology Network: Recommendations for a National Medical Device Evaluation System: Strategically Coordinated Registry Networks to Bridge Clinical Care and Research. (2015). Available at: http://www.mdepinet.org/wp-content/uploads/Recommendations-for-a-National-Medical-Device-Evaluation-System_24-Aug-2015.pdf (Accessed May 23, 2020).

26. LeRoy H, Gressler LE, Liebeskind DS, Brooks CE, Siddiqui A, Ansari S, et al. Developing the foundation for assessment of devices used for acute ischemic stroke interventions (DAISI) using a coordinated registry network. BMJ Surg Interv Health Technol. (2022) 4:e000113. doi: 10.1136/bmjsit-2021-000113

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Baird CE, Guiahi M, Chudnoff S, Loyo-Berrios N, Garcia S, Jung M, et al. Building blocks for the long-acting and permanent contraceptives coordinated registry network. BMJ Surg Interv Health Technol. (2022) 4:e000075. doi: 10.1136/bmjsit-2020-000075

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Baird CE, Myers E, Jacoby V, Gressler LE, Venable S, O’Neill A, et al. Development of a core minimum data set to advance real-world evidence generation for uterine fibroids treatment technologies. BMJ Surg Interv Health Technol. (2022) 4:e000094. doi: 10.1136/bmjsit-2021-000094

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Baird C, Jung M, Andrews S, Ferriter A, Cornelison T, Chughtai B, et al. Development of a coordinated registry network for pelvic organ prolapse technologies. BMJ Surg Interv Health Technol. (2022) 4(Suppl 1):e000076. doi: 10.1136/bmjsit-2020-000076

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Wilentz JB, Cowley AWJ. How can precision medicine be applied to temporomandibular disorders and its comorbidities? Mol Pain. (2017) 13:1744806917710094. doi: 10.1177/1744806917710094

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Maixner W, Fillingim RB, Williams DA, Smith SB, Slade GD. Overlapping chronic pain conditions: implications for diagnosis and classification. J Pain. (2016) 17:T93–107. doi: 10.1016/j.jpain.2016.06.002

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Brown BB. Delphi process: a methodology used for the elicitation of opinions of experts. CA: RAND Corporation PP—Santa Monica (1968).

33. Myers S, Kaimal S, Springsteen J, Ferreira J, Ko C-C, Fricton J. Development of a national TMJ implant registry and repository– NIDCR’s TIRR. Northwest Dent. (2007) 86:13–8. PMID: 18240532

PubMed Abstract | Google Scholar

34. Hoffmann RG, Kotchen JM, Kotchen TA, Cowley T, Dasgupta M, Cowley AW. Temporomandibular disorders and associated clinical comorbidities. Clin J Pain. (2011) 27:268–74. doi: 10.1097/AJP.0b013e31820215f5

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Australian Orthopaedic Association (AOA). Australian Orthopaedic Association National Joint Replacement Registry (AOANJRR). (n.d.). Available at: https://AoanjrrSahmriCom https://aoanjrr.sahmri.com (Accessed January 29, 2023).

36. Idle MR, Green J, Lowe D, Rogers SN, Sidebottom A, Speculand B, et al. National case registration of temporomandibular joint replacement: preliminary outcome data report. Br J Oral Maxillofac Surg. (2012) 50:S17. doi: 10.1016/j.bjoms.2012.04.190

CrossRef Full Text | Google Scholar

37. ClinicalTrials.gov. Alloplastic Total Temporomandibular Joint (TMJ) Replacement Registry. ClinicalTrialsGov Identifier: NCT03991728. (2019).

38. The Office of the National Coordinator for Health Information Technology (ONC). United States Core Data for Interoperability (USCDI). HealthITGov. (2023). Available at: https://www.healthit.gov/isa/united-states-core-data-interoperability-uscdi (Accessed February 4, 2023).

39. Simonyan V, Chumakov K, Dingerdissen H, Faison W, Goldweber S, Golikov A, et al. High-performance integrated virtual environment (HIVE): a robust infrastructure for next-generation sequence data analysis. Database (Oxford). (2016) 2016:baw022. doi: 10.1093/database/baw022

PubMed Abstract | CrossRef Full Text | Google Scholar

40. U.S. Department of Health and Human Services—Office of the Assistant Secretary for Health. Health Insurance Portability and Accountability Act (HIPPA) of 1996. (1996). Available at: https://aspe.hhs.gov/reports/health-insurance-portability-accountability-act-1996

41. Horizon Framework Programme of the European Union. General Data Protection Regulation (GDPR). (2018).

42. Cybersecurity and Infrastructure Security Agency. Federal Information Security Modernization Act (FISMA). (2014). Available at: https://www.cisa.gov/federal-information-security-modernization-act (Accessed January 29, 2023)

43. Balestroni G, Bertolotti G. EuroQol-5D (EQ-5D): an instrument for measuring quality of life. Monaldi Arch Chest Dis. (2012) 78:155–9. doi: 10.4081/monaldi.2012.121

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Ohrbach R, Larsson P, List T. The jaw functional limitation scale: development, reliability, and validity of 8-item and 20-item versions. J Orofac Pain. (2008) 22:219–30. PMID: 18780535

PubMed Abstract | Google Scholar

45. Durham J, Steele JG, Wassell RW, Exley C, Meechan JG, Allen PF, et al. Creating a patient-based condition-specific outcome measure for temporomandibular disorders (TMDs): oral health impact profile for TMDs (OHIP-TMDs). J Oral Rehabil. (2011) 38:871–83. doi: 10.1111/j.1365-2842.2011.02233.x

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Nugent SM, Lovejoy TI, Shull S, Dobscha SK, Morasco BJ. Associations of pain numeric rating scale scores collected during usual care with research administered patient reported pain outcomes. Pain Med. (2021) 22:2235–41. doi: 10.1093/pm/pnab110

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Turner-Bowker D, Hogue SJ. Short form 12 health survey (SF-12). Encyclopedia of quality of life and well-being research. Dordrecht: Springer Netherlands. (2014). p. 5954–7. doi: 10.1007/978-94-007-0753-5_2698

48. Gonzalez YM, Schiffman E, Gordon SM, Seago B, Truelove EL, Slade G, et al. Development of a brief and effective temporomandibular disorder pain screening questionnaire: reliability and validity. J Am Dent Assoc. (2011) 142:1183–91. doi: 10.14219/jada.archive.2011.0088

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Hsu C-C, Sandford B. The delphi technique: making sense of consensus. Practical assessment. Res Eval. (2007) 12:1–8. doi: 10.7275/pdz9-th90

CrossRef Full Text | Google Scholar

50. Yousuf M. Using Experts’ opinions through delphi technique. Pract Assess Res Eval. (2007) 12:1–8. doi: 10.7275/rrph-t210

CrossRef Full Text | Google Scholar

51. U.S. Food and Drug Administration Center for Devices and Radiological Health. 2022–2025 Strategic Priorities. (2022). Available at: https://www.fda.gov/media/155888/download

52. 114th Congress. H.R.34–21st Century Cures Act. CongressGov2016. (n.d.). Available at: https://www.congress.gov/bill/114th-congress/house-bill/34/text (Accessed January 10, 2020).