The interRAI Suite of Mental Health Assessment Instruments: An Integrated System for the Continuum of Care

The lives of persons living with mental illness are affected by psychological, biological, social, economic, and environmental factors over the life course. It is therefore unlikely that simple preventive strategies, clinical treatments, therapeutic interventions, or policy options will succeed as singular solutions for the challenges of mental illness. Persons living with mental illness receive services and supports in multiple settings across the health care continuum that are often fragmented, uncoordinated, and inadequately responsive. Appropriate assessment is an important tool that health systems must deploy to respond to the strengths, preferences, and needs of persons with mental illness. However, standard approaches are often focused on measurement of psychiatric symptoms without taking a broader perspective to address issues like growth, development, and aging; physical health and disability; social relationships; economic resources; housing; substance use; involvement with criminal justice; stigma; and recovery. Using conglomerations of instruments to cover more domains is impractical, inconsistent, and incomplete while posing considerable assessment burden. interRAI mental health instruments were developed by a network of over 100 researchers, clinicians, and policy experts from over 35 nations. This includes assessment systems for adults in inpatient psychiatry, community mental health, emergency departments, mobile crisis teams, and long-term care settings, as well as a screening system for police officers. A similar set of instruments is available for child/youth mental health. The instruments form an integrated mental health information system because they share a common assessment language, conceptual basis, clinical emphasis, data collection approach, data elements, and care planning protocols. The key applications of these instruments include care planning, outcome measurement, quality improvement, and resource allocation. The composition of these instruments and psychometric properties are reviewed, and examples related to homeless are used to illustrate the various applications of these assessment systems.

Introduction

The lives of persons living with mental illness are affected by the interplay of a broad range intrinsic and extrinsic factors emerging over the life course. From the earliest stages of life to the person's final moments, these factors can influence opportunities for growth and development, access to resources, engagement in interpersonal relationships, participation in community, and achieving an overall sense of well-being.

Mental health concerns are both pervasive and heterogeneous. Mental illness and substance use disorders are the world's leading cause of disability (1) accounting for up to 32% of years lived with disability and 13% of disability-adjusted life years (2). Unipolar depression was the fourth leading cause of total disease burden in 2000, after perinatal conditions, lower respiratory infections, and HIV/AIDS (3). With many underlying psychological, biological, social, economic, and environmental causes, it is unlikely that preventive strategies, clinical treatments, therapeutic interventions, or policy options will succeed as singular solutions for the challenges of mental illness.

Persons living with mental illness tend to receive services and supports in multiple settings across the health care continuum that are often fragmented, uncoordinated, and inadequately responsive. Needs may remain unaddressed (4) as the person navigates a path to many providers that function in narrowly defined siloes. Patient flow through episodic service environments (e.g., hospitals) may be constrained (5) and access to appropriate services may be delayed due to lengthy waiting lists (6).

To be effective, any health system's approach must include strategies for identifying and responding to mental illness and related dimensions of health and well-being throughout the life course and in all parts of the health care continuum. As such, the ability for different providers, organizations and sectors to communicate with one another is crucial.

This paper provides an overview of the interRAI suite of mental health instruments, which is designed to function as an integrated assessment and screening system to provide a holistic view of the person's strengths, preferences, and needs. It begins with an examination of the range of factors that must be considered beyond psychiatric symptoms in order to support a person-centered approach to shared decision-making. Next, it describes the design, psychometric properties, and applications of interRAI mental health assessments using examples related to homelessness and trans-institutionalization.

Assessing the Bigger Picture: The Need to Look Beyond Psychiatric Symptoms

System integration requires the use of a common language for describing needs, monitoring service use, and tracking outcomes over time. Even within a specific care setting, unstructured narrative charts have little value when they are simply “electronic paper records” (7). Medical charts are incomplete, cumbersome, and overly narrow in their focus (8–10). While natural language processing might reduce the burden of reading volumes of narrative notes (11, 12), unstructured charts often have too many information gaps to have value (13). Standardized clinical assessment data can be more useful if they cover the relevant domains, use psychometrically sound data elements, and can be accessed and interpreted for immediate use.

Some clinicians are reluctant to use standardized assessment tools (14) even though their role in evidence-informed practice is recognized (15). Their reasons might include a perceived lack of benefit over clinical judgment, concerns over psychometric properties, and practicality (14). Even if these issues are addressed to one clinician's satisfaction, inconsistency across settings in the choices of measures prevents communication between organizations serving the same individual.

Given the complexity of mental illness over the life course and the likelihood of engaging with diverse service providers, the data requirements for assessment are not rudimentary. Obviously, psychiatric symptoms, cognitive function, diagnosis, and behavioral issues will be highly relevant in most clinical contexts, particularly during acute episodes or mental health crises. However, if the view is widened to the person's broader experience, ability to function, and overall well-being, more domains must be considered as part of a comprehensive assessment of the “whole person”.

Life Course Perspective: Growth, Development, and Aging

About 20% of the Canadian population experiences problems with mental health or addictions annually and about half will face mental health problems by age 40 (16). Estimates from New Zealand's Dunedin cohort studies suggest that only 17% of the population will have enduring lifetime freedom from mental illness (17). This is explained, in part, by factors at both ends of life.

About 70% of mental health problems are reported to begin in childhood or adolescence (18), many of which persist over the lifetime (19, 20). Early childhood development sets the stage for social and emotional functioning, academic achievement, and interpersonal relationships (21, 22). As an individual's identity develops in adolescence (23), social networks and peer pressure (24), substance abuse (25), parenting styles (26), and school performance come into play (27). Age-based service restrictions mean the transition from youth to adulthood is often met with fewer mental health resources (28).

Aging is associated with the onset of a number of conditions that can affect mental health, with dementia as the most obvious example (29, 30). Mental illness may accelerate the aging process resulting in shorter life expectancy and greater years of life lost compared with the general population (31–33). Some, but not all, of this is attributable to deaths by suicide (34). Other causes include higher smoking rates leading to increased cancer and cardiovascular disease (35, 36) and higher rates of diabetes arising from poor diet, physical inactivity, and drug related side effects (37). Persons with psychiatric diagnoses who do survive to later life have greater odds of being the most frail (38).

Physical Health and Disability

Physical comorbidities are often neglected needs in persons living with mental illness (39, 40). Health care providers are commonly divided according to whether they provide physical health or mental health services. This bifurcation occurs to the detriment of persons living with mental illness who tend to receive lower quality medical care for physical health problems (33, 41). For example, mental health problems are often overlooked or stigmatized in acute hospital settings (33), cancer screening tends to be inadequate (42), and effective therapeutic regimens are less likely to be received by patients with mental illness who have acute myocardial infarction (43). In mental health settings, treatment of physical health problems is often delayed or inadequate (44, 45), and health promotion may be regarded as a low priority (46).

Mental health issues may be both consequences and causes of physical disability. A four-nation study showed that impaired physical functioning and dual sensory loss were associated with greater odds of depressive symptoms in home care clients (47). Suicide attempts may cause disability among survivors, but also there is increased risk of suicide among persons living with physical disabilities (48–51) for reasons that include pain and a sense of burden to others (52, 53).

Social Relationships

Social isolation and loneliness are increasingly recognized as important risk factors for the physical and mental health of persons of all ages (54–56). For example, social support networks may play an important role in facilitating recovery from mental health problems (57) and they may provide instrumental support with tasks such as child care, transportation, or medical management (58). On the other hand, mental health problems may pose barriers to forming or maintaining close personal relationships over the life course (59, 60). In addition, social relationships may be a source of stress, trauma, or abuse (61–63) that can have immediate and long-term consequences for mental health. Hence, interventions must consider both the social resources that may be beneficial and relationships that may be detrimental to the person's well-being.

Economic Resources

The pathways between poverty, deprivation and mental illness are multifold. Low income households are associated with higher rates of depression and antisocial behavior in children, but the impact may lessen if household resources improve (64). Job loss may pose tremendous stress on individuals and families resulting in worsened physical and mental health (65). However, for persons with pre-existing mental illness, unemployment and poverty are pervasive (66) resulting in reduced access to adequate food, shelter, clothing, and medical treatment. The stigma associated with poverty and mental illness (67, 68) may be further magnified in low resource nations (69).

Homelessness and Housing

Stable housing is an important prerequisite for recovery. Persons who lack stable housing, particularly the homeless, commonly experience higher rates of physical and mental illness, poor quality of life and high mortality (70–74). At the core of homelessness are the processes of marginalization and exclusion, with multiple risk factors driving these experiences (75–78). For instance, persons with childhood trauma, mental illness, and substance use are at greater risk for housing instability and homelessness (79). Homelessness arises from an interplay of individual vulnerabilities, interpersonal, structural, and systemic factors. Individual factors consist of psychosocial vulnerabilities connected to family background, health, and trauma. Interpersonal factors are linked to behaviors like substance use. Poverty, unemployment, and lack of affordable housing are examples of the structural factors; lack of social security of the systemic ones.

Substance Use

The relationship between substance use and mental illness is multifaceted. Based on a recent meta-analysis, more than half of persons with mental illness have a comorbid substance use disorder (80). This comorbidity leads to poorer clinical outcomes, worsened physical and mental health; higher levels of disability; increased risk of suicidal behavior (81), homelessness, and psychiatric readmissions (82); and greater difficulties in interpersonal and family relationships (80). Symptoms of mental illness may interfere with substance use treatment and the reverse may also be true (83).

Proposed mechanisms for this comorbidity include self-medication where people use substances to treat their own mental health symptoms; the reverse-causal pathway where substance use causes or worsens the symptoms of mental illness or side effects of substances produce symptoms similar to mental illness; and a shared environmental or genetic vulnerability where factors like poverty or childhood trauma increase risk for both (84). Considering the complexity of these relationships, comprehensive assessment is needed to understand the context of substance use in relation to symptoms of mental illness.

Trans-Institutionalization and the Criminal Justice System

Deinstitutionalization of mental health services began in Canada and other countries in the 1960s–70s with the rapid closure of psychiatric hospitals followed by further reductions in general hospital admissions for mental health in the 1990s (85). In most countries, the motivations for this change included breakthroughs in pharmaceutical treatments (86) combined with an emphasis on human rights and social inclusion of persons with mental illness (87), and the expectation of better outcomes in the community (88). However, legal concerns and fiscal decisions aimed at reducing health care expenditures (85, 89) were also important drivers. Globally, there is considerable variability in rates of deinstitutionalization (90, 91) despite broad based philosophical support (92, 93).

Deinstitutionalization is associated with improved life expectancy for persons with mental disorders (94), disability reduction (90, 91), better quality of life, and greater autonomy (95). Adverse consequences include a reported rise in trans-institutionalization, homelessness, and criminalization of persons with mental illness (96–99). While there is debate about the extent to which deinstitutionalization has resulted in negative outcomes (100, 101), contact with the criminal justice system is now a common experience for persons with mental illness.

Police have become the first line of contact for many persons living with mental illness (102, 103), with substance use and impulse control issues as common causes (104). In some jurisdictions, the large-scale expansion of inpatient forensic mental health services has been driven by increased admissions of younger individuals from ethno-racial minorities with low levels of violence and substance use offences (105). Specialized mental health courts provide alternative mechanisms for dealing with offenders with mental illness (106), and more advanced analytics of data based on high quality assessments will be essential for the improvement of forensic mental health services (107).

Recovery

The shift to community-based mental health services has been accompanied by a transition from a biomedical care model to one that emphasizes personal recovery (108–110). While there is no common global definition of recovery (111), there are agreed-upon common core principles. For example, that mental health is viewed in terms of a continuum of functioning; recovery may be an iterative or non-linear process; and belonging, community engagement, hope, connectedness, identity, freedom from stigma, meaning in life, and empowerment are key markers to consider (108). Providing a supportive environment that builds on strengths and is calibrated to needs is a priority, rather than stopping services (112). Treatment of symptoms is important, but “clinical recovery” is either not enough or not fully possible in some cases. Therefore, assessment must consider the person in a broader social context to reach beyond symptoms alone.

Stigma

Stigma associated with mental health problems often hinders full citizenship and inclusion in society (113, 114). At the societal level, public stigma includes labeling persons with mental illness as dangerous or less competent than the general population. At the individual level, self-stigma occurs when persons apply psychiatric labels to their identity, often with negative consequences. Those who experience perceived stigma or discrimination may apply stereotypes to themselves resulting in underestimation of their own potential and a sense of helplessness (115). This may lead to avoidance of necessary care, increasing problems in multiple domains, and worsened prognosis (116, 117).

Conglomeration Versus Integration of Assessment

If we accept that standardized assessment has value, we must consider how to define that standard. In many health care settings, usual practice has been to create a conglomerated information system made up of home-grown intake forms combined with a collection of clinical scales used to measure singular issues (e.g., cognition, depression) on an “as-needed” basis.

Data from these conglomerated systems are typically not automated and forms completed separately are not linked. For the service recipient, this can lead to repetitive measurement of the same issues and limited access to clinically relevant information for team members in the circle of care. Longitudinal case histories cannot be constructed easily, linkages between clinical issues tracked on different forms may go unrecognized, and person-level analytics cannot be readily applied to the clinical record. Data compiled this way have little utility for population level analyses because converting them into a complete, electronic database is a daunting task. Handwritten notes on incomplete forms stored in locked filing cabinets serve little purpose beyond single-case after-the-fact chart audits.

Even if data are automated, when the choice of which scale to use for a particular concern is left to individual clinicians, there will almost certainly be no measurement consistency at the organization or system levels. The Handbook of Psychiatric Measures (118) contains 26 chapters with over 275 distinct psychiatric measures, and that list was almost surely incomplete at the time of publication. Assuming one could resolve the controversy over whether the Montreal Cognitive Assessment should be used in place of the Mini Mental State Examination (119), what is the likelihood of a consensus on which of the nine depression measures listed in the Handbook should be used with all patients, in all settings, during all episodes of care? At the population level, inconsistent data have diminished value. Imagine the response by an organization ranked negatively in an outcome-based quality indicator where different providers use different tools. The fact that an indicator was “not measured the same way” across settings will be the immediate defense used to negate the validity of any differences in performance suggested by the indicator.

Using specific scales on an “as-needed” basis is also a flawed strategy. We do not always know who has a given problem without asking. Indeed, one of the values of comprehensive assessments is that they force the clinicians to ask about underlying issues that go beyond the immediate, presumed problem. Persons with substance use problems are not only found in substance use programs; depression affects people other than those in a mood disorder program; cognitive impairment spans the barriers of a dementia unit. Under-detection of issues like pain, depression and dyspnea is an important problem throughout the health care system (120–122).

If we succeeded in achieving a consensus on a set of tools to measure psychiatric symptoms using a collection of single-focus measures, the data volume and time to complete assessment could still be unwieldy. For example, a set of 15 well-known mental health instruments would measure cognition, (123), delirium (124), depression (125), anxiety (126), psychosis (127), mania (128), trauma (129), pain (130), sleep (131), behavior (132), substance use (133), instrumental activities of daily living (134), functional status (135), suicidality (136), and aggression (137). Despite comprising more than 400 different items, this set does not address many of the key issues highlighted previously.

Further, intellectual property and licensing requirements may be difficult to resolve. Within a single clinical setting the task of securing permissions for 15 or more different instruments would be a challenge, but to do so on a national level is impractical. This would be further complicated by electronic health record vendors licensing requirements, as well as the challenge of training clinical staff on coding conventions that likely differ dramatically among instruments. Finally, though the conglomerated set of assessments may measure the severity of an issue at a given time and can be used on a repeated basis to monitor changes, these measures generally do not invoke a clinical response in reaction to their numeric scores.

The alternative to a conglomeration of stand-alone tools is to use a single, integrated assessment system that can be applied across care settings to persons of any age. The system should serve multiple applications for multiple audiences, including triggering a clinical response leading to shared decision-making in support of key recovery goals.

The interRAI Suite of Mental Health Instruments: An Integrated Assessment System

interRAI (www.interRAI.org) is a not-for-profit network of over 100 researchers, clinicians and policy experts from over 35 countries in North and South America, Europe, Asia, Africa and Oceania (138–140). It was founded with an initial focus on geriatric research in the early 1990s, but its scope broadened to include vulnerable persons of all ages. interRAI's multinational program of research aims to develop and implement comprehensive assessment and screening systems to support improved quality of care and quality of life among persons of all ages with complex needs across the continuum of health and social service settings. This includes the creation of psychometrically sound measurement systems, the application of data from those assessments to support multiple uses by multiple audiences, and completion of multinational comparative analyses and “natural policy experiments.”

The initial innovation behind interRAI assessments is that they were designed not only to describe status or severity at a given time, but also to invoke a clinical response through the use of embedded triggering algorithms and care planning guidelines (141).

The first interRAI instruments were designed to be used in a single sector at a time (142, 143), and the release of interRAI's mental health instrument for in-patient psychiatry (144) represented its first effort to target the general adult population. In 2000, interRAI launched a major effort to redesign all of its assessment instruments to function as a fully integrated suite of measures (145–148). The most recent developments by interRAI include the creation of a parallel suite of instruments for children and youth (149, 150), screening systems for use by non-health professionals (151), and a set of self-report tools to measure patient experience (152–154) and patient reported outcome measures(155, 156).

The interRAI suite of assessment instruments includes over 20 comprehensive assessments, supplementary assessments, and screeners. All of these systems include indicators related to mental health (particularly cognition and depressive symptoms). However, the focus in the following section will be on the adult versions of the mental health instruments using homelessness to illustrate how they can provide insights about strengths, preferences, and needs.

Assessment and Screening Instruments for Adults

Mental Health Settings

Inpatient Psychiatry

The interRAI Mental Health (MH) assessment (157, 158) supports care plan development in 20 domains, and is used in in-patient mental health settings at admission, discharge (if more than 7 days after admission), every 90 days (for long-stay patients), and when there is a clinically significant change in the person's status that is not a self-limiting and could require modifications to the care plan. The instrument is available in English, Canadian and Belgian French, Flemish, Icelandic, and Finnish.

The first version of the instrument was released in 1999, with a major update in 2002. The most recent version 9.1.2 was published in 2012 (157) with revisions designed to make it fully compatible with the interRAI suite (147). The MH was pilot tested in Nordic countries and the US, but the primary implementation has been in two Canadian provinces (Ontario, Newfoundland and Labrador). Local implementations have occurred in two other provinces (Quebec, Manitoba). To date over 1.4 million assessments have been completed on over 320,000 unique individuals in Canada.

The target audience is adults aged 18 years and older, including acute, long-stay, forensic and geriatric patients. Canadian provinces have also implemented the MH in adult-designated beds, even if those are occupied by persons under 18 years of age. This proved to be a helpful source of data for publications that pre-dated the completion of the child-youth suite of mental health instruments (159).

The assessment is comprised of 396 items in several domains (see Table 1). It is mainly completed by mental health professionals (typically nurses or social workers), and includes a limited number of self-report items. Most items employ a standard 3-day look-back period, although service use and therapies use a standard 7-day window. Other items use 30-day, 90-day or lifetime estimates depending on the nature of the issue. The average time to complete the assessment is 1 hour, and it can be used as a replacement for a standard nursing intake assessment. The MH instrument deals with items that usually would be addressed in mental health assessments, but also expands the scope to look at broader issues within a time frame for completion that is consistent with conventional practice. It contains about the same number of items as the hypothetical conglomerate of stand-alone tools described earlier, but it substantially increases the scope of issues addressed.

TABLE 1

Table 1 Item counts by domain area and interRAI mental health system for adult populations (age 18+ years).

Community Mental Health

The interRAI Community Mental Health (CMH) assessment (160) supports care plan development in 20 domains. It is designed for community agencies employing mental health clinicians, including those with case managed mental health services and assertive community treatment programs. The assessment can be used at intake, discharge, every 6 months depending on length of stay, and after a change in the person's status that requires care plan modification. If the person is admitted to the community agency from an inpatient setting using the MH, the discharge MH assessment is shared to support continuity of care (161) to allow additional time for community staff to establish a clinical relationship with the person. The target audience for the CMH is also adults aged 18 years and older, including a broad range of service recipients. Although community mental health agencies are the typical service provider, it has been used in consultation liaison services for long-term care to deal with psychiatric and behavioral issues outside the usual scope of practice for clinicians in those settings as well as a Dutch study of homeless services recipients (162). The original version of the CMH released in 2007 was pilot tested in Ontario, Canada, New York State, Finland, Belgium, Chile and Hong Kong. The most current version 9.2 published in 2012 is compatible with the interRAI suite. Newfoundland and Labrador have begun provincial implementation of the system, as well as regions in Ontario and Quebec. The Swiss Home Care association has announced plans for national implementation. The instrument is available in English, French (Canadian, Belgian, Swiss), Swiss German, Swiss Italian, Flemish, Finnish, and Chinese. For this paper, 12,862 assessments from pilot studies are available for analyses. Because of constraints related to European data protection laws, it was not possible to complete pooled analyses with those data (162).

The CMH includes 405 items dealing with comparable domains and look-back periods as used in the MH. The two instruments share 330 common items, but the CMH has 75 items not present in the MH and the MH has 66 items not present in the CMH. The main differences relate to issues that are encountered in one, but not the other, service environment. For example, the CMH includes an expanded range of items on social relationships and supports in the community, and on the home environment. The MH includes items on control interventions (e.g., restraints) not used in community settings.

The average time to complete the CMH is also an hour, but completion time may be affected by the lack of access to another informant for persons who have difficulty responding. This issue poses a greater challenge than noted in the interRAI Home Care assessment (163) where caregivers are routinely available as additional sources of information.

Emergency and Crisis Services

The interRAI Emergency Screener for Psychiatry (ESP) (164) is designed for general emergency departments, psychiatric emergency departments, and mobile crisis teams. Like the MH and CMH, the ESP is typically completed by nurses, social workers, or clinicians other than psychiatrists. It is used at the time of crisis/emergency, with the expectation that a follow-up assessment for persons remaining on service would revert to the MH or CMH once the person is stabilized.

The first version of the ESP was pilot tested in Ontario in 2004, and the interRAI suite version 9.1 was published in 2011. The ESP's 141 items are a subset of those in the MH and CMH. The instrument has been pilot tested in Ontario and Quebec, Canada and Finland. Two regional mental health services in Canada have begun implementation of the ESP and a child-youth variant of the instrument. The current data holdings available at the time of writing comprised 5,249 ESPs completed in Canadian organizations.

The target audience for the ESP is also adults aged 18 years and older; however, the clinical focus of the ESP is narrower. Whereas the MH and CMH focus on care plan development in 20 areas, the emphasis of the ESP is on patient safety issues and acute symptoms. Thus, the ESP has a 24-hour look back period to focus on immediate presenting concerns. It triggers care plans in three areas of safety (harm to self, harm to others, inability to care for self), and in substance related withdrawal symptoms.

The average time to complete the ESP is 30 min; however, the acute nature of illness may pose barriers to continuous completion of the assessment if other informants are not available.

Police, Emergency Medical Services, and Other Non-Mental Health Settings

The interRAI Brief Mental Health Screener (BMHS) is a short screening tool intended to be used by non-mental health professionals who may be the first line of contact for persons with mental health needs (165). The initial target audience for the BMHS is police services (151) with the aim of facilitating improved communications among polices officers, emergency mental health services, and community mental health agencies (166). The BMHS is a 46-item subset of the ESP, but the training manual includes additional materials for training non-mental health staff in the use of appropriate terminology to describe the person's presenting symptoms. It is designed to be used at a single time point, and it employs a 24-hour look back period.

The first draft version of the BMHS was developed using analyses of 41,019 MH admission assessments to determine the minimum subset of items needed to identify persons who would be admitted to inpatient psychiatry due a combination of disordered thought and danger to public safety (167). The draft BMHS was tested with two police services and five hospitals in Ontario in 2011. The finalized version 9.3 of the BMHS was published in English and French in 2015, and the screener is now in extensive use by over 40 police agencies (local, provincial, and federal) in four Canadian provinces. A pilot study of the BMHS in one US region is expected to launch in 2020. For the purposes of the present paper, 70,005 Canadian BMHS assessments were used.

Unlike the previous assessments, the BMHS is not a care planning tool. It is designed to provide a systematic means of summarizing the observations of police officers (and other non-mental health staff) using the same items that are employed in the interRAI mental health suite. It can also be considered a mental health training intervention, because the screener provides guidance to those using the instrument on how to identify and describe mental health symptoms. The BMHS includes three patient safety algorithms related to harm to self, harm to others, and inability to care for self based on machine learning analyses of MH and BMHS data (168). These algorithms help police and others to communicate acuity to hospitals and community mental health agencies in real time. The Ontario Human Services and Justice Coordinating Committee recommended the province-wide use of the BMHS in 2019 in order to facilitate more timely and appropriate transfers of the care of persons with mental illness from police control to mental health agencies (169).

There is considerable potential to employ the BMHS in other settings. Emergency Medical Services are also in frequent contact with persons with mental health crises in the community and face similar decision-making challenges. In addition, a non-police version of the BMHS is available that could be used with peer-led agencies, shelters, food banks, or other settings for persons with mental illness.

Forensic and Addictions Services

The interRAI MH and CMH instruments both address a broad range of adult service recipients, including persons with addictions and those in contact with the criminal justice system (170–172). However, in forensic and addictions programs, there may be value in obtaining additional information about the severity of the problem (e.g., criminal convictions) or items that would not be asked routinely for all service recipients (e.g., readiness for change, health symptoms associated with substance abuse, dynamic and static forensic risk factors).

To that end, interRAI has two new supplements for the MH and CMH to expand depth in criminal justice and addictions with 33- and 35-items, respectively. These items provide additional severity measures, risk algorithms and specialized care planning triggers for these clinical subpopulations only. At the time of writing, both supplements are in beta versions, with expected publication in 2020.

Non-Mental Health Settings for Adults

All interRAI assessment and screening systems, including those for non-mental health settings, have at least a core set of mental health items. A brief description of some of the main adult instruments follows with examples of mental health research done in those settings.

Intellectual Disability

The interRAI Intellectual Disability (ID) assessment (173, 174) is used with adults aged 18 years and older with intellectual and developmental disabilities (e.g., Down syndrome, autism) in community and residential services. It comprises 320 items, including 188 in common with the interRAI MH, and it is usually completed by developmental services workers. The ID includes support planning protocols for abuse by others, communication, continence, injurious behavior, meaningful activities, mental illness, and social relationships (175). The mental health content of the ID includes measures of psychological well-being, stressful life events and trauma, cognition, psychosis, extrapyramidal symptoms, mood, anxiety, negative symptoms, sleep disturbance, behaviors, violence, and previous psychiatric hospitalizations. These indicators have been the focus of a number of studies in ID settings (176–180) as well as cross-national comparison of persons with ID (181).

The interRAI ID has been implemented in the state of New York (USA) and in Prince Edward Island (Canada). It was used in Ontario, Canada to support the closure of the province's last three large institutions for persons with intellectual disabilities (179, 180, 182–184). Several other jurisdictions in the USA and Canada have announced plans to implement the ID. ID supplements to the MH and CMH instruments are currently being pilot tested in Ontario and in Belgium.

All interRAI mental health assessments include items on intellectual disability, since persons with dual psychiatric and ID diagnoses are an important subpopulation in mental health settings. Several interRAI papers have examined persons with intellectual disabilities in trans-institutional settings (179, 185–189).

Home Care and Nursing Homes

interRAI's Long Term Care Facility (LTCF) and Home Care (HC) instruments (142, 143, 163, 190, 191) were first developed more than 25 year ago. By the end of 2018, over 9 million interRAI home care and nursing home assessments had been completed in Canada alone. In the US, since 1990, over 100 million interRAI nursing home assessments have been completed. The use of LTCF and HC is worldwide; other nations with large scale implementations of one or both of these instruments include Iceland, Finland, Belgium, France, Switzerland, Italy, Hong Kong, and New Zealand.

The LTCF and HC contain 322 and 318 items, respectively, and are completed by health professionals (typically nurses or social workers). The mental health items assessed in both instruments include cognition, delirium, mood, behaviors, psychosocial well-being, psychosis, alcohol and psychotropic medications. The 27 care planning protocols triggered by the HC and LTCF include ones dealing with cognitive loss, delirium, mood, behavior, abusive relationships, tobacco and alcohol use, and appropriate medications (192).

The mental health research in long-term care with the LTCF and HC includes depression (193), bipolar disorder (194, 195), suicidal behaviors (196, 197), traumatic brain injury (198–200), delirium(201, 202), aggressive behavior (203–205), and cognitive impairment (206–210).

Two quality of care issues of considerable interest are use of physical restraints (211–218) and potentially inappropriate use of antipsychotics (211–214, 219–225). The Canadian Institute for Health Information (CIHI) now reports interRAI's risk adjusted quality indicators for restraint and antipsychotic use in nursing homes in seven provinces/territories. This will expand to all other jurisdictions except Quebec once current implementations are complete.

Instruments for Children and Youth

The newest instruments in the interRAI mental health suite are those designed for children and youth. Implementation of the interRAI MH in all adult inpatient beds in Ontario was mandated in 2005 in response to recommendations from the province's Joint Policy and Planning Committee (JPPC) of the Ministry of Health and Long Term Care and the Ontario Hospital Association. The JPPC also called for development and implementation of a compatible system for children and youth. In response, interRAI researchers developed the interRAI Child/Youth Mental Health (ChYMH) assessment system for children aged 4–18 in mental health settings (226) as well as a shorter screener (227).

The ChYMH has 382 items and 31 summary scales (e.g., internalizing, externalizing, distractibility and hyperactivity, aggression, anxiety, social disengagement, depression severity index), and risk assessments to inform the intensity and nature of the child or youth's service needs (149, 159, 228–236). Additionally, the ChYMH has 30 care planning protocols and a preliminary system to describe resource intensity (237). An Adolescent Supplement covers items not addressed with younger children (e.g., substance use) and a Youth Justice Supplement is for youth in correctional settings. A variant of the ChYMH is also available for children/youth with intellectual and developmental disabilities (238). Finally, a new instrument for newborns to three-year olds is nearing completion (239).

The interRAI ChYMH has already been adopted by 90 children's agencies in Ontario, and three other provinces have expressed interests in adopting the system. In addition, there were 16,588 MH assessments on children aged 13–17 in adult mental health beds in Ontario by December 2018.

The availability of an integrated suite of mental health assessment systems that span the life course from newborns to centenarians provides enormous potential to improve care with various transitions that occur over the life span. In addition, at the person-level these compatible assessment systems can provide a rich continuous clinical picture of the person's growth, development and aging from a comprehensive perspective. At the population level, the large-scale implementations of these instruments portend the emergence of a new, detailed longitudinal database of large cohorts of individuals living with mental illness in the earliest stages of their lives until the latest stages and end of life.

What Makes interRAI an Integrated Mental Health Information System?

The following factors make interRAI's mental health assessments and screeners an integrated health information system that spans the continuum of care for persons of all ages. In this and subsequent sections, interRAI data holdings are used to illustrate concepts discussed below. Table 2 provides a summary of the data sources used in these examples.

TABLE 2

Table 2 Data sources used to illustrate concepts in manuscript.

Common Language

All interRAI instruments use common terminology to define the same items across all settings. Mood, psychosis, cognition, pain, function and physical health systems use the same definitions, inclusion/exclusion criteria, and phrasing wherever they appear irrespective of the type of setting. Items in the child/youth instruments only differ from the adult instruments when a developmental rationale requires the difference (e.g., performance of activities of daily living). Also, most instruments employ a standard 3-day look back period; except the ESP and BMHS use a 24-h look back due to the acute nature of the conditions addressed.

Common Conceptual Basis

All interRAI assessment instruments are designed first and foremost to support care planning, by embedding data elements where algorithms trigger the need for further intervention in a given area. The development process for instruments always begins with identifying the focal domains for care planning and then identifying the minimum item set to trigger the need for additional investigation. Secondary consideration is given to items that are not used for care planning, but could be used for outcome measurement, resource allocation, of quality management. Specific items to only track matters of research interest are not included in these systems.

Common Clinical Emphasis

The primary focus of interRAI assessments is on function and symptoms rather than diagnosis. These instruments track provisional and finalized psychiatric and medical diagnoses and they provide rich clinical information that is relevant to diagnosis, but they are not intended to be diagnostic systems. Rather they focus on what are the person's strengths and needs, how the person relates to others and engages in the community, what the person can do independently, and where support needs are required.

Common Data Collection Methods

All interRAI systems employ a common and consistent assessment methodology. For the clinician-administered instruments, the assessor is provided with specific item definitions, time frames, inclusion/exclusion criteria, lists of examples, coding guidelines, and instructions for the assessment approach. The assessments do not use a fixed narrative set of questions, and the order of completion can be adapted to the natural flow of the assessment process on a case-by-case basis. Assessors are aware of the information they need to acquire for a given item, but they are free to adapt approaches to each item to be culturally or gender-appropriate. Assessors use all sources of information available to them, including direct questions posed to the person, observation of the person's behavior and mannerisms, interviews with family or other members of the support network (where appropriate), information exchanges with other members of the circle of care, and medical or other records. When these information sources provide conflicting evidence, clinicians exercise their best judgement to record what response they believe to be the most correct answer for a given item. It is also possible to parcel out portions of the assessment to different staff members. For example, clerical staff could readily complete administrative and tracking elements or data on historical service utilization. Other mental health disciplines may complete some but not all portions of the assessment. For auditing practice, one individual must sign-off that the assessment is complete and as accurate as possible given available information.

This assessment approach is standardized for all interRAI systems and shared items are assessed in the same manner. What differs among instruments is the curated item sets and associated scales and algorithms. Thus, information between sectors can be compared directly, and staff with experience in interRAI systems in one sector can easily learn another interRAI instrument if they move to another sector. While items sets and clinical issues may be new, the approach to completing items remains consistent.

Common Core Elements

Mood, pain, cognition, and functional status are basic human characteristics that are relevant in any setting and any age group. In that light, there is a set of common items that are measured in almost all instruments. The main exceptions are screeners that are intended to use a reduced item set for limited targeting purposes, and the newborn to three year-old instrument that has items dealing with earliest stages of human development.

This consistency of measurement allows for the examination of some clinical issues at the population level. For example, Figure 1 shows the distribution of interRAI's Cognitive Performance Scale for about 1.7 million individuals across a variety of adult service settings in Canada (see Table 2 for data sources). Scores range from zero for cognitively intact to six for very severely cognitively impaired. There is a clear transition from less to greater cognitive impairment as one moves from populations whose usual residence is in the community to facility-based settings. The most severely cognitive impaired population is nursing home residents with a previous history of psychiatric hospital admissions. The third most severely impaired population is inpatient psychiatry patients who were admitted from nursing homes. Both of these trans-institutional populations have much more severe levels of cognitive impairment than the general inpatient psychiatry population.

FIGURE 1

Figure 1 Percentage distribution of cognitive performance scale scores across Canadian care settings.

Common Care Planning Protocols

interRAI's clinical assessment protocols are designed to inform the care planning process for issues identified in the assessment. These protocols are grouped according to clusters of care settings serving persons with common clinical issues who may nonetheless be at different points in the treatment trajectory. The MH, CMH, and ESP are adjacent sectors of the mental health system that use the same set of mental health protocols to address the needs of their shared populations. Similarly, the LTCF, HC and interRAI Community Health Assessment share common protocols for a population that is generally affected by geriatric issues or issues related to physical disability paired with medical complexity and/or cognitive impairment. The protocols for supporting adults with ID can be used in residential and community settings, and the ChYMH instruments have shared protocols for community and facility based settings for children/youth.

Psychometric Properties of interRAI Mental Health Instruments

One of the main value propositions for adopting research-based assessment systems over “home grown” bespoke forms is that decision-makers must have confidence in the veracity of the findings generated by the assessment. The data must be meaningful, accurate, and relevant to the issues affecting the person who has been assessed. Reliability and validity are two basic psychometric properties to be considered and the cornerstone for instrument development (240).

Reliability

Inter-Rater Reliability

In any health care system it is essential that independent assessments by different trained clinicians yield consistent answers. To be cost-effective and to minimize assessment burden, it should be feasible to use assessment information completed by another colleague (including from a different care setting) to inform our own clinical decision-making process.

To appraise this, interRAI has a long history of engaging in inter-rater reliability testing of its assessment systems (142, 143, 191, 241–248). The approach used has been to conduct independent assessments of service recipients without contact between the assessors and with a limited time gap between assessments. This conservative, yet pragmatic, approach mimics what may occur in usual practice in health care settings. Although it is convenient to use vignettes in reliability research they are avoided because they do not capture the nuances and complexities of “real life”. Similarly, assessors do not work together in pairs because that approach violates assumptions of independence between observers (249), which could inflate reliability estimates and does not reflect usual practice patterns in health care.

Recognizing that interRAI assessments share a common core of data elements that are combined with specialized items unique to a given sector, it is feasible to use inter-rater reliability evidence from multiple sectors to evaluate instrument performance. In a 12-nation study of inter-rater reliability of five different interRAI instruments (MH, LTCF, HC, Palliative Care, and interRAI Post-Acute Care), the mean weighted kappa for the core items common to all instruments was 0.75 and the kappa of the specialized mental health items was 0.64 (145). Both results suggest “substantial” inter-rater reliability (250).

Internal Consistency

Challenges with inter-rater reliability studies are that they are expensive, intrusive to the person being assessed twice, and of no clinical benefit to that person. Therefore, it is useful to employ other statistical approaches to reliability evaluation that can be done continuously without additional assessment. When instruments use parallel form scales to measure underlying domains with multiple items, it is possible to compute measures of internal consistency using Cronbach's alpha (251). This approach is used to evaluate new scales developed for the interRAI suite, but it can also be deployed to monitor data quality continuously with large scale implementations as has been done with other interRAI assessments (252, 253).

Table 3 describes several clinical summary scales that are available in the interRAI mental health suite and Table 4 shows the internal consistency results for seven parallel form scales from four interRAI mental health instruments in three countries. The scales include symptoms typically of interest in mental health settings; however, the two measures of functional status (activities of daily living (ADL) and instrumental activities of daily living (IADL)) are relevant to understanding disability in the assessed populations. Virtually all scales in all settings examined in Table 4 meet or exceed the standard alpha value of 0.70 for good internal consistency and, in several instances, exceed 0.80 indicating excellent reliability. Good reliability is evident in both acute and longer-term service settings. In addition, the Positive Symptoms Scale, as assessed by police officers, has comparable reliability to that found when administered by mental health clinicians. The two exceptions in performance are the Aggressive Behavior Scale (ABS) in US community mental health settings and the Mania Scale in Canadian ESP settings. The US findings may be a function of an attenuated distribution of the ABS in the US pilot site (see Table 5). Site specific analyses of the Mania scale performance indicate that the problem was with one ESP pilot site, suggesting that training may have been a concern. Put differently, 39 of 41 evaluations of scale reliability in diverse mental health settings met or exceeded standard cut-points for good reliability.

TABLE 3

Table 3 Summary of scales and algorithms in interRAI mental health instruments.

TABLE 4

Table 4 Internal consistency of clinical scales derived from interRAI Mental Health Instruments, by country.

TABLE 5

Table 5 Univariate distributions of selected clinical scales derived from interRAI instruments by country.

Validity

Validity is a more complex psychometric issue than reliability, requiring evaluation through a number of methods. The key questions of interest include: does this item or scale measure what I think it measures? Does the assessment address the important characteristics affecting the person's well-being? How does the instrument compare with other widely used systems? Do risk indicators actually predict future outcomes of interest?

Face and Content Validity

Face and content validity are necessary, but not sufficient criteria to meet for developing an assessment system. The interRAI development process typically addresses these through extensive involvement of front-line practitioners and researchers combined with reviews of the current literature. For example, as part of the development effort for the care planning protocols, key informants from several nations were asked: a) How consistent is the MH-CAP with the Recovery Model as used by your organization? (83% rated the protocol reviewed to be mostly or completely consistent); b) How consistent is the MH CAP with guidelines/best practices used by your organization? (92% rated them as mostly or completely consistent); and c) How would you rate this CAP in terms of its use as a support for care planning in this area? (90% rated them as good or excellent). In addition, critical feedback from informants was used to inform final revisions to the penultimate versions of these care planning protocols.

Convergent Validity

The patterns of associations in a dataset can provide insights about whether the instrument measures what one thinks it measures. The initial version of the MH was successfully evaluated by examining the presence of expected associations with age of cognitive loss and functional decline, the relationship between depression measures in suicide related indicators, readmission rates and problems with medication management, and cognitive performance with behavior (248).

As an extension of this approach, one might examine the extent to which hallmark clinical symptoms of various psychiatric diagnoses are associated with the presence of those diagnoses in the assessment data. A clear positive association would provide reassurance that the instrument measures symptoms that have meaning to clinicians. Table 6 shows the relationship that could be expected between various mental status indicators and provisional diagnoses of neurocognitive disorders; substance related and addictive disorders; schizophrenia spectrum and other psychotic disorders; and depressive disorders. For the first three diagnoses in all three settings, the anticipated relationships with symptoms are strong, in the appropriate directions, and the c-statistics are generally at the 0.80 level or higher. For depression diagnoses, the odds ratios for the depressive severity index and social withdrawal index are in the appropriate directions, but the associations with depressive symptoms are stronger than with social withdrawal (which may relate to other diagnoses). The c-statistics for depression diagnoses are between 0.64 and 0.70.

TABLE 6

Table 6 Odds ratios (95% CL) for provisional psychiatric diagnoses by associated symptoms and setting, Canada.

Criterion Validity

A common approach to validation is to compare a new instrument to another scale that is recognized as a “gold standard” measure. The challenge in mental health is that few measures are universally accepted as a gold standard, and biological markers are not particularly useful as they might be in somatic illness. Previous interRAI research has established the criterion validity of the following scales and items: pain scale vs Visual Analogue Scale (254); Cognitive Performance Scale versus Mini Mental State Examination and Montreal Cognitive Assessment (255, 256); Depression Rating Scale versus Hamilton and Cornell Scales and psychiatrists ratings (257); Aggressive Behavior Scale versus Cohen Mansfield Agitation Inventory (203); delirium algorithms versus the Confusion Assessment Method (258); recorded diagnoses versus acute hospital medical records (259); and mental health care planning triggers versus clinician judgement (260). In a developmental project to refine an earlier version of the interRAI MH, criterion validity studies with 876 patients in 11 psychiatric hospitals/units yielded the following correlations: Aggressive Behavior Scale with Neuropsychiatric Inventory Total Score r = 0.50 and with PANSS Aggression Risk Profile r = 0.58; Pain Scale with McGill Pain Questionnaire r = 0.64; Positive Symptoms Scale with PANSS Positive Symptoms r = 0.62; Negative Symptoms Scale with PANSS Negative Symptoms r = 0.49 (261).

Predictive Validity

Arguably the most important (and difficult) form of validity to establish for an assessment system is predictive validity. Presumably, the ultimate purpose of assessment is to guide interventions that will have an impact on a future clinical trajectory of change. This approach was used extensively to identify triggering rules for interRAI's mental health care planning protocols (see discussion below). Examples of publications reporting on this type of validity include studies of inpatient violence (262), re-hospitalization (82); and suicide behaviors (196, 197).

Applications of interRAI Instruments

Critically, all interRAI assessment instruments are designed to be used as part of normal clinical practice to serve multiple functions for multiple audiences (263), including: a) care planning; b) outcome measurement; c) resource allocation; and d) quality improvement. In addition, the aggregated data compiled as part of regular use can be employed for policy analysis, evaluation and research. Examples of peer-reviewed health services research based on interRAI MH data include: mental health needs of prisoners (264, 265); mental health service planning (266); access to psychiatrists by linguistic minorities (267) and nursing home residents (268); use of ECT by psychiatric inpatients (269); prolonged stays (270, 271); length of stay (272, 273); and restrictions in movement in forensic patients (172). Examples of clinically oriented research with these data include studies of: sexual dysfunction (274, 275); incontinence (276, 277); discharges against medical advice (229, 278); medication non-adherence and misuse (229, 279, 280); restraint and acute control medication use (281); and pharmacoepidemiological studies (282).

Care Planning

The Clinical Assessment Protocols (CAPs) associated with the suite of interRAI assessments are care planning guidelines designed with a common structure and clinical approach, but they are adapted to the populations and service settings for which they should be used (175, 192, 283–286). Each CAP contains five main components: a) a description of the importance of the clinical issue; b) goals of care underlying CAP design; c) triggering algorithms that use the assessment items to classify persons for different care planning approaches; d) guidelines to consider for both immediate actions to deal with safety concerns and longer-term strategies that may be used; and e) additional reference materials and resources that may be consulted.

The CAP development process involves three main approaches. First, interRAI's large international network is deployed to engage with clinicians, researchers, policy makers, and advocacy groups to discuss priorities and approaches to care recommended in different countries. Second, reviews of literature and best practice guidelines help identify practical, evidence-based interventions for specific clinical concerns. Third, interRAI's large longitudinal data holdings are used to develop and validate the predictive validity of CAP “triggering” algorithms. The results of these analyses are included in the mental health CAP descriptions to identify expected triggering rates and outcomes for inpatient, community and emergency settings. Finally, agreement with clinician ratings was also used to validate and refine the CAP triggering rules (287).

CAPs are designed using several key principles. The intervention guidelines must be rooted in empirical evidence from the peer reviewed literature in multiple continents so as not to reflect only one system of care. In addition, all mental health CAPs are framed on recovery principles (108, 111), including shared decision-making and support of the person's autonomy calibrated to his/her current level of functioning. The approach engages the person and, where appropriate, the informal support network in collaborative discussion to review the assessment findings about the person's strengths and needs and to identify the person's preferences for how CAP issues will be addressed, if at all. CAPs are not intended to be robotic care planning libraries. Rather, they provide qualitative “interview guides” that outline potential responses to the quantitatively defined triggering algorithms derived from the standardized assessment. Similarly, the CAPs are not a diagnostic system; they are designed to focus on intervention strategies at the person, family, and community levels that might enhance the person's quality of life in as many dimensions as possible.

The CAPs triggered by the interRAI mental health suite are listed in Table 7. The interRAI MH and CMH trigger all CAPs, but have somewhat different protocols for informal supports. The interRAI ESP triggers mainly the patient safety related CAPs as priority issues for crisis situations. Previous publications have highlighted triggering patterns and outcomes associated with the CAPs for traumatic life events (288), education and employment (289), and harm to others (262). Figure 2 shows the triggering rates for patient safety CAPs by homelessness in different service settings. The triggering rates for these CAPs in homeless persons in four settings (community dwelling recipients of Dutch homeless services, Canadian community mental health, emergency/mobile crisis, and inpatient settings) are lowest for the purposeful self-harm CAP and highest for the self-care CAP. With the transition from community to acute hospital-based services, the triggering rates for purposeful self-harm and harm to others are higher for all groups. Conversely, self-care triggers at the highest rate for homeless persons in community mental health settings. For non-homeless populations, the triggering rates for patient safety issues are generally lower than with homeless persons, but the rates and severity levels are also higher in emergency and hospital settings than in the community.

TABLE 7

Table 7 List of clinical assessment protocols triggered by different interrai mental health instruments.

FIGURE 2

Figure 2 Triggering rates for three patient safety CAPs, by homelessness, setting and country.

The CAPs can be used at the person-level to inform care planning, or aggregated at the organization or population levels to provide estimates for need analyses in various domains. The CAPs focus on current issues that increase the risk of adverse outcomes or those that might be addressed to support recovery. For example, the CAP for traumatic life events (288) targets two sub-groups for intervention: a) persons who face immediate threats to their safety due to assaults, abuse or criminal victimization that has occurred within the last 7 days; and b) persons who experienced potentially traumatic life events and who described those events as inducing a sense of intense fear or horror. The latter group were found to have elevated rates of PTSD related symptoms (288). However, from a population perspective it is also sometimes useful to examine patterns of lifetime exposure to adverse circumstances to inform policy development. Figure 3 shows the lifetime rates of criminal victimization (excluding assaults, abuse), and three types of assault or abuse by homelessness and gender. Whether considering the homeless population receiving services in the community or psychiatric inpatients admitted from homeless settings, the lifetime rates of criminal victimization, assault or abuse are higher than in the non-homeless population. Moreover, women (whether homeless or not) are systematically more likely to have experienced the three types of abuse in their lives.

FIGURE 3

Figure 3 Percentage distributions of lifetime history of victimization by homelessness and service setting, Canada and Netherlands.

Outcome Measurement

interRAI instruments contain numerous embedded scales (see Tables 3 and 5) and individual items that summarize the presence/absence and severity of needs at a given point in time. These may be examined longitudinally with reassessments or when linked to previous interRAI assessments done in other care settings. At the person-level, these longitudinal changes provide insights about the person's changes in strengths and needs, response to treatment, and progression or recovery from illness.

Figure 4 shows the distribution of the Positive Symptoms Scale (see Table 3 for scale description) for homeless and non-homeless persons in five care settings. About 30% of persons in homeless services in the Netherlands, who receive only limited mental health supports, have signs of psychosis compared with almost half of the homeless persons in Canadian community mental health agencies. Severity of positive symptoms increases among those in contact with hospital or emergency mental health services, but it is most pronounced for homeless persons at the time of contact with police. In all settings, except police contacts, the severity of positive symptoms is greater for homeless than non-homeless persons.

FIGURE 4

Figure 4 Percentage distributions of positive symptoms scale score by homelessness and service setting, Canada and Netherlands.

Figure 5 shows the transitions in the Positive Symptoms Scores by homelessness from the time of admission to discharge for persons with stays of less than 90 days. Both populations improved substantially in symptom severity, but the scores were worse at admission and discharge for homeless persons.

FIGURE 5

Figure 5 Percentage distribution of positive symptoms scale scores at admission and discharge within 90 days, inpatient psychiatry, Canada.

Quality Indicators

interRAI has an extensive history of developing risk-adjusted outcome-based quality indicators. While scales and items tracked over time for individuals can be useful for person-level care planning, the longitudinal data may be aggregated at the population level to benchmark performance for internal quality improvement, accreditation and public reporting (290–297). Although interRAI's nursing home quality indicators are reported on-line nationally (298, 299), its mental health quality indicators (MHQIs) are currently provided to hospitals for internal use only. A detailed summary of these risk-adjusted quality indicators is provided elsewhere (300). The majority of these indicators are outcome-based, with the exception of a limited number of process indicators dealing with restraint and acute control medication use. There are two main outcomes: a) improvement in symptoms for persons who have non-zero values for the scale of interest at admission; b) failure to improve or worsening for persons who do not have maximum scores at admission. The outcomes include common psychiatric symptoms (e.g., depressive symptoms), functional indicators (e.g., medication management), and others that are less commonly tracked (e.g., pain).

Figure 6A shows the population-based risk-adjusted MHQI rates over time for three improvement indicators stratified by homelessness (homeless vs not homeless) in Canadian inpatient psychiatry. The rates are more volatile for the homeless indicators due to smaller sample sizes, but there are only small differences between the two subgroups for improvement in hallucinations and depressive symptoms. Both of these indicators show improvement rates to be above 0.70 as the risk-adjusted proportion improving from baseline to follow-up. On the other hand, for the homeless group, improvement rates are much lower for capacity to manage finances, despite this group having a somewhat better improvement rate than non-homeless persons.

FIGURE 6

Figure 6 Temporal changes in three risk adjusted mental health quality indicators by year, inpatient psychiatry, Canada. Figure 6A shows the risk adjusted rates of improvement in hallucinations, depressive symptoms, and capacity to manage finances in the last 90 days or at discharge (if discharged less than 90 days since baseline assessment). Figure 6B shows the risk adjusted rates of worsening of or failure to improve in hallucinations, depressive symptoms, and capacity to manage finances in the last 90 days or at discharge (if discharged less than 90 days since baseline assessment).

Figure 6B shows three risk-adjusted MHQIs for worsening or failure to improve in the same three domains. Rates of worsening or failure to improve increased (indicating poorer care) over time with notably higher rates for homeless persons. The indicator for depressive symptoms also showed poorer performance over time, but no substantive difference between subpopulations. On the other hand, rates of worsening or failure to improve in managing finances declined over time for both groups.

Resource Allocation

interRAI systems may also be used to inform decisions about the allocation of health care resources at the person and population levels. In non-mental health settings, interRAI systems have been developed to prioritize access to community and institutional services (301) and for eligibility determination in long-term care (302). Mental health counterparts to this work include a level of care framework based on the interRAI MH (303), analyses of service complexity for children with mental health needs and developmental disability based on the ChYMH-DD (304), and a decision-support algorithm describing resource intensity based on the interRAI ChYMH (305).

Case-mix systems have considerable value for informing a variety of health care decisions, including funding methodologies (306). These systems tend to have two main components: a) a classification system, based on clinical characteristics, to group service users into categories with similar levels of resource use; and b) a weighting system (case-mix indexes or CMIs) assigns ratio-level numeric values to these groups that can, among other functions, be applied to funding formulas in payment systems. interRAI-based case-mix systems are available for nursing homes (307–309), home care (310), intellectual disability services (311, 312), and intervener/interpreter services for dual sensory loss (313).

The Diagnosis Related Groups (314) system is widely used in acute general hospitals, but a consensus was reached over three decades ago that the system was inadequate for describing resource use in psychiatry (315). A number of studies pointed to the potential to use per-diem based case-mix systems that estimate costs of care per day of stay, rather than episode-based systems that attempt to predict length of stay (316–318). Most research of this type in mental health has been in hospital settings with only modest progress in community mental health services (319). In addition, although better than episode-based models, the ability to explain variance in resource use in psychiatry is lower than in more homogeneous care settings such as nursing homes (320).

The System for Classification of In-Patient Psychiatry (SCIPP) is a per-diem case-mix system for inpatient mental health services based on an earlier version of the interRAI MH (321–324). SCIPP was developed through a staff time measurement study of about 2,000 patients in 34 psychiatric hospitals in three Canadian provinces. The System for Classification of In-Patient Psychiatry (SCIPP) includes about 100 variables for a 47-group algorithm (see Figure 7) explaining about 26% of variance in per diem resource use among adult psychiatric patients. There is an 8.4 to 1 range in CMIs across the SCIPP groups. Careful attention was paid to avoiding the use of service variables, facility variables, gameable items and items that had poor psychometric properties. The SCIPP algorithm provides an important step forward in case-mix research for psychiatry. It achieves a higher explained variance than has been possible in episode-based systems, and does so without the use of independent variables that would be problematic to administer as part of a prospective payment system.

FIGURE 7

Figure 7 System for classification of inpatient psychiatry (SCIPP) schematic. Blue boxes represent decision points, gold ovals represent terminal SCIPP groups.

Figure 8 shows historical trends in the mean CMI scores in Canadian psychiatric hospitals/units using the MH as part of routine practice. Between 2006 and 2017, the mean admission CMIs rose from 1.567 to 1.657 equating to a 5.7% increase in resource intensity. On the other hand, the discharge CMIs were virtually unchanged over that time period at about 0.940. This indicates two main points: a) there is roughly a 55% drop in resource intensity from admission to discharge associated with the alleviation of symptoms related to mental health and co-morbid conditions; and b) hospitals admitted heavier patients over time without changes in their resource intensity when discharged. In addition, homeless persons were consistently more resource intensive at admission compared with the general populations with relative differences ranging between 5.6 and 4.2%.

FIGURE 8

Figure 8 Trends in mean in (95% CL) SCIPP case-mix index values, inpatient psychiatry, Canada.

Future Directions

Although much has been accomplished through interRAI's two decades of research on its mental health instruments, some limitations of the existing instruments must be addressed in order to continue to improve their utility.

There is a clear need for more cross-national research and implementation of the mental health suite. Low-middle income nations are of particular interest, but adaptations to lower resource environments are likely to be necessary. Many such countries are undertaking dramatic reforms of their mental health systems to place greater emphasis on decentralized networks and primary care (325–328) where interRAI's mental health systems have not yet been used. Another gap is the lack of a triage and screening tool for intake services and public “help” lines. These gaps may be addressed, in part, through the development of companion self-report instruments to reach populations not receiving mental health services. Such instruments might also be useful for settings with few available mental health professionals.

Another limitation is that the translation of interRAI instruments into normal clinical practice is heavily dependent on robust implementation strategies led by highly committed mental health professionals. The ability to realize the full potential of all clinical and management applications can be hindered by implementations that emphasize the data collection aspects of the system rather than its clinical use (329, 330). Moreover, even though interRAI instruments are designed to function as an integrated, cross-sectoral system, the sharing of assessment results between clinicians may not occur for reasons including ineffective communication and collaboration between those sectors (160). Further, relatively little work has been done to date on how best to present interRAI assessment results to patients as part of the shared decision-making process.

Also, while a great deal of validation work has been done, there is need for additional research on criterion validity related to mania, anxiety, and trauma. Predictive validity studies related to suicide and forensic risk are underway. It would be useful to complete inter-rater reliability studies between assessments completed by nurses or social workers and those done by clinical psychologists and psychiatrists. Of particular importance is the need for validation and refinement studies for the SCIPP system, particularly for specialized populations (e.g., forensics). Also, there is no case-mix system available at this stage for the community mental health instrument.

The availability of rich longitudinal datasets with over 1 million observations of over 400 clinical variables creates opportunities for applying new machine learning and artificial intelligence tools. When combined with real-time analytic capabilities in electronic medical records, the potential to create new personalized medicine applications that place the person's data in the context of population level data is considerable. In addition, linkage to other data sources—wearable devices, registry and administrative data, biomarker data (e.g., laboratory values, genetic data)—provides great opportunity for novel insights and innovative improvements to mental health services. Such discoveries have already been made by linking Icelandic genetic data and interRAI nursing home data to examine Alzheimer disease and cognitive decline (331, 332).

Finally, if the next 20 years of use of interRAI mental health systems sees the same degree of growth in its child/youth instruments, the future opportunities for understanding mental health from a life course perspective can be realized. The availability of scientifically sound, standardized, and fully compatible measures that follow persons living with mental illness from the earliest life stages throughout adulthood will be an unprecedented new opportunity to develop solutions to alleviate the impact of mental illness for persons of all ages.

Author’s Note

The authors are all part of the interRAI network.

Ethics Statement

Ethics clearance for secondary analyses of interRAI data gathered by other organizations was obtained from the University of Waterloo (ORE#30173).

Author Contributions

All authors contributed to the formulation of the ideas presented in the study and provided critical feedback to the manuscript. JPH drafted the first version of the manuscript, with major sections contributed by CE, MF, JH, AH, LM, CP, SS and CA. JPH, JF, MJ, BF, and LM made editorial changes to initial drafts and in response to reviewers' comments. JPH, BF, and CE conducted data analyses of Canadian, US and Dutch data, respectively.

Funding

Some aspects of this research were funded by the Health Transition Fund, Health Canada G03-05691. Research on homelessness in the Netherlands was commissioned and funded with support from various municipalities and human services organizations. Maastricht University and Radboud University, Nijmegen provide core financial support for the scientific use of data from these studies. Data from implementations in Canada, United States and Finland were made available based on data sharing agreements with interRAI.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be considered as a potential conflict of interest.

Acknowledgments

The authors would like to acknowledge the support of Jonathan Chen, Micaela Jantzi, and Britta Sohlman who provided analytic support toward portions of this manuscript. We thank the staff who conducted pilot implementations of various interRAI systems in their initial stages. We are particularly grateful to Dr. Paula Reaume-Zimmer, Integrated VP, Mental Health Services and the other staff of Bluewater Health and CMHA Lambton Kent who have played a major leadership role in the implementation and use of the full suite of interRAI Mental Health Instruments. Finally, the authors wish to acknowledge the contributions of Dr. Edgardo Perez as co-lead author of the original RAI-Mental Health assessment instrument.

Supplementary Material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/fpsyt.2019.00926/full#supplementary-material

References

1. Kessler RC, Aguilar-Gaxiola S, Alonso J, Chatterji S, Lee S, Ormel J, et al. The global burden of mental disorders: an update from the WHO World Mental Health (WMH) surveys. Epidemiol Psichiatry Soc (2009) 18(1):23–3. doi: 10.1017/S1121189X00001421

CrossRef Full Text | Google Scholar

2. Vigo D, Thornicroft G, Atun R. Estimating the true global burden of mental illness. Lancet Psychiatry (2016) 3(2):171–8. doi: 10.1016/S2215-0366(15)00505-2

PubMed Abstract | CrossRef Full Text | Google Scholar

3. World Health Organization. Regional Office for Europe. Mental health : facing the challenges, building solutions: report from the WHO European Ministerial Conference. Copenhagen, Denmark: World Health Organization, Regional Office for Europe (2005). xi, 181 pp.

Google Scholar

4. Sheppard R, Deane FP, Ciarrochi J. Unmet need for professional mental health care among adolescents with high psychological distress. Aust New Z J Psychiatry (2018) 52(1):59–7. doi: 10.1177/0004867417707818

CrossRef Full Text | Google Scholar

5. Murrell K. Lean Flow and the Impact of the Long-Stay Patient. In: Big Book of Emergency Department Psychiatry: A Guide to Patient Centered Operational Improvement, CRC Press: Taylor & Francis Group (2017). vol. 10. doi: 10.1201/b21955-20

CrossRef Full Text | Google Scholar

6. Steinman KJ, Shoben AB, Dembe AE, Kelleher KJ. How long do adolescents wait for psychiatry appointments? Community Ment Health J (2015) 51(7):782–89. doi: 10.1007/s10597-015-9897-x

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Price M, Singer A, Kim J. Adopting electronic medical records: are they just electronic paper records? Can Family Physician (2013) 59(7):e322–e9.

Google Scholar

8. Jonsson PV, Finne-Soveri H, Jensdottir AB, Ljunggren G, Bucht G, Grue EV, et al. Co-morbidity and functional limitation in older patients underreported in medical records in Nordic Acute Care Hospitals when compared with the MDS-AC instrument. Age Ageing (2006) 35(4):434–8. doi: 10.1093/ageing/afj060

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Singer A, Kroeker AL, Yakubovich S, Duarte R, Dufault B, Katz A. Data quality in electronic medical records in Manitoba: do problem lists reflect chronic disease as defined by prescriptions? Can Fam Physician (2017) 63(5):382–89. doi: 10.1093/jamia/ocw013

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Singer A, Yakubovich S, Kroeker AL, Dufault B, Duarte R, Katz A. Data quality of electronic medical records in Manitoba: do problem lists accurately reflect chronic disease billing diagnoses? J Am Med Inform Assoc (2016) 23(6):1107–12. doi: 10.1038/tp.2015.182

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Rumshisky A, Ghassemi M, Naumann T, Szolovits P, Castro VM, McCoy TH, et al. Predicting early psychiatric readmission with natural language processing of narrative discharge summaries. Trans Psychiatry (2016) 6(10):e921. doi: 10.1136/bmj.h1885

CrossRef Full Text | Google Scholar

12. Liao KP, Cai T, Savova GK, Murphy SN, Karlson EW, Ananthakrishnan AN, et al. Development of phenotype algorithms using electronic medical records and incorporating natural language processing. BMJ (2015) 350:h1885. doi: 10.1097/00001888-199908000-00012

PubMed Abstract | CrossRef Full Text | Google Scholar

13. Friedman C, Hripcsak G. Natural language processing and its future in medicine. Acad Med (1999) 74(8):890–95. doi: 10.1016/j.cbpra.2013.12.005

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Bjaastad JF, Jensen-Doss A, Moltu C, Jakobsen P, Hagenberg H, Joa I. Attitudes toward standardized assessment tools and their use among clinicians in a public mental health service. Nordic J Psychiatry (2019), 7: 1–10. doi: 10.1080/08039488.2019.1642383

CrossRef Full Text | Google Scholar

15. Youngstrom EA, Choukas-Bradley S, Calhoun CD, Jensen-Doss A. Clinical guide to the evidence-based assessment approach to diagnosis and treatment. Cogn Behav Practice (2015) 22(1):20–5. doi: 10.1037/abn0000232

CrossRef Full Text | Google Scholar

16. Smetanin P, Briante C, Stiff D, Ahmad S, Khan M. The life and economic impact of major mental illnesses in Canada. Ment Health Comm Canada (2011).

Google Scholar

17. Schaefer JD, Caspi A, Belsky DW, Harrington H, Houts R, Horwood LJ, et al. Enduring mental health: prevalence and prediction. J Abnorm Psychol (2017) 126(2):212–24. doi: 10.1016/S0140-6736(13)62116-9

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Public Health Agency of Canada. The human face of mental health and mental illness in Canada, 2006. Ottawa: Public Health Agency of Canada (2006). xi, 188 p. doi: 10.1016/S0010-440X(00)80018-8

CrossRef Full Text | Google Scholar

19. Patton GC, Coffey C, Romaniuk H, Mackinnon A, Carlin JB, Degenhardt L, et al. The prognosis of common mental disorders in adolescents: a 14-year prospective cohort study. Lancet (2014) 383(9926):1404–11. doi: 10.1080/09297049.2011.578572

PubMed Abstract | CrossRef Full Text | Google Scholar

20. Wittchen HU, Lieb R, Pfister H, Schuster P. The waxing and waning of mental disorders: evaluating the stability of syndromes of mental disorders in the population. Compr Psychiatry (2000) 41(2 Suppl 1):122–32. doi: 10.1016/j.appdev.2016.02.002

PubMed Abstract | CrossRef Full Text | Google Scholar

21. Willoughby MT, Kupersmidt JB, Voegler-Lee ME. Is preschool executive function causally related to academic achievement? Child Neuropsychol (2012) 18(1):79–1. doi: 10.1111/josh.12603

PubMed Abstract | CrossRef Full Text | Google Scholar

22. Darling-Churchill KE, Lippman L. Early childhood social and emotional development: advancing the field of measurement. J Appl Dev Psychol (2016) 45:1–7. doi: 10.1016/j.drugpo.2014.02.002

CrossRef Full Text | Google Scholar

23. Lee CK, Corte C, Stein KF. Drinker identity: key risk factor for adolescent alcohol use. J School Health (2018) 88(3):253–60. doi: 10.1080/1067828X.2017.1292977

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Studer J, Baggio S, Deline S, N'Goran AA, Henchoz Y, Mohler-Kuo M, et al. Peer pressure and alcohol use in young men: a mediation analysis of drinking motives. Int J Drug Policy (2014) 25(4):700–8. doi: 10.1080/09595230802392790

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Lutz HR, McClure K, Armstrong S. Social problem solving and adolescent alcohol use within the context of well-established risk factors for adolescent alcohol use. J Child Adolesc Subst Abuse (2017) 26(3):229–41. doi: 10.1136/bmjopen-2017-016060

CrossRef Full Text | Google Scholar

26. Montgomery C, Fisk JE, Craig L. The effects of perceived parenting style on the propensity for illicit drug use: the importance of parental warmth and control. Drug Alcohol Rev (2008) 27(6):640–49. doi: 10.1080/00981389.2017.1318800

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Hodder RK, Freund M, Bowman J, Wolfenden L, Campbell E, Dray J, et al. Effectiveness of a pragmatic school-based universal resilience intervention in reducing tobacco, alcohol and illicit substance use in a population of adolescents: cluster-randomised controlled trial. BMJ Open (2017) 7(8):e016060. doi: 10.1186/s13195-016-0188-8

PubMed Abstract | CrossRef Full Text | Google Scholar

28. Milen MT, Nicholas DB. Examining transitions from youth to adult services for young persons with autism. Soc Work Health Care (2017) 56(7):636–48. doi: 10.1016/j.jalz.2012.11.007

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Prince M, Ali GC, Guerchet M, Prina AM, Albanese E, Wu YT. Recent global trends in the prevalence and incidence of dementia, and survival with dementia. Alzheimers Res Ther (2016) 8(1):23. doi: 10.1371/journal.pone.0019590

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Prince M, Bryce R, Albanese E, Wimo A, Ribeiro W, Ferri CP. The global prevalence of dementia: a systematic review and metaanalysis. Alzheimers Dement (2013) 9(1):63–75 e2. doi: 10.1016/S2215-0366(17)30078-0

PubMed Abstract | CrossRef Full Text | Google Scholar

31. Chang CK, Hayes RD, Perera G, Broadbent MT, Fernandes AC, Lee WE, et al. Life expectancy at birth for people with serious mental illness and other major disorders from a secondary mental health care case register in London. PLoS One (2011) 6(5):e19590. doi: 10.1192/bjp.bp.111.092718

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Hjorthoj C, Sturup AE, McGrath JJ, Nordentoft M. Years of potential life lost and life expectancy in schizophrenia: a systematic review and meta-analysis. Lancet Psychiatry (2017) 4(4):295–301. doi: 10.1111/acps.12842

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Thornicroft G. Physical health disparities and mental illness: the scandal of premature mortality. Br J Psychiatry (2011) 199(6):441–42. doi: 10.1001/jama.298.15.1794

PubMed Abstract | CrossRef Full Text | Google Scholar

34. Jokinen J, Talback M, Feychting M, Ahlbom A, Ljung R. Life expectancy after the first suicide attempt. Acta Psychiatr Scand (2018) 137(4):287–95. doi: 10.1016/j.amepre.2016.06.007

PubMed Abstract | CrossRef Full Text | Google Scholar

35. Newcomer JW, Hennekens CH. Severe mental illness and risk of cardiovascular disease. JAMA (2007) 298(15):1794–6. doi: 10.1038/nrendo.2014.203

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Tam J, Warner KE, Meza R. Smoking and the reduced life expectancy of individuals with serious mental illness. Am J Prev Med (2016) 51(6):958–66. doi: 10.3138/8774-758W-702Q-2531

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Holt RI, Mitchell AJ. Diabetes mellitus and severe mental illness: mechanisms and clinical implications. Nat Rev Endocrinol (2015) 11(2):79–89. doi: 10.1016/j.jagp.2017.02.007

PubMed Abstract | CrossRef Full Text | Google Scholar

38. Andrew MK, Rockwood K. Psychiatric illness in relation to frailty in community-dwelling elderly people without dementia: a report from the Canadian Study of Health and Aging. Can J Aging (2007) 26(1):33–8. doi: 10.3138/8774-758w-702q-2531

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Gallo JJ. Multimorbidity and Mental Health. Am J Geriatr Psychiatry (2017) 25(5):520–1. doi: 10.1136/bmjopen-2011-000778

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Chong SA, Abdin E, Nan L, Vaingankar JA, Subramaniam M. Prevalence and impact of mental and physical comorbidity in the adult Singapore population. Ann Acad Med Singapore (2012) 41(3):105–14. doi: 10.1016/S1470-2045(10)70085-1

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Bjorkenstam E, Ljung R, Burstrom B, Mittendorfer-Rutz E, Hallqvist J, Weitoft GR. Quality of medical care and excess mortality in psychiatric patients–a nationwide register-based study in Sweden. BMJ Open (2012) 2:e000778. doi: 10.1001/archpsyc.58.6.565

PubMed Abstract | CrossRef Full Text | Google Scholar

42. Howard LM, Barley EA, Davies E, Rigg A, Lempp H, Rose D, et al. Cancer diagnosis in people with severe mental illness: practical and ethical issues. Lancet Oncol (2010) 11(8):797–804. doi: 10.1016/S1470-2045(10)70085-1

PubMed Abstract | CrossRef Full Text | Google Scholar

43. Druss BG, Bradford WD, Rosenheck RA, Radford MJ, Krumholz HM. Quality of medical care and excess mortality in older patients with mental disorders. Arch Gen Psychiatry (2001) 58(6):565–72. doi: 10.1177/1359786810382058

PubMed Abstract | CrossRef Full Text | Google Scholar

44. Naylor C, Das P, Ross S, Honeyman M, Thompson J, Gilburt H. The case for change: 10 areas where integration is needed most. In: . Bringing together physical and mental health - A new frontier for integrated care. The King's Fund: London, U.K (2016). p. 20–3. doi: 10.1186/s12913-014-0618-x

CrossRef Full Text | Google Scholar

45. Lawrence D, Kisely S. Inequalities in healthcare provision for people with severe mental illness. J Psychopharmacol (2010) 24(4 Suppl):61–8. doi: 10.1371/journal.pone.0155073

PubMed Abstract | CrossRef Full Text | Google Scholar

46. Kulkarni M, Huddlestone L, Taylor A, Sayal K, Ratschen E. A cross-sectional survey of mental health clinicians' knowledge, attitudes, and practice relating to tobacco dependence among young people with mental disorders. BMC Health Serv Res (2014) 14:618. doi: 10.1016/j.jad.2016.11.026

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Guthrie DM, Declercq A, Finne-Soveri H, Fries BE, Hirdes JP. The health and well-being of older adults with Dual Sensory Impairment (DSI) in four countries. PLoS One (2016) 11(5):e0155073. doi: 10.1007/s40266-014-0233-x

PubMed Abstract | CrossRef Full Text | Google Scholar

48. Kavalidou K, Smith DJ, O'Connor RC. The role of physical and mental health multimorbidity in suicidal ideation. J Affect Disord (2017) 209:80–5. doi: 10.1007/s10597-015-9911-3

PubMed Abstract | CrossRef Full Text | Google Scholar

49. Pompili M, Venturini P, Lamis DA, Giordano G, Serafini G, Belvederi Murri M, et al. Suicide in stroke survivors: epidemiology and prevention. Drugs Aging (2015) 32(1):21–9. doi: 10.1016/j.dhjo.2009.09.001

PubMed Abstract | CrossRef Full Text | Google Scholar

50. McConnell D, Hahn L, Savage A, Dube C, Park E. suicidal ideation among adults with disability in western canada: a brief report. Community Ment Health J (2016) 52(5):519–26. doi: 10.1016/j.copsyc.2017.07.018

PubMed Abstract | CrossRef Full Text | Google Scholar

51. Giannini MJ, Bergmark B, Kreshover S, Elias E, Plummer C, O'Keefe E. understanding suicide and disability through three major disabling conditions: intellectual disability, spinal cord injury, and multiple sclerosis. Disabil Health J (2010) 3(2):74–8. doi: 10.1007/s00127-016-1178-7

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Khazem LR. Physical disability and suicide: recent advancements in understanding and future directions for consideration. Curr Opin Psychol (2018) 22:18–2. doi: 10.1177/1745691614568352

PubMed Abstract | CrossRef Full Text | Google Scholar

53. Khazem LR, Jahn DR, Cukrowicz KC, Anestis MD. Physical disability and the interpersonal theory of suicide. Death Stud (2015) 39(10):641–46. doi: 10.1111/hsc.12311

PubMed Abstract | CrossRef Full Text | Google Scholar

54. Matthews T, Danese A, Wertz J, Odgers CL, Ambler A, Moffitt TE, et al. Social isolation, loneliness and depression in young adulthood: a behavioural genetic analysis. Soc Psychiatry Psychiatr Epidemiol (2016) 51(3):339–48. doi: 10.1093/jurban/78.3.458

PubMed Abstract | CrossRef Full Text | Google Scholar

55. Holt-Lunstad J, Smith TB, Baker M, Harris T, Stephenson D. Loneliness and social isolation as risk factors for mortality: a meta-analytic review. Perspect Psychol Sci (2015) 10(2):227–37. doi: 10.1093/bjsw/bcr076

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Courtin E, Knapp M. Social isolation, loneliness and health in old age: a scoping review. Health Soc Care Comm (2017) 25(3):799–802. doi: 10.1016/j.jad.2014.12.049

CrossRef Full Text | Google Scholar

57. Tew J, Ramon S, Slade M, Bird V, Melton J, Le Boutillier C. Social factors and recovery from mental health difficulties: a review of the evidence. Br J Soc Work (2012) 42(3):443–60. doi: 10.1038/nrn4005

CrossRef Full Text | Google Scholar

58. Santini ZI, Koyanagi A, Tyrovolas S, Mason C, Haro JM. The association between social relationships and depression: a systematic review. J Affect Disord (2015) 175:53–5. doi: 10.3928/00485713-20050501-06

PubMed Abstract | CrossRef Full Text | Google Scholar

59. Smith TF, Hirdes JP. Predicting social isolation among geriatric psychiatry patients. Int Psychogeriat (2009) 21(1):50–9. doi: 10.1111/jgs.13454

CrossRef Full Text | Google Scholar

60. Green MF, Horan WP, Lee J. Social cognition in schizophrenia. Nat Rev Neurosci (2015) 16(10):620–31. doi: 10.4324/9781315783147

PubMed Abstract | CrossRef Full Text | Google Scholar

61. Van der Kolk BA. Developmental trauma disorder: toward a rational diagnosis for children with complex trauma histories. Psychiatr Ann (2017) 35(5):401–8. doi: 10.1177/002214650504600404

CrossRef Full Text | Google Scholar

62. Dong XQ. Elder abuse: systematic review and implications for practice. J Am Geriat Soc (2015) 63(6):1214–38. doi: 10.1016/j.jhealeco.2015.07.003

PubMed Abstract | CrossRef Full Text | Google Scholar

63. Harris M, Landis CL. Sexual Abuse in the Lives of Women Diagnosed withSerious Mental Illness. Routledge (2016). doi: 10.1080/2156857X.2015.1134629

CrossRef Full Text | Google Scholar

64. Strohschein L. Household income histories and child mental health trajectories. J Health Soc Behav (2005) 46(4):359–75. doi: 10.1177/1078390307307830

PubMed Abstract | CrossRef Full Text | Google Scholar

65. Schaller J, Stevens AH. Short-run effects of job loss on health conditions, health insurance, and health care utilization. J Health Econ (2015) 43:190–3. doi: 10.3233/WOR-2009-0890

PubMed Abstract | CrossRef Full Text | Google Scholar

66. Topor A, Ljungqvist I, Strandberg EL. Living in poverty with severe mental illness coping with double trouble. Nordic Soc Work Res (2016) 6(3):201–10. doi: 10.1016/S0140-6736(14)61132-6

CrossRef Full Text | Google Scholar

67. Perese EF. Stigma, poverty, and victimization: Roadblocks to recovery for individuals with severe mental illness. J Am Psychiatr Nurses Assoc (2007) 13(5):285–95. doi: 10.1371/journal.pmed.0050225

CrossRef Full Text | Google Scholar

68. Krupa T, Kirsh B, Cockburn L, Gewurtz R. Understanding the stigma of mental illness in employment. Work (2009) 33(4):413–25. doi: 10.1016/S0140-6736(11)60747-2

PubMed Abstract | CrossRef Full Text | Google Scholar

69. Trani J-F, Bakhshi P, Kuhlberg J, Narayanan SS, Venkataraman H, Mishra NN, et al. Mental illness, poverty and stigma in India: a case–control study. BMJ Open (2015) 5(2):e006355. doi: 10.1016/S0140-6736(17)31869-X

PubMed Abstract | CrossRef Full Text | Google Scholar

70. Fazel S, Geddes JR, Kushel M. The health of homeless people in high-income countries: descriptive epidemiology, health consequences, and clinical and policy recommendations. Lancet (2014) 384(9953):1529–40. doi: 10.1177/1403494818759839

PubMed Abstract | CrossRef Full Text | Google Scholar

71. Fazel S, Khosla V, Doll H, Geddes J. The prevalence of mental disorders among the homeless in western countries: systematic review and meta-regression analysis. PLoS Med (2008) 5(12):e225. doi: 10.1177/0042098015587818

PubMed Abstract | CrossRef Full Text | Google Scholar

72. Nielsen SF, Hjorthøj CR, Erlangsen A, Nordentoft M. Psychiatric disorders and mortality among people in homeless shelters in Denmark: a nationwide register-based cohort study. Lancet (2011) 377(9784):2205–14. doi: 10.1016/S2214-109X(18)30539-4

PubMed Abstract | CrossRef Full Text | Google Scholar

73. Aldridge RW, Story A, Hwang SW, Nordentoft M, Luchenski SA, Hartwell G, et al. Morbidity and mortality in homeless individuals, prisoners, sex workers, and individuals with substance use disorders in high-income countries: a systematic review and meta-analysis. Lancet (2018) 391(10117):241–50. doi: 10.1080/02673037.2017.1344957

PubMed Abstract | CrossRef Full Text | Google Scholar

74. Benjaminsen L, Birkelund JF. Explaining excess morbidity amongst homeless shelter users: a multivariate analysis for the danish adult population. Scand. J Public Health (2018) 1403494818759839. doi: 10.1177/0042098014548138

CrossRef Full Text | Google Scholar

75. Benjaminsen L. Homelessness in a scandinavian welfare state: the risk of shelter use in the danish adult population. Urban Stud (2016) 53(10):2041–63. doi: 10.1177/0042098012452329

CrossRef Full Text | Google Scholar

76. Baranyi G, Scholl C, Fazel S, Patel V, Priebe S, Mundt AP. Severe mental illness and substance use disorders in prisoners in low-income and middle-income countries: a systematic review and meta-analysis of prevalence studies. Lancet Global Health (2019) 7(4):e461–e71. doi: 10.1111/dar.12448

CrossRef Full Text | Google Scholar

77. Bramley G, Fitzpatrick S. Homelessness in the UK: who is most at risk? Housing Stud (2018) 33(1):96–116. doi: 10.1111/dar.12419

CrossRef Full Text | Google Scholar

78. Piat M, Polvere L, Kirst M, Voronka J, Zabkiewicz D, Plante M-C, et al. Pathways into homelessness: understanding how both individual and structural factors contribute to and sustain homelessness in Canada. Urban Stud (2015) 52(13):2366–82. doi: 10.4088/PCC.15m01784

CrossRef Full Text | Google Scholar

79. Fitzpatrick S, Bramley G, Johnsen S. Pathways into multiple exclusion homelessness in seven UK cities. Urban Stud (2013) 50(1):148–68. doi: 10.1016/j.jsat.2007.01.011

CrossRef Full Text | Google Scholar

80. Kingston REF, Marel C, Mills KL. A systematic review of the prevalence of comorbid mental health disorders in people presenting for substance use treatment in Australia. Drug Alcohol Rev (2017) 36(4):527–39. doi: 10.1016/B978-0-12-398336-7.00055-3

PubMed Abstract | CrossRef Full Text | Google Scholar

81. Prior K, Mills K, Ross J, Teesson M. Substance use disorders comorbid with mood and anxiety disorders in the Australian general population. Drug Alcohol Rev (2017) 36(3):317–24. doi: 10.1177/070674370404900405

PubMed Abstract | CrossRef Full Text | Google Scholar

82. Perlman CM, Hirdes JP, Vigod S. psychiatric rehospitalization: development of a person-level indicator for care planning and quality assurance. Prim Care Compan CNS Disord (2015) 17(4). doi: 10.1001/jama.2010.616

CrossRef Full Text | Google Scholar

83. Drake RE, O'Neal EL, Wallach MA. A systematic review of psychosocial research on psychosocial interventions for people with co-occurring severe mental and substance use disorders. J Subst Abuse Treat (2008) 34(1):123–38. doi: 10.3109/09638237.2015.1036973

PubMed Abstract | CrossRef Full Text | Google Scholar

84. Deady M, Teesson M, Brady KT. Impact of substance use on the course of serious mental disorders. Principles Addiction: Compr Addictive Behav Disord (2013) 1:525–32. doi: 10.1080/00207411.1982.11448926

CrossRef Full Text | Google Scholar

85. Sealy P, Whitehead PC. Forty years of deinstitutionalization of psychiatric services in Canada: an empirical assessment. Can J Psychiatry (2004) 49(4):249–57. doi: 10.7870/cjcmh-2004-0012

PubMed Abstract | CrossRef Full Text | Google Scholar

86. Patel V, Prince M. Global mental health: a new global health field comes of age. JAMA (2010) 303(19):1976–77. doi: 10.1016/j.healthplace.2019.01.006

PubMed Abstract | CrossRef Full Text | Google Scholar

87. van Hoof F, Knispel A, Aagaard J, Schneider J, Beeley C, Keet R, et al. The role of national policies and mental health care systems in the development of community care and community support: an international analysis. J Ment Health (2015) 24(4):202–7. doi: 10.1080/00207411.2016.1167489

PubMed Abstract | CrossRef Full Text | Google Scholar

88. Richman A, Harris P. Mental hospital deinstitutionalization in Canada: A national perspective with some regional examples. Int J Ment Health (1982) 11(4):64–3. doi: 10.1002/j.2051-5545.2011.tb00060.x

CrossRef Full Text | Google Scholar

89. Morrow M. Mental health reform, economic globalization and the practice of citizenship. Can J Commun Ment Health (2004) 23(2):39–0. doi: 10.1080/23288604.2017.1356424

PubMed Abstract | CrossRef Full Text | Google Scholar

90. Hudson CG. Deinstitutionalization of mental hospitals and rates of psychiatric disability: An international study. Health Place (2019) 56:70–9. doi: 10.1093/eurpub/ckr068

PubMed Abstract | CrossRef Full Text | Google Scholar

91. Hudson CG. A model of deinstitutionalization of psychiatric care across 161 nations: 2001–2014. Int J Ment Health (2016) 45(2):135–53. doi: 10.1016/j.eurpsy.2016.11.011

CrossRef Full Text | Google Scholar

92. Semrau M, Barley EA, Law A, Thornicroft G. Lessons learned in developing community mental health care in Europe. World Psychiatry (2011) 10(3):217–25. doi: 10.1177/0887403414547043

PubMed Abstract | CrossRef Full Text | Google Scholar

93. Shen GC, Eaton J, Snowden LR. Mainstreaming Mental Health Care in 42 Countries. Health Syst Reform (2017) 3(4):313–24. doi: 10.1192/bjp.176.4.332

PubMed Abstract | CrossRef Full Text | Google Scholar

94. Westman J, Gissler M, Wahlbeck K. Successful deinstitutionalization of mental health care: increased life expectancy among people with mental disorders in Finland. Eur J Public Health (2012) 22(4):604–6. doi: 10.5694/j.1326-5377.2006.tb00581.x

PubMed Abstract | CrossRef Full Text | Google Scholar

95. Taylor Salisbury T, Killaspy H, King M. The relationship between deinstitutionalization and quality of care in longer-term psychiatric and social care facilities in Europe: a cross-sectional study. Eur Psychiatry (2017) 42:95–102. doi: 10.1080/14789940802594445

PubMed Abstract | CrossRef Full Text | Google Scholar

96. Kim D-Y. Psychiatric deinstitutionalization and prison population growth: A critical literature review and its implications. Criminal Justice Policy Rev (2016) 27(1):3–21. doi: 10.1192/bjp.bp.114.161943

CrossRef Full Text | Google Scholar

97. Gunn J. Future directions for treatment in forensic psychiatry. Br J Psychiatry (2000) 176(4):332–8. doi: 10.1001/jamapsychiatry.2014.2444

PubMed Abstract | CrossRef Full Text | Google Scholar

98. White P, Whiteford H. Prisons: mental health institutions of the 21st century? Med J Australia (2006) 185(6):302–3. doi: 10.1176/appi.ps.53.10.1266

PubMed Abstract | CrossRef Full Text | Google Scholar

99. Sirdifield C, Gojkovic D, Brooker C, Ferriter M. A systematic review of research on the epidemiology of mental health disorders in prison populations: a summary of findings. J Forensic Psychiatry Psychol (2009) 20(S1):S78–S101. doi: 10.1080/10439463.2016.1219734

CrossRef Full Text | Google Scholar

100. Winkler P, Barrett B, McCrone P, Csemy L, Janouskova M, Hoschl C. Deinstitutionalised patients, homelessness and imprisonment: systematic review. Br J Psychiatry (2016) 208(5):421–8. doi: 10.1016/j.ijlp.2016.02.013

PubMed Abstract | CrossRef Full Text | Google Scholar

101. Lamb HR. Does deinstitutionalization cause criminalization?: The penrose hypothesis. JAMA Psychiatry (2015) 72(2):105–6. doi: 10.1007/s00127-018-1619-6

PubMed Abstract | CrossRef Full Text | Google Scholar

102. Lamb HR, Weinberger LE, DeCuir WJ Jr. The police and mental health. Psychiatr Serv (2002) 53(10):1266–71. doi: 10.1176/appi.ps.52.4.477

PubMed Abstract | CrossRef Full Text | Google Scholar

103. Wood JD, Watson AC. Improving police interventions during mental health-related encounters: past, present and future. Policing Soc (2017) 27(3):289–99. doi: 10.3928/00485713-20180111-02

PubMed Abstract | CrossRef Full Text | Google Scholar

104. Roy L, Crocker AG, Nicholls TL, Latimer E, Gozdzik A, O'Campo P, et al. Profiles of criminal justice system involvement of mentally ill homeless adults. Int J Law Psychiatry (2016) 45:75–8. doi: 10.1192/bjp.bp.110.083733

PubMed Abstract | CrossRef Full Text | Google Scholar

105. Penney SR, Seto MC, Crocker AG, Nicholls TL, Grimbos T, Darby PL, et al. Changing characteristics of forensic psychiatric patients in Ontario: a population-based study from 1987 to 2012. Soc Psychiatry Psychiatr Epidemiol (2019) 54(5):627–38. doi: 10.3109/09540261.2011.644847

PubMed Abstract | CrossRef Full Text | Google Scholar

106. Watson A, Hanrahan P, Luchins D, Lurigio A. Mental health courts and the complex issue of mentally ill offenders. Psychiatr Services (2001) 52(4):477–81. doi: 10.1007/s00127-012-0484-y

CrossRef Full Text | Google Scholar

107. Nierenberg AA. A Future Foretold for Forensic Psychiatry: Toward Evidence-Based Risk Assessment and Prediction. Psychiatr Annals (2018) 48(2):70–. doi: 10.1017/S2045796012000133

CrossRef Full Text | Google Scholar

108. Leamy M, Bird V, Le Boutillier C, Williams J, Slade M. Conceptual framework for personal recovery in mental health: systematic review and narrative synthesis. Br J Psychiatry (2011) 199(6):445–52. doi: 10.1002/wps.20084

PubMed Abstract | CrossRef Full Text | Google Scholar

109. Slade M, Adams N, O'Hagan M. Recovery: past progress and future challenges. Int Rev Psychiatry (2012) 24(1):1–4. doi: 10.3109/09540261.2011.644847

PubMed Abstract | CrossRef Full Text | Google Scholar

110. Williams J, Leamy M, Bird V, Harding C, Larsen J, Le Boutillier C, et al. Measures of the recovery orientation of mental health services: systematic review. Soc Psychiatry Psychiatr Epidemiol (2012) 47(11):1827–35. doi: 10.1192/bjp.bp.106.025791

PubMed Abstract | CrossRef Full Text | Google Scholar

111. Slade M, Leamy M, Bacon F, Janosik M, Le Boutillier C, Williams J, et al. International differences in understanding recovery: systematic review. Epidemiol Psychiatr Sci (2012) 21(4):353–64. doi: 10.1002/j.2051-5545.2009.tb00218.x

PubMed Abstract | CrossRef Full Text | Google Scholar

112. Slade M, Amering M, Farkas M, Hamilton B, O'Hagan M, Panther G, et al. Uses and abuses of recovery: implementing recovery-oriented practices in mental health systems. World Psychiatry (2014) 13(1):12–0. doi: 10.1016/j.schres.2009.01.008

PubMed Abstract | CrossRef Full Text | Google Scholar

113. Thornicroft G. Shunned: Discrimination against people with mental illness. Oxford university press: Oxford; (2006). doi: 10.1016/j.jad.2015.02.013

CrossRef Full Text | Google Scholar

114. Thornicroft G, Rose D, Kassam A, Sartorius N. Stigma: ignorance, prejudice or discrimination? Br J Psychiatry (2007) 190:192–93. doi: 10.1192/bjp.bp.106.025791

PubMed Abstract | CrossRef Full Text | Google Scholar

115. Corrigan PW, Larson JE, Rusch N. Self-stigma and the "why try" effect: impact on life goals and evidence-based practices. World Psychiatry (2009) 8(2):75–1. doi: 10.1017/S1041610211001839

PubMed Abstract | CrossRef Full Text | Google Scholar

116. Farooq S, Large M, Nielssen O, Waheed W. The relationship between the duration of untreated psychosis and outcome in low-and-middle income countries: a systematic review and meta analysis. Schizophr Res (2009) 109(1–3):15–3. doi: 10.1016/j.jcgg.2013.09.001

PubMed Abstract | CrossRef Full Text | Google Scholar

117. Reynders A, Kerkhof AJ, Molenberghs G, Van Audenhove C. Help-seeking, stigma and attitudes of people with and without a suicidal past. A comparison between a low and a high suicide rate country. J Affect Disord (2015) 178:5–11. doi: 10.1093/gerona/53A.2.M92

PubMed Abstract | CrossRef Full Text | Google Scholar

118. Rush AJ Jr., First MB, Blacker D. Handbook of psychiatric measures. Arlington VA: American Psychiatric Pub (2009). doi: 10.1164/rccm.201805-0996OC

CrossRef Full Text | Google Scholar

119. Larner AJ. Screening utility of the Montreal Cognitive Assessment (MoCA): in place of–or as well as–the MMSE? Int Psychogeriat (2012) 24(3):391–6. doi: 10.1016/0022-3956(75)90026-96

CrossRef Full Text | Google Scholar

120. Miu DKY, Chan KC. Under-detection of pain in elderly nursing home residents with moderate to severe dementia. J Clin Gerontol Geriat (2014) 5(1):23–7. doi: 10.7326/0003-4819-113-12-941

CrossRef Full Text | Google Scholar

121. Garrard J, Rolnick SJ, Nitz NM, Luepke L, Jackson J, Fischer LR, et al. Clinical detection of depression among community-based elderly people with self-reported symptoms of depression. J Gerontol Ser A: Biol Sci Med Sci (1998) 53(2):M92–M101. doi: 10.1037/t00742-000

CrossRef Full Text | Google Scholar

122. Gentzler ER, Derry H, Ouyang DJ, Lief L, Berlin DA, Xu CJ, et al. Underdetection and undertreatment of dyspnea in critically ill patients. Am J Resp Crit Care Med (2019) 199(11):1377–84. doi: 10.1037/0022-006X.56.6.893

PubMed Abstract | CrossRef Full Text | Google Scholar

123. Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res (1975) 12(3):189–98. doi: 10.1093/schbul/13.2.261

PubMed Abstract | CrossRef Full Text | Google Scholar

124. Inouye SK, van Dyck CH, Alessi CA, Balkin S, Siegal AP, Horwitz RI. Clarifying confusion: the confusion assessment method: a new method for detection of delirium. Ann Internal Med (1990) 113(12):941–48. doi: 10.1192/bjp.133.5.429

CrossRef Full Text | Google Scholar

125. Beck AT, Steer RA, Brown GK. Beck depression inventory-II. San Antonio (1996) 78(2):490–8. doi: 10.1037/t05175-000

CrossRef Full Text | Google Scholar

126. Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol (1988) 56(6):893. doi: 10.1016/0304-3959(87)91074-8

PubMed Abstract | CrossRef Full Text | Google Scholar

127. Kay SR, Fiszbein A, Opler LA. The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophr Bull (1987) 13(2):261–76. doi: 10.1016/0165-1781(89)90047-4

PubMed Abstract | CrossRef Full Text | Google Scholar

128. Young RC, Biggs JT, Ziegler VE, Meyer DA. A rating scale for mania: reliability, validity and sensitivity. Br J Psychiatry (1978) 133(5):429–35. doi: 10.1017/S1041610297003530

PubMed Abstract | CrossRef Full Text | Google Scholar

129. Falsetti SA, Resnick HS, Resick PA, Kilpatrick DG. The modified PTSD symptom scale: a brief self-report measure of posttraumatic stress disorder. Behav Ther (1993). 16: 161–162. doi: 10.1097/00005053-198001000-00006

CrossRef Full Text | Google Scholar

130. Melzack R. The short-form McGill pain questionnaire. Pain (1987) 30(2):191–7. doi: 10.1093/geront/9.3_Part_1.179

PubMed Abstract | CrossRef Full Text | Google Scholar

131. Buysse DJ, Reynolds Iii CF, Monk TH, Berman SR, Kupfer DJ. The Pittsburgh Sleep Quality Index: a new instrument for psychiatric practice and research. Psychiatry Res (1989) 28(2):193–3. doi: 10.1007/BF00783424

PubMed Abstract | CrossRef Full Text | Google Scholar

132. Cohen-Mansfield J. Conceptualization of agitation: results based on the Cohen-Mansfield agitation inventory and the agitation behavior mapping instrument. Int Psychogeriat (1997) 8(S3):309–15. doi: 10.1176/appi.ajp.2011.10111704

CrossRef Full Text | Google Scholar

133. McLellan AT, Luborsky L, Woody GE, O'Brien CP. An improved diagnostic evaluation instrument for substance abuse patients: the Addiction Severity Index. J Nervous Ment Disease (1980). 1: 26–33. doi: 10.1037/t04911-000

CrossRef Full Text | Google Scholar

134. Lawton MP, Brody EM. Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontol (1969) 9(3_Part_1):179–86. doi: 10.1093/geront/9.3_Part_1.179

CrossRef Full Text | Google Scholar

135. Janca A, Kastrup M, Katschnig H, López-Ibor JJ, Mezzich JE, Sartorius N. The World Health Organization Short Disability Assessment Schedule (WHO DAS-S): a tool for the assessment of difficulties in selected areas of functioning of patients with mental disorders. Soc Psychiatry Psychiatr Epidemiol (1996) 31(6):349–54.

PubMed Abstract | Google Scholar

136. Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, et al. The Columbia–Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry (2011) 168(12):1266–77. doi: 10.1176/appi.ajp.2011.10111704

PubMed Abstract | CrossRef Full Text | Google Scholar

137. Yudofsky SC, Silver JM, Jackson W, Endicott J, Williams D. The Overt Aggression Scale for the objective rating of verbal and physical aggression. Am J Psychiatry (1986) 1: 35–39. doi: 10.1111/j.1532-5415.2007.01207.x

CrossRef Full Text | Google Scholar

138. Steel K, Jonsson PV, Dupasquier JN, Gilgen R, Hirdes J, Schroll M, et al. Systems of care for frail older persons. InterRAI. Trans Am Clin Climatological Assoc (1999) 110:30. doi: 10.1093/geront/30.3.293

CrossRef Full Text | Google Scholar

139. Bernabei R, Gray L, Hirdes J, Pei X, Henrard JC, Jonsson PV, et al. International gerontology. Hazzard's Geriatric Med Gerontol (2009) 69–76. doi: 10.1111/j.1532-5415.1997.tb02975.x

CrossRef Full Text | Google Scholar

140. Carpenter GI, Hirdes JP. Using interRAI assessment systems to measure and maintain quality of long-term care. In: . A good life in old age? Monitoring and improving quality in long-term care. OECD Health Policy Studies. 28. OECD Publishing: Paris (2013). p. 2017. doi: 10.12927/hcq.2000.16756

CrossRef Full Text | Google Scholar

141. Fries BE, Morris JN, Bernabei R, Finne-Soveri H, Hirdes J. Rethinking the resident assessment protocols. Journal of the American Geriatric Society (2007). 55(7): 113–114. doi: 10.1111/j.1532-5415.2007.01207.x

CrossRef Full Text | Google Scholar

142. Morris JN, Hawes C, Fries BE, Phillips CD, Mor V, Katz S, et al. Designing the national resident assessment instrument for nursing homes. Gerontologist (1990) 30(3):293–7. doi: 10.1093/ageing/afl036

PubMed Abstract | CrossRef Full Text | Google Scholar

143. Morris JN, Fries BE, Steel K, Ikegami N, Bernabei R, Carpenter GI, et al. Comprehensive clinical assessment in community setting: applicability of the MDS-HC. J Am Geriat Soc (1997) 45(8):1017–24. doi: 10.1186/1472-6963-9-71

PubMed Abstract | CrossRef Full Text | Google Scholar

144. Hirdes JP, Marhaba M, Smith TF, Clyburn L, Mitchell L, Lemick RA, et al. Development of the Resident Assessment Instrument–Mental Health (RAI-MH). Hosp Q (2000) 4(2):44–1. doi: 10.1016/S0840-4704(10)60164-0

CrossRef Full Text | Google Scholar

145. Hirdes JP, Ljunggren G, Morris JN, Frijters DH, Finne Soveri H, Gray L, et al. Reliability of the interRAI suite of assessment instruments: a 12-country study of an integrated health information system. BMC Health Serv Res (2008) 8:277. doi: 10.1007/s10578-017-0751-y

PubMed Abstract | CrossRef Full Text | Google Scholar

146. Hirdes JP. Addressing the health needs of frail elderly people: Ontario"s experience with an integrated health information system. Age and Ageing. Oxford University Press 35(4): 329–331. (2006). doi: 10.4137/HSI.S30775

CrossRef Full Text | Google Scholar

147. Gray LC, Berg K, Fries BE, Henrard JC, Hirdes JP, Steel K, et al. Sharing clinical information across care settings: the birth of an integrated assessment system. BMC Health Serv Res (2009) 9:71. doi: 10.1016/j.ijlp.2016.02.031

PubMed Abstract | CrossRef Full Text | Google Scholar

148. Hirdes JP, Fries BE, Morris JN, Steel K, Mor V, Frijters D, et al. Integrated health information systems based on the RAI/MDS series of instruments. Healthcare Manage Forum (1999) 12(4):30–40. doi: 10.1016/j.jamda.2017.08.010

CrossRef Full Text | Google Scholar

149. Lau C, Stewart SL, Saklofske DH, Tremblay PF, Hirdes J. Psychometric Evaluation of the interRAI Child and Youth Mental Health Disruptive/Aggression Behaviour Scale (DABS) and Hyperactive/Distraction Scale (HDS). Child Psychiatry Hum Dev (2018) 49(2):279–89. doi: 10.1017/S0714980814000579

PubMed Abstract | CrossRef Full Text | Google Scholar

150. Phillips CD, Hawes C. The interRAI Pediatric Home Care (PEDS HC) Assessment: evaluating the long-term community-based service and support needs of children facing special healthcare challenges. Health Serv Insights (2015) 8:17–24. doi: 10.1177/0898264315594138

PubMed Abstract | CrossRef Full Text | Google Scholar

151. Hoffman R, Hirdes J, Brown GP, Dubin JA, Barbaree H. The use of a brief mental health screener to enhance the ability of police officers to identify persons with serious mental disorders. Int J Law Psychiatry (2016) 47:28–35. doi: 10.1016/j.ijlp.2016.02.031

PubMed Abstract | CrossRef Full Text | Google Scholar

152. Morris JN, Declercq A, Hirdes JP, Finne-Soveri H, Fries BE, James ML, et al. Hearing the voice of the resident in long-term care facilities-an internationally based approach to assessing quality of life. J Am Med Directors Assoc (2018) 19(3):207–15. doi: 10.4236/health.2013.53A084

CrossRef Full Text | Google Scholar

153. Kehyayan V, Hirdes JP, Tyas SL, Stolee P. Residents' self-reported quality of life in long-term care facilities in Canada. Can J Aging (2015) 34(2):149–64. doi: 10.1017/S0714980814000579

PubMed Abstract | CrossRef Full Text | Google Scholar

154. Kehyayan V, Hirdes JP, Tyas SL, Stolee P. Predictors of long-term care facility residents' self-reported quality of life with individual and facility characteristics in Canada. J Aging Health (2016) 28(3):503–29. doi: 10.12927/hcq.2000.16756

PubMed Abstract | CrossRef Full Text | Google Scholar

155. Morris J, Howard E, Geffen L, Hirdes J, Hogeveen S, Berg K, et al. interRAI Check-Up (CU) Assessment, Supplement, and Self-Reported Forms and User’s Manual. Version 10.1. Washington DC: interRAI (2018). doi: 10.1177/0840470415581240

CrossRef Full Text | Google Scholar

156. Naus TE, Hirdes JP. Psychometric properties of the interRAI subjective quality of life instrument for mental health. Health (2013) 5(03):637. doi: 10.4236/health.2013.53A084

CrossRef Full Text | Google Scholar

157. Hirdes JP, interRAI (Organization). MH Coordinating Committee. interRAI mental health (MH) assessment form and user's manual for in-patient psychiatry. 9.1. ed. Ann Arbor, Mich.: interRAI; 2010. vii, 151 p. (2010). doi: 10.1186/1472-6963-14-457

CrossRef Full Text | Google Scholar

158. Hirdes JP, Marhaba M, Smith TF, Clyburn L, Mitchell L, Lemick RA, et al. Development of the resident assessment instrument–mental health (RAI-MH). Hosp Q (2000) 4(2):44–51. doi: 10.12927/hcq.2000.16756

CrossRef Full Text | Google Scholar

159. Stewart SL, Hirdes JP. Identifying mental health symptoms in children and youth in residential and in-patient care settings. Healthc Manage Forum (2015) 28(4):150–56. doi: 10.1177/0840470415581240

PubMed Abstract | CrossRef Full Text | Google Scholar

160. Hirdes JP, Morris JN, interRAI (Organization). CMH Coordinating Committee., interRAI (Organization). Instrument and Systems Development Committee. interRAI community mental health (CMH) assessment form and user's manual. 9.1, interRAI standard ed. Ann Arbor, Mich.: interRAI; 2010. vii, 151 p. (2010).

Google Scholar

161. Martin L, Hirdes JP. Exploring the impact of common assessment instrumentation on communication and collaboration in inpatient and community-based mental health settings: a focus group study. BMC Health Serv Res (2014) 14:457. doi: 10.1186/1472-6963-14-457

PubMed Abstract | CrossRef Full Text | Google Scholar

162. van Everdingen C. Daklozenzorg in Den Haag: de keten is zoek! (2018) doi: 10.1177/0840470415581257

CrossRef Full Text | Google Scholar

163. Morris JN. interRAI (Organization). interRAI home care (HC) assessment form and user's manual. 9.1. ed. Ann Arbor, MI: interRAI; 2010. vii, 109 p. p.

Google Scholar

164. Rabinowitz T, Morris JN, interRAI (Organization). ESP Coordinating Committee., interRAI (Organization). Instrument and Systems Development Committee. interRAI emergency screener for psychiatry (ESP) assessment form and user's manual. 9.1. ed. Ann Arbor, Mich.: interRAI; 2013. vii, 67 p. (2013).

Google Scholar

165. Hirdes J, Hoffman R, Brown GP, Barbaree H, Curtin-Telegdi N, Morris J, et al. interRAI Brief Mental Health Screener (BMHS) Assessment Forms and User"s Manual: A Screening Level Assessment for Use by Police Officers and Other Front-line Service Providers (version 9.3). Washington DC: interRAI (2015).

Google Scholar

166. Pizzingrilli B, Hoffman R, Hirdes DP. A protocol to reduce police wait times in the emergency department. Healthc Manage Forum (2015) 28(4):134–8. doi: 10.1177/0840470415581257

PubMed Abstract | CrossRef Full Text | Google Scholar

167. Hoffman RE. Development of the interRAI brief mental health screener to enhance the ability of police officers to identify persons with serious mental disorder. (2013). doi: 10.1023/B:LAHU.0000046435.98590.55

CrossRef Full Text | Google Scholar

168. Pearson Hirdes D. The Potential for Machine Learning in Mental Health Policing: Predicting Outcomes of Mental Health Related Calls for Service. McMaster University: Hamilton, ON, Canada (2019). doi: 10.1177/0840470415581232

CrossRef Full Text | Google Scholar

169. Committee HSaJC. Tools for Developing Police-Hospital Transition Protocols in Ontario. Toronto, ON Canada; 2019 May 2019.

Google Scholar

170. Mathias K-L. Gender-based analysis of criminogenic risk and clinical need among Ontario forensic patients. University of Waterloo: Waterloo, ON, Canada (2015). doi: 10.1111/j.1741-1130.2006.00094.x

CrossRef Full Text | Google Scholar

171. Seto MC, Harris GT, Rice ME. The criminogenic, clinical, and social problems of forensic and civil psychiatric patients. Law Hum Behav (2004) 28(5):577–86. doi: 10.1023/b:lahu.0000046435.98590.55

PubMed Abstract | CrossRef Full Text | Google Scholar

172. Mathias K, Hirdes JP. Gender differences in the rate of restriction to room among Ontario forensic patients. Healthc Manage Forum (2015) 28(4):157–62. doi: 10.1016/j.ridd.2008.07.010

PubMed Abstract | CrossRef Full Text | Google Scholar

173. Hirdes J, Martin L, Curtin-Telegdi N, Fries B, James M, Rabinowitz T, et al. interRAI Intellectual Disability (ID) Assessment Form and User"s Manual. Version 9.2. interRAI. Washington DC: interRAI (2013). doi: 10.1080/19315860802029154

CrossRef Full Text | Google Scholar

174. Martin L, Hirdes JP, Fries BE, Smith TF. Development and psychometric properties of an assessment for persons with intellectual disability—the interRAI ID. J Policy Pract Intellect Disabil (2007) 4(1):23–9. doi: 10.1111/j.1365-2788.2008.01041.x

CrossRef Full Text | Google Scholar

175. Martin LA, Morris JN, interRAI (Organization). ID CAP Development Committee., interRAI (Organization). Instrument and Systems Development Committee. interRAI Intellectual Disability Collaborative Action Plans (CAPs) : for use with the intellectual disability assessment instrument. 9.2. ed. Ann Arbor, Michigan: interRAI; 2013. xi, 71. p. (2013).

Google Scholar

176. Ashworth M, Hirdes JP, Martin L. The social and recreational characteristics of adults with intellectual disability and pica living in institutions. Res Dev Disabil (2009) 30(3):512–20. doi: 10.1515/ijdhd.2012.008

PubMed Abstract | CrossRef Full Text | Google Scholar

177. Ashworth M, Martin L, Hirdes JP. Prevalence and Correlates of Pica Among Adults with Intellectual Disability in Institutions. J Ment Health Res (2008) 1(3):176–90. doi: 10.1111/jir.12588

CrossRef Full Text | Google Scholar

178. Langlois L, Martin L. Relationship between diagnostic criteria, depressive equivalents and diagnosis of depression among older adults with intellectual disability. J Intellect Disabil Res (2008) 52(11):896–904. doi: 10.1515/IJDHD.2010.033

PubMed Abstract | CrossRef Full Text | Google Scholar

179. Martin L, Hirdes J, ME P. Healthcare needs of community- and institution-based older adults with intellectual disabilities. J Policy Pract Intellect. Dev Disabil (2008). 5 (1): 28–29 doi: 10.1186/s12889-019-6912-0

CrossRef Full Text | Google Scholar

180. Martin L, Ashworth M, Montague P. A longitudinal study of institutional downsizing and challenging behaviors among adults with intellectual disability. ijdhd (2012) 11(1):51. doi: 10.1111/j.1741-1130.2010.00261.x

CrossRef Full Text | Google Scholar

181. Carfi A, Vetrano DL, Mascia D, Meloni E, Villani ER, Acampora N, et al. Adults with Down syndrome: a comprehensive approach to manage complexity. J Intellect Disabil Res (2019) 63(6):624–9. doi: 10.1179/2047387713Y.0000000021

PubMed Abstract | CrossRef Full Text | Google Scholar

182. Ashworth M, Martin L, Montague P. A longitudinal study of institutional downsizing and activity involvement among adults with intellectual disability. ijdhd (2010) 9(4):319. doi: 10.1515/IJDHD.2010.033

CrossRef Full Text | Google Scholar

183. Martin L. Changes before the transition to the community: experiences of persons in Ontario's closing institutions. J Dev Disabil (2008) 14(1):127–37. doi: 10.1111/jppi.12171

CrossRef Full Text | Google Scholar

184. Martin L, Ashworth M. Deinstitutionalization in Ontario, Canada: understanding who moved when. J Policy Pract Intel (2010) 7(3):167–76. doi: 10.1080/23297018.2018.1443023

CrossRef Full Text | Google Scholar

185. Martin L. Re-examining the relationship between intellectual disability and receipt of inpatient psychiatry and complex continuing care services in Ontario, Canada. Int J Dev Disabil (2014) 60(2):65–9. doi: 10.1179/2047387713Y.0000000021

CrossRef Full Text | Google Scholar

186. Martin L, Hirdes JP, Fries BE. Examining the characteristics of persons with intellectual disability receiving hospital services, Part 2: Complex continuing care hospitals/units. J Dev Disabil (2007) 13(3):105–17.

Google Scholar

187. Martin L, Ouellette-Kuntz H, McKenzie K. Care in the community: home care use among adults with intellectual and developmental disabilities over time. J Policy Pract Intel (2017) 14(3):251–4. doi: 10.1111/j.1532-5415.1997.tb02974.x

CrossRef Full Text | Google Scholar

188. Martin L, Ouellette-Kuntz H, McKenzie K. Use of home care services among adults with intellectual and developmental disabilities: does where you live matter? Res Pract Intellect (2018) 5(2):192–201. doi: 10.1080/23297018.2018.1443023

CrossRef Full Text | Google Scholar

189. Martin LR, Hirdes JP, Fries BE. Examining the characteristics of persons with intellectual disability receiving hospital services: Part 1 - Psychiatric hospitals/units. J Dev Disabil (2007) 13(3):89–103. doi: 10.1017/S0714980800013878

CrossRef Full Text | Google Scholar

190. Morris JN. interRAI (Organization). interRAI long-term care facilities (LTCF) assessment form and user's manual. 9.1. ed. Ann Arbor, MI: interRAI; 2010. vii, 123 p. (2010) doi: 10.1111/bdi.12511

CrossRef Full Text | Google Scholar

191. Morris JN, Nonemaker S, Murphy K, Hawes C, Fries BE, Mor V, et al. A commitment to change: revision of HCFA's RAI. J Am Geriatr Soc (1997) 45(8):1011–6. doi: 10.1111/j.1532-5415.1997.tb02974.x

PubMed Abstract | CrossRef Full Text | Google Scholar

192. Morris JN, interRAI (Organization). Instrument and Systems Development Committee. InterRAI clinical assessment protocols (CAPs) for use with community and long-term care assessment instruments. 9.1. ed. Ann Arbor, MI: interRAI; 2010. xi, 179 p. (2010) doi: 10.1177/0840470415581233

CrossRef Full Text | Google Scholar

193. Hirdes JP, Ikegami N, Jonsson PV, Topinkova E, Maxwell CJ, Yamauchi K. Cross-national comparisons of antidepressant use among institutionalized older persons based on the minimum data set (MDS). Can J Aging-Revue Can Du Vieillissement (2000) 19:18–27. doi: 10.1002/gps.4259

CrossRef Full Text | Google Scholar

194. Schluter PJ, Lacey C, Porter RJ, Jamieson HA. An epidemiological profile of bipolar disorder among older adults with complex needs: A national cross-sectional study. Bipolar Disord (2017) 19(5):375–85. doi: 10.5770/cgj.20.228

PubMed Abstract | CrossRef Full Text | Google Scholar

195. Hirdes JP, Fries BE, Rabinowitz T, Morris JN. Comprehensive assessment of persons with bipolar disorder in long-term care settings: the potential of the interRAI family of instruments. In: Sajatovic M, Blow FC, editors. Bipolar disorders in late life. Johns Hopkins University Press: Baltimore, MD (2007). p. 27–9. doi: 10.1177/0898264317708072

CrossRef Full Text | Google Scholar

196. Neufeld E, Hirdes JP, Perlman CM, Rabinowitz T. A longitudinal examination of rural status and suicide risk. Healthc Manage Forum (2015) 28(4):129–33. doi: 10.1097/HTR.0000000000000112

PubMed Abstract | CrossRef Full Text | Google Scholar

197. Neufeld E, Hirdes JP, Perlman CM, Rabinowitz T. Risk and protective factors associated with intentional self-harm among older community-residing home care clients in Ontario, Canada. Int J Geriatr Psych (2015) 30(10):1032–40. doi: 10.1002/gps.2770

CrossRef Full Text | Google Scholar

198. McGuire C, Kristman VL, Martin L, Bedard M. Characteristics and incidence of traumatic brain injury in older adults using home care in Ontario from 2003-2013. Can Geriatr J (2017) 20(1):2–9. doi: 10.1586/14737175.3.3.343

PubMed Abstract | CrossRef Full Text | Google Scholar

199. McGuire C, Kristman VL, Martin L, Bedard M. The association between depression and traumatic brain injury in older adults: a nested matched case control study. J Aging Health (2018) 30(7):1156–68. doi: 10.1111/j.1532-5415.2008.02048.x

PubMed Abstract | CrossRef Full Text | Google Scholar

200. Colantonio A, Hsueh J, Petgrave J, Hirdes JP, Berg K. A profile of patients with traumatic brain injury within home care, long-term care, complex continuing care, and institutional mental health settings in a publicly insured population. J Head Trauma Rehab (2015) 30(6):E18–9. doi: 10.1016/j.jamda.2009.06.004

CrossRef Full Text | Google Scholar

201. Boorsma M, Joling KJ, Frijters DH, Ribbe ME, Nijpels G, van Hout HP. The prevalence, incidence and risk factors for delirium in Dutch nursing homes and residential care homes. Int J Geriatr Psychiatry (2012) 27(7):709–15. doi: 10.1016/j.pain.2011.10.007

PubMed Abstract | CrossRef Full Text | Google Scholar

202. Rabinowitz T, Murphy KM, Nagle KJ, Bodor CI, Kennedy SM, Hirdes JP. Delirium: pathophysiology, recognition, prevention and treatment. Expert Rev Neurother (2003) 3(3):343–55. doi: 10.1046/j.1532-5415.2001.49035.x

PubMed Abstract | CrossRef Full Text | Google Scholar

203. Perlman CM, Hirdes JP. The aggressive behavior scale: a new scale to measure aggression based on the minimum data set. J Am Geriatr Soc (2008) 56(12):2298–303. doi: 10.1017/S0714980816000325

PubMed Abstract | CrossRef Full Text | Google Scholar

204. Volicer L, Van der Steen JT, Frijters DH. Modifiable factors related to abusive behaviors in nursing home residents with dementia. J Am Med Directors Assoc (2009) 10(9):617–22. doi: 10.1080/13607863.2016.1247426

CrossRef Full Text | Google Scholar

205. Tosato M, Lukas A, van der Roest HG, Danese P, Antocicco M, Finne-Soveri H, et al. Association of pain with behavioral and psychiatric symptoms among nursing home residents with cognitive impairment: Results from the SHELTER study. Pain (2012) 153(2):305–10. doi: 10.1093/geronj/49.4.M174

PubMed Abstract | CrossRef Full Text | Google Scholar

206. Ahronheim JC, Mulvihill M, Sieger C, Park P, Fries BE. State practice variations in the use of tube feeding for nursing home residents with severe cognitive impairment. J Am Geriatr Soc (2001) 49(2):148–52. doi: 10.1177/0891988715598231

PubMed Abstract | CrossRef Full Text | Google Scholar

207. Bansal S, Hirdes JP, Maxwell CJ, Papaioannou A, Giangregorio LM. Identifying fallers among home care clients with dementia and parkinson's disease. Can J Aging (2016) 35(3):319–31. doi: 10.1002/gps.2232

PubMed Abstract | CrossRef Full Text | Google Scholar

208. Garms-Homolova V, Notthoff N, Declercq A, van der Roest HG, Onder G, Jonsson P, et al. Social and functional health of home care clients with different levels of cognitive impairments. Aging Ment Health (2017) 21(1):18–3. doi: 10.1111/ggi.12707

PubMed Abstract | CrossRef Full Text | Google Scholar

209. Morris JN, Fries BE, Mehr DR, Hawes C, Phillips C, Mor V, et al. MDS cognitive performance scale. J Gerontol (1994) 49(4):M174–82. doi: 10.1111/j.1532-5415.1997.tb02970.x

PubMed Abstract | CrossRef Full Text | Google Scholar

210. Morris JN, Howard EP, Steel K, Perlman C, Fries BE, Garms-Homolova V, et al. Updating the Cognitive Performance Scale. J Geriatr Psych Neur (2016) 29(1):47–5. doi: 10.3233/JPD-160931

CrossRef Full Text | Google Scholar

211. Feng Z, Hirdes JP, Smith TF, Finne-Soveri H, Chi I, Du Pasquier JN, et al. Use of physical restraints and antipsychotic medications in nursing homes: a cross-national study. Int J Geriatr Psychiatry (2009) 24(10):1110–8. doi: 10.1002/gps.2232

PubMed Abstract | CrossRef Full Text | Google Scholar

212. Freeman S, Spirgiene L, Martin-Khan M, Hirdes JP. Relationship between restraint use, engagement in social activity, and decline in cognitive status among residents newly admitted to long-term care facilities. Geriatr Gerontol Int (2017) 17(2):246–55. doi: 10.2105/AJPH.83.3.342

PubMed Abstract | CrossRef Full Text | Google Scholar

213. Hawes C, Mor V, Phillips CD, Fries BE, Morris JN, Steele-Friedlob E, et al. The OBRA-87 nursing home regulations and implementation of the resident assessment instrument: effects on process quality. J Am Geriatr Soc (1997) 45(8):977–85. doi: 10.1097/00005650-199611000-00008

PubMed Abstract | CrossRef Full Text | Google Scholar

214. Heckman GA, Crizzle AM, Chen J, Pringsheim T, Jette N, Kergoat MJ, et al. Clinical complexity and use of antipsychotics and restraints in long-term care residents with parkinson's disease. J Parkinsons Dis (2017) 7(1):103–15. doi: 10.1093/ageing/26.suppl_2.43

PubMed Abstract | CrossRef Full Text | Google Scholar

215. Phillips CD, Hawes C, Fries BE. High costs of restraints. Provider (1994) 20(2):33–4. doi: 10.1186/s12877-015-0127-8

CrossRef Full Text | Google Scholar

216. Phillips CD, Hawes C, Fries BE. Reducing the use of physical restraints in nursing homes: will it increase costs? Am J Public Health (1993) 83(3):342–48. doi: 10.1016/j.jamda.2014.07.012

PubMed Abstract | CrossRef Full Text | Google Scholar

217. Phillips CD, Hawes C, Mor V, Fries BE, Morris JN, Nennstiel ME. Facility and area variation affecting the use of physical restraints in nursing homes. Med Care (1996) 34(11):1149–62. doi: 10.1001/archinte.165.6.696

PubMed Abstract | CrossRef Full Text | Google Scholar

218. Ljunggren G, Phillips CD, Sgadari A. Comparisons of restraint use in nursing homes in eight countries. Age Ageing (1997) 26(suppl_2):43–7. doi: 10.4088/JCP.v64n0918

CrossRef Full Text | Google Scholar

219. Foebel A, Ballokova A, Wellens NI, Fialova D, Milisen K, Liperoti R, et al. A retrospective, longitudinal study of factors associated with new antipsychotic medication use among recently admitted long-term care residents. BMC Geriatr (2015) 15:128. doi: 10.4088/JCP.15m10303

PubMed Abstract | CrossRef Full Text | Google Scholar

220. Foebel AD, Liperoti R, Onder G, Finne-Soveri H, Henrard JC, Lukas A, et al. Use of antipsychotic drugs among residents with dementia in European long-term care facilities: results from the shelter study. J Am Med Dir Assoc (2014) 15(12):911–7. doi: 10.1186/s12877-017-0636-8

PubMed Abstract | CrossRef Full Text | Google Scholar

221. Liperoti R, Gambassi G, Lapane KL, Chiang C, Pedone C, Mor V, et al. Conventional and atypical antipsychotics and the risk of hospitalization for ventricular arrhythmias or cardiac arrest. Arch Internal Med (2005) 165(6):696–71. doi: 10.1177/2158244019835942

CrossRef Full Text | Google Scholar

222. Liperoti R, Mor V, Lapane KL, Pedone C, Gambassi G, Bernabei R. The use of atypical antipsychotics in nursing homes. J Clin Psychiatry (2003) 64(9):1106–12. doi: 10.4088/jcp.v64n0918

PubMed Abstract | CrossRef Full Text | Google Scholar

223. Liperoti R, Sganga F, Landi F, Topinkova E, Denkinger MD, van der Roest HG, et al. Antipsychotic drug interactions and mortality among nursing home residents with cognitive impairment. J Clin Psychiatry (2017) 78(1):e76–2. doi: 10.4088/JCP.15m10303

PubMed Abstract | CrossRef Full Text | Google Scholar

224. Rios S, Perlman CM, Costa A, Heckman G, Hirdes JP, Mitchell L. Antipsychotics and dementia in Canada: a retrospective cross-sectional study of four health sectors. BMC Geriatr (2017) 17(1):244. doi: 10.4236/psych.2017.814157

PubMed Abstract | CrossRef Full Text | Google Scholar

225. Hirdes JP, Major J, Didic S, Quinn C, Sinclair C, Bucek J, et al. Study protocol and baseline comparisons for a Pan-Canadian initiative to reduce inappropriate use of antipsychotics in long-term care homes. SAGE Open (2019) 9(1):2158244019835942. doi: 10.1016/j.psychres.2013.09.024

CrossRef Full Text | Google Scholar

226. interRAI (Organization). ChYMH Coordinating Committee, interRAI (Organization). Instrument and Systems Development Committee, interRAI (Organization). interRAI child and youth mental health (ChYMH) assessment form and user's manual: for use with in-patient and community-based assessments; includes: interRAI adolescent supplement (ChYMH-A). 9.3. ed. Ann Arbor, Michigan: interRAI; (2015). viii, 191 pages p.doi: 10.1016/j.psychres.2017.05.018

CrossRef Full Text | Google Scholar

227. Stewart SL, Hirdes JP, McKnight M, interRAI (Organization). ChYMH Screener Coordinating Committee, interRAI (Organization). ChYMH Coordinating Committee, interRAI (Organization). Instrument and Systems Development Committee, et al. InterRAI child and youth mental health-screener (ChYMH-S): assessment form and user's manual. 9.3. ed. Washington, DC: interRAI; (2017). p. p.doi: 10.1016/j.chiabu.2017.04.011

CrossRef Full Text | Google Scholar

228. Arbeau K, Theall L, Willoughby K, Berman JMJ, Stewart SL. What happened? Exploring the relation between traumatic stress and provisional mental health diagnoses for children and youth. Psychology (2017) 8(14):2485–95. doi: 10.1007/s10578-017-0751-y

CrossRef Full Text | Google Scholar

229. Baiden P, den Dunnen W, Stewart SL. Discharge of adolescents with mental health problems against medical advice: Findings from adult mental health inpatient facilities across Ontario, Canada. Psychiatry Res (2013) 210(3):1161–67. doi: 10.1080/14999013.2015.1014528

PubMed Abstract | CrossRef Full Text | Google Scholar

230. Baiden P, Stewart SL, Fallon B. The mediating effect of depressive symptoms on the relationship between bullying victimization and non-suicidal self-injury among adolescents: Findings from community and inpatient mental health settings in Ontario, Canada. Psychiatry Res (2017) 255:238–47. doi: 10.1186/s12913-016-1970-9

PubMed Abstract | CrossRef Full Text | Google Scholar

231. Baiden P, Stewart SL, Fallon B. The role of adverse childhood experiences as determinants of non-suicidal self-injury among children and adolescents referred to community and inpatient mental health settings. Child Abuse Neglect (2017) 69:163–76. doi: 10.1016/j.chiabu.2017.04.011

PubMed Abstract | CrossRef Full Text | Google Scholar

232. Lau C, Stewart SL, Saklofske DH, Tremblay PF, Hirdes J. Psychometric evaluation of the interRAI child and youth mental health disruptive/aggression behaviour scale (DABS) and Hyperactive/Distraction Scale (HDS). Child Psychiat Hum D (2018) 49(2):279–89. doi: 10.1016/j.psychres.2019.06.013

CrossRef Full Text | Google Scholar

233. Stewart SL, Baiden P, den Dunnen W, Hirdes JP, Perlman CM. Prevalence and correlates of criminal activity in adolescents treated in adult inpatient mental health beds in Ontario, Canada. Int J Forensic Ment (2015) 14(1):33–4. doi: 10.1177/1178632919827930

CrossRef Full Text | Google Scholar

234. Stewart SL, Hamza CA. The Child and Youth Mental Health Assessment (ChYMH): an examination of the psychometric properties of an integrated assessment developed for clinically referred children and youth. BMC Health Serv Res (2017) 17: 82. doi: 10.1186/s12913-016-1970-9

PubMed Abstract | CrossRef Full Text | Google Scholar

235. Stewart SL, Klassen J. Emerging mental health diagnoses and school disruption: an examination among clinically referred children and youth. Exceptionality Educ Int (2016) 26(2):5–20.

Google Scholar

236. Lau C, Stewart SL, Saklofske DH, Hirdes J. Scale development and psychometric properties of internalizing symptoms: The interRAI Child and Youth Mental Health Internalizing Subscale. Psychiat Res (2019). 278: 235–241. doi: 10.1080/13607863.2017.1408773

CrossRef Full Text | Google Scholar

237. Laspina N, Stewart SL. Examining service complexity in children with intellectual disability and mental health problems who receive inpatient or outpatient services. J Intellec Dev Disabil (2019) 44(4): 464–473.

Google Scholar

238. interRAI (Organization). ChYMH-DD Coordinating Committee, interRAI (Organization). Instrument and Systems Development Committee, interRAI (Organization). InterRAI child and youth mental health and developmental disability (ChYMH-DD) assessment form and user's manual : for use with in-patient and community-based assessments ; includes: interRAI adolescent supplement (ChYMH-A). 9.3. ed. Ann Arbor, Michigan: interRAI; (2015). viii, 201 pages p. doi: 10.1093/ageing/afl038

CrossRef Full Text | Google Scholar

239. Stewart SL, Morris JN, Berg K, Björkgren M, Declercq A, Finne-Soveri H, et al. interRAI 0-3 Collaborative Action Plans (CAPs) for use with the interRAI 0-3 Assessment Instrument. Version 9.3, Standard Edition. interRAI: Washington DC (In press). doi: 10.1007/BF03353428

CrossRef Full Text | Google Scholar

240. Carmines EG, Zeller RA. Reliability and validity assessment. Sage Publications;: Beverly Hills, Calif (1979). p. 70. doi: 10.1111/ggi.12330

CrossRef Full Text | Google Scholar

241. Carpenter GI. Accuracy, validity and reliability in assessment and in evaluation of services for older people: the role of the interRAI MDS assessment system. Age Ageing (2006) 35(4):327–29. doi: 10.1093/ageing/26.suppl_2.27

PubMed Abstract | CrossRef Full Text | Google Scholar

242. Carpenter GI, Teare GF, Steel K, Berg K, Murphy K, Bjornson J, et al. A new assessment for elders admitted to acute care: reliability of the MDS-AC. Aging (Milano) (2001) 13(4):316–30. doi: 10.1016/j.archger.2011.07.005

PubMed Abstract | CrossRef Full Text | Google Scholar

243. Hawes C, Morris JN, Phillips CD, Mor V, Fries BE, Nonemaker S. Reliability estimates for the Minimum Data Set for nursing home resident assessment and care screening (MDS). Gerontologist (1995) 35(2):172–78. doi: 10.1016/S0840-4704(10)60127-5

PubMed Abstract | CrossRef Full Text | Google Scholar

244. Kim H, Jung YI, Sung M, Lee JY, Yoon JY, Yoon JL. Reliability of the interRAI Long Term Care Facilities (LTCF) and interRAI Home Care (HC). Geriatr Gerontol Int (2015) 15(2):220–8. doi: 10.1007/BF02287348

PubMed Abstract | CrossRef Full Text | Google Scholar

245. Sgadari A, Morris JN, Fries BE, Ljunggren G, Jonsson PV, DuPaquier JN, et al. Efforts to establish the reliability of the Resident Assessment Instrument. Age Ageing (1997) 26Suppl 2:27–0. doi: 10.1093/ageing/26.suppl_2.27

PubMed Abstract | CrossRef Full Text | Google Scholar

246. Wellens NI, Van Lancker A, Flamaing J, Gray L, Moons P, Verbeke G, et al. Interrater reliability of the interRAI Acute Care (interRAI AC). Arch Gerontol Geriatr (2012) 55(1):165–72. doi: 10.2307/2529310

PubMed Abstract | CrossRef Full Text | Google Scholar

247. Poss JW, Jutan NM, Hirdes JP, Fries BE, Morris JN, Teare GF, et al. A review of evidence on the reliability and validity of Minimum Data Set data 2008. SAGE Publications Sage CA: Los Angeles, CA. (2008) doi: 10.5116/ijme.4dfb.8dfd

CrossRef Full Text | Google Scholar

248. Hirdes JP, Smith TF, Rabinowitz T, Yamauchi K, Perez E, Telegdi NC, et al. The Resident Assessment Instrument-Mental Health (RAI-MH): inter-rater reliability and convergent validity. J Behav Health Serv Res (2002) 29(4):419–32. doi: 10.1186/s12911-017-0547-9

PubMed Abstract | CrossRef Full Text | Google Scholar

249. Sim J, Wright CC. The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther (2005) 85(3):257–68. doi: 10.1186/1472-6947-13-27

PubMed Abstract | CrossRef Full Text | Google Scholar

250. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics (1977) 33(1):159–74. doi: 10.1093/geront/41.2.173

PubMed Abstract | CrossRef Full Text | Google Scholar

251. Tavakol M, Dennick R. Making sense of Cronbach's alpha. Int J Med Educ (2011) 2:53. doi: 10.1177/070674371005501108

PubMed Abstract | CrossRef Full Text | Google Scholar

252. Hogeveen SE, Chen J, Hirdes JP. Evaluation of data quality of interRAI assessments in home and community care. BMC Med Inform Decis Mak (2017) 17(1):150. doi: 10.1007/s12603-012-0439-8

PubMed Abstract | CrossRef Full Text | Google Scholar

253. Hirdes JP, Poss JW, Caldarelli H, Fries BE, Morris JN, Teare GF, et al. An evaluation of data quality in Canada's Continuing Care Reporting System (CCRS): secondary analyses of Ontario data submitted between 1996 and 2011. BMC Med Inform Decis Mak (2013) 13:27. doi: 10.1093/ageing/29.2.165

PubMed Abstract | CrossRef Full Text | Google Scholar

254. Fries BE, Simon SE, Morris JN, Flodstrom C, Bookstein FL. Pain in U.S. Nurs Homes: Valid. Pain Scale Minimum Data Set. Gerontologist (2001) 41(2):173–79. doi: 10.1007/s12603-012-0074-4

CrossRef Full Text | Google Scholar

255. Jones K, Perlman CM, Hirdes JP, Scott T. Screening cognitive performance with the resident assessment instrument for mental health cognitive performance scale. Can J Psychiat (2010) 55(11):736–40. doi: 10.1186/1472-6963-13-457

CrossRef Full Text | Google Scholar

256. Travers C, Byrne GJ, Pachana NA, Klein K, Gray L. Validation of the interRAI cognitive performance scale against independent clinical diagnosis and the mini-mental state examination in older hospitalized patients. J Nutr Health Aging (2013) 17(5):435–9. doi: 10.1111/j.1365-2850.2009.01429.x

PubMed Abstract | CrossRef Full Text | Google Scholar

257. Burrows AB, Morris JN, Simon SE, Hirdes JP, Phillips C. Development of a minimum data set-based depression rating scale for use in nursing homes. Age Ageing (2000) 29(2):165–72. doi: 10.1093/ageing/29.2.165

PubMed Abstract | CrossRef Full Text | Google Scholar

258. Salih SA, Paul S, Klein K, Lakhan P, Gray L. Screening for delirium within the interRAI acute care assessment system. J Nutr Health Aging (2012) 16(8):695–700. doi: 10.1007/s11414-011-9271-x

PubMed Abstract | CrossRef Full Text | Google Scholar

259. Foebel AD, Hirdes JP, Heckman GA, Kergoat MJ, Patten S, Marrie RA. Diagnostic data for neurological conditions in interRAI assessments in home care, nursing home and mental health care settings: a validity study. BMC Health Serv Res (2013) 13:457. doi: 10.1177/0840470415581231

PubMed Abstract | CrossRef Full Text | Google Scholar

260. Martin L, Hirdes JP, Morris JN, Montague P, Rabinowitz T, Fries BE. Validating the Mental Health Assessment Protocols (MHAPs) in the Resident Assessment Instrument Mental Health (RAI-MH). J Psychiatr Ment. Hlt (2009) 16(7):646–53. doi: 10.1177/0306624X13507040

CrossRef Full Text | Google Scholar

261. Hirdes JP, Smith T, Smith T, Arocha J, Perez E, Nadkarni S, et al. Technical Report to Primary Health Care Transition Fund, Health Canada: “Enhancing the Use of interRAI Instruments in Mental Health - Making an Integrated Mental Health Information System Happen”. University of Waterloo;: Waterloo, ON Canada (2007). Contract No.: Reference Number: G03-05691 x-ref G03-02877. doi: 10.1016/j.ijlp.2013.04.008

CrossRef Full Text | Google Scholar

262. Neufeld E, Perlman CM, Hirdes JP. Predicting inpatient aggression using the InterRAI risk of harm to others clinical assessment protocol: a tool for risk assessment and care planning. J Behav Health Serv Res (2012) 39(4):472–80. doi: 10.1016/S0840-4704(10)60291-8

PubMed Abstract | CrossRef Full Text | Google Scholar

263. Martin L, Perlman C, Bieling P. Using the RAI-MH to support shared decision-making in mental healthcare (2015). SAGE Publications Sage CA: Los Angeles, CA. doi: 10.1177/0840470415581244

CrossRef Full Text | Google Scholar

264. Brown GP, Hirdes JP, Fries BE. Measuring the prevalence of current, severe symptoms of mental health problems in a canadian correctional population: implications for delivery of mental health services for inmates. Int J Offender Ther Comp Criminol (2015) 59(1):27–0. doi: 10.1016/j.jamda.2019.01.121

PubMed Abstract | CrossRef Full Text | Google Scholar

265. Fries BE, Schmorrow A, Lang SW, Margolis PM, Heany J, Brown GP, et al. Symptoms and treatment of mental illness among prisoners: a study of Michigan state prisons. Int J Law Psychiatry (2013) 36(3–4):316–25. doi: 10.1097/YCT.0000000000000461

PubMed Abstract | CrossRef Full Text | Google Scholar

266. Martin L, Hirdes JP. Mental health needs and service use in Ontario. Healthc Manage Forum (2009) 22(1):40–6. doi: 10.1177/0840470415581226

CrossRef Full Text | Google Scholar

267. Tempier R, Bouattane el M, Hirdes JP. Access to psychiatrists by French-speaking patients in Ontario hospitals: 2005 to 2013. Healthc Manage Forum (2015) 28(4):167–71. doi: 10.1007/s10488-018-0898-2

PubMed Abstract | CrossRef Full Text | Google Scholar

268. Perlman C, Kirkham J, Velkers C, Leung RH, Whitehead M, Seitz D. Access to Psychiatrist Services for Older Adults in Long-Term Care: A Population-Based Study. J Am Med Directors Assoc (2019) 20(5):610–6. doi: 10.5430/jha.v4n2p15

CrossRef Full Text | Google Scholar

269. Knight J, Jantzi M, Hirdes J, Rabinowitz T. Predictors of electroconvulsive therapy use in a large inpatient psychiatry population. J ECT (2018) 34(1):35–9. doi: 10.1111/camh.12022

PubMed Abstract | CrossRef Full Text | Google Scholar

270. Little J, Hirdes JP, Daniel I. ALC status in in-patient mental health settings: evidence based on the Ontario mental health reporting system. Healthc Manage Forum (2015) 28(4):146–9. doi: 10.1080/0092623X.2011.582437

PubMed Abstract | CrossRef Full Text | Google Scholar

271. Little J, Hirdes JP, Perlman CM, Meyer SB. Clinical predictors of delayed discharges in inpatient mental health settings across Ontario. Adm Policy Ment Health (2019) 46(1):105–14. doi: 10.1176/appi.psy.48.4.309

PubMed Abstract | CrossRef Full Text | Google Scholar

272. Ismail Z, Arenovich T, Grieve C, Willett P, Addington D, Rajji TK, et al. Predicting hospital length of stay for geriatric and adult patients with schizophrenia. J Hosp Adm (2015) 4:15–2. doi: 10.1176/ps.62.1.pss6201_0097

CrossRef Full Text | Google Scholar

273. Stewart SL, Kam C, Baiden P. Predicting length of stay and readmission for psychiatric inpatient youth admitted to adult mental health beds in Ontario, Canada. Child Adolesc Ment Health (2014) 19(2):115–21. doi: 10.1111/j.1365-2702.2011.03862.x

CrossRef Full Text | Google Scholar

274. Martin L, Perlman CM, Hirdes JP. Social relationships and activities among married psychiatric inpatients with sexual difficulties. J Sex Marital Ther (2011) 37(4):307–22. doi: 10.1002/erv.762

PubMed Abstract | CrossRef Full Text | Google Scholar

275. Perlman CM, Martin L, Hirdes JP, Curtin-Telegdi N, Perez E, Rabinowitz T. Prevalence and predictors of sexual dysfunction in psychiatric inpatients. Psychosomatics (2007) 48(4):309–18. doi: 10.3109/10826084.2013.765482

PubMed Abstract | CrossRef Full Text | Google Scholar

276. Rabinowitz T, Martin L, Montague P, Hirdes JP. Prevalence and correlates of urinary incontinence in a large cohort of psychiatric inpatients. Psychiatr Serv (2011) 62(1):97–100. doi: 10.1016/j.psychres.2013.01.017

PubMed Abstract | CrossRef Full Text | Google Scholar

277. Martin L, Rabinowitz T, Montague P. Urinary incontinence in inpatient psychiatry: prevalence, change and relationship to length of stay. J Clin Nursing (2012) 21(15–16):2219–27. doi: 10.1177/0840470415581230

CrossRef Full Text | Google Scholar

278. Masson PC, Perlman CM, Ross SA, Gates AL. Premature termination of treatment in an inpatient eating disorder programme. Eur Eating Disord Review: Prof J Eating Disord Association (2007) 15(4):275–82. doi: 10.1002/erv.762

CrossRef Full Text | Google Scholar

279. Stewart SL, Baiden P, den Dunnen W. Prescription medication misuse among adolescents with severe mental health problems in Ontario, Canada. Subst Use Misuse (2013) 48(5):404–14.

PubMed Abstract | Google Scholar

280. Stewart SL, Baiden P. An exploratory study of the factors associated with medication nonadherence among youth in adult mental health facilities in Ontario, Canada. Psychiatry Res (2013) 207(3):212–17.

PubMed Abstract | Google Scholar

281. Mah TM, Hirdes JP, Heckman G, Stolee P. Use of control interventions in adult in-patient mental health services. Healthc Manage Forum (2015) 28(4):139–45.

PubMed Abstract | Google Scholar

282. Zuo S, Fries BE, Szafara K, Regal R. Valproic acid as a potentiator of metabolic syndrome in institutionalized residents on concomitant antipsychotics: fat chance, or slim to none? P T (2015) 40(2):126–32.

PubMed Abstract | Google Scholar

283. Hirdes JP, Morris JN, interRAI (Organization). MH CAP Coordinating Committee., interRAI (Organization). Instrument and Systems Development Committee. interRAI Mental Health Clinical Assessment Protocols (CAPs): for use with community and hospital-based mental health assessment instruments. 9.1. ed. Ann Arbor, Mich.: interRAI; (2011). xv, 197 p. p. doi: 10.1111/j.1365-2850.2009.01429.x

CrossRef Full Text | Google Scholar

284. interRAI (Organization). ChYMH CAP Development Committee, interRAI (Organization). Instrument and Systems Development Committee, interRAI (Organization). InterRAI child and youth mental health collaborative action plans (CAPs): for use with the child and youth mental health assessment instrument. 9.3. ed. Washington, DC: interRAI; (2015). xvi, 239 pages p. doi: 10.1007/s10597-010-9308-2

CrossRef Full Text | Google Scholar

285. interRAI (Organization). ChYMH-DD CAP Development Committee, interRAI (Organization). Instrument and Systems Development Committee, interRAI (Organization). InterRAI child and youth mental health and developmental disability collaborative action plans (CAPS): for use with the ChYMH-DD assessment instrument : version 9.3. Washington, DC: interRAI; (2016). xvi, 188 pages p. doi: 10.1007/s10597-014-9694-y

CrossRef Full Text | Google Scholar

286. Steel RK, Morris JN, interRAI (Organization), interRAI (Organization). Palliative Care CAP Coordinating Committee., interRAI (Organization). Instrument and Systems Development Committee. interRAI palliative care clinical assessment protocols (PC CAPs): for use with the palliative care assessment instrument. 9.1. ed. Ann Arbor, Mich.: interRAI; 2013. xi, 57 p. p. (2013) doi: 10.1097/00006199-200603001-00012

CrossRef Full Text | Google Scholar

287. Martin L, Hirdes JP, Morris JN, Montague P, Rabinowitz T, Fries BE. Validating the Mental Health Assessment Protocols (MHAPs) in the Resident Assessment Instrument Mental Health (RAI-MH). J Psychiatr Ment Health Nurs (2009) 16(7):646–53. doi: 10.1186/s12877-015-0146-5

PubMed Abstract | CrossRef Full Text | Google Scholar

288. Mathias K, Hirdes JP, Pittman D. A care planning strategy for traumatic life events in community mental health and inpatient psychiatry based on the InterRAI assessment instruments. Community Ment Health J (2010) 46(6):621–7. doi: 10.1007/s10597-010-9308-2

PubMed Abstract | CrossRef Full Text | Google Scholar

289. Doran DM, Harrison MB, Laschinger HS, Hirdes JP, Rukholm E, Sidani S, et al. Nursing-sensitive outcomes data collection in acute care and long-term-care settings. Nurs Res (2006) 55(2 Suppl):S75–81. doi: 10.1186/1472-6963-10-96

PubMed Abstract | CrossRef Full Text | Google Scholar

290. Foebel AD, van Hout HP, van der Roest HG, Topinkova E, Garms-Homolova V, Frijters D, et al. Quality of care in European home care programs using the second generation interRAI Home Care Quality Indicators (HCQIs). BMC Geriatr (2015) 15:148. doi: 10.1186/s12877-018-0842-z

PubMed Abstract | CrossRef Full Text | Google Scholar

291. Gray L, Morris JN, interRAI (Organization). Instrument and Systems Development Committee., interRAI (Organization). interRAI clinical and management applications manual for use with the interRAI acute care assessment instrument: scales, screeners, problems, clinical action points, and quality indicators. 9.1. ed. Ann Arbor, Mich.: interRAI; (2013). 75 p. doi: 10.1186/1471-2318-13-127

CrossRef Full Text | Google Scholar

292. Guthrie DM, Harman LE, Barbera L, Burge F, Lawson B, McGrail K, et al. Quality indicator rates for seriously Ill home care clients: analysis of resident assessment instrument for home care data in six canadian provinces. J Palliat Med (2019). 11: 1346–1356. doi: 10.1093/geront/44.5.665

CrossRef Full Text | Google Scholar

293. Jones RN, Hirdes JP, Poss JW, Kelly M, Berg K, Fries BE, et al. Adjustment of nursing home quality indicators. BMC Health Serv Res (2010) 10:96. doi: 10.1186/1472-6963-10-96

PubMed Abstract | CrossRef Full Text | Google Scholar

294. Morris JN, Berg K, Topinkova E, Gray LC, Schachter E. Developing quality indicators for in-patient post-acute care. BMC Geriatr (2018) 18(1):161. doi: 10.1186/s12877-018-0842-z

PubMed Abstract | CrossRef Full Text | Google Scholar

295. Morris JN, Fries BE, Frijters D, Hirdes JP, Steel RK. interRAI home care quality indicators. BMC Geriatr (2013) 13:127. doi: 10.1186/1472-6963-13-15

PubMed Abstract | CrossRef Full Text | Google Scholar

296. Hirdes JP, Fries BE, Morris JN, Ikegami N, Zimmerman D, Dalby DM, et al. Home care quality indicators (HCQIs) based on the MDS-HC. Gerontologist (2004) 44(5):665–79. doi: 10.1186/1741-7015-6-9

PubMed Abstract | CrossRef Full Text | Google Scholar

297. White N, Poss J. An embarrassment of data: how e-assessments are supporting front line clinical decisions and quality management across Canada and around the world. Stud Health Technol Inform (2009) 143:155–60. doi: 10.1093/geront/42.4.462

PubMed Abstract | CrossRef Full Text | Google Scholar

298. Information CIfH. Your Health System - Antipsychotic Use Ottawa: CIHI; 2019 [Available from: https://yourhealthsystem.cihi.ca/hsp/indepth;jsessionid=78ktV1ghtjJhqDtSoo0tjIKQ.yhs?lang=en#/indicator/008/2/C5001/ . (2019) doi: 10.1007/s11414-007-9088-9

CrossRef Full Text | Google Scholar

299. Perlman CM, Hirdes JP, Barbaree H, Fries BE, McKillop I, Morris JN, et al. Development of mental health quality indicators (MHQIs) for inpatient psychiatry based on the interRAI mental health assessment. BMC Health Serv Res (2013) 13:15. doi: 10.1111/jir.12423

PubMed Abstract | CrossRef Full Text | Google Scholar

300. Hirdes JP, Poss JW, Curtin-Telegdi N. The Method for Assigning Priority Levels (MAPLe): a new decision-support system for allocating home care resources. BMC Med (2008) 6(1):9. doi: 10.1177/1178632919827930

PubMed Abstract | CrossRef Full Text | Google Scholar

301. Fries BE, Shugarman LR, Morris JN, Simon SE, James M. A screening system for Michigan's home-and community-based long-term care programs. Gerontologist (2002) 42(4):462–74. doi: 10.1177/0840470414551866

PubMed Abstract | CrossRef Full Text | Google Scholar

302. Gibbons C, Dubois S, Ross S, Parker B, Morris K, Lim T, et al. Using the resident assessment instrument-mental health (RAI-MH) to determine levels of care for individuals with serious mental illness. J Behav Health Serv Res (2008) 35(1):60–0. doi: 10.1111/j.1532-5415.1991.tb05899.x

PubMed Abstract | CrossRef Full Text | Google Scholar

303. Stewart SL, Falah Hassani K, Poss J, Hirdes J. The determinants of service complexity in children with intellectual disabilities. J Intellect Disability Res (2017) 61(11):1055–68. doi: 10.1097/00005650-199407000-00002

CrossRef Full Text | Google Scholar

304. Stewart SL, Poss JW, Thornley E, Hirdes JP. Resource Intensity for Children and Youth: The Development of an Algorithm to Identify High Service Users in Children's Mental Health. Health Serv Insights (2019) 12: 1–11. doi: 10.1177/1178632919827926

CrossRef Full Text | Google Scholar

305. Costa AP, Poss JW, McKillop I. Contemplating case mix: a primer on case mix classification and management. SAGE Publications Sage CA: Los Angeles, CA (2015). doi: 10.1017/S0714980800013921

CrossRef Full Text | Google Scholar

306. Fries BE, Ljunggren G, Winblad B. International comparison of long-term care: the need for resident-level classification. J Am Geriatr Soc (1991) 39(1):10–6. doi: 10.1177/1178632919856011

PubMed Abstract | CrossRef Full Text | Google Scholar

307. Fries BE, Schneider DP, Foley WJ, Gavazzi M, Burke R, Cornelius E. Refining a case-mix measure for nursing homes: Resource Utilization Groups (RUG-III). Med Care (1994) 32(7):668–85. doi: 10.1177/1744629511413506

PubMed Abstract | CrossRef Full Text | Google Scholar

308. Turcotte LA, Poss J, Fries B, Hirdes JP. An Overview of International Staff Time Measurement Validation Studies of the RUG-III Case-mix System. Health Serv Insights (2019) 12:1–11. doi: 10.1186/1472-6963-13-137

CrossRef Full Text | Google Scholar

309. Björkgren MA, Fries BE, Shugarman LR. A RUG-III case-mix system for home care. Can J Aging/La Rev Can du Vieillissement (2000) 19(S2):106–25. doi: 10.1017/S0174980800013921

CrossRef Full Text | Google Scholar

310. Fries BE, James ML, Martin L, Head MJ, Park PS. A case-mix system for adults with developmental disabilities. Health Serv Insights (2019) 12: 1–8. doi: 10.1097/00005650-198407000-00002 1178632919856011

CrossRef Full Text | Google Scholar

311. Martin L, Fries BE, Hirdes JP, James M. Using the RUG-III classification system for understanding the resource intensity of persons with intellectual disability residing in nursing homes. J Intellect Disabil (2011) 15(2):131–41. doi: 10.1097/00005650-198905000-00009

PubMed Abstract | CrossRef Full Text | Google Scholar

312. Guthrie DM, Poss JW. Development of a case-mix funding system for adults with combined vision and hearing loss. BMC Health Serv Res (2013) 13:137. doi: 10.1186/1472-6963-13-137

PubMed Abstract | CrossRef Full Text | Google Scholar

313. Fetter RB, Shin Y, Freeman JL, Averill RF, Thompson JD. Case mix definition by diagnosis-related groups. Med Care (1980) 18(2):i–53. doi: 10.1371/journal.pone.0152669

PubMed Abstract | CrossRef Full Text | Google Scholar

314. Taube C, Lee ES, Forthofer RN. DRGs in psychiatry: An empirical evaluation. Med Care (1984), 22(7):597–600. doi: 10.1177/1178632919862248

PubMed Abstract | CrossRef Full Text | Google Scholar

315. Ashcraft ML, Fries BE, Nerenz DR, Falcon SP, Srivastava SV, Lee CZ, et al. A psychiatric patient classification system. An alternative to diagnosis-related groups. Med Care (1989) 27(5):543–57. doi: 10.1017/S204579601300067X

PubMed Abstract | CrossRef Full Text | Google Scholar

316. Fries BE, Durance PW, Nerenz DR, Ashcraft ML. A comprehensive payment model for short- and long-stay psychiatric patients. Health Care Financ Rev (1993) 15(2):31–40.

PubMed Abstract | Google Scholar

317. Wolff J, McCrone P, Patel A, Normann C. Determinants of per diem hospital costs in mental health. PloS One (2016) 11(3):e0152669.

PubMed Abstract | Google Scholar

318. Tran N, Poss JW, Perlman C, Hirdes JP. Case-mix classification for mental health care in community settings: a scoping review. Health Serv Insights (2019) 12:1–12. 1178632919862248

Google Scholar

319. Wolff J, McCrone P, Koeser L, Normann C, Patel A. Cost drivers of inpatient mental health care: a systematic review. Epidemiol Psychiatr Sci 24 (2015). p. 78–9. doi: 10.1017/S204579601300067X

CrossRef Full Text | Google Scholar

320. Minoletti A, Alegría M, Barrioneuvo H, Grana D, Hirdes JP, Perez E, et al. Translating psychiatric diagnosis and classificiation into public health usage. Public Health Aspects Diagnosis Class. Ment Behav Disord (2012), 201–41. doi: 10.1590/1982-02752017000400006

CrossRef Full Text | Google Scholar

321. Hirdes JP, Fries BE, Botz C, Ensley C, Marhaba M, Perez E. The System for Classification of In-Patient Psychiatry (SCIPP): a new case-mix methodology based on the RAI–Mental Health (RAI-MH). Toronto ON, Canada: interRAI (2003). doi: 10.1590/1413-81232014201.21152013

CrossRef Full Text | Google Scholar

322. Ontario Joint Policy and Planning Committee MHTWG. Grouper and weighting methodology for adult inpatient mental health care in Ontario a technical report. Toronto, ON Canada: JPPC; (2008). doi: 10.26633/RPSP.2018.136

CrossRef Full Text | Google Scholar

323. CIHI. Ontario Mental Health Reporting System (OMHRS): Interpreting SWPD Reports, 2012–2013. Ottawa ON, Canada: Canadian Institute for Health Information; 2013. doi: 10.4103/0019-5545.69220

CrossRef Full Text | Google Scholar

324. Hirdes A, Silva MKRD. Matrix support in mental health in Primary Health Care: Barriers and facilitating factors. Estudos Psicologia (Campinas) (2017) 34(4):499–1. doi: 10.1177/070674371205701106

CrossRef Full Text | Google Scholar

325. Oliveira MMD. Campos GWdS. Matrix support and institutional support: analyzing their construction. Ciencia Saude Coletiva (2015) 20(1):229–38. doi: 10.1177/070674371405900708

PubMed Abstract | CrossRef Full Text | Google Scholar

326. Minoletti A, Soto-Brandt G, Sepúlveda R, Toro O, Irarrázaval M. Capacidad de respuesta de la atención primaria en salud mental en Chile: una contribución a Alma-Ata. Rev Panamericana Salud Pública (2018) 42:e136. doi: 10.1038/nature11283

CrossRef Full Text | Google Scholar

327. Math SB, Srinivasaraju R. Indian Psychiatric epidemiological studies: Learning from the past. Indian J Psychiatry (2010) 52(Suppl1):S95. doi: 10.1056/NEJMoa1211103

PubMed Abstract | CrossRef Full Text | Google Scholar

328. Urbanoski KA, Mulsant BH, Willett P, Ehtesham S, Rush B. Real-world evaluation of the Resident Assessment Instrument-Mental Health assessment system. Can J Psychiatry (2012) 57(11):687–95. doi: 10.1177/070674371205701106

PubMed Abstract | CrossRef Full Text | Google Scholar

329. Perlman C, Martin L, Hirdes J. Using routinely collected clinical assessments in mental health services: the resident assessment instrument-mental health. Can J Psychiatry-Revue Can Psychiatrie (2014) 59(7):399.

Google Scholar

330. Jonsson T, Atwal JK, Steinberg S, Snaedal J, Jonsson PV, Bjornsson S, et al. A mutation in APP protects against Alzheimer's disease and age-related cognitive decline. Nature (2012) 488(7409):96.

PubMed Abstract | Google Scholar

331. Jonsson T, Stefansson H, Steinberg S, Jonsdottir I, Jonsson PV, Snaedal J, et al. Variant of TREM2 associated with the risk of Alzheimer's disease. New Engl J Med (2013) 368(2):107–16. doi: 10.1056/NEJMoa1211103

PubMed Abstract | CrossRef Full Text | Google Scholar

332. Kawachi I, Berkman LF. Social ties and mental health. J Urban Health (2001) 78(3):458–67. doi: 10.1017/S1041610208007850

PubMed Abstract | CrossRef Full Text | Google Scholar

333. van der Wolf E, van Hooren SAH, Waterink W, Lechner L. Well-being in elderly long-term care residents with chronic mental disorder: a systematic review. Aging Ment Health (2019) 23(3):287–96. doi: 10.1093/geront/35.2.172

PubMed Abstract | CrossRef Full Text | Google Scholar

334. Kehyayan V, Hirdes JP, Perlman CM. Education and employment needs and receipt of services in community and inpatient mental health settings. Community Ment Health J (2014) 50(6):637–45. doi: 10.1089/jpm.2019.0022

PubMed Abstract | CrossRef Full Text | Google Scholar