Co-design of the neurodevelopment assessment scale

Introduction: Neurodevelopmental disorders (NDDs) have high comorbidity rates and shared etiology. Nevertheless, NDD assessment is diagnosis-driven and focuses on symptom profiles of individual disorders, which hinders diagnosis and treatment. There is also no evidence-based, standardized transdiagnostic approach currently available to provide a full clinical picture of individuals with NDDs. The pressing need for transdiagnostic assessment led to the development of the Neurodevelopment Assessment Scale (NAS).

Methods: This paper describes the co-design process used in the development of NAS prototype with stakeholders including individuals with NDDs, parents of children with NDDs, and health professionals.

Results and discussion: Results indicated stakeholder consensus that NAS would be useful for NDD assessment, and included recommendations for fine-tuning the way some questions were asked (e.g., child's diagnoses), question flow (e.g., branching logic), and the language and presentation of the prototype (e.g., readability). Stakeholders also suggested the administration protocol should be flexible using electronic, face-to-face, online formats etc.

1 Introduction

Neurodevelopmental disorders (NDDs) are a heterogeneous group of conditions which include cerebral palsy (CP), intellectual disability (ID), autism spectrum disorder (ASD), attention-deficit/hyperactive disorder (ADHD), tic disorders, as well as motor, communication and specific learning disorders (1, 2). These disorders share risk-conferring genetic variants (3–7), as well as environmental risk factors (8, 9). Furthermore, NDDs often share similar symptoms, for example, sleeping issues (e.g., 10), executive function (e.g., 11, 12), attention (e.g., 13, 14), and social information processing discrepancies (e.g., 15). The term “Early Symptomatic Syndromes Eliciting Neurodevelopmental Clinical Examinations” (ESSENCE) has been proposed, highlighting the fact individuals with NDDs share common symptoms (16).

The close relation between NDDs is also evidenced by the high co-occurrence rate of two or more NDD diagnoses (i.e., comorbidity) (17–21);. For example, Eapen and Robertson (22) found that 86.5% of individuals with Tourette syndrome had comorbidities such as ADHD. Similarly, a review found that Tourette syndrome frequently co-occurs with ADHD (17%–68% of the cohort with Tourette syndrome also had ADHD) (23). Furthermore, ADHD also frequently co-occurs with ASD, learning disorders, and tic disorders (24), while ASD also frequently co-occur with ADHD, ID, and developmental coordination disorder (DCD) (25). The close relation between NDDs has led to the hypothesis that NDDs are a continuous spectrum of disorders (26, 27).

Despite the close relationship between NDDs, NDDs have been traditionally categorized as individual disorders (1). The diagnostic process of NDDs is often conducted in silos (28). Assessment tools, procedures, and diagnostic criteria for NDDs are also often designed to diagnose a specific NDD (29, 30). This approach is prone to the omission of different aspects of clinical profiles (11, 12). As a result, the diagnosis of comorbidities could be delayed (31, 32). Furthermore, treatment of NDDs could be compromised by misdiagnoses caused by overlapping symptoms across NDDs, and a failure to address associated symptoms and dysfunctions (2, 33). In the broader sense of knowledge advancement, siloed diagnostic approaches limit understanding of NDDs as a whole, as they do not disentangle the complex presentation of individual NDDs and the interface with comorbidities (5, 27).

To reduce the compartmentalization of NDD diagnoses which creates an illusion that NDDs are mutually exclusive, a comprehensive and transdiagnostic tool should be developed. We report on the co-design process used to develop an evidence-based and standardized transdiagnostic Neurodevelopment Assessment Scale (NAS; protocol as described by Masi and colleagues) (34). NAS is designed to be used across NDD symptoms, broader comorbidity symptoms, associated distress, and family history, in a single assessment.

NAS was co-designed with stakeholders to ensure the needs and requirements of future end-users (individuals with NDDs, parents/careers, and health care professionals) were considered and incorporated into NAS. Co-design refers to designing and deriving a solution to a question together with stakeholders (35). Compared to the traditional researcher-orientated designing process, co-design enhances the understanding of the problem, and provides additional perspectives in the development of the solution (36). For NAS development, stakeholders provided feedback through survey items with options as well as qualitative feedback as part of the co-design process. This paper summarizes stakeholder feedback on the NAS prototype (second consultation as described below).

2 Materials and methods

The ethics approval of this study was obtained from the UNSW Human Research Ethics Committee (reference number: HC210260).

2.1 Participants/stakeholders

Stakeholders were recruited via promotion through practitioner networks, disability service providers, advocacy groups, and word-of-mouth. Participation was voluntary, and participants could withdraw anytime without consequences. Stakeholders were given an information sheet and provided consent if they agreed to participate. Eligibility criteria included: (a) being 18 years old or over; and (b) being a parent/caregiver of a child with an NDD, an adult with an NDD, a health professional or clinician working with children with NDDs, or a representative of a disability service provider. Participants were ineligible to participate if they were non-English speakers, although no consenting stakeholders were excluded on this basis.

The initial pool (see “Design”) of stakeholder participants consisted of nine health practitioners/disability service providers, seven parents of children with NDDs, as well as seven adults with NDDs (i.e., n = 23). Participants were invited to introduce themselves, and amongst adults with NDDs, NDDs they reported having included: ASD (n = 5), ADHD (n = 4), CP (n = 1), dyslexia (n = 1), developmental coordination disorder (n = 1), and NDDs that were not further elaborated (n = 1). Other diagnoses included: anxiety symptoms (n = 2), obsessive-compulsive disorder (n = 1), brain injury (n = 1), and unspecified mental health issues (n = 1). In terms of parents and caregivers, three participants reported that they were mothers of children with NDDs, two reported that they were fathers, and two did not specify their relationship with their children with NDDs. The type of NDDs that they reported their children having included: ASD (n = 6), tic disorders (n = 2), ADHD (n = 1), ID (n = 2), learning difficulties (n = 1), and stuttering (n = 1). Other conditions that parents reported their children having included premature birth (n = 2), obsessive-compulsive disorder (n = 1), anxiety symptoms (n = 1), bipolar disorder (n = 1), and pontocerebellar hypoplasia (n = 1). Regarding health professionals, clinicians, and representatives of disability service providers, they reported that they were: pediatricians (n = 4), speech therapists (n = 2), social worker (n = 1), disability physiotherapist (n = 1), and unspecified clinician (n = 1). Although all health professionals reported that they work with individuals with NDDs, most did not specify the type of NDDs (n = 5), and examples given included ID (n = 2), ADHD (n = 1), ASD (n = 1), and learning disorders (n = 1).

The second consultation included six stakeholders within each respective participant group [three health practitioners/disability service providers, one parent, and one adult with NDD(s) withdrew from participation], yielding a total of 18 stakeholders. Due to the anonymous nature of the study, it was not possible to find out the demographics of the participants who dropped out using data from the first consultation. Stakeholders were given AUD20 worth of supermarket vouchers as a token of appreciation.

2.2 Design

The study involved a mixed-methods co-design process. A cluster analysis of archived data sets, which contained assessment results of individuals with NDDs, was conducted to investigate how symptoms of different NDDs could be categorized. The identified categories of symptoms were used to guide the structure of NAS. In addition, stakeholders provided anonymous comments via questionnaires, and comments were aggregated to inform the formation (first consultation) and modification (second consultation) of the first NAS prototype. The first consultation explored stakeholders' thoughts on what was important in the process of assessing NDDs, as well as stakeholders' preferences on NDD assessments through open-ended questions. Details of the second stakeholder consultation are described in the “Second Stakeholder Consultation” section below. This iterative consultation process allowed unique perspectives of stakeholders to be captured and considered for NAS development.

2.2.1 Development of the first NAS prototype

A team of psychiatrists, medical practitioners and researchers utilized the results of the first stakeholder consultation and cluster analysis to produce the first NAS prototype. To ensure the validity of the questionnaire, the development of NAS was guided by a list of questionnaires that were used in the data included in the cluster analysis and/or items relating to core symptoms of neurodevelopmental disorders. The list of questionnaires included (in alphabetical order): the Antisocial Process Screening Device (APSD) (37); Autism Diagnostic Observation Schedule, Second Edition (ADOS-2) (38); Autism Diagnostic Interview-Revised (ADI-R (39); Brief Symptom Inventory (BSI) (40); Conners' Parent Rating Scale—Revised (41); Child Behaviour Checklist (CBCL) (42); Children's Sleep Habits Questionnaire (CSHQ) (43); Griffith Empathy Measure (GEM) (44); Mullen Scales of Early Learning (MSEL) (45); Social Communication Questionnaire (SCQ) (46); Strengths and Difficulties Questionnaire (SDQ) (47); Short Sensory Profile, Second Edition (SSP) (48); Preschool Form and School-Age Form of Social Responsiveness Scale, 2nd Edition (SRS-2) (49); Vineland Adaptive Behaviour Scales (Second Edition; Vineland-II) (50); and Wechsler Intelligence Scale for Children, Fifth Edition (WISC-V) (51). Developmental regression questions were also informed by Furley and colleague's (52) review.

The NAS questionnaire prototype included a demographics section to capture socio-demographics information (age, gender), medications currently used, known physical and mental health conditions, and family medical history. The second part of the prototype encompassed a detailed assessment of twelve categories of NDD-related symptoms and abilities, including: (a) communication, (b) emotions, (c) social, (d) behavior (including hyperactivity, impulsivity, hostility, conduct issues, callousness, as well as emotional and behavioral regulations), (e) cognitive, (f) obsession and compulsion, (g) repetitive restricted behaviors, (h) tics, (i) sensory, (j) motor ability, (k) adaptive functioning, and (l) physical health. Screening questions were included in all but the physical health section, such that detailed questions were only displayed if a responder selected “yes” to any of the screening questions, indicating potential concerns in a particular symptom or ability category. The NAS questionnaire was also designed so that questions irrelevant to the responders' age and/or ability could be skipped. The NAS prototype can be provided upon request.

2.2.2 Second stakeholder consultation

The draft NAS questionnaire generated from the above analysis was then provided for a second stakeholder consultation to understand whether the NAS prototype met the needs of the stakeholders and aspects that needed to be revised. The second consultation involved the same group of stakeholders evaluating the first NAS prototype via an online questionnaire (see Supplementary Table S1) to understand: (a) if the screening questions in the initial draft could effectively guide responders to fill out relevant sections only (Yes/No question); (b) if the questions were important and suitable for NDD assessments (Yes/No questions, followed by open-ended feedback if “No” was selected); healthcare professionals were also asked whether the order of the questions was appropriate (open-ended question), if the NAS questions could effectively capture relevant NDD symptoms (Yes/No question), and if they would recommend adding/removing any questions (Yes/No question and open-ended question); (c) if language, structure, and response format of the questionnaire were appropriate (multiple selections, Yes/No questions, and/or open-ended questions); (d) (for parents or adults with NDDs) the ideal environmental settings for NDD assessment (open-ended question); (e) how important it is for NAS to detect NDD compared to obtaining a details profile of the person being assessed (multiple selections); and (f) how best to present the NAS results (multiple selections). Participants were given opportunities to provide suggestions and elaborate on their responses. The qualitative data was collected between June and October, 2023, via an online questionnaire hosted in REDCap electronic data capture tool hosted and managed by Research Technology Services (UNSW Sydney). Responses from 18 participants were collected, that is, all of the stakeholders who agreed to participate in the second consultation, and 78.26% of the initial pool of 23 stakeholders.

2.3 Data analyses

Quantitative data was analyzed using IBM Statistical Package for the Social Sciences (SPSS) and Microsoft Excel. Qualitative responses were coded independently by at least two of the three authors (TYW, SIA, and CLC) using NVivo 14 software. Each coder coded the responses to the questionnaires independently and grouped the codes into themes and sub-themes guided by the grounded theory method (53, 54). Any emerging new themes were compared against the existing themes, and either became their own new themes or were used to expand and/or modify existing themes. The themes and subthemes between the three coders were then compared. In case of discrepancies, codes and questionnaire responses were reviewed and refined, and new themes were formed through consensus.

3 Results

3.1 Quantitative feedback on the prototype

Quantitative data was collected from stakeholders (parents/caregivers of children with NDDs, adults with NDDs, health professionals or clinicians working with children with NDDs, and a representative of a disability service provider). In terms of NAS prototype content, stakeholders tended to agree that the prototype was relevant and appropriate. All stakeholders (n = 18) agreed that the sections in the NAS prototype are important for the assessment of a child with NDD. Furthermore, all health practitioners and service providers (n = 6) agreed that the NAS prototype effectively captured the type of symptoms a child with NDD(s) may have. A minority of participants (n = 5; 27.78% of all 18 participants) suggested there were items/sections in the draft that were not suitable, and provided further comments. In addition, two health practitioners/service providers suggested additional questions could be included in the prototype. Suggestions were summarized as qualitative feedback. Finally, while all adults with NDD(s) and parents/caregivers (n = 12) agreed that NAS was important or very important for detecting NDDs, 50% of health practitioners/service providers (n = 3) also thought that NAS was important or very important in detecting NDDs and comorbidities (as opposed to building an assessment profile). All qualitative suggestions will be discussed in the “Qualitative Feedback on the Prototype” section below.

In terms of the logistics of assessment, 83.33% of all 18 participants (n = 15) agreed that the screening questions provided enough information to allow subsequent irrelevant questions to be skipped. In addition, most participants endorsed a 4-point scale to respond to NAS questions (“None of the time”, “Some of the time”, “Most of the time”, and “Not applicable”), regardless of whether the questions were about the child's daily function (endorsing participants n = 9; 50%), caretakers' level of distress (endorsing participants n = 11; 61.11%), or how a child's symptoms impacts people around them (endorsing participants n = 11; 61.11%). Furthermore, participants reported that they preferred NAS assessment results to be reported using graphs with visual indicators of the normal range (n = 11; 61.11%).

3.2 Qualitative feedback on the prototype

Three themes and six sub-themes were identified from the second consultation. The main themes were: (a) suitable features of the NAS draft, (b) modification suggestions on the NAS questionnaire; and (c) suggestions on the administration of NAS. The themes and associated sub-themes are illustrated in Table 1 and are discussed below.

Table 1

Table 1. Themes and sub-themes emerged during the second stakeholder consultation.

3.2.1 Theme 1: suitable features of the NAS draft

Adults with NDD(s), parents, health practitioners, identified positive attributes of NAS. Some participants commented that the NAS covered a comprehensive range of symptoms, and was useful as an assessment scale for disorders. For instance, an adult with NDD(s) mentioned that the scale “should be helpful for both getting a diagnosis, and possibly an assessment tool that could be used for say NDIS to allocated resources”. Note that NDIS, or National Disability Insurance Scheme in full, is an Australian scheme that provides financial support for disability interventions. It was also mentioned that the order of the questionnaire was appropriate, and the items were easily understandable.

3.2.2 Theme 2: modification suggestions on the NAS questionnaire

Stakeholders suggested some adjustments to the NAS prototype, which include modifying the content of the questionnaire (Sub-theme 1), showing relevant questions only (Sub-theme 2), and improving the language, grammar, and presentation of the questionnaire (Sub-theme 3).

3.2.2.1 Sub-theme 1: comprehensive diagnostic content

Participants suggested modifications to various aspects of the questionnaire prototype. Regarding the content of the questionnaire, it was suggested that we should fine-tune the child's diagnoses, family medical history, physical pain questions, as well as the “social communication” and “communication” domains. Suggestions to child's diagnosis questions include asking for more detailed information, especially: (a) allowing NAS responders to indicate disorder(s) being diagnosed prenatal; (b) providing a chance for NAS responders to disagree with a child's diagnosis; (c) asking about who made the reported diagnosis/diagnoses; and (d) allowing reported speculated but yet-to-be diagnosed disorder(s). For example, a disability service provider mentioned that “[w]ith conditions such as Down syndrome and some other syndromes which can now be detected prenatally, the question relating to age of onset/diagnosis may need to include a “prenatally detected” option”. (corresponding to point a). In regard to point (b), an adult with NDD(s) suggested “asking the parents if they agree with the child's diagnosis”. Concerning who diagnosed a child (point c), a health practitioner suggested that “do you need to ascertain if the diagnosis has been made by a health professional OR are you ok if the parent has made a diagnosis by Dr Google/internet?”. Finally, a disability service provider raised the question “I wonder if there's value in adding an option of “suspected”?” (point d). It is worth noting that parents did not provide suggestions on the child's diagnostic questions.

Suggestions were also made on modifying the family medical history, physical pain questions, and the “social communication” and “communication” domains. Regarding family medical history, it was suggested that cousins' conditions should also be taken into consideration, and that “other” and “unknown” options should be provided. It was also suggested that the question “level of the impact of [a family member's disorder] on the functioning of the child's [diagnosis]” should be reworded as an adult with NDD(s) suggested that he/she “was confused by the wording of the questions.” Note that no health professionals or parents provided feedback on the family medical history questions. Aside from family history questions, an adult with NDD(s) suggested that physical symptoms should also be assessed, in case “if a person's NDD is physical this is something that could have a significant impact upon both their functioning and their wellbeing”. Regarding the “social communication” and “communication” domains, an adult with NDD(s) suggested that for the social communication domain, “there should be a question about if the child can maintain long[-]term friendships”. Another health practitioner suggested that the “communication” domain should be expanded to cover a wider spectrum of language difficulties, “e.g., speech sound disorder and fluency problems”.

Furthermore, adults with NDDs, health practitioners, and parents suggested that NAS respondents should be allowed to enter free text and to elaborate on their responses, and provided with further information for some questions. For example, regarding free text, a health practitioner said “I have selected an open-ended response in the above [all categories of] questions as I think it allows more space for people to describe what felt important to them”. However, the same health practitioner also noted that they were “aware that it might not be suitable for every parent or caregiver”. Similarly, regarding questions related to a child's feelings or emotions, an adult with NDD(s) suggested providing an option for parents/caretakers to flag that the emotion reported was observed by the parent/caretaker instead of being expressed by the child being assessed. In addition, adult with NDD(s) and health practitioner participants suggested that examples should be given to explain the questions better, for example, a health practitioner said, “I wonder whether examples might be provided”.

In addition, it was recommended that the strengths and all aspects of challenges faced by children with NDDs should be considered. Regarding strengths, one adult with NDD(s) said “[y]ou could include a mention to child's strengths and talents”. Similarly, an adult with NDD(s) also suggested taking a child's academic excellence into account: “[in the] cognitive domain an option for parents to express that their child excels academically may be useful”. Concerning challenges, an adult with NDD(s) suggested that NAS has to “capture the spectrum of different challenges people may be experiencing … as well as things they find tricky”.

There are some other suggestions related to the content of NAS in general. For example, when assessing family medical history, we should be mindful that formal diagnoses of some disorders might not have previously existed. Furthermore, we should be aware that some disorders are not well-understood by the general public (e.g., mood issues and oppositional defiant disorder), symptoms of some disorders are hard to distinguish (e.g., ASD and obsessive-compulsive disorder), and some disorders are traditionally diagnosed by a pediatrician (e.g., global developmental delay and ASD). A health practitioner also reminded that we should reconsider the rationale of having screening questions. On the note of comorbidities, it was suggested that an algorithm system could be employed to identify the primary diagnosis and to assess comorbidities if ASD was identified. A health practitioner also proposed that if the goal of NAS was to collect all symptom information, then formal diagnoses may not need to be reported in NAS. Finally, regarding tics and Tourette's Syndrome, it was suggested that enquiries about the improvement of the condition is irrelevant due to the fluctuating nature of the symptoms.

3.2.2.2 Sub-theme 2: applicable questions

Participants suggested that is it important to only show relevant questions to NAS users. One adult with NDD(s) suggested that “I think the branching logic is great and very important … as not all questions will be relevant for each individual”. A health practitioner also pointed out that NAS could be revised to only show age-appropriate questions, as questions such as “is your child able to stop and think before acting?” and “does your child experience intense emotions that are difficult to control?” might not be useful in identifying symptoms in younger children. Note that no parents provided comments on this topic.

3.2.2.3 Sub-theme 3: readability and language of the questionnaire

There were suggestions on the language, grammar, and presentation of the questionnaire prototype. Some participants commented that the domain titles were inappropriate. For example, an adult with NDD(s) said, “[m]ight be a bit confronting and unhelpful to be reading this when you are just wanting people to answer based on what their child's experience is (regardless of what labels they have been exposed to)”. Another health practitioner mentioned “[t]he heading for the sub-sections might also create response bias to the respondents”, which echoed the comment from an adult with NDD(s): “I wonder whether the inclusion of domain and subdomain headings on the final version of the questionnaire will influence people's responses?”. Nevertheless, parents did not raise any concerns about the domain titles.

All three groups of participants also raised suggestions on the word choices of the NAS prototype. Some words that were suggested to be replaced included “overly/excessively” (because they have negative connotations), “normal” (an adult with NDDs mentioned that “if the NAS is filled out by a parent who is used to their child's behaviours … their answers may be slightly skewed because they may view those behaviours as “normal” for that child”), “acceptable” (what is regarded as acceptable can be subjective), and “disease”. It was suggested that the choice of words in the social domain was too arbitrary at times and should be revised. Furthermore, all three groups of participants raised concerns regarding the use of phrases “age of onset” and “trigger” in the child's health history questions, as (a) these phrases could lead to confusion, such as “is this referring to age of diagnosis, or age of when symptoms first started being noticed?” (comment from an adult with NDDs), and (b) that some disorders are congenital and therefore do not have a time of onset or triggers.

In terms of readability, participants suggested that the language of the questionnaire was too difficult at times, and some minor grammatical mistakes should be fixed. For instance, a parent reported that “I would say across the entire survey the language level is quite high and most people who need to complete this would require a higher level of literacy”. A health practitioner commented “I am thinking whether items under cognitive domains might be a bit challenging for parents to understand just by reading”. Similarly, an adult with NDD(s) suggested titles such as “interpersonal sensitivity” and “affective empathy” should be written in lay language. To improve the readability, it was suggested the team invite individuals with NDD to review the questions, and/or have an easy-to-read version for NAS. In addition to readability, it was suggested that the team go through the NAS questionnaire and correct grammatical errors.

All three participant groups also provided feedback on the format of NAS to enhance its accessibility. Suggestions include (a) adding clear instructions of what the next question would be based on the current response; (b) using headings, different font sizes and colors to distinguish between sections; (c) re-ordering questions to present sections that are more easily understood first (e.g., physical health); and (d) consider using a 10-point scale for responses (cf. the quantitative findings under “Quantitative Feedback on the Prototype”).

Finally, participants from all three groups were concerned about the length of the questionnaire. Nevertheless, some parent and health practitioner participants thought the length was acceptable to cover the complex NDD presentations. To shorten the questionnaire, participants recommended grouping or truncating questions on the child's and family's mental health and medical history.

3.2.3 Theme 3: pragmatic administration of NAS

Participants also provided suggestions on how NAS should be administered. Specifically, participants suggested that NAS administration should be flexible (Sub-theme 4). They also provided opinions on assessment contexts (Sub-theme 5) and practitioner attributes (Sub-theme 6).

3.2.3.1 Sub-theme 4: flexible administration

Participants recommended providing extra information or support whilst individuals fill in the NAS questionnaire, with one such example being the purpose of the questionnaire. An adult with NDD(s) suggested that “[it] would also really help with increasing understanding of why people should complete [NAS]”. Similarly, it was suggested that the purpose of the NAS results should be better explained to participants. In addition, participants also suggested providing extra resources or support whilst individuals fill in the NAS questionnaire. For instance, support should be provided to ensure that the questionnaire respondents understand the NAS questions. A health practitioner commented that “I wonder if there could be ways to make sure the questions are understood in the ways that they are intended to. I think parents might need help when they are filling in the survey”. It was also suggested that support or information should also be given if NAS responders find certain questions upsetting or confronting.

Further to providing information and support, participants also advised NAS administrators to consider individual needs and ways to accommodate them accordingly when administering NAS. Special considerations included: (a) preparing multiple versions of NAS to accommodate a child's age and the responders' physical ability/accessibility requirements (e.g., providing different delivery formats including an audio version/screen reader, allowing a person to respond to NAS verbally or physically using a computer or on paper, and providing appropriate font size), (b) allowing assessments to be done across multiple sessions and providing extra response time if needed, (c) allowing assessment without children and/or provide an assessment location friendly to children, and (d) measures (e.g., inform about the assessment progress) should be put in place to keep interviewees engaged).

3.2.3.2 Sub-theme 5: administration contexts

Participants were also concerned about the assessment location. Participants suggested that the NAS assessment be done in a comfortable location. For example, an adult with NDD(s) and a parent suggested that NAS should be administered in a familiar location/environment. The assessment environment should also be flexible according to the characteristics of the child, and should be comfortable for the assessment scale responder. Other suggested environmental features include having low light, no clock, and being in a “not-crowded” setting. Some responders suggested that assessments should be done face-to-face, and suggested locations such as clinics, home, and schools. However, a parent mentioned that an online or telehealth option should be provided for individuals to suit individual needs, for example “[O]nline/digital appointments would be a great idea for many with TS [i.e., Tourette syndrome]”.

3.2.3.3 Sub-theme 6: practitioner attributes

Participants also discussed some practitioner attributes that they would appreciate. Adults with NDD(s) suggested that they preferred the assessment to be administered by a health practitioner that they are familiar with. One of the participants explained that this was because “[i]f an older child is self-completing needs support to answer questions, it would be important that this support person knows them well and is familiar with their communication style”. Some other important attributes included that the assessment atmosphere should be welcoming, and the experience with the clinician should be pleasant.

4 Discussion

The current study aimed to describe the co-design and developmental process of NAS, with a focus on obtaining stakeholder feedback on the first NAS prototype. The prototype covered symptoms in the areas of communication, emotions, social, behavior (including hyperactivity, impulsivity, hostility, conduct issues, callousness, as well as emotional and behavioral regulations), cognitive, obsession and compulsion, repetitive restricted behaviors, tics, sensory, motor ability, adaptive functioning, and physical health. Comments from stakeholders suggested that while NAS would be useful, further adjustments were necessary to improve the content, assessment flow (only show relevant questions), the language/clarity, and the presentation of the prototype. Suggestions were also made on how NAS should be administered. It was advised that the assessment arrangement and location should be flexible, and the practitioner should be welcoming and familiar to the person being assessed. NAS was then modified according to stakeholders' feedback.

The results of the study highlighted the need to develop a transdiagnostic neurodevelopmental disorder assessment scale. This was evidenced not only by participants' feedback on NAS being comprehensive and potentially useful, but also from the modification suggestions. Specifically, there were suggestions of including extra symptoms and information, which underscored the fact that a wide range of presentations should be considered while assessing a child with NDDs. This is consistent with the findings that individuals with a primary NDD diagnosis could show symptoms of other NDD diagnoses, even those symptoms were not part of the diagnostic criteria of the primary NDD diagnosis (55). In fact, symptoms of NDDs evolve across the developmental span of individuals (56, 57). Specifically, some symptom constellations may take prominence and meet the diagnostic criteria, while others may remain sub-threshold at a given time-point (58, 59). In addition, it was found that symptoms “irrelevant” to a primary diagnosis could nonetheless predict the primary disorder being diagnosed in the future (e.g., early motor development delay associated with future ADHD diagnosis) (60). The symptom manifestation pattern highlights the fact that NDD assessment should be transdiagnostic and should capably consider a board range of symptoms, in order to provide necessary support, timely discovery of comorbidities, and prevent symptoms from developing into severe psychopathology.

4.1 Strengths, limitations, and future directions

A strength of the current study is that stakeholders were substantially involved in co-designing NAS. Co-design can enhance the developmental process of a health problem by providing new insight into the problem and possible solutions (36). The benefits are also observed in the current project. For example, stakeholders provided comments on what information should be gathered by NAS, how to adjust the language to make it more acceptable to future end-users, and how to streamline the NAS assessment process. Comments from stakeholders ensure the rigor of the NAS scale and allow it to be both efficient and useful to individuals with NDDs, as a career and/or diagnosis makers. Overall, the study results underscore the importance of involving comments from stakeholders, as stakeholders provided unique insight from their experience as end-users.

The current study design has several limitations. Due to recruitment difficulties, some stakeholders were lost between the two stakeholder consultations of the study. Although we successfully gathered comments from 18 participants in the second consultation, which is an acceptable number (61, 62), the loss of information from attrition may bias the information received. Moreover, stakeholders' socioeconomic background was not taken into account in the current study. Future studies should investigate how NAS could be adapted to suit the needs of individuals from different socioeconomic backgrounds. In addition, while NAS prototype has face validity after the rigorous development process by stakeholders and researchers, its psychometrics properties remain unknown. Future work will be conducted to investigate its validity, reliability, and real-world applicability. Furthermore, future work will continue with this theme of co-design upon completion of the NAS scale to allow for further fine-tuning of NAS administration.

5 Conclusion

The current article described the co-design process used in the development of NAS, a transdiagnostic NDD assessment, with a focus on the stakeholder feedback on the first NAS prototype. The results presented underscore the importance of having a transdiagnostic NDD assessment that covers a wide range of symptoms, and suggestions for further modifications to be made to improve the content, assessment flow, language, and presentation of NAS. NAS prototype will be amended according to stakeholders' feedback, and the validity and reliability of NAS will be investigated in future work.

Data availability statement

The raw data supporting the conclusions of this article will be made available by the authors, without undue reservation.

Ethics statement

The studies involving humans were approved by the University of New South Wales HREC Committee A. The studies were conducted in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author contributions

TYW: Conceptualization, Data curation, Formal Analysis, Investigation, Methodology, Project administration, Software, Visualization, Writing – original draft, Writing – review & editing. SA: Conceptualization, Formal Analysis, Investigation, Project administration, Writing – review & editing, Writing – original draft, Methodology. CLC: Formal Analysis, Methodology, Project administration, Supervision, Writing – review & editing, Conceptualization, Investigation, Writing – original draft, Resources. PL: Supervision, Writing – review & editing, Methodology, Project administration, Writing – original draft, Conceptualization, Investigation. AM: Conceptualization, Funding acquisition, Investigation, Supervision, Writing – review & editing, Writing – original draft, Methodology. AlM: Conceptualization, Funding acquisition, Investigation, Methodology, Supervision, Writing – review & editing, Writing – original draft. AnM: Conceptualization, Funding acquisition, Investigation, Methodology, Resources, Supervision, Writing – review & editing, Writing – original draft, Project administration. KM: Conceptualization, Funding acquisition, Investigation, Supervision, Writing – review & editing, Writing – original draft, Methodology. AW: Conceptualization, Funding acquisition, Investigation, Supervision, Writing – review & editing, Writing – original draft, Methodology. MD: Conceptualization, Investigation, Supervision, Writing – review & editing, Resources, Writing – original draft, Methodology. VE: Conceptualization, Data curation, Funding acquisition, Investigation, Methodology, Resources, Supervision, Writing – review & editing, Writing – original draft, Formal Analysis, Project administration.

Funding

The author(s) declare financial support was received for the research, authorship, and/or publication of this article. This work was funded by the Cerebral Palsy Alliance Research Foundation under Grant Number: PRG11219.

Conflict of interest

AW received support from the Investigator Grant from the National Health and Medical Research Council (1173896).

The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

Supplementary material

The Supplementary Material for this article can be found online at: https://www.frontiersin.org/articles/10.3389/frcha.2025.1497632/full#supplementary-material

References

1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 5th ed. Washington, DC: American Psychiatric Association Publishing (2022). p. 1120. doi: 10.1176/appi.books.9780890425787

Crossref Full Text | Google Scholar

2. Ismail FY, Shapiro BK. What are neurodevelopmental disorders? Curr Opin Neurol. (2019) 32:611–6. doi: 10.1097/WCO.0000000000000710

PubMed Abstract | Crossref Full Text | Google Scholar

3. Lee PH, Anttila V, Won H, Feng Y-CA, Rosenthal J, Zhu Z, et al. Genomic relationships, novel loci, and pleiotropic mechanisms across eight psychiatric disorders. Cell. (2019) 179:1469–1482.e11. doi: 10.1016/j.cell.2019.11.020

PubMed Abstract | Crossref Full Text | Google Scholar

4. Michetti C, Falace A, Benfenati F, Fassio A. Synaptic genes and neurodevelopmental disorders: from molecular mechanisms to developmental strategies of behavioral testing. Neurobiol Dis. (2022) 173:105856. doi: 10.1016/j.nbd.2022.105856

PubMed Abstract | Crossref Full Text | Google Scholar

5. Parenti I, Rabaneda LG, Schoen H, Novarino G. Neurodevelopmental disorders: from genetics to functional pathways. Trends Neurosci. (2020) 43:608–21. doi: 10.1016/j.tins.2020.05.004

PubMed Abstract | Crossref Full Text | Google Scholar

6. Price KM, Wigg KG, Feng Y, Blokland K, Wilkinson M, He G, et al. Genome-wide association study of word Reading: overlap with risk genes for neurodevelopmental disorders. Genes Brain Behav. (2020) 19:e12648. doi: 10.1111/gbb.12648

PubMed Abstract | Crossref Full Text | Google Scholar

7. Wang T, Hoekzema K, Vecchio D, Wu H, Sulovari A, Coe BP, et al. Large-scale targeted sequencing identifies risk genes for neurodevelopmental disorders. Nat Commun. (2020) 11:4932. doi: 10.1038/s41467-020-18723-y

PubMed Abstract | Crossref Full Text | Google Scholar

8. Doi M, Usui N, Shimada S. Prenatal environment and neurodevelopmental disorders. Front Endocrinol (Lausanne). (2022) 13:860110. doi: 10.3389/fendo.2022.860110

PubMed Abstract | Crossref Full Text | Google Scholar

9. Tran NQV, Miyake K. Neurodevelopmental disorders and environmental toxicants: epigenetics as an underlying mechanism. Int J Genomics Proteomics. (2017) 2017:7526592. doi: 10.1155/2017/7526592

PubMed Abstract | Crossref Full Text | Google Scholar

10. Shelton AR, Malow B. Neurodevelopmental disorders commonly presenting with sleep disturbances. Neurotherapeutics. (2021) 18:156–69. doi: 10.1007/s13311-020-00982-8

PubMed Abstract | Crossref Full Text | Google Scholar

11. Crisci G, Caviola S, Cardillo R, Mammarella IC. Executive functions in neurodevelopmental disorders: comorbidity overlaps between attention deficit and hyperactivity disorder and specific learning disorders. Front Hum Neurosci. (2021) 15:594234. doi: 10.3389/fnhum.2021.594234

PubMed Abstract | Crossref Full Text | Google Scholar

12. Dajani DR, Llabre MM, Nebel MB, Mostofsky SH, Uddin LQ. Heterogeneity of executive functions among comorbid neurodevelopmental disorders. Sci Rep. (2016) 6:36566. doi: 10.1038/srep36566

PubMed Abstract | Crossref Full Text | Google Scholar

13. Keehn B, Lincoln AJ, Müller R-A, Townsend J. Attentional networks in children and adolescents with autism spectrum disorder. J Child Psychol Psychiatry. (2010) 51:1251–9. doi: 10.1111/j.1469-7610.2010.02257.x

PubMed Abstract | Crossref Full Text | Google Scholar

14. Kooistra L, Crawford S, Gibbard B, Kaplan BJ, Fan J. Comparing attentional networks in fetal alcohol spectrum disorder and the inattentive and combined subtypes of attention deficit hyperactivity disorder. Dev Neuropsychol. (2011) 36:566–77. doi: 10.1080/87565641.2010.549978

PubMed Abstract | Crossref Full Text | Google Scholar

15. Vandewouw MM, Choi E, Hammill C, Arnold P, Schachar R, Lerch JP, et al. Emotional face processing across neurodevelopmental disorders: a dynamic faces study in children with autism spectrum disorder, attention deficit hyperactivity disorder and obsessive-compulsive disorder. Transl Psychiatry. (2020) 10:375. doi: 10.1038/s41398-020-01063-2

PubMed Abstract | Crossref Full Text | Google Scholar

16. Gillberg C. The ESSENCE in child psychiatry: early symptomatic syndromes eliciting neurodevelopmental clinical examinations. Res Dev Disabil. (2010) 31:1543–51. doi: 10.1016/j.ridd.2010.06.002

PubMed Abstract | Crossref Full Text | Google Scholar

17. Anckarsäter H, Larson T, Hansson SL, Carlström E, Ståhlberg O, Gillberg C, et al. Child neurodevelopmental and behavioural problems are intercorrelated and dimensionally distributed in the general population. Open Psychiatr J. (2008) 2:5–11. doi: 10.2174/1874354400802010005

Crossref Full Text | Google Scholar

18. Craig F, Savino R, Trabacca A. A systematic review of comorbidity between cerebral palsy, autism spectrum disorders and attention deficit hyperactivity disorder. Eur J Paediatr Neurol. (2019) 23:31–42. doi: 10.1016/j.ejpn.2018.10.005

PubMed Abstract | Crossref Full Text | Google Scholar

19. Dewey D. What is comorbidity and why does it matter in neurodevelopmental disorders? Curr Dev Disord Rep. (2018) 5:235–42. doi: 10.1007/s40474-018-0152-3

Crossref Full Text | Google Scholar

20. Francés L, Quintero J, Fernández A, Ruiz A, Caules J, Fillon G, et al. Current state of knowledge on the prevalence of neurodevelopmental disorders in childhood according to the DSM-5: a systematic review in accordance with the PRISMA criteria. Child Adolesc Psychiatry Ment Health. (2022) 16:27. doi: 10.1186/s13034-022-00462-1

PubMed Abstract | Crossref Full Text | Google Scholar

21. Hansen BH, Oerbeck B, Skirbekk B, Petrovski BÉ, Kristensen H. Neurodevelopmental disorders: prevalence and comorbidity in children referred to mental health services. Nord J Psychiatry. (2018) 72:285–91. doi: 10.1080/08039488.2018.1444087

PubMed Abstract | Crossref Full Text | Google Scholar

22. Eapen V, Robertson MM. Are there distinct subtypes in tourette syndrome? Pure-tourette syndrome versus tourette syndrome-plus, and simple versus complex tics. Neuropsychiatr Dis Treat. (2015) 11:1431–6. doi: 10.2147/NDT.S72284

PubMed Abstract | Crossref Full Text | Google Scholar

23. Cravedi E, Deniau E, Giannitelli M, Xavier J, Hartmann A, Cohen D. Tourette syndrome and other neurodevelopmental disorders: a comprehensive review. Child Adolesc Psychiatry Ment Health. (2017) 11:59. doi: 10.1186/s13034-017-0196-x

PubMed Abstract | Crossref Full Text | Google Scholar

24. Gnanavel S, Sharma P, Kaushal P, Hussain S. Attention deficit hyperactivity disorder and comorbidity: a review of literature. World J Clin Cases. (2019) 7:2420–6. doi: 10.12998/wjcc.v7.i17.2420

PubMed Abstract | Crossref Full Text | Google Scholar

25. Saito M, Hirota T, Sakamoto Y, Adachi M, Takahashi M, Osato-Kaneda A, et al. Prevalence and cumulative incidence of autism spectrum disorders and the patterns of co-occurring neurodevelopmental disorders in a total population sample of 5-year-old children. Mol Autism. (2020) 11:35. doi: 10.1186/s13229-020-00342-5

PubMed Abstract | Crossref Full Text | Google Scholar

26. Eapen V. Neurodevelopmental genes have not read the DSM criteria: or, have they? Front Psychiatry. (2012) 3:75. doi: 10.3389/fpsyt.2012.00075

PubMed Abstract | Crossref Full Text | Google Scholar

27. Morris-Rosendahl DJ, Crocq M-A. Neurodevelopmental disorders-the history and future of a diagnostic concept. Dialogues Clin Neurosci. (2020) 22:65–72. doi: 10.31887/DCNS.2020.22.1/macrocq

PubMed Abstract | Crossref Full Text | Google Scholar

28. McDowell M. Child with multiple problems: clinical complexity and uncertainty. J Paediatr Child Health. (2018) 54:1084–9. doi: 10.1111/jpc.14151

PubMed Abstract | Crossref Full Text | Google Scholar

29. Antshel KM, Russo N. Autism spectrum disorders and ADHD: overlapping phenomenology, diagnostic issues, and treatment considerations. Curr Psychiatry Rep. (2019) 21:34. doi: 10.1007/s11920-019-1020-5

PubMed Abstract | Crossref Full Text | Google Scholar

30. Jeste SS. Neurodevelopmental behavioral and cognitive disorders. Continuum. (2015) 21:690–714. doi: 10.1212/01.CON.0000466661.89908.3c

PubMed Abstract | Crossref Full Text | Google Scholar

31. Kentrou V, de Veld DM, Mataw KJ, Begeer S. Delayed autism spectrum disorder recognition in children and adolescents previously diagnosed with attention-deficit/hyperactivity disorder. Autism. (2019) 23:1065–72. doi: 10.1177/1362361318785171

PubMed Abstract | Crossref Full Text | Google Scholar

32. Manohar H, Kuppili PP, Kandasamy P, Chandrasekaran V, Rajkumar RP. Implications of comorbid ADHD in ASD interventions and outcome: results from a naturalistic follow up study from south India. Asian J Psychiatr. (2018) 33:68–73. doi: 10.1016/j.ajp.2018.03.009

PubMed Abstract | Crossref Full Text | Google Scholar

33. Martino D, Malaty I, Müller-Vahl K, Nosratmirshekarlou E, Pringsheim TM, Shprecher D, et al. Treatment failure in persistent tic disorders: an expert clinicians’ consensus-based definition. Eur Child Adolesc Psychiatry. (2023) 32:859–72. doi: 10.1007/s00787-021-01920-5

PubMed Abstract | Crossref Full Text | Google Scholar

34. Masi A, Azim SI, Lam-Cassettari C, Dadds M, Mendoza Diaz A, Henry G, et al. Co-design of a neurodevelopment assessment scale: a study protocol. Int J Environ Res Public Health. (2021) 18:12837. doi: 10.3390/ijerph182312837

PubMed Abstract | Crossref Full Text | Google Scholar

35. Vargas C, Whelan J, Brimblecombe J, Allender S. Co-creation, co-design, co-production for public health: a perspective on definitions and distinctions. Public Health Res Pract. (2022) 32:e3222211. doi: 10.17061/phrp3222211

Crossref Full Text | Google Scholar

36. Steen M, Manschot M, De Koning N. Benefits of co-design in service design projects. Int J Des. (2011) 5:53–60. Available online at: http://www.ijdesign.org/index.php/IJDesign/article/view/890/346 (accessed May 16, 2023).

Google Scholar

37. Vitacco MJ, Rogers R, Neumann CS. The antisocial process screening device—an examination of its construct and criterion-related validity. Assessment. (2003) 10:143–50. doi: 10.1177/1073191103252347

PubMed Abstract | Crossref Full Text | Google Scholar

38. Lord C, Rutter M, DiLavore PC, Risi S, Gotham K, Bishop SL, et al. Autism Diagnostic Observation Schedule. 2nd ed. Torrance, CA: Western Psychological Services (2012).

Google Scholar

39. Rutter M, Le Couteur A, Lord C. Autism Diagnostic Interview-Revised. Los Angeles, CA: Western Psychological Services (2003).

Google Scholar

40. Derogatis LR. Brief Symptom Inventory. Minneapolis, MN: Pearson Assessments (2004).

Google Scholar

41. Conners K. Conners’ Parent Rating Scale—revised. North Tonawanda, NY: Multi-Health Systems (1997).

Google Scholar

42. Achenbach TM, Rescorla LA. Manual for the ASEBA School-age Forms & Profiles: Child Behavior Checklist for Ages 6-18, Teacher’s Report Form, Youth Self-report: An Integrated System of Multi-informant assessment. Burlington, VT: University of Vermont, Research Center for Children, Youth & Families (2001).

Google Scholar

43. Owens JA, Spirito A, McGuinn M. The children’s sleep habits questionnaire (CSHQ): psychometric properties of a survey instrument for school-aged children. Sleep. (2000) 23:1043–51. doi: 10.1093/sleep/23.8.1d

PubMed Abstract | Crossref Full Text | Google Scholar

44. Dadds MR, Hunter K, Hawes DJ, Frost ADJ, Vassallo S, Bunn P, et al. A measure of cognitive and affective empathy in children using parent ratings. Child Psychiatry Hum Dev. (2008) 39:111–22. doi: 10.1007/s10578-007-0075-4

PubMed Abstract | Crossref Full Text | Google Scholar

45. Mullen EM. Mullen Scales of Early Learning. AGS ed. Bloomington, MN: Pearson Clinical Assessment (1995).

Google Scholar

46. Rutter M, Bailey A, Berument SK, Lord C. Social Communication Questionnaire. Los Angeles, CA: Western Psychological Services (2003).

Google Scholar

47. Goodman R. The strengths and difficulties questionnaire: a research note. J Child Psychol Psychiatry. (1997) 38:581–6. doi: 10.1111/j.1469-7610.1997.tb01545.x

PubMed Abstract | Crossref Full Text | Google Scholar

48. Dunn W. Short Sensory Profile. 2nd ed. Bloomington, MN: Pearson Clinical Assessment (2014).

Google Scholar

49. Constantino JN. Social Responsiveness Scale. 2nd ed. Torrance, CA: Western Psychological Services (2012).

Google Scholar

50. Sparrow SS, Cicchetti D, Balla DA. Vineland Adaptive Behavior Scales. 2nd ed. AGS Publishing (2005).

Google Scholar

51. Wechsler D. Wechsler Intelligence Scale for Children. 5th ed. Bloomington, MN: PsychCorp (2014).

Google Scholar

52. Furley K, Mehra C, Goin-Kochel RP, Fahey MC, Hunter MF, Williams K, et al. Developmental regression in children: current and future directions. Cortex. (2023) 169:5–17. doi: 10.1016/j.cortex.2023.09.001

PubMed Abstract | Crossref Full Text | Google Scholar

53. Braun V, Clarke V. Can I use TA? Should I use TA? Should I not use TA? Comparing reflexive thematic analysis and other pattern-based qualitative analytic approaches. Couns Psychother Res. (2021) 21:37–47. doi: 10.1002/capr.12360

Crossref Full Text | Google Scholar

54. Corbin J, Strauss A. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory. 4th ed. Thousand Oaks, CA: SAGE (2015).

Google Scholar

55. Operto FF, Smirni D, Scuoppo C, Padovano C, Vivenzio V, Quatrosi G, et al. Neuropsychological profile, emotional/behavioral problems, and parental stress in children with neurodevelopmental disorders. Brain Sci. (2021) 11:584. doi: 10.3390/brainsci11050584

PubMed Abstract | Crossref Full Text | Google Scholar

56. Danielsson H, Imms C, Ivarsson M, Almqvist L, Lundqvist L-O, King G, et al. A systematic review of longitudinal trajectories of mental health problems in children with neurodevelopmental disabilities. J Dev Phys Disabil. (2024) 36:203–42. doi: 10.1007/s10882-023-09914-8

Crossref Full Text | Google Scholar

57. Nishimura T, Takei N, Tsuchiya KJ, Asano R, Mori N. Identification of neurodevelopmental trajectories in infancy and of risk factors affecting deviant development: a longitudinal birth cohort study. Int J Epidemiol. (2016) 45:543–53. doi: 10.1093/ije/dyv363

PubMed Abstract | Crossref Full Text | Google Scholar

58. Antolini G, Colizzi M. Where do neurodevelopmental disorders go? Casting the eye away from childhood towards adulthood. Healthcare. (2023) 11:1015. doi: 10.3390/healthcare11071015

PubMed Abstract | Crossref Full Text | Google Scholar

59. Moffitt TE, Houts R, Asherson P, Belsky DW, Corcoran DL, Hammerle M, et al. Is adult ADHD a childhood-onset neurodevelopmental disorder? Evidence from a four-decade longitudinal cohort study. Am J Psychiatry. (2015) 172:967–77. doi: 10.1176/appi.ajp.2015.14101266

PubMed Abstract | Crossref Full Text | Google Scholar

60. Micai M, Fulceri F, Caruso A, Guzzetta A, Gila L, Scattoni ML. Early behavioral markers for neurodevelopmental disorders in the first 3 years of life: an overview of systematic reviews. Neurosci Biobehav Rev. (2020) 116:183–201. doi: 10.1016/j.neubiorev.2020.06.027

PubMed Abstract | Crossref Full Text | Google Scholar

61. Guest G, Namey E, McKenna K. How many focus groups are enough? Building an evidence base for nonprobability sample sizes. Field Methods. (2017) 29:3–22. doi: 10.1177/1525822X16639015

Crossref Full Text | Google Scholar

62. Shang Z. Use of Delphi in health sciences research: a narrative review. Medicine. (2023) 102:e32829. doi: 10.1097/MD.0000000000032829

PubMed Abstract | Crossref Full Text | Google Scholar