Study Protocol for the COVID-19 Pandemic Adjustment Survey (CPAS): A Longitudinal Study of Australian Parents of a Child 0–18 Years

Background: The COVID-19 pandemic presents significant risks to the mental health and wellbeing of Australian families. Employment and economic uncertainty, chronic stress, anxiety, and social isolation are likely to have negative impacts on parent mental health, couple and family relationships, as well as child health and development.

Objective: This study aims to: (1) provide timely information on the mental health impacts of the emerging COVID-19 crisis in a close to representative sample of Australian parents and children (0–18 years), (2) identify adults and families most at risk of poor mental health outcomes, and (3) identify factors to target through clinical and public health intervention to reduce risk. Specifically, this study will investigate the extent to which the COVID-19 pandemic is associated with increased risk for parents’ mental health, lower well-being, loneliness, and alcohol use; parent-parent and parent-child relationships (both verbal and physical); and child and adolescent mental health problems.

Methods: The study aims to recruit a close to representative sample of at least 2,000 adults aged 18 years and over living in Australia who are parents of a child 0–4 years (early childhood, N = 400), 5–12 years (primary school N = 800), and 13–18 years (secondary school, N = 800). The design will be a longitudinal cohort study using an online recruitment methodology. Participants will be invited to complete an online baseline self-report survey (20 min) followed by a series of shorter online surveys (10 min) scheduled every 2 weeks for the duration of the COVID-19 pandemic (i.e., estimated to be 14 surveys over 6 months).

Results: The study will employ post stratification weights to address differences between the final sample and the national population in geographic communities across Australia. Associations will be analyzed using multilevel modeling with time-variant and time-invariant predictors of change in trajectory over the testing period.

Conclusions: This study will provide timely information on the mental health impacts of the COVID-19 crisis on parents and children in Australia; identify communities, parents, families, and children most at risk of poor outcomes; and identify potential factors to address in clinical and public health interventions to reduce risk.

Introduction

The World Health Organization declared COVID-19 a pandemic on the 11^th of March 2020. Consistent with government responses around the world, the Australian federal and state governments introduced an increasingly strict regime of social distancing/isolation measures to slow the rate of infection (1). These measures may present significant risks to the population, over and above the health threat associated with COVID-19 (2, 3). Findings from a cross-sectional study of 2,077 participants recruited in 22 countries in late March and early April 2020 indicated that adult mental health symptoms at that time were elevated compared to historical norms, with participants’ concern about the COVID-19 pandemic and loss of employment associated with higher levels of mental health problems (4). It is as yet unknown what the full impact of the pandemic will be on Australian families.

The COVID-19 pandemic represents an unprecedented confluence of risk in Australia and globally in this century, including: (1) a high level of uncertainty in regard to the parameters, time frames, and outcomes of the pandemic; (2) high rates of unemployment or underemployment, and housing and economic uncertainty; (3) threat to, or reduction of protective factors, such as social and community connection, physical activity, access to greenspace, and other co-curricular activities; and restricted access to clinical, community, family, and other supports and services; (4) increased pressure on parents to supervise and/or home-school children while juggling working from home; and (5) risk associated with being ‘locked in’ with family members in close quarters. It is unknown what effect the combination of these risks may have on the population of parents. However, each of these factors have an evidence-base demonstrating potential risks to adult and child mental health and wellbeing (5–16). There is evidence showing increased risks of mental health problems, drug and alcohol use, and family violence during and after crisis events and disasters (5–7, 17). Job loss, employment uncertainty, and difficulties in juggling work and family roles are associated with increases in parent mental health problems, couple conflict, and child mental health problems (8–16). Finally, there is evidence that quarantine is associated with a range of negative psychological outcomes including post-traumatic stress symptoms, confusion, and anger (3).

It is important to understand the experiences and consequences of the COVID-19 pandemic for all Australian families in order to plan for appropriate intervention and support, both during and after the pandemic period. However, the pandemic is likely to have a disproportionate effect on vulnerable parents and families. There is an urgent need to understand the impact for families with pre-existing risk factors to ensure that any public health interventions are appropriately tailored to these subgroups (2). Mental health problems are highly prevalent, affecting approximately one in five adults in Australia (18). It will be important to understand how adults with a pre-existing mental health problem or other personal vulnerabilities, such as difficulties in managing relationships and emotions (i.e., attachment insecurity and difficulties regulating emotions), respond to the COVID-19 pandemic. In addition, approximately one in seven children and adolescents experience a mental health or neurodevelopmental disorder, such as attention-deficit/hyperactivity disorder or autism spectrum disorder, equating to about 560,000 young people in Australia (19). In Australia and other nations, child mental health problems are clustered in places of disadvantage (20). To88 date, there is limited evidence as to how place‐based epidemic management affects disadvantaged communities. This study represents an important opportunity to understand how Australian communities and families affected by such conditions adjust to a global pandemic. Further, adults with chronic physical health conditions (such as diabetes, cardiovascular disease, and autoimmune conditions) are also at increased risk of negative outcomes via the potential for (1) more serious illness outcomes (21), (2) exacerbation of their health condition(s) caused by psychosocial stress and depression (22–24); and increased risk of infection in context of immune system impairment (25) or immunosuppressive treatments (26).

This study will investigate the impact of COVID-19 on the health and wellbeing of parents, children, and families. Specifically, the study will examine:

1. The extent to which the developing COVID-19 pandemic over time is increasing risk for:

a. Parent mental health problems, poor wellbeing, loneliness, and alcohol use;

b. Parent-parent (verbal and physical conflict) and parent-child relationship problems;

c. Child and adolescent mental health problems.

2. Whether some families and communities have a higher risk of experiencing these problems over time compared to other families, including:

a. Families with a member with a pre-existing mental health problems, attachment insecurity, and/or recent stressful life events;

b. Families living with or supporting those with a physical health condition or disability;

c. Families experiencing financial strain, crisis-associated job loss, and/or on low incomes or government benefits.

3. Whether there are modifiable factors that moderate families’ experience of risk over time, that could be targeted to strengthen families during and after the crisis, including:

a. Individual: promoting emotion-regulation, sleep quality, physical activity, and healthy screen-use;

b. Couple: promoting supportive relationships and constructive management of conflict; Familial: promoting nurturant parenting and positive familial communication.

Methods

Design

This is a longitudinal cohort study of Australian parents of a child aged 0–18 years. The study comprises two sets of online surveys scheduled on a regular basis for the duration of the COVID-19 pandemic. The surveys include:

1. A repeated baseline survey (20 min) scheduled at baseline and at 3-month intervals and

2. A brief longitudinal survey (10 min) scheduled every 2 weeks.

The time frame of the study will extend across the duration of the social distancing measures implemented by the Australian federal and state governments to manage the COVID-19 pandemic in Australia. The federal government released a statement estimating that the likely time frame will be a period of six months from March 2020 to September 2020 (27). The regularity and time-frame of the longitudinal surveys will be reviewed every 2–3 months to ensure that benefits of regular follow-up are weighed against potential for participant burden and fatigue.

Eligibility

Participants will be eligible to participate if they are an Australian resident, 18 years or over, and are a parent of a child aged 0–18 years. Survey information and advertisements will be written in English, so it is expected that people with adequate English fluency will complete the survey.

Recruitment

Parents will be recruited via paid and unpaid social media advertisements. A range of methods will be used to target specific groups to increase the representativeness of the sample (e.g., targeting via postcodes and demographic factors). The style and wording of advertisements is important in determining recruitment success. Consistent with prior research, this study will employ advertisements that: (1) refer to research; include the Deakin University affiliation, refer to the incentive (as detailed below), and are written in engaging yet plain language (28).

Participants will primarily be recruited via the social media platform, Facebook, given demonstrated success in recruiting hard-to-reach populations on this platform (29, 30). A project ‘business’ Facebook page will be established to maintain contact with participants, affiliate organizations, and the wider public. The page will be monitored regularly by project staff and any content/comments deemed inappropriate or offensive will be promptly removed. Both paid and unpaid recruitment strategies on Facebook will be used in the current study. Unpaid strategies will include making contact with established interest groups, parenting groups, and organizations on Facebook via the project Facebook page and/or Deakin University email (i.e., where email addresses are provided), and requesting that these sites endorse our project by posting the project advertisement so that it is visible to their group members. Paid strategies will involve using Facebook’s systems to target recruitment to specific sub-populations via demographic variables (e.g., parents of children 0–18 years; fathers, remote/regional postcodes, and parents speaking a language other than English), posting paid advertisements on all available platforms, including Facebook and Instagram. We will also use other social media platforms (e.g., Reddit, Twitter, Instagram, and WhatsApp) following the same protocols to post both paid and unpaid advertisements as per our current use of Facebook.

Expected Sample Size

The study aims to recruit a minimum of 2,000 parents of a child 0–4 years (early childhood, N = 400), 5–12 years (primary, N = 800), or 13–18 years (secondary, N = 800).

Procedures

Baseline Survey

The advertisements used for recruitment will contain a web hyperlink which will direct participants to an initial Qualtrics survey website. The landing page for the survey will contain a brief description of the purpose of the research. On the next page, participants will be asked two eligibility questions, checking that they are a parent of a child 0–18 years and that they currently live in Australia. If participants are not eligible to participate, they will be directed out of the survey with an explanation of the eligibility criteria. Eligible participants will then be presented with a Plain Language Statement and Online Consent form available for download as a PDF document. On this page, participants will be asked to check a box that confirms that they have read the Plain Language Statement, which they understand its contents, and consent to participate in the study. Participants will then be asked to provide contact information with details of their first name, phone number, and email address. A brief explanation will be provided that this information will assist the research team in contacting the participants for the follow-up survey, sending reminders, and contacting winners of the monthly prize draw. On completion of the baseline survey, participants will be automatically allocated a unique ID number, which will be embedded in their subsequent surveys to identify them and link their data.

An invitation (and web link) will be included at the end of the Qualtrics baseline survey inviting the potential participant to ‘friend’ the CPAS page on Facebook. This is intended as a strategy to maximize participant retention rates and promote participant connectedness to the study. Facebook allows a stable means of communication where participants can be contacted for future time points of the study regardless of changes in contact details. This request would be a means of keeping the study in the minds of participants as study updates and news would appear on the participant’s own Facebook ‘News Feed.’ Only one email request would be sent with no follow-ups, even if the request is declined or ignored. No changes would be made to the previously approved Facebook privacy settings.

Fortnightly Longitudinal Survey

Participants will be re-contacted every 2 weeks after completion of the baseline survey via an automated email invitation. Participants will be recruited on a rolling basis to maximize reach and sample size. Regardless of whether a participant responds in a given week, participants will remain on the active list and will continue to receive survey invites and reminders. All emails to participants will contain an opt-out link with two options: to opt-out from the survey or to opt-out of the study entirely.

Participant Reminders

Participants who open the baseline survey, consent to participate, and who have provided their contact details but did not complete the full version of the online baseline survey will be sent an email reminder about completing the survey 24 hours later. Participants will be sent an email reminder 24 hours after each fortnightly longitudinal survey is sent. If participants have not completed a survey or made contact with the study team over a period of three consecutive surveys, the team will use a range of methods to attempt to re-engage participants in the study. This may include sending an additional follow-up email, sending an SMS reminder and/or calling the participant on their mobile phone number, or contacting the participant via Facebook (refer to section Facebook Tracing, below). We will limit all contacts to a maximum of 1 direct contact (i.e., involving communication from the participant) within a week, via email, SMS, or voicemail message. In order to understand reasons for participant drop out, we will ask participants two brief questions when making contact via phone, a question asking about the participants’ reasons for not completing follow-up surveys (“Day to day life is very busy”; “Want to complete but forget or never get around to it”; “Change in your circumstances—decrease in job hours/loss job; increase in job hours, gained employment, started studying, stopped studying, change in caregiving responsibilities”; “Lost interest in the survey”; “Other”) and a question assessing participants’ level of functioning (“Compared to when you first completed the survey in April – this was around the beginning to middle of the most restrictive period in Australia – would you say you are going about the same, better, or worse right now? “).

Facebook Tracing

For participants whom we are not able to contact (no email response or a return to sender email; and no evidence that we reached the correct participant’s phone—i.e., no identifying voicemail message or the number was disconnected), we propose attempting contact via Facebook. Facebook searches will be conducted to generate evidence from which to identify participants. Only publicly available information will be viewed based on information publicly visible on users’ profiles, “Liked Pages,” “Groups,” or “Check-ins” to verify the location of the participant, compared to their last known residential address. If the study team has strong evidence to link a Facebook user with the identity of a previous participant, participants will be contacted through Facebook Messenger.

Remuneration for Participation

Research has shown benefits associated with the use of incentives in social media recruitment via Facebook (28, 31). Participants will be entered into a prize draw for 1 of 10 AU $50 online gift vouchers if they have completed at least one survey for every month of the survey. We have estimated vouchers based on a study of six months’ duration (6 prize draws, 10 vouchers offered at each draw, a total of 60 vouchers).

Consent

Consent will be obtained at baseline. Participants will also complete separate (optional) consent to be contacted for future research participation. Participants will be informed that they are under no obligation to participate and advised that they are free to withdraw at any time without consequences.

Data Management

Study data will be managed using Qualtrics, hosted at Deakin University (32). Data will be downloaded from the Qualtrics server on a weekly basis and stored on servers maintained by Deakin University.

Measures

Table 1 provides an outline of study measures. Where possible, measures will be harmonized with the Longitudinal Study of Australian Children (LSAC), a population representative sample of Australian families. LSAC includes two cohorts of children and families recruited in 2005 and followed biennially on an ongoing basis (altogether, N = 10,000 at baseline) (52).

TABLE 1

Table 1 Overview of measures included in the COVID-19 Pandemic Adjustment Survey (CPAS).

Demographic and COVID-19 Variables

Identifiable Information (First Baseline Survey Only)

First name, email address, mobile number, and postcode.

Demographics (Baseline Survey Only)

About adult: Age, gender, country of birth, Aboriginal and Torres Strait Islander status, language other than English spoken at home, education, relationship status, whether living with partner, and number of children in the household. Demographics prior to COVID-19: employment, study, household income, source of income, and shortage of money. Items about housing: type of dwelling, owned or rented, number of bedroom, number of people living in house, satisfaction with quality of housing, and access to private outdoor space at current home.

About partner: Gender; partner’s relationship to child, employment, and education.

About child: Age, gender, and education setting.

COVID-19 Factors (Baseline and Fortnightly Survey)

Items adapted from the CoRonavIruS Health Impact Survey (CRISIS) V0.1 (33).

Household: COVID-19 diagnosis, test result, or symptoms.

About adult: Participant or family members affected by COVID-19 (fallen ill, hospitalized, self-quarantine, and passed away), financial problems or housing and food insecurity related to COVID-19, working from home, frequency and type of contact with work colleagues, impact on family life, food/medical shortages, use of media, feelings and attitudes about COVID-19, impact of COVID-19 on family life (short-answer question, “How has COVID-19 affected your family life?”), coping strategies (short-answer question, “What strategies are helping you to stay calm in the current situation?”), frequency of use of news sources (newspapers, television, social media, radio, rated on 6-point scale from ‘not at all’ to ‘multiple times per day’), appraisals of COVID-19 as a serious health risk, and whether likely to catch COVID-19 (both rated on a 7-point scale from ‘strongly disagree’ to ‘strongly agree’).

About child: Presence of a daily routine at home, time outside home (going to stores, parks, etc.), child’s relationship quality with their friends (rated on 5-point scale from ‘a lot worse’ to ‘a lot better’). Whether school classes are running on campus, school attendance on campus or online. For children home-schooling: whether child home with parent while they work, child’s internet/computer access at home, whether they have assignments to complete from home, amount of school work completed each day, and parents rating of how well they are managing child’s home learning (4-point scale from ‘very poorly’ to ‘very well’).

Adult Outcomes

Wellbeing (Baseline and Fortnightly Survey)

Personal Wellbeing Index (34) (seven items). seven domains: standard of living, personal health, achieving in life, personal relationships, personal safety, community-connectedness, and future security. Example item: “How satisfied are you with … your standard of living?” Rated on a 11-point scale from ‘no satisfaction at all’ to ‘completely satisfied’.

Personality (Baseline Survey Only)

Introvert/extrovert (one item, designed for the current study) “Do you consider yourself an introvert?” rated on a 7-point scale from ‘introvert’ to ‘extrovert.’

Mental Health (Baseline and Fortnightly Survey)

Depression and Anxiety Scale (DASS) 21-item version (35). Three subscales: depression, stress, and anxiety (seven items each). Example item: “I found it hard to wind down.” Rated on a 4-point scale from ‘did not apply to me at all’ to ‘applied to me very much, or most of the time.’

Mental or Physical Health Diagnosis (Baseline and One Fortnightly Survey Only)

One item (baseline): “Have you ever had a professional diagnose or treat you for a mental or physical health condition? What was the condition?” One item (presented at one fortnightly survey): Have you ever been treated or diagnosed for any of the following chronic physical conditions by a health professional? Ulcerative Colitis, Crohn’s disease, endometriosis, cardiovascular disease (e.g., coronary heart disease, stroke, and heart failure); hypertension (clinically high blood pressure), type 1 diabetes, type 2 diabetes, and other.

Emotion Regulation (Baseline Survey Only)

Difficulties in Emotion Regulation Scale-16 Item Version (36) (16 items). Five subscales: strategies, non-acceptance, impulse control, goals, and clarity. Example item: “I have difficulty making sense out of my feelings.” Rated on a 5-point scale from ‘almost never’ to ‘almost always.’

Positive Effect (Baseline Survey Only)

Positive Affect Subscale from the positive and negative effect schedule short form (37) (five items). Example item: “Thinking about yourself in the past 4 weeks, about how often did you feel … alert?” Rated on a 5-point scale from ‘very slightly or not at all’ to ‘extremely.’

Physical Health (Baseline and Fortnightly Survey)

Physical activity (one item) from the Longitudinal Study of Australian Children (LSAC). Item: “About how many days each week do you do at least 30 min of moderate or vigorous physical activity (like walking briskly, riding a bike, gardening, tennis, swimming, running, etc)?” Rated from 1 to 7 days.

Sleep (one item) from LSAC. Item: “During the past month, how would you rate your sleep quality overall?” Rated on a 4-point scale from ‘very good’ to ‘very bad.’

Substance Use (Baseline and Fortnightly Survey)

Alcohol consumption (one item) from LSAC. Item: “How often do you have a drink containing alcohol?” Rated on a 7-point scale from ‘never’ to ‘every day.’

Cigarette smoking (one item) from LSAC. Item: “How often do you smoke cigarettes?” Rated on a 3-point scale from ‘do not smoke at all’ to ‘at least once a day.’

Adult Attachment (Baseline Survey Only)

Experiences in Close Relationships Scale–Relationship Structures (ECR-RS) (38) (nine items). Two subscales: attachment anxiety and attachment avoidance. Example item: “It helps to turn to people in times of need.” Rated on a 7-point scale from ‘strongly disagree’ to ‘strongly agree.’

Resilience (Baseline Survey Only)

Brief resilience scale (BRS) (39) (six items). Example item: “I tend to bounce back quickly after hard times.” Rated on a 5-point scale from ‘strongly disagree’ to ‘strongly agree.’

Loneliness (Baseline and Fortnightly Survey)

UCLA loneliness scale (40) (six items). Example item: “I lack companionship.” Rated on a 4-point scale from ‘never’ to ‘always.’

Utopian Thinking (Baseline and Fortnightly Survey)

Utopian thinking (one item). Item: “I often think about what an ideal society might look like.” Rated on a 7-point scale from ‘strongly disagree’ to ‘strongly agree.’

Family, Couple, and Parenting Outcomes

Family Expressiveness (baseline and fortnightly survey)

Adapted short-form of the Self-Expressiveness in the Family Questionnaire (41) (11 items were selected according to a consensus of three independent expert ratings evaluating item relevance in relation to the COVID-19 pandemic). Two subscales: positive and negative expressiveness. Example item: “Showing contempt for another’s actions.” Rated on a 9-point scale from ‘not at all frequently in my family’ to ‘very frequently in my family.’

Stressful Life Events (Baseline Survey Only)

Stressful life events over the past 12 months (42) (eight items). Example items: “In the last year, have any of the following happened to you (or your partner)? You became pregnant or had a baby; You moved house.” Items rated Yes/No.

Couple Conflict (Baseline and Fortnightly Survey)

Argumentative Relationship Scale used in LSAC (43) (five items). Example item: “How often do you and your partner disagree about basic household issues?” Rated on a 5-point scale from ‘never’ to ‘always.’

Relationship Quality (Baseline and Fortnightly Survey)

Perceived Relationships Quality Component (PRQC) Questionnaire (44) (six items measured in baseline survey and one item in fortnightly survey). Example item (and item in fortnightly survey): “How satisfied are you with your relationship?” Rated on a 7-point scale from ‘not at all’ to ‘extremely.’.

Social Support (Baseline and Fortnightly Survey)

Social support (one item) from LSAC. Item: “Overall how do you feel about the amount of support or help you get from family or friends living elsewhere?” Rated on a 4-point scale from ‘I get enough help’ to ‘I don’t get any help at all’ and ‘I don’t need any help.’

Social Provisions Scale (45) (one item selected). Item: “When I am feeling stressed about a new or unknown situation, I can rely on my partner to comfort me.” Rated on a 7-point scale from ‘strongly disagree’ to ‘strongly agree.’

Secure Base Characteristics Scale (46) (one item selected). Item: “My partner encourages me to draw on my skills and abilities to deal with challenges”. Rated on a 7-point scale from ‘strongly disagree’ to ‘strongly agree.’

Neighborhood Disadvantage (Baseline Survey Only)

Postcodes used to derive neighborhood disadvantage according to the Socio-Economic Indexes for Areas (SEIFA) advantage and disadvantage (47).

Parenting (Baseline and Fortnightly Survey)

Interpersonal Mindfulness in Parenting (IEM-P) (48) (three items). Example item: “When I’m upset with my child, I notice how I am feeling before I take action.” Rated on a 5-point scale from ‘almost never’ to ‘almost always.’

Emotion-Focused Parenting (three items). Example item: “When my child experiences strong emotions (sad, angry, scared), I connect with them and provide comfort and support.” Rated on a 5-point scale from ‘almost never’ to ‘almost always.’

Parenting Irritability (five items) from LSAC. Example item: “In the past 6 months, how often would you say … I have raised my voice with or shouted at this child.” Rated on a 10-point scale from ‘not at all’ to ‘all the time.’

Child Outcomes

Physical Health (Baseline and Fortnightly Survey)

Global child health from LSAC. Item: “In general, is your child’s current health…” Rated on a 5-point scale from ‘excellent’ to ‘poor.’

Child Diagnosis (Baseline Survey Only)

Professional diagnosis or treatment (one item). Item: “Has your child ever been diagnosed or treated for any of the following by a health professional?” Response options (rated Yes/No): ADHD; autism, Asperger’s, other autism spectrum; oppositional defiant or conduct disorder; speech or language disorder; head injury, epilepsy, seizure(s), febrile convulsions; disability; and other (free text).

Mental Health (Baseline and Fortnightly Survey)

The Short Mood and Feelings Questionnaire (SMFQ) (50) (13 items). One scale: Depression. Example item: “Your child felt miserable or unhappy.” Rated on a 3-point scale from ‘not true’ to ‘true.’

Modified Brief Spence Children’s Anxiety Scale (51) (four selected items). One scale: Anxiety. Example item: “My child worries about things.” Rated on a 4-point scale from ‘never’ to ‘always’.

Swanson, Nolan, and Pelham –IV Questionnaire (SNAP-IV) (53) Parent Rating Scale, Opposition/Defiance Subscale (four selected items). Example item: “Often actively defies or refuses adult requests or rules” Rated on a 4-point scale from ‘not at all’ to ‘very much.’

Loneliness (one item) adapted from the CoRonavIruS Health Impact Survey (CRISIS) (33). Item: “During the past 2 weeks, how lonely has your child been?”

Irritability (one item) adapted from the CoRonavIruS Health Impact Survey (CRISIS) (33) Item: “During the past 2 weeks, how irritable or easily angered has your child been?”

Child mood (eight items) (fortnightly survey only). Item: “Please indicate below how your child is feeling: happy, sad, content, bored, excited, anxious, alert, tired.” Rated on a 11-point scale from ‘not at all’ to ‘very much.’

Physical Health (Baseline and Fortnightly Survey)

Physical activity (one item) adapted from LSAC. Item: “About how many days each week does your child do at least 30 min of moderate or vigorous physical activity (like walking briskly, riding a bike, swimming, running, etc)?” Rated from 1 to 7 days.

Sleep pattern (one item) from LSAC. Item: “How much is your child’s sleeping pattern or habits a problem for you?” Rated on a 4-point scale from ‘not a problem at all’ to ‘a large problem.’

Sleep regularity (one item) from LSAC. Item: “Does the study child go to bed at regular times?” Rated on a 5-point scale from ‘never’ to ‘always.’

Screen-Time (Baseline and Fortnightly Survey)

Screen time (two items) adapted from LSAC. “About how many hours on a typical weekday does your child watch TV or videos at home not for educational purposes? (e.g., YouTube, Instagram, TikTok, streaming services such as Netflix).” Rated on a sliding scale from 1 to 24 hours.

Intervention Willingness (Baseline and Fortnightly Survey)

Online intervention (three items). Items: “The COVID-19 pandemic and the associated measures to increase social distancing have caused many people to feel stressed and worried. How likely would you be to use an online or smartphone intervention for the following reasons: Mental health support for yourself, mental health support for your child, and parenting support.” Rated on a 5-point scale from ‘not at all’ to ‘extremely likely.’

Mental health intervention (two items). Items: “Should you experience a mental health difficulty in the future, how likely are you to use a … Self-guided internet- or smartphone-app based treatment program? Therapist-assisted internet- or smartphone-app based treatment program?” Rated on a 5-point scale from ‘extremely likely’ to ‘extremely unlikely.’

Analysis Approach

Quantitative Data

Data Preparation

Data will be prepared in Stata version 16 (54). Missing data will be addressed using either full information maximum likelihood estimation or multiple imputation by chained equations, depending on the analysis. Both methods rely on the untestable assumption that missingness is ignorable. Sensitivity analyses (e.g., in the form of selection models or pattern mixture models) will be conducted to evaluate impact of violation of this assumption on modeled results (55).

Data Analysis

Analyses will be conducted in Stata version 16, or where relevant, in Mplus version 8 (56). The planned approach for testing Aims 1–3 is outlined below. Where relevant, all associations will be investigated in unadjusted analyses, and then in adjusted analyses, the latter controlling for the baseline effects of factors known to be associated with adult socio-emotional adjustment (gender, age, health, family demographic factors). Decisions about the inclusion of specific covariates in each model will be made using directed acyclic graphs (DAGs) (57). Associations will be analyzed using multilevel modeling in either a latent variable or mixed effects framework to: (i) account for the clustered nature of time points within individuals while (ii) also modeling between-individual differences in rate of change over time. In these models, we will regress an outcome (e.g., mental health) on to ‘time,’ any moderator variables of interest, and background covariates. We anticipate ‘time’ being treated as a continuous predictor in all models (with the baseline time-point coded as 0 and then numbered consecutively), but we will also consider treating ‘time’ as a categorical variable with discrete categories of time demarcated by important events that may occur during the pandemic window. The influence of potential moderators on the relationship between these associations will be investigated by including interaction terms (e.g., moderator x time).

Population Weighting

We will use post-stratification weights, generated through a raking approach (58) to compensate for differences between the final sample and the national population across geographic community clusters, parent age, gender, educational attainment, and country of birth (Australia/New Zealand versus other). We will ensure that strata sample sizes are large enough to not unduly influence the overall results.

Power Calculation

Power is demonstrated for our key analyses involving within person relationships during the longitudinal study. Given the clustered nature of the study of time points nested within participants, the Effective Sample Size (ESS) for the study is given by ESS = nm/(1+(m-1)ρ) (59), where n = number of participants aiming to be recruited, m = number of data points per cluster, and ρ = the within cluster correlation. Based on a 6-month window of data collection and fortnightly assessments (estimated 14 assessments), the smallest sample of 400 participants (parents of a child 0–4 years) has an ESS = 746 assuming a conservative within cluster correlation of ρ = .5. Using Monte Carlo simulation (10,000 draws) in Mplus 8, an ESS = 746 would provide 98.2% power to detect a true effect of interest (e.g., time related change in parent mental health problems) of even small magnitude (β = .14, representing just ~2% extra variance accounted for in the outcome above a base level of ~10% by other variables in the model; at α = .05, two-tailed). Thus, the study is well powered for even small true effects of interest. Note that even if participants only complete two of the assessments (ESS = 533), this would still provide 92.7% power to detect the above-mentioned effect for our smallest age stratified group (parents of a child 0–4 years). Additionally, for any between person relationships (e.g., differences between families), even the minimum sample size of 400 would provide 84.8% power to detect effects of the above-mentioned size. Thus, the study is well powered.

Qualitative Analysis

Qualitative data will be analyzed using thematic analyses to determine the common themes that arise from the participant answers to the two short-answer questions posed regarding parent’s coping strategies and impact of COVID-19 on family life (60), Thematic analysis is a method of analyzing qualitative data that is focused on identifying, examining, and recording major patterns or themes in the data.

Research Study Administration

Ethics Statement

The current study has been approved by the Deakin University Human Ethics Advisory Group (Project number: HEAG-H 52_2020).

Ethical Issues

We use brief screening measures to assess adult and child functioning. These measures are routinely used in population-level, large scale, longitudinal surveys, but are not designed to collect clinical information, thus the scales cannot be used to diagnose physical or mental health conditions. Participants will be provided with a Plain Language Statement that outlines the key constructs assessed in the study, reminds participants they can withdraw at any time, and provides information on where participants can seek help if any of the questions do cause them discomfort or distress. It will be possible for participants to skip any of the questions/items in the survey, and to facilitate this, none of the special case assessment items on the online survey form will be coded as a ‘forced’ answer. In the event that a participant expresses significant risk to themselves or others (e.g., suicidal ideation) in free-text comments, such as in the qualitative data, the lead investigator (EW), a registered clinical psychologist, will contact the participant to offer information on support services and referral options.

Dissemination of Outcomes

Results will be disseminated in peer reviewed journals, via the media, online, and at academic conferences. A plain language summary of results from the study will be made available to participants upon request. Participants are advised of the process to request a plain language summary of the results in the Plain Language Statement.

Future Research and Data Sharing

Participants are invited to provide optional consent to be contacted for future research, such as further follow-up beyond 6 months. This process would involve a new ethics application. Participants will also be invited to consent to their de-identified information being stored on public repositories for the purposes of data sharing. If consent is provided, participant data will be stored securely. All information about the study (including publication preprints, data access, and analytic code) will be available at https://osf.io/78g5t/.

Project Closure

At the conclusion of the study, recruitment materials, the project landing page, and online survey materials will be deactivated or removed. All data will remain securely stored on Deakin University servers. Information collected in this research project involves children who are under 18 years old, thus data will be kept until the youngest child turns 33 years of age.

Recruitment Progress

The study was launched on the 8^th of April, 2020. As at the 26^th of April, 2,375 eligible participants had completed the baseline survey.

Discussion

The COVID-19 pandemic presents significant risks to the mental health and wellbeing of Australian families. This project seeks to investigate the manifold impacts of the pandemic, including the impacts for families in regards to job loss, employment conditions, home-schooling, and unprecedented lifestyle changes associated with social distancing measures. Chronic stress and social isolation have potential risks for adult mental health, couple and family relationships, and children’s health and development (8–13). The novel contribution of the current study will be the repeated measures design, which will facilitate the tracking of changes in mental health over time in relation to the developing situation around the world.

This project is designed to provide timely information to government and communities on the mental health effects of the emerging COVID-19 crisis on Australian parents and children. This information can then be used to inform the development of assessment and screening tools to identify those parents, families, and children who may be most at risk. Furthermore, the findings of this research can guide health practitioners and policy makers regarding the factors that should be the focus of clinical and public health interventions to reduce risks of adult mental health, family breakdown, and child maladjustment when faced with such health crises in the future. Finally, the findings from this study can be used to develop practical information and advice for families in how to deal with such crises and create positive family environments to buffer against mental health problems, family dysfunction, and child maladjustment.

Ethics Statement

The studies involving human participants were reviewed and approved by Deakin University Human Ethics Advisory Group (project number: HEAG-H 52_2020). The patients/participants provided their written informed consent to participate in this study.

Author Contributions

All authors contributed to the conceptualization of the study, drafting of the study protocol, and selection of survey items/measures. EW drafted the manuscript, and together with TB, developed all study materials, including the online Qualtrics survey and the study adverts. EW, GK, JAM, LO, GY, CG, AM-W, ES, SE, MF-T, RC, DH, GM, JF, ST, AW, JT, TB, JL, PE, MS, and CO wrote or revised sections of the manuscript. All authors contributed to the article and approved the submitted version.

Funding

The current study was funded through the Centre for Social and Early Emotional Development, a Strategic Research Centre at Deakin University.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank all participating parents and their families for their involvement in the study.

References

1. Australian Government, Department of Health, Coronavirus (COVID-19) health alert. In: Department of Health AG. Available online at: https://www.health.gov.au/news/health-alerts/novel-coronavirus-2019-ncov-health-alert (Accessed 26 August 2020).

Google Scholar

2. Holmes EA, O’Connor RC, Perry VH, Tracey I, Wessely S, Arseneault L, et al. Multidisciplinary research priorities for the COVID-19 pandemic: a call for action for mental health science. Lancet Psychiatry (2020) 7(6):547–60. doi: 10.1016/S2215-0366(20)30168-1

PubMed Abstract | CrossRef Full Text | Google Scholar

3. Brooks SK, Webster RK, Smith LE, Woodland L, Wessely S, Greenberg N, et al. The psychological impact of quarantine and how to reduce it: rapid review of the evidence. Lancet (2020) 395:912–20. doi: 10.1016/S0140-6736(20)30460-8

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Nelson BW, Pettitt AK, Flannery J, Allen NB. Psychological and Epidemiological Predictors of COVID-19 Concern and Health-Related Behaviors. PsyArXiv (2020). doi: 10.31234/osf.io/jftze

CrossRef Full Text | Google Scholar

5. Reifels L, Mills K, Dückers MLA, O’Donnell ML. Psychiatric epidemiology and disaster exposure in Australia. Epidemiol Psychiatr Sci (2019) 28(3):310–20. doi: 10.1017/S2045796017000531

PubMed Abstract | CrossRef Full Text | Google Scholar

6. Parkinson D, Zara C. The hidden disaster: Domestic violence in the aftermath of natural disaster. Aust J Emergency Management (2013) 28(2):28.

Google Scholar

7. Molyneaux R, Gibbs L, Bryant RA, Humphreys C, Hegarty K, Kellett C, et al. Interpersonal violence and mental health outcomes following disaster. BJ Psych Open (2020) 6(1):e1. doi: 10.1192/bjo.2019.82

CrossRef Full Text | Google Scholar

8. Lupien SJ, Juster R-P, Raymond C, Marin M-F. The effects of chronic stress on the human brain: From neurotoxicity, to vulnerability, to opportunity. Front Neuroendocrinol (2018) 49:91–105. doi: 10.1016/j.yfrne.2018.02.001

PubMed Abstract | CrossRef Full Text | Google Scholar

9. Bakusic J, Schaufeli W, Claes S, Godderis L. Stress, burnout and depression: A systematic review on DNA methylation mechanisms. J Psychosom Res (2017) 92:34–44. doi: 10.1016/j.jpsychores.2016.11.005

PubMed Abstract | CrossRef Full Text | Google Scholar

10. Wang J, Lloyd-Evans B, Giacco D, Forsyth R, Nebo C, Mann F, et al. Social isolation in mental health: a conceptual and methodological review. Soc Psychiatry Psychiatr Epidemiol (2017) 52(12):1451–61. doi: 10.1007/s00127-017-1446-1

PubMed Abstract | CrossRef Full Text | Google Scholar

11. Leigh-Hunt N, Bagguley D, Bash K, Turner V, Turnbull S, Valtorta N, et al. An overview of systematic reviews on the public health consequences of social isolation and loneliness. Public Health (2017) 152:157–71. doi: 10.1016/j.puhe.2017.07.035

PubMed Abstract | CrossRef Full Text | Google Scholar

12. Marin M-F, Lord C, Andrews J, Juster R-P, Sindi S, Arsenault-Lapierre G, et al. Chronic stress, cognitive functioning and mental health. Neurobiol Learn Memory (2011) 96(4):583–95. doi: 10.1016/j.nlm.2011.02.016

CrossRef Full Text | Google Scholar

13. Olesen SC, Butterworth P, Leach LS, Kelaher M, Pirkis J. Mental health affects future employment as job loss affects mental health: findings from a longitudinal population study. BMC Psychiatry (2013) 13(1):144. doi: 10.1186/1471-244X-13-144

PubMed Abstract | CrossRef Full Text | Google Scholar

14. Westrupp EM, Strazdins L, Martin A, Cooklin AR, Zubrick S, Nicholson JM. Maternal work-family conflict and psychological distress: Reciprocal relationships over 8 years. J Marriage Family (2015) 78(1):107–26. doi: 10.1111/jomf.12262

CrossRef Full Text | Google Scholar

15. Vahedi A, Krug I, Westrupp EM. Crossover of parents’ work-family conflict to family functioning and child mental health. J Appl Dev Psychol (2019) 62:38–49. doi: 10.1016/j.appdev.2019.01.001

CrossRef Full Text | Google Scholar

16. Dinh H, Cooklin AR, Leach LS, Westrupp EM, Nicholson JM, Strazdins L. Parents’ transitions into and out of work-family conflict and children’s mental health: Longitudinal influence via family functioning. Soc Sci Med (2017) 194:42–50. doi: 10.1016/j.socscimed.2017.10.017

PubMed Abstract | CrossRef Full Text | Google Scholar

17. North CS, Ringwalt CL, Downs D, Derzon J, Galvin D. Postdisaster course of alcohol use disorders in systematically studied survivors of 10 disasters. Arch Gen Psychiatry (2011) 68(2):173–80. doi: 10.1001/archgenpsychiatry.2010.131

PubMed Abstract | CrossRef Full Text | Google Scholar

18. Australian Bureau of Statistics. National Survey of Mental Health and Wellbeing 2007: Summary of results. ABS. Canberra, Australia: Australian Bureau of Statistics (2008).

Google Scholar

19. Lawrence D, Johnson S, Hafekost J, Boterhoven de Haan K, Sawyer M, Ainley J, et al. Report on the second Australian child and adolescent survey of mental health and wellbeing. Department of Health: Australia (2015).

Google Scholar

20. Stirling K, Toumbourou JW, Rowland B. Community factors influencing child and adolescent depression: A systematic review and meta-analysis. Aust New Z J Psychiatry (2015) 49(10):869–86. doi: 10.1177/0004867415603129

CrossRef Full Text | Google Scholar

21. World Health Organisation. Coronavirus disease 2019 (COVID-19) Situation Report – 51. World Health Organization (2020). Available at: https://apps.who.int/iris/bitstream/handle/10665/331475/nCoVsitrep11Mar2020-eng.pdf.

Google Scholar

22. Bunker SJ, Colquhoun DM, Esler MD, Hickie IB, Hunt D, Jelinek VM, et al. “Stress” and coronary heart disease: psychosocial risk factors. Med J Aust (2003) 178(6):272–6. doi: 10.5694/j.1326-5377.2003.tb05193.x

PubMed Abstract | CrossRef Full Text | Google Scholar

23. Shedd OL, Sears SF, Harvill JL, Arshad A, Conti JB, Steinberg JS, et al. The World Trade Center attack: increased frequency of defibrillator shocks for ventricular arrhythmias in patients living remotely from New York City. J Am Coll Cardiol (2004) 44(6):1265–7. doi: 10.1016/j.jacc.2004.04.058

PubMed Abstract | CrossRef Full Text | Google Scholar

24. Kloner RA. Lessons learned about stress and the heart after major earthquakes. Am Heart J (2019) 215:20–6 doi: 10.1016/j.ahj.2019.05.017

PubMed Abstract | CrossRef Full Text | Google Scholar

25. Doran MF, Crowson CS, Pond GR, O’Fallon WM, Gabriel SE. Frequency of infection in patients with rheumatoid arthritis compared with controls: a population-based study. Arthritis Rheum (2002) 46(9):2287–93. doi: 10.1002/art.10524

PubMed Abstract | CrossRef Full Text | Google Scholar

26. Kirchgesner J, Lemaitre M, Carrat F, Zureik M, Carbonnel F, Dray-Spira R. Risk of Serious and Opportunistic Infections Associated With Treatment of Inflammatory Bowel Diseases. Gastroenterology (2018) 155(2):337–46.e10. doi: 10.1053/j.gastro.2018.04.012

PubMed Abstract | CrossRef Full Text | Google Scholar

27. Media Statement 22 Mar 2020 Prime Minister. Update on coronavirus measures [press release]. Department of the Prime Minister and Cabinet: Canberra, Australia (2020).

Google Scholar

28. Thornton L, Batterham PJ, Fassnacht DB, Kay-Lambkin F, Calear AL, Hunt S. Recruiting for health, medical or psychosocial research using Facebook: Systematic review. Internet Interv (2016) 4:72–81. doi: 10.1016/j.invent.2016.02.001

PubMed Abstract | CrossRef Full Text | Google Scholar

29. Batterham RW, Buchbinder R, Beauchamp A, Dodson S, Elsworth GR, Osborne RH. The OPtimising HEalth LIterAcy (Ophelia) process: study protocol for using health literacy profiling and community engagement to create and implement health reform. BMC Public Health (2014) 14(1):694. doi: 10.1186/1471-2458-14-694

PubMed Abstract | CrossRef Full Text | Google Scholar

30. Carlini BH, Safioti L, Rue TC, Miles L. Using Internet to recruit immigrants with language and culture barriers for tobacco and alcohol use screening: a study among Brazilians. J Immigrant Minority Health (2015) 17(2):553–60. doi: 10.1007/s10903-013-9934-1

CrossRef Full Text | Google Scholar

31. Topolovec-Vranic J, Natarajan K. The Use of Social Media in Recruitment for Medical Research Studies: A Scoping Review. J Med Internet Res (2016) 18(11):e286. doi: 10.2196/jmir.5698

PubMed Abstract | CrossRef Full Text | Google Scholar

32. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. A metadata-driven methodology and workflow process for providing translational research informatics support. J BioMed Inform (2009) 42(2):377–81. doi: 10.1016/j.jbi.2008.08.010

PubMed Abstract | CrossRef Full Text | Google Scholar

33. Merikangas K, Milham M, Stringaris A. The CoRonavIruS Health Impact Survey (CRISIS) V0.1 2020. Github (2020). Available at: https://github.com/nimh-mbdu/CRISIS.

Google Scholar

34. Cummins RA, Eckersley R, Pallant J, Van Vugt J, Misajon R. Developing a national index of subjective wellbeing: The Australian Unity Wellbeing Index. Soc Indic Res (2003) 64(2):159–90. doi: 10.1023/A:1024704320683

CrossRef Full Text | Google Scholar

35. Lovibond PF, Lovibond SH. The structure of negative emotional states: Comparison of the Depression Anxiety Stress Scales (DASS) with the Beck Depression and Anxiety Inventories. Behav Res Ther (1995) 33(3):335–43. doi: 10.1016/0005-7967(94)00075-U

PubMed Abstract | CrossRef Full Text | Google Scholar

36. Bjureberg J, Ljótsson B, Tull MT, Hedman E, Sahlin H, Lundh L-G, et al. Development and validation of a brief version of the Difficulties in Emotion Regulation Scale: the DERS-16. J Psychopathol Behav Assess (2016) 38(2):284–96. doi: 10.1007/s10862-015-9514-x

PubMed Abstract | CrossRef Full Text | Google Scholar

37. Thompson ER. Development and validation of an internationally reliable short-form of the positive and negative affect schedule (PANAS). J Cross-Cultural Psychol (2007) 38(2):227–42. doi: 10.1177/0022022106297301

CrossRef Full Text | Google Scholar

38. Fraley RC, Heffernan ME, Vicary AM, Brumbaugh CC. The experiences in close relationships—Relationship Structures Questionnaire: A method for assessing attachment orientations across relationships. psychol Assess (2011) 23(3):615. doi: 10.1037/a0022898

PubMed Abstract | CrossRef Full Text | Google Scholar

39. Smith BW, Dalen J, Wiggins K, Tooley E, Christopher P, Bernard J. The brief resilience scale: assessing the ability to bounce back. Int J Behav Med (2008) 15(3):194–200. doi: 10.1080/10705500802222972

PubMed Abstract | CrossRef Full Text | Google Scholar

40. Russell D, Peplau LA, Cutrona CE. The revised UCLA Loneliness Scale: concurrent and discriminant validity evidence. J Pers Soc Psychol (1980) 39(3):472. doi: 10.1037/0022-3514.39.3.472

PubMed Abstract | CrossRef Full Text | Google Scholar

41. Halberstadt AG, Cassidy J, Stifter CA, Parke RD, Fox NA. Self-expressiveness within the family context: Psychometric support for a new measure. psychol Assess (1995) 7(1):93. doi: 10.1037/1040-3590.7.1.93

CrossRef Full Text | Google Scholar

42. Brugha T, Cragg D. The list of threatening experiences: the reliability and validity of a brief life events questionnaire. Acta Psychiatrica Scand (1990) 82(1):77–81. doi: 10.1111/j.1600-0447.1990.tb01360.x

CrossRef Full Text | Google Scholar

43. Australian Institute of Family Studies. Growing up in Australia: The Longitudinal Study of Australian Children: 2004 Annual Report. In: Department of Family and Community Services. Melbourne, Australia: Australian Institute of Family Studies (2005).

Google Scholar

44. Fletcher GJ, Simpson JA, Thomas G. The measurement of perceived relationship quality components: A confirmatory factor analytic approach. Pers Soc Psychol Bull (2000) 26(3):340–54. doi: 10.1177/0146167200265007

CrossRef Full Text | Google Scholar

45. Cutrona CE, Russell DW. The provisions of social relationships and adaptation to stress. Adv Pers Relat (1987) 1(1):37–67.

Google Scholar

46. Feeney BC, Thrush RL. Relationship influences on exploration in adulthood: the characteristics and function of a secure base. J Pers Soc Psychol (2010) 98(1):57. doi: 10.1037/a0016961

PubMed Abstract | CrossRef Full Text | Google Scholar

47. Australian Bureau of Statistics. Technical Paper Socio-Economic Indexes for Areas (SEIFA) 2016. Canberra: Australian Bureau of Statistics (2018).

Google Scholar

48. Duncan LG. Assessment of mindful parenting among parents of early adolescents: Development and validation of the Interpersonal Mindfulness in Parenting scale. Doctor of Philosophy Dissertation. PennState University, Human Development and Family Studies. Available at: https://etda.libraries.psu.edu/catalog/7740. Accessed 26 August 2020.

Google Scholar

49. Zubrick SR, Lucas N, Westrupp EM, Nicholson JM. Parenting measures in the Longitudinal Study of Australian Children: Construct validity and measurement quality, Waves 1 to 4. (LSAC Technical Paper No. 12). Australian Government Department of Social Services: Canberra, ACT (2014).

Google Scholar

50. Sharp C, Goodyer IM, Croudace TJ. The Short Mood and Feelings Questionnaire (SMFQ): a unidimensional item response theory and categorical data factor analysis of self-report ratings from a community sample of 7-through 11-year-old children. J Abnormal Child Psychol (2006) 34(3):365–77. doi: 10.1007/s10802-006-9027-x

CrossRef Full Text | Google Scholar

51. Reardon T, Spence SH, Hesse J, Shakir A, Creswell C. Identifying children with anxiety disorders using brief versions of the Spence Children’s Anxiety Scale for children, parents, and teachers. psychol Assess (2018) 30(10):1342. doi: 10.1037/pas0000570

PubMed Abstract | CrossRef Full Text | Google Scholar

52. Soloff C, Lawrence D, Johnstone R. Sample Design (LSAC Technical Paper No.1). In: Studies AIoF. Melbourne: Australian Institute of Family Studies (2005).

Google Scholar

53. Bussing R, Fernandez M, Harwood M, Wei H, Garvan CW, Eyberg SM, et al. Parent and teacher SNAP-IV ratings of attention deficit hyperactivity disorder symptoms: psychometric properties and normative ratings from a school district sample. Assessment (2008) 15(3):317–28. doi: 10.1177/1073191107313888

PubMed Abstract | CrossRef Full Text | Google Scholar

54. StataCorp. Stata Statistical Software. Release 12 ed. StataCorp LP: College Station: TX (2011).

Google Scholar

55. Little RJ, D’Agostino R, Cohen ML, Dickersin K, Emerson SS, Farrar JT, et al. The Prevention and Treatment of Missing Data in Clinical Trials. New Engl J Med (2012) 367(14):1355–60. doi: 10.1056/NEJMsr1203730

PubMed Abstract | CrossRef Full Text | Google Scholar

56. Muthén LK, Muthén BO. Mplus User"s Guide. Muthén & Muthén: Los Angeles, CA (1998-2010).

Google Scholar

57. Shrier I, Platt RW. Reducing bias through directed acyclic graphs. BMC Med Res Methodol (2008) 8(1):70. doi: 10.1186/1471-2288-8-70

PubMed Abstract | CrossRef Full Text | Google Scholar

58. Kolenikov S. Calibrating survey data using iterative proportional fitting (raking). Stata J (2014) 14(1):22–59. doi: 10.1177/1536867X1401400104

CrossRef Full Text | Google Scholar

59. Ahn C, Heo M, Zhang S. Sample size calculations for clustered and longitudinal outcomes in clinical research. London: CRC Press (2014).

Google Scholar

60. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol (2006) 3(2):77–101. doi: 10.1191/1478088706qp063oa

CrossRef Full Text | Google Scholar