Reliability and validity of Japanese versions of KIDSCREEN-27 and KIDSCREEN-10 questionnaires

Abstract

Objective

This study aimed to assess the reliability and validity of Japanese versions of the KIDSCREEN-27 (J-KIDSCREEN-27) and KIDSCREEN-10 (J-KIDSCREEN-10) questionnaires, which are shorter versions of the KIDSCREEN-52 (J-KIDSCREEN-52).

Methods

The present analyses are based on a pre-existing dataset of the J-KIDSCREEN-52 validation study, including 1564 children and adolescents aged 8–18 years and their 1326 parents. All were asked to complete the J-KIDSCREEN and Pediatric Quality of Life Inventory (PedsQL) questionnaires. Test–retest reliability was assessed with Intraclass Correlation Coefficients (ICCs) in a one-way random effects model, and internal consistency reliability was measured using Cronbach’s alpha coefficients. Agreement between child and parent scores was evaluated using ICCs in a two-way mixed effects model. To assess concurrent validity, a sub-sample of 535 parents evaluated their child’s mental health status using the Strengths and Difficulties Questionnaire (SDQ).

Results

For children, test–retest ICCs were ≥0.60 and Cronbach’s alpha ≥0.70 for every dimension of both instruments. Correlations of corresponding dimensions between the J-KIDSCREEN-27 or -10 and the PedsQL were acceptable. For parents, test–retest ICCs were ≥0.60, Cronbach’s alpha ≥0.70, and ICCs between child and parent scores ≥0.41 in every dimension of both instruments. In multivariate logistic regression models, after adjusting for confounders, lower health-related QOL in every dimension of both instruments, except Physical Well-being, was significantly associated with higher odds ratios for borderline and clinical ranges of the SDQ.

Conclusion

The child/adolescent and parent/proxy versions of the J-KIDSCREEN-27 and J-KIDSCREEN-10 demonstrated acceptable levels of reliability and validity.

Introduction

Health-related quality of life (HRQOL) refers to an individual’s perception and subjective evaluation of their health and well-being; therefore, it is important to obtain responses by self-report whenever possible [1]. Various self-report measurements for children and adolescents have been developed and are used as outcome measures in epidemiological and clinical studies [2, 3].

The KIDSCREEN is a generic questionnaire designed to measure subjective HRQOL in children and adolescents, and is conceptually based on the definition of QOL as a multidimensional construct including physical, emotional, mental, social, and behavioral components of well-being and functioning as perceived by children and adolescents [4]. The first instrument developed by the KIDSCREEN Group [4] was a 52-item questionnaire (KIDSCREEN-52) covering 10 dimensions of HRQOL that has been shown to have good psychometric properties [5, 6]. A 27-item questionnaire (KIDSCREEN-27) was derived from the KIDSCREEN-52 using psychometric methods to provide an instrument that might be useful in epidemiological and clinical studies and reduce the response burden and to save administration costs [7, 8]. Similarly, a 10-item version (KIDSCREEN-10) was derived from the KIDSCREEN-27. The KIDSCREEN-10 allows summarization of scores into single values which can be useful for examining overall HRQOL changes in situations such as monitoring in clinical and school settings [9].

Since these shorter versions are practical to use and have a low administrative burden while maintaining many of the advantages of the KIDSCREEN-52 [10], they have been used in many countries for large-scale population-based studies [11, 12], studies that investigate factors that impact the HRQOL of children with chronic diseases or conditions [13, 14], and interventional [15] and longitudinal studies [16, 17]. In particular, the use of these questionnaires for studies of HRQOL of children with mental health problems has increased [18–20]. Since mental health problems such as attention deficits, learning difficulties, and conduct problems can strongly impact HRQOL at school and at home in everyday life [19, 21], in addition to the potential difficulties in maintaining attention when performing tasks [22], a brief and reliable HRQOL measure is indispensable for evaluating therapeutic and educational interventions. However, only a few well-validated instruments exist in Japan [23, 24].

The Japanese versions of KIDSCREEN questionnaires (J-KIDSCREEN-52, -27, and -10) were simultaneously developed from the English KIDSCREEN questionnaires in collaboration with the KIDSCREEN Group, and the J-KIDSCREEN-52 has already been validated [25]. This study aimed to investigate the reliability and validity of the shorter versions (J-KIDSCREEN-27 and -10) in order to make these instruments available for use in Japan.

Materials and methods

Participants and settings

The present analyses are based on a pre-existing data set that was used to validate the J-KIDSCREEN-52 [25]. A total of 1564 children and adolescents (hereafter, both children and adolescents are referred to as ‘children’) aged 8–18 years attending five schools and 1326 of their parents and proxies (hereafter, both parents and proxies are referred to as ‘parents’) participated in the study. The study was carried out between April and June 2013. All children and parents were asked to answer the J-KIDSCREEN questionnaire and the Pediatric Quality of Life Inventory 4.0 Generic Core Scales (PedsQL) [24]. A total of 492 child-parent pairs participated in the test–retest study within a 3–4-week interval. A more detailed description of the sampling for the J-KIDSCREEN-52 validity study is provided elsewhere [25]. A sub-sample of 535 parents was asked to complete an additional assessment of their child’s mental health status using the Strengths and Difficulties Questionnaire (SDQ) [26]. All schools that participated in this study were recruited for reasons of convenience. Details of the study were explained to the principal of each school. The test–retest study was conducted at three schools and an additional assessment using the SDQ was conducted at two schools when their cooperation was obtained.

Informed consent forms were distributed to all parents via the children and obtained from all parents included in the study. All children were informed of the purpose of the study, that their responses would be treated anonymously, and that their participation was completely voluntary. The present study protocol was approved by the Nara Medical University Ethics Committee.

Questionnaires

• 1. The KIDSCREEN questionnaires [4]

The KIDSCREEN questionnaires are available in three versions: the original long version (consists of 52 items), a 27-item version, and a 10-item version. These instruments were designed to be used in populations aged 8–18 years and both self-complete (child/adolescent) and proxy (parent/proxy) versions are available. Items in all versions are answered on a five-point Likert-type scale assessing frequency: never (1), seldom (2), sometimes (3), often (4), and always (5), or intensity: not at all (1), slightly (2), moderately (3), very (4), and extremely (5), with a 1-week recall period. Scores are coded from 1 to 5, negatively formulated items are recoded, and the sum scores for respective dimensions (for one index in the KIDSCREEN-10) are transformed to T scores with a mean of 50 and a standard deviation (SD) of 10. Higher scores indicate better HRQOL. Figure 1 shows the dimensions of the KIDSCREEN instruments and relationships between versions [10]. The KIDSCREEN-27 and -10 require approximately 10 and 5 min to complete, respectively, compared to 15 min for the KIDSCREEN-52.

Fig. 1

Dimensions of the KIDSCREEN and relationships between versions [10]

• 2. KIDSCREEN-27 [4, 7, 8]

The KIDSCREEN-27 is the mid-length version of the KIDSCREEN instruments. It is embedded within the KIDSCREEN-52 and consists of 27 items that measure five dimensions: Physical Well-being (same 5 items as in the longer version) explores the level of the child’s physical activity, energy, and fitness, as well as the extent to which a child feels unwell and complains of poor health. Psychological Well-being (7 items from Psychological Well-being, Moods & Emotions, and Self-Perception of the longer version) examines psychological well-being of the child, including positive emotions and satisfaction with life as well as the absence of feelings such as loneliness and sadness. Parent Relations & Autonomy (7 items from Autonomy, Parent Relations & Home Life, and Financial Resources of the longer version) explores relationships with parents, the atmosphere at home, and feelings of having enough age-appropriate freedom to choose as well as feeling satisfied with their own financial resources. Social Support & Peers (4 items from the longer version) examines the quality of the interaction between the child and peers as well as their perceived support. School Environment (4 items from the longer version) explores the child’s perceptions of his/her cognitive capacity, learning and concentration, and feelings about school.

• 3. KIDSCREEN-10 [4, 9]

The KIDSCREEN-10 is the shortest version of the KIDSCREEN instruments, and is embedded within the KIDSCREEN-27. This unidimensional measure contains 10 items and represents a global score (General HRQOL Index) for the dimensions of the longer KIDSCREEN versions. Items 1 and 2 explore the level of the child’s physical activity, energy, and fitness. Items 3 and 4 explore the absence of feelings such as loneliness and sadness. Items 5 and 6 ask about feelings of having enough age-appropriate freedom to choose. Item 7 explores the relationship between child and parent. Item 8 examines the quality of the interaction between the child and peers. Finally, items 9 and 10 explore the child’s perceptions of his/her cognitive capacity, learning, and concentration.

• 4. PedsQL

The PedsQL [27] is a generic HRQOL measure that comprises 23 items from the following four dimensions: Physical functioning, Emotional functioning, Social functioning, and School functioning. Items are reverse-scored and linearly transformed to a 0–100 scale. Higher scores indicate better HRQOL. Each dimension score (0–100) was hypothesized to correlate with the corresponding J-KIDSCREEN-27 dimension score: Physical functioning correlates with Physical Well-being, Emotional functioning with Psychological Well-being, Social functioning with Social Support & Peers, and School functioning with School Environment. Summary scores of the PedsQL, which include the Psychosocial summary score (0–300), sum of scores in Emotional, Social, School functioning, and the Total score (0–400), the sum of the scores of the four dimensions, were hypothesized to correlated with the J-KIDSCREEN-10. The Japanese version of the PedsQL has been validated in a previous study [24].

• 5. SDQ

Children’s mental health problems were assessed using the parent-reported SDQ [28]. The SDQ is a brief screening questionnaire designed for children and adolescents that asks about their mental health symptoms and positive attitudes over the previous 6 months. We used the total difficulties score, which is the sum of four components: emotional symptoms (5 items), conduct problems (5 items), hyperactivity/inattention (5 items), and peer problems (5 items). Each item of the SDQ is scored on a three-point scale: not true (0), somewhat true (1), or certainly true (2). Higher scores indicate more significant problems and the total difficulties score (0–40) was classified into a normal range (0–12), borderline range (13–15), and clinical range (16–40) [26]. The Japanese parent-rated SDQ has been validated for a large community-based sample of 4 to 12-year-old children [26].

Statistical analysis

The mean T score and standard deviation (SD) were calculated for each dimension. Differences between gender (boys/girls) or age group (8–11 years/12–18 years) in reporting HRQOL were evaluated with the t test of differences (Δ) in mean T score, and unbiased Cohen’s effect size (d) corrected for sample size. A positive Δ or d indicates a higher level of HRQOL reported by boys and the younger age group (8–11 years). P < 0.05 was considered statistically significant. A |d| value between 0.21 and 0.5 was considered a small effect size, between 0.51 and 0.8 moderate, and >0.8 large [29].

Test–retest reliability was assessed with Intraclass Correlation Coefficients (ICCs) between each score of the initial test and retest. A coefficient ≥0.60 was considered evidence for adequate test–retest stability [30]. Internal consistency of each dimension was evaluated by Cronbach’s alpha coefficients, and those ≥0.70 were considered adequate [31]. Convergent validity was assessed by Pearson’s correlation coefficients of corresponding scores between the J-KIDSCREEN-27 or -10 and the PedsQL. Correlation coefficients between 0.10 and 0.30 were considered low, those between 0.31 and 0.50 moderate, and those over 0.50 high [7]. Correlations of corresponding dimensions between the J-KIDSCREEN-27 and -52, or those between the J-KIDSCREEN-10 and -27, were assessed by Pearson’s correlation coefficients. A coefficient ≥0.70 was considered satisfactory [7]. Agreement between ratings from the child and the parent was analyzed using ICCs in a two-way mixed effects model. ICC values <0.4 were considered poor to fair, between 0.41 and 0.6 moderate, and >0.6 substantial to almost perfect [30].

To evaluate concurrent validity, parent-rated SDQ scores (normal, borderline, and clinical ranges) were used to assess the child’s mental health status. The analysis of variance (ANOVA) was used to compare means of HRQOL by mental health status. The Dunnett tests for multiple comparison (normal vs borderline or clinical ranges) were used when p < 0.05 in ANOVA. The prevalence odds ratio (OR) was expressed as a point estimate with a 95 % confidence interval (CI). Logistic regression models included mental health status (SDQ categories; borderline or clinical range/normal range) as dependent variables and lower HRQOL (each defined by a score less than the 25th percentile score in each dimension of the J-KIDSCREEN-27 and -10) as independent variables. Covariates included gender (boy/girl), age (one-year increase), self-reported chronic illness (yes/no), and responder to the questionnaire (other/mother). Statistical analyses were performed using SPSS for Windows (Version 19, IBM).

Results

The analysis included data from 1058 children, 880 parents, and 681 child-parent pairs. Of the 1058 children (mean age 11.7 years), 458 (43.3 %) were boys, and 71 (6.7 %) reported having medical conditions, such as allergy-related diseases. Mean T scores of the children and differences in mean T values by gender (boys/girls) and age group (8–11 years/12–18 years) are shown in Table 1. No significant differences were found in gender proportions between age groups and in age group proportions between gender by the Chi square test. Girls scored higher than boys in Social Support & Peers of the J-KIDSCREEN-27 with a small effect size (|d| = 0.36). As for age group differences, the younger age group (8–11 years) scored higher than the older age group (12–18 years) in the J-KIDSCREEN-27, with moderate effect sizes in Physical Well-being (|d| = 0.74), Psychological Well-being (0.60), and School Environment (0.73), and in the J-KIDSCREEN-10 (0.61).

Table 1 Mean T values of the children and differences by gender and age group

Table 2 shows results on test–retest ICCs, Cronbach’s alpha values, correlation of the J-KIDSCREEN-27 with each dimension of the PedsQL and the J-KIDSCREEN-52, and correlation of the J-KIDSCREEN-10 with each dimension of the PedsQL and the J-KIDSCREEN-27 in children. Test–retest ICCs were ≥0.60 (range 0.73–0.79) and Cronbach’s alpha values were ≥0.70 (range 0.78–0.87) for every dimension of the J-KIDSCREEN-27 and -10. Correlations between the J-KIDSCREEN-27 and PedsQL dimensions were moderate in the hypothesized direction. Specifically, the PedsQL physical functioning dimension showed higher correlation with Physical Well-being of the J-KIDSCREEN-27 (r = 0.34) than other dimension scores of the PedsQL, followed by that between Emotional functioning and Psychological Well-being (r = 0.45), Social functioning and Social Support & Peers (r = 0.35), and School functioning and School Environment (r = 0.45). Correlations between the J-KIDSCREEN-27 and the corresponding dimensions of the J-KIDSCREEN-52 ranged from 0.71 to 0.98, but some dimensions of the J-KIDSCREEN-52 were below the a priori specified threshold of 0.70 (i.e., r = 0.58 for Self-Perception and 0.64 for Financial Resources). The correlation between the J-KIDSCREEN-10 and the PedsQL were moderate in the hypothesized direction. Specifically, the Psychosocial summary score (r = 0.46) and Total score (r = 0.47) of the PedsQL showed higher correlations with the J-KIDSCREEN-10 than other dimension scores of the PedsQL. Correlations between the J-KIDSCREEN-10 and -27 were ≥0.70 for three dimensions (r = 0.82 for Psychological Well-being, 0.71 for Parent Relations & Autonomy, and 0.77 for School Environment) of the KIDSCREEN-27.

Table 2 Mean T values, test–retest ICCs, Cronbach’s alpha, and correlations of the J-KIDSCREEN-27 with the PedsQL or J-KIDSCREEN-52, and the J-KIDSCREEN-10 with the PedsQL or J-KIDSCREEN-27 in children

Of 880 parents, 816 (92.7 %) respondents were mothers, 60 (6.8 %) were fathers, and 4 (0.5 %) were significant others. Table 3 shows mean T values, test–retest ICCs, Cronbach’s alpha values among parents, and ICCs between child and parent scores of the J-KIDSCREEN-27 and -10. Test–retest ICCs were ≥0.60 (range 0.65–0.76) and Cronbach’s alpha values were ≥0.70 (range 0.73–0.89). ICCs between child and parent scores ranged from moderate (0.42 for Parent Relations & Autonomy) to substantial (0.62 for Physical Well-being).

Table 3 Mean T values, test–retest ICCs, Cronbach’s alpha, and child’s and parent’s rating ICCs of the J-KIDSCREEN-27 and the J-KIDSCREEN-10 in parents

Of 535 child-parent pairs, 364 returned both the SDQ and the J-KIDSCREEN (response rate: 68.0 %), and data from 320 pairs were valid for analysis. There were 30 (9.4 %) borderline range children and 21 (6.6 %) clinical range children whose mental health status was assessed by parent-rated SDQ (Table 4). Mean total difficulties scores (SD) were 5.9 (3.2) in the normal range, 13.7 (0.8) in the borderline range, and 19.8 (3.4) in the clinical range. The self (child)-reported J-KIDSCREEN-27 and -10 scores showed significant or marginal (p = 0.056 between normal and clinical range in Parent Relations & Autonomy, and p = 0.065 between normal and borderline range in School Environment; data not shown in Table 4) differences by mental health status (normal vs borderline or clinical ranges) in every dimension except Physical Well-being.

Table 4 Differences in J-KIDSCREEN dimension scores by mental health status (SDQ)

Multiple logistic regression analysis (Table 5), adjusted for gender, age, presence of chronic illness, and SDQ respondent, revealed that ORs for lower mental health status (borderline or clinical range in the SDQ) were significantly associated with lower HRQOL (quartiles in each dimension of the J-KIDSCREEN-27 and -10) in four dimensions of the J-KIDSCREEN-27 (Psychological Well-being: OR, 2.31; 95 % CI, 1.21–4.41; Parent Relations & Autonomy: OR, 2.14; 95 % CI, 1.12–4.09; Social Support & Peers: OR, 3.03; 95 % CI, 1.62–5.69; School Environment: OR, 2.55; 95 % CI, 1.33–4.88) and J-KIDSCREEN-10 (OR, 2.02; 95 % CI, 1.07–3.84), but not with Physical Well-being.

Table 5 Multiple logistic regression analysis for the association between HRQOL and lower mental health status (borderline or clinical range in the SDQ categories)

Discussion

In the present study, we identified differences in HRQOL by gender and age groups of children. In the original KIDSCREEN studies [7, 9], boys tended to score higher in HRQOL than girls, except in Social Support & Peers and School Environment. In contrast, boys in the present study scored higher only in Physical Well-being. Although the effect sizes were none or small either in our study and the aforementioned previous studies, further research will be needed to determine whether gender differences were different from other countries and what aspects, such as cultural and socioeconomic factors, could explain the discrepancies between studies [32]. Consistent with the original studies, age group differences were evident in every dimension [7, 9]. In general, most cultures show more symptoms and lower health perception in most HRQOL scales with increasing age [7, 9, 32]. When growing up, children are confronted with various physical and social transitions and need to adapt to their changing bodies. This may lead to impairments in HRQOL [32].

The J-KIDSCREEN-27 showed good reliability and validity in children. Test–retest reliability and internal consistency of each dimension showed adequate. Convergent validity was indicated by correlations between the J-KIDSCREEN-27 and the PedsQL. Correlations were generally highest for those pairs of dimensions in which higher correlations were a priori expected. This was considered to reflect reasonable convergence, while correlations between theoretically different dimensions were low to moderate, indicating poor discriminant validity. Notwithstanding, these results were similar to the original study [7], in which some questions in the PedsQL were thought to overlap the dimensions of the KIDSCREEN. Correlations between the J-KIDSCREEN-27 and corresponding dimensions of the J-KIDSCREEN-52 were acceptable with a few exceptions. Self-Perception of the J-KIDSCREEN-52 correlated below the a priori specified threshold with the corresponding dimension of Psychological Well-being of the J-KIDSCREEN-27. This is consistent with the original study because fewer items from the long version were incorporated into the shorter version [7]. In contrast, Financial Resources of the J-KIDSCREEN-52 correlated to a slightly lower degree than in the original study with Parent Relations & Autonomy of the J-KIDSCREEN-27 (r = 0.71) [7]. The reason for this discrepancy is unclear and should be addressed in future studies that include socioeconomic information. Use of the J-KIDSCREEN-27 should take into account that the Self-Perception and Financial Resources dimensions are less well represented.

Test–retest reliability and internal consistency were adequate for the J-KIDSCREEN-10 in children. Convergent validity was acceptable since correlations with the Total score and Psychosocial summary score of the PedsQL were higher than other dimension scores. Correlations with the J-KIDSCREEN-27 were ≥0.70 for the three dimensions that were well represented by the J-KIDSCREEN-10, but other dimensions such as Physical Well-being and Social Support & Peers had slightly lower coefficients. Our results for the J-KIDSCREEN-10 were generally consistent with the original study that reported the KIDSCREEN-10 was more focused on aspects of HRQOL related to mental health [9].

Among parents, test–retest reliability and internal consistency were adequate for both the J-KIDSCREEN-27 and -10. In addition, parent scores agreed well with child scores, and therefore both the J-KIDSCREEN-27 and -10 parent/proxy versions can be considered reliable measures to assess a child’s health status.

Overall, our results were similar to those of the original KIDSCREEN studies [4, 7, 9], and demonstrate that the J-KIDSCREEN-27 and -10 questionnaires are acceptable to use for children and parents in Japan. The J-KIDSCREEN-27 represented seven dimensions of the J-KIDSCREEN-52 well: Physical Well-being, Psychological Well-being, Moods & Emotions, Autonomy, Parent Relations & Home life, Social Support & Peers, and School Environment. The J-KIDSCREEN-10 represented three dimensions of the J-KIDSCREEN-27 well: Psychological Well-being, Parent Relations & Autonomy, and School Environment. No items relating to Social Acceptance (Bullying) of the KIDSCREEN-52 were included in the 27-item version, and no items relating to Social Acceptance (Bullying), Psychological Well-being, Self-Perception, and Financial Resources of the KIDSCREEN-52 were included in the 10-item version. Thus, the loss of information relating to some psychosocial aspects of the 52-item version should be considered when deciding which version to apply.

The association between the J-KIDSCREEN-27 or -10 with mental health status was assessed using the parent-rated SDQ. Since SDQ subscales include problem scores (emotional symptoms, conduct problems, hyperactivity/inattention, and peer problems), higher total difficulties scores indicate mental health problems or difficulties in daily life. The current study population included 9.4 % borderline and 6.6 % clinical range children whose mental health status was assessed by their parents. In comparison with a study reporting 9.9 % borderline and 5.2 % clinical range children in European countries [18], participants of the present study included a similar percentage of children with lower mental health status. Borderline and clinical range children showed the same tendency to report marginally to significantly lower mean scores in psychological and social health dimensions, but not in the physical well-being dimension compared to normal range children. Even after adjusting for gender, age, presence of chronic illness, and questionnaire respondent, a significant association was found between lower HRQOL in the psychosocial dimensions of the J-KIDSCREEN-27 or -10 and lower mental health status. These results indicate that the J-KIDSCREEN-27 and -10 could sensitively capture and serve as an instrument for children with mental health problems to assess such psychosocial burdens in the school and home life, as reported in previous studies [4, 7, 9, 18]. Mental health problems in childhood are known to confer a higher risk for various negative outcomes, such as behavioral problems, increased school absences, and comorbidity of psychiatric problems [18, 22, 33], but educational interventions, medical management, and social support can improve these outcomes [22, 34–36]. In future research, these measurements will offer more knowledge about the impact of particular mental health constraints and adaptations to clinical practice or public health practice by comparing the HRQOL of different health conditions, medical treatments, and health care services in Japan.

Some limitations of the present study are worth noting. First, our study was conducted among school-based participants recruited through non-random sampling. Only a small number of children with chronic illness were included, and thus most participants may have been relatively healthy. Future validity studies will be needed that target the general population or are conducted in clinical settings. Second, because information about potentially influential variables such as socio-economic status and parental health problems were not acquired, we could not analyze differences in HRQOL based on these factors. Moreover, the lack of more objective health measures such as those obtainable from physical examinations or medical records may be less reliable to define a child’s physical and/or mental health condition. Finally, this was a cross-sectional study and we were unable to confirm whether these instruments were sensitive enough to detect the changes in the child’s health conditions.

In spite of these limitations, our study population was sufficiently large to allow us to conclude that the child/adolescent and parent/proxy versions of the J-KIDSCREEN-27 and -10 questionnaires achieved most attributes proposed by the KIDSCREEN Group [4], and demonstrated acceptable levels of reliability and validity. Although the loss of information from some aspects relative to the 52-item version should be kept in mind, the availability of the shorter versions will make them adaptable tools that can be used in many different settings such as clinical environments, schools, or the respondent’s home, in the fields of public health, epidemiology, and medicine in Japan.