Reliability and validity of the Japanese version of the KIDSCREEN-52 health-related quality of life questionnaire for children/adolescents and parents/proxies

Abstract

Objectives

The present study aimed to assess the reliability and validity of the Japanese version of KIDSCREEN-52 (J-KIDSCREEN-52), a generic questionnaire used to assess health-related quality of life (HRQOL) among children/adolescents and parents/proxies.

Methods

We conducted a school-based study, in which 1564 children and adolescents aged 8–18 years and their 1326 parents participated from five schools. They were asked to complete two questionnaires (the J-KIDSCREEN-52 and the Pediatric Quality of Life Inventory (PedsQL)), and the Oslo 3-Item Social Support (OSS-3) scale. Internal consistency reliability was measured using the Cronbach’s alpha coefficient. Test–retest reliability was assessed by the Intraclass Correlation Coefficient (ICC) in the one-way random effects model in sub-samples taken approximately three to four weeks apart. Agreement between the ratings of the child and parent was evaluated using the ICC in the two-way mixed effects model among 681 pairs.

Results

For the overall sample, Cronbach’s alpha values of 10 dimensions were ≥0.70, except for one dimension. Test–retest ICCs were ≥0.60 for nearly all dimensions. Correlation coefficients between the J-KIDSCREEN-52 and the PedsQL dimensions indicated a reasonable convergent validity. Parent ratings corresponded well with child ratings (ICC = 0.38–0.62). Statistically significant differences in mean T scores were dependent on gender for seven dimensions, age group for all dimensions, and health status for two dimensions.

Conclusions

The J-KIDSCREEN-52 questionnaires child/adolescent and parent/proxy versions demonstrated acceptable levels of reliability and validity.

Introduction

A great advance in pediatric medicine has increased the number of children who live with chronic disease, disability, or even life-threatening conditions in a community. Health-related quality of life (HRQOL) measures can determine the burden of such diseases or disabilities [1], and the combination of clinical and HRQOL measures can provide a complete assessment of the impact of disease and treatment on child’s overall well-being [2]. In addition, research conducted within a general population can help identify at-risk children who may be subject to child abuse, neglect, disrupted families, or other problems within the community [3]. Some children also deal with bullying, developmental disorders [4], poor peer relations [5], and other problems in school. HRQOL measures can be used to identify hidden or unexpected health problems, evaluate health service needs, and thereby influence public policy decisions such that early prevention and, if necessary, intervention to address these concerns is a possibility [1].

Conceptually, assessment of a child’s HRQOL should utilize measures which contain multi-dimensional aspects from a child’s point of view. Particularly in social and psychological functioning, it is important to realize that children are embedded within various social contexts such as family, peer groups, the classroom, and the community [2]. Methodologically, a child’s emotional development, cognitive capacity, and reading skills should be taken into account when a measure is developed [6]. While a child’s self-report is the ideal way to obtain their subjective perspective, a combination of the ratings from both the child and parent may provide the most comprehensive information [2].

The KIDSCREEN-52, a 52-item questionnaire, was developed from the KIDSCREEN project funded by the European Commission (2001–2004) and aims to measure a child’s perspective on health status [7]. This questionnaire is advantageous in that it (1) is available in a cross-cultural setting because it was developed simultaneously in several different countries and was subsequently tested among a large number of representatives [8]; (2) covers the physical, psychological, social, and behavioral aspects of well-being and functioning from child’s point of view [9]; (3) has corresponding versions (self-reported and parent-reported versions) in which each item of the parent version is reworded to the self (child)-reported questionnaire [10], and (4) applies a combination of classical test theory with modern techniques, such as item response theory, within the process of development [9].

While the interest level in HRQOL assessment is certainly increasing, there are still very few reliable instruments, particularly for children, established for cross-cultural use. We developed the Japanese version of KIDSCREEN-52 HRQOL questionnaire (J-KIDSCREEN-52) [11] in collaboration with the KIDSCREEN Group. The KIDSCREEN-52 can evaluate the aspects of Social Acceptance (Bullying), Financial Resources, and Autonomy, which are rarely included in the existing measurements for children. The present study examined whether the proposed scale structure was equivalent to that of the original questionnaire and evaluated the reliability and validity of the J-KIDSCREEN-52.

The present study protocol was approved by the Nara Medical University Ethics Committee.

Materials and methods

Subjects and settings

A cross-sectional study was conducted from April to June, 2013. Participants comprised 773 children aged 8–11 years, 791 adolescents aged 12–18 years (hereafter, both children and adolescents are referred to as ‘children’), and 1326 of their parents. The children were students from two public schools, two private schools, and one national school in Nara and Kyoto, Japan. The five schools that participated in this study were recruited for reasons of convenience. We explained our study to the principal of each school and requested their cooperation. Informed consent forms and questionnaires for parents were distributed to all parents via the children. Questionnaire responses were made by 569 children during school hours, and by the remaining 995 children and 1326 parents at home. Returned questionnaires were collected at school.

The J-KIDSCREEN-52 child/adolescent [11] and the Pediatric Quality of Life Inventory 4.0 Generic Core Scales (PedsQL) self-report versions [12] were used for all children, and the Oslo 3-Item Social Support (OSS-3) scale [13] was used for subjects aged 12–18 years. We also inquired about the presence of a chronic health condition (yes/no). Parents were asked to specify their relationship with the child and to complete the J-KIDSCREEN-52 parent/proxy [11] and the PedsQL parent-proxy report versions [12].

To determine test–retest reliability, the J-KIDSCREEN-52 questionnaire was distributed to a sub-sample of 492 pairs of children and their parents. These were completed by 213 children in school, and by 279 adolescents and 492 parents at home, on two separate occasions, approximately three to four weeks apart.

All children were informed the purpose of the study, that this was not a test so that there were no wrong answers, that their responses would be treated anonymously, and that their participation was completely voluntary.

Questionnaires

J-KIDSCREEN-52

The KIDSCREEN-52 comprises 52 items, each of which is categorized as one of the following ten dimensions: Physical Well-being, Psychological Well-being, Moods & Emotions, Self-Perception, Autonomy, Parent Relations & Home Life, Financial Resources, Social Support & Peers, School Environment, and Social Acceptance (Bullying). The time frame refers to the week up to the time when the questionnaire was completed. It assesses either the frequency of behavior/feeling or intensity of an attitude, using a five-point Likert response scale. Scores are coded from 1 to 5, negatively formulated items are recoded, and item scores for each respective dimension are summed up. The scores for each of the ten dimensions are transformed into T scores with a mean of 50 and a standard deviation (SD) of 10, based on a representative sample of the European general population [3]. Higher scores indicate a better HRQOL.

After obtaining permission of copyright from Dr. U. Ravens-Sieberer, we developed the J-KIDSCREEN-52 from the English KIDSCREEN-52 questionnaire in collaboration with the KIDSCREEN Group following international guidelines [14]. The first step employed a forward–backward–forward translation technique. Specifically, the English questionnaire was translated into Japanese by two researchers working independently. The items of these national reconciled forward translations were then back-translated for the purpose of subsequent comparison with items of the original English draft. This comparison was designed to produce the final Japanese questionnaire. Thereafter, the degree of conceptual equivalence of the final Japanese forward translation was checked on an international basis to ensure cross-cultural harmonization. Twenty subjects underwent the pre-test followed by cognitive interviews in order to ensure the feasibility of the pilot questionnaire. After resolving inadequate concepts of translation or terminology due to cultural discrepancies between Japan and the KIDSCREEN Group, we came to an agreement on the final item formulation [11].

PedsQL and OSS-3

Convergent validity was assessed by comparing J-KIDSCREEN-52 with the PedsQL and the OSS-3 scale. The PedsQL is a generic HRQOL measure and comprises 23 items from the following four scales: (1) Physical functioning (8 items), (2) Emotional functioning (5 items), (3) Social functioning (5 items), and (4) School functioning (5 items). Items are reverse scored and linearly transformed to a 0–100 scale. Those with higher scores indicate better HRQOL. The Japanese version of PedsQL has been validated by a previous study [12].

The OSS-3 was adapted to assess the level of social support. This scale is based on (1) how many people can provide a sense of security and support to the child (1 item) and (2) the level of emotional and instrumental support provided by those people (2 items). The total score is calculated from the sum of those three items. Scores range from 3 to 14 points, with higher scores indicating stronger social support. The questionnaire was translated into Japanese by the authors.

Statistical analysis

Differences in mean T scores between boys and girls in each age group (8–11 and 12–18 years old) were evaluated by the t test in respective dimensions of the J-KIDSCREEN-52 and the PedsQL. Exploratory factor analysis using the principal factor method and promax rotation were conducted on the 52 items of J-KIDSCREEN to evaluate factorial validity. Any factor with an eigenvalue of 1.0 or greater was considered significant. Internal consistency of each dimension was evaluated by Cronbach’s alpha coefficients, and those ≥0.70 were considered acceptable [15]. To determine test–retest reliability, we examined the Intraclass Correlation Coefficient (ICC) between corresponding scores of the initial test and retest in the one-way random effects model. A coefficient of ≥0.60 was considered evidence for adequate test–retest stability [16]. The mean T score, standard deviation (SD), percentage of missing values, and proportion of responders with the lowest (floor) and highest (ceiling) possible scores were calculated for each dimension. Missing rates under 5 % and floor and ceiling effects lower than 15 % were considered acceptable [3]. Convergent validity was assessed by Pearson’s correlation coefficients between the J-KIDSCREEN-52 dimension scores and either the PedsQL or the OSS-3 dimension scores. Correlation coefficients between 0.10 and 0.30 were considered low, those between 0.31 and 0.50 moderate, and those over 0.50 high [9]. Convergent validity was assumed when correlations between comparable dimensions were higher than those between theoretically different dimensions [9]. Agreement between ratings from the child and the parent was analyzed using the ICC in the two-way mixed effects model. ICC values < 0.4 were considered poor to fair, between 0.41 and 0.6 moderate, and >0.6 substantial to almost perfect [16]. To evaluate the differences between children and parent responses, gender (boys/girls), age group (8–11 years/12–18 years), or health status (healthy/chronic condition) in reporting HRQOL, the t test of the differences (Δ) in mean T score, as well as unbiased Cohen’s effect size (d) corrected for sample size, was calculated. A positive Δ or d indicated a higher level of HRQOL reported by children, boys, younger age group (8–11 years), or healthy group. P < 0.05 was considered statistically significant. ∣d∣ between 0.21 and 0.5 was considered a small effect size, between 0.51 and 0.8 moderate, and >0.8 large [17]. All statistical analyses were performed using SPSS for Windows (Version 19; IBM).

Results

A total of 1,274 children (81.5 % response rate) and 982 parents (74.1 %) returned the questionnaires. Of these, 1058 children submitted questionnaires with no missing values. Mean age for these respondents was 11.7 years, and 458 (43.3 %) were boys. According to the child’s self-reported answers, 71 children (6.7 %) had a long-term disability or medical condition (allergy-related diseases, anemia, and others). Table 1 shows subject characteristics by age and sex. Response rates were higher in both sexes in the younger age group than in the older age group. However, in terms of percentages of respondents without missing values, boys (76.0 %) and girls (72.8 %) in the younger age group had lower values than boys (92.2 %) and girls (90.8 %) in the older group. Gender differences of mean T scores were found in 3 dimensions of J-KIDSCREEN-52 in the younger age group, and in 7 dimensions of the older age group.

Table 1 Subject characteristics by age and sex

Of the 880 parents who completed the questionnaire, 816 (92.7 %) were mothers, 60 (6.8 %) were fathers, and 4 (0.5 %) were significant others. Of the 845 pairs for whom the IDs were equivalent, responses from 681 child–parent pairs were available for analysis.

Exploratory factor analysis identified a 10-factor structure in both children and parents, as was the case for the original KIDSCREEN questionnaires.

As Table 2 shows, of the 259 children and 244 parents who completed the questionnaires twice, the ICC for test–retest reliability met the criteria of ≥0.60, except for the dimension of Social Acceptance in parents (0.56). Cronbach’s alpha coefficients ≥0.70 were found for all dimensions except for the Self-Perception in children (0.68). The percentage of missing values ranged from 0.3 to 5.0 %. No major floor effects were found, but ceiling effects greater than 15 % were found in seven dimensions for children and in two dimensions for parents.

Table 2 Test–retest reliability, internal consistency, and scale descriptions

Table 3 shows the results of the convergent validity analysis. For children, the correlations between the J-KIDSCREEN-52 and the PedsQL dimensions were moderate to high in the hypothesized direction: the PedsQL Physical functioning dimension showed the highest correlation with the J-KIDSCREEN-52 Physical Well-being (r = 0.36), followed by that between Emotional functioning and Moods & Emotions (r = 0.52), Social functioning and Social Acceptance (r = 0.44), and School functioning and School Environment (r = 0.44). For parents, the J-KIDSCREEN-52 and the PedsQL dimensions generally showed a moderate to high correlation in the hypothesized direction: the PedsQL Physical functioning dimension was most highly correlated with the J-KIDSCREEN-52 Physical Well-being (r = 0.22), Emotional functioning with Moods & Emotions (r = 0.56), Social functioning with Social Acceptance (r = 0.46), and School functioning with School Environment (r = 0.52). The J-KIDSCREEN-52 and the OSS-3 for children aged 12–18 years were more highly correlated with School Environment (r = 0.46) and Social Support & Peers (r = 0.45) than with other dimensions (r = 0.25 to 0.38) (data not shown).

Table 3 Correlation between the J-KIDSCREEN-52 and the PedsQL

Table 4 shows the discrepancies and agreements between ratings by the children and parents. With the exception of Financial Resources and Social Acceptance, self-reported scores for all dimensions were higher than parent-reported scores. Statistically significant discrepancies were found for seven dimensions, but the effect sizes were either none or small (∣d∣ = 0.12 to 0.36). The ICCs were moderate to excellent (range, 0.41–0.62) for all dimensions except Moods & Emotions and Social Acceptance (0.40).

Table 4 Discrepancies and agreements between child and parent ratings

Table 5 shows the mean (SD) T scores in each dimension of the J-KIDSCREEN-52, with the differences stratified by gender (boys/girls), age group (8–11 years/12–18 years), and self-reported health status (healthy/chronic condition). Girls scored higher than boys in seven dimensions, but the effect sizes were either none or small (∣d∣ = 0.13–0.33). In contrast, age group differences were more pronounced, with younger children (8–11 years) reporting higher scores than the older children (12–18 years) in almost every dimension. Especially in the four dimensions of Physical Well-being, Psychological Well-being, Self-Perception, and School Environment, the effect size was moderate (∣d∣ = 0.58–0.78). Older children’s scores were higher only in Financial Resources. Children with chronic conditions scored lower in Physical Well-being and Autonomy than healthy children, but the effect size was small (∣d∣ = 0.24–0.33).

Table 5 Differences in mean T values for the J-KIDSCREEN-52 by gender, age group, and health status

Discussion

The KIDSCREEN-52 HRQOL questionnaire was developed using strict methodology which complied with the internationally agreed guidelines submitted by a group of experts on HRQOL [7]. To date, this questionnaire has been translated into 38 different languages [10], and the validity has been reported for each version that was not created by a member of the original study group [18–20]. Several cross-sectional [21, 22] and longitudinal studies [23, 24] have been conducted using the questionnaires. The present study aimed to confirm whether or not the Japanese version of KIDSCREEN-52 developed by the authors is similar in structure to the original questionnaire and is thereby reliable for use in Japan.

A 10-factor structure was identified from 52 items of the questionnaire by exploratory factor analysis. The items included in each factor were, at a basic level, similar to those of the original research [9], which may indicate that each factor was represented by similar concepts as those observed in the international study.

We found acceptable levels of reliability for each dimension. Only the Self-Perception dimension among children showed Cronbach’s alpha coefficient of 0.68, which was slightly lower than those of the criteria, similar to those observed in another country [20]. Some of the questions in this dimension may have been slightly abstract (e.g., “Have you been happy with the way you are?”) and thus difficult to answer. Missing values in each dimension were low and noted at acceptable levels both in children and parents. Severe ceiling effects for Social Acceptance (43.5 % in children and 42.8 % in parents) in the present study were found, and were consistent with those found in the international study (49.1 % in children and 45.2 % in parents), but ceiling effects for six other dimensions that were not identified in the international study were found in children [10]. This is likely because the present study subjects were all children attending school, and thus they were in relatively good health. A comparison of J-KIDSCREEN-52 dimensions with PedsQL or OSS-3 scales showed the highest correlation for each similar concept, resulting in a satisfactory convergent validity. However, some dimensions such as Self-Perception, Autonomy, Parent Relations & Home Life, and Financial Resources were not evaluated because of the absence of the well-validated measurements. Future studies are needed for this issue.

Scores from children and parents agreed well, especially those for Physical Well-being. This was fairly consistent with results from the international study [25] and indicated that the parent/proxy version of the J-KIDSCREEN-52 could be used as a fair approximation for the child’s HRQOL. However, parents underestimated their children’s health for Physical Well-being, Psychological Well-being, Moods & Emotions, Autonomy, and Parent Relations & Home Life. Levels of agreement or discrepancy between the parent and child perspectives on HRQOL could serve as additional information regarding how a health condition or treatment affects the lives of a child and their family [25]. In addition, the parent’s perspective is inherently valuable as the parent is the one to assess the impact on the child’s health when deciding their health needs or treatment [2].

The perceived levels of well-being reported by children in the present study were similar to those reported by European children across almost every HRQOL dimension [10]. Mean T scores for individual dimensions were higher in Japan than in international studies for School Environment (55.1), Social Support & Peers (54.7), and Physical Well-being (54.0). Lower mean T scores were found in our study for Self-Perception (47.0), Social Acceptance (47.3), and Financial Resources (47.5) compared to those in international studies. A previous study reported that children in Asian countries had a similar conceptualization of QOL to that held by children in Western countries [26]. However, studies of cultural differences between European countries and Japan that might influence a child’s HRQOL are limited, and future studies are needed to clarify this.

Of the present study subjects, girls reported a higher HRQOL than boys in three dimensions: Psychological Well-being, Parent Relations & Home Life, and Social Support & Peers, although Cohen’s d values were small for these dimensions. Several studies have reported that adolescent girls have worse subjective health compared to boys [18, 19, 27] as they are confronted with the physical transition and are required to adapt to changes in their physique and gender identities, which are often more dramatic than those experienced by boys [27, 28]. Conversely, adolescent boys have been found to have fewer people with whom they can consult about these issues, and experience more depression than girls [29]. Further studies may clarify whether gender differences among Japanese children differ from those in other countries. In the present study, the younger group showed generally higher HRQOL relative to the older group, with the exception of Financial Resources. International studies have also found that in most cultures, more symptoms and lower health perception in most dimensions occur with increasing age, with the exception of Financial Resources and Social Acceptance [27].

Several HRQOL measures for children and adolescents have been developed, primarily in North America and Europe [30, 31]. Disease-specific measures are assumed to be more sensitive to the implications of a single condition when evaluating the impact of a change in treatment or the outcomes in clinical trials [30]. In contrast, generic measures are suitable for QOL assessment, regardless of the child’s specific condition, and can be used to compare patient groups with different diagnoses or a general population in order to determine the impact of a health condition or disease on HRQOL [2]. In Japan, several disease-specific self-reported measures are available for diseases such as asthma [32], cancer [33], brain tumor [34] and behavioral problems [35]. In addition, some generic measures are available for young adults (e.g., SF-36 [36] ), but to date, very few well-validated measures are focused on children. Exceptions include the PedsQL for 5- to 25-year-olds [12] and the Questionnaire for Measuring Health-Related Quality of Life In Children, Revised Version (Kid-KINDL^R) [37] for 8- to 12-year-olds. Although these questionnaires are used widely, the KIDSCREEN is a truly cross-national instrument, offers three versions of different lengths which can be used according to content and setting, and covers more aspects of a child’s HRQOL, including important topics such as social acceptance/rejection (bullying) [21], beyond the existing measurements [10]. Overall, the current study showed the J-KIDSCREEN-52 to be a reliable instrument, and we anticipate that this will help promote a better understanding of comprehensive health status in children and adolescents as well as enable international comparisons.

Some limitations of the present study are worth noting. First, our study was conducted among school-based participants and included small numbers of children with chronic conditions. Future validation studies should be conducted in a clinical setting. Second, we recruited our subjects through non-random sampling and did not obtain information on the family psychosocial background. Third, as this was a cross-sectional study, we were unable to confirm whether the instrument was sensitive enough to gauge changes in the child’s health condition.

Despite these limitations, the study population was sufficiently large, allowing us to confirm acceptable levels of reliability and validity for the J-KIDSCREEN-52 child/adolescent and parent/proxy versions. Further studies should assess this in settings where clinical information is available, and confirm the sensitivity to change in a longitudinal study.