Example: air traffic controller

RELIABLITY OF RESPONSES IN QUESTIONNAIRE RESEARCH …

RELIABLITY OF RESPONSES IN QUESTIONNAIRE RESEARCH WITHCHILDREN( Coding scheme: a technical report : appended ) of HoxUtrecht UniversityChildren are no longer neglected as respondents in large-scale surveys. Researchers areconvinced that information about perspectives, attitudes, and behaviors of children should becollected from the children themselves. Regarding adults, empirical evidence shows thatrespondent characteristics as well as question characteristics affect response quality. Becausechildren are in process of developing their cognitive and social skills, it is expected thatanswering questions in surveys brings along additional problems concerning response far however, methodological expertise on surveying children is scarce, and researchers relyon ad-hoc knowledge from fields such as child psychiatry and educational testing, or onmethodological knowledge on surveying this article we report results of a secondary analysis on several data sets.

3 provide a satisfactory response instead of an optimal one. Using a satisficing strategy may lead to less reliable responses than using an optimizing strategy.

Tags:

  Research, Questionnaire, Response, Relability, Reliablity of responses in questionnaire research

Information

Domain:

Source:

Link to this page:

Please notify us if you found a problem with this document:

Other abuse

Transcription of RELIABLITY OF RESPONSES IN QUESTIONNAIRE RESEARCH …

1 RELIABLITY OF RESPONSES IN QUESTIONNAIRE RESEARCH WITHCHILDREN( Coding scheme: a technical report : appended ) of HoxUtrecht UniversityChildren are no longer neglected as respondents in large-scale surveys. Researchers areconvinced that information about perspectives, attitudes, and behaviors of children should becollected from the children themselves. Regarding adults, empirical evidence shows thatrespondent characteristics as well as question characteristics affect response quality. Becausechildren are in process of developing their cognitive and social skills, it is expected thatanswering questions in surveys brings along additional problems concerning response far however, methodological expertise on surveying children is scarce, and researchers relyon ad-hoc knowledge from fields such as child psychiatry and educational testing, or onmethodological knowledge on surveying this article we report results of a secondary analysis on several data sets.

2 The multi levelmeta-analysis was directed on the (interaction) effects of child characteristics and questioncharacteristics on the reliability of words: special population, question characteristics, respondent characteristics; responsequality, survey resaerch 1 Corresponding author: Natacha BorgersFaculty of Social and Behavioral Sciences, University of AmsterdamWibautstraat 4NL-1091 CM AmsterdamThe Netherlandstel. +31 20 5251526fax +31 20 5251200email: general surveys, procedures to enhance response quality are well documented (Biemer,Groves, Lyberg, Mathiowetz, & Sudman, 1991; Groves, 1989; Lyberg et al., 1997). However,surveying an adult population is far from simple. Even slight variations in question wordingaffects RESPONSES . There is an increasing body of empirical evidence that both respondentcharacteristics and question characteristics affect the reliability of RESPONSES in surveys(Krosnick, 1991; Krosnick & Fabrigar, 1997; Schwarz & Hippler, 1995).

3 However, there isstill a lack of methodological knowledge on procedures to enhance response quality in surveyresearch with children, whereas survey researchers increasingly collect perspectives, attitudesand behaviors from the children themselves (Scott, 1997).Krosnick (1991) utilizes a satisficing theory to explain why the reliability of RESPONSES differsbetween respondents, and why it can be affected by question wording. The satisficing theoryelaborates on a standard question answering process-model developed by Tourangeau (1988)( , Miller, & Oksenberg, 1990; Krosnick, Narayan, & Smith, 1996; Schwarz,Kn uper, & Park, 1998; Sudman, Bradburn, & Schwarz, 1996). Four steps characterize anoptimal question answering and interpreting the question being the relevant information from this information into a summarized judgment4 Reporting this judgment by translating it to the format of the presented response to this satisficing theory there are three factors that affect the question answeringprocess.

4 The first is the motivation of the respondent to perform the task, the second is thedifficulty of the task, and the third is the respondent s cognitive ability to perform the , satisficing theory identifies two processes that explain differences in reliability ofresponses, namely optimizing and satisficing. Optimizing means that the respondent goesthrough all the four cognitive steps needed to answer a survey question. Contrary tooptimizing, satisficing means that a respondent gives more or less superficial RESPONSES thathowever, appear reasonable or acceptable to the researcher, without going through all thesteps involved in the question-answering process. Satisficing is related to the motivation ofthe respondent, the difficulties of the task and the cognitive abilities of the questions, low cognitive abilities, and low motivation may lead respondents to3provide a satisfactory response instead of an optimal one.

5 Using a satisficing strategy maylead to less reliable RESPONSES than using an optimizing , the satisficing theory assumes an interaction effect between respondentcharacteristics and question characteristics, which can be described as follows: the lesscognitively sophisticated respondents are, the more sensitive to difficult or cognitivelydemanding questions they will be, and the less reliable their RESPONSES will be. The fewprojects that reported on the interaction between respondent and question characteristicspresented empirical evidence for this effect (Borgers, Leeuw, & Hox, 1999; Borgers, Leeuw,& Hox, 2000; Kn uper, Belli, Hill, & Herzog, 1997; Marsh, 1986; Schwarz et al., 1998).These studies showed that the less cognitively sophisticated respondents, elderly or youngchildren gave less reliable RESPONSES on more difficult questions than the more the last decade methodological knowledge on the effects of reduced cognitive ability onresponse reliability, increased.

6 This knowledge is mainly based on RESEARCH results reportingspecial populations in survey RESEARCH , such as the effects of the cognitive decline of elderly(Herzog & Rodgers, 1992; Kn uper et al., 1997). Several studies show that reduction incognitive functioning is associated with a decline in the reliability of RESPONSES (Alwin &Krosnick, 1991; Kn uper et al., 1997; Krosnick, 1991; Schwarz et al., 1998).Like ageing, growing up involves changes in cognitive functioning. Within childrencognitive, communicative and social skills are developing. As a consequence, cognitiveability varies considerably across children. These differences across children can result in theuse of different strategies in answering questions and by that, differences in the reliability ofresponses. It is reasonable that answering questions in surveys brings along additionalproblems concerning response quality.

7 However, methodological knowledge on how tosurvey children is still scarce, while this kind of data collection displays a rapid growth (Scott,1997). The few studies that are known (Borgers, 1998; Leeuw & Otter, 1995; Otter,Mellenberg, & Glopper, 1995; Otter, 1993) support the hypothesis that growing up is relatedto an increase of the reliability of RESPONSES . Therefore it is necessary to increase themethodological knowledge on response quality in survey RESEARCH with study investigates the effects of child and question characteristics on the reliability ofchildren s RESPONSES in self-administered questionnaires. Besides, the interaction effectbetween both characteristics on the reliability of RESPONSES , which follows from satisficingtheory, will be researched. Based on satisficing theory and empirical results, hypotheses can4be formulated for the effect of child characteristics and question characteristics on thereliability of RESPONSES .

8 Less cognitively sophisticated children (youngest, less years ofeducation) produce less reliable RESPONSES than the more cognitively sophisticated overview of the hypothesized direction of the expectations for each question characteristicis given in Table 1. Besides their operationalization of the question characteristics are given inthis table. The question characteristics are categorized according to the step of the questionanswering process model, were the concerned characteristic cognitively appeal 1:Summary of the question characteristics, direction of the hypotheses andoperationalizationComprehension and interpretation of the questiondirection of thehypothesis2 Operationalization Question lengthnegativenumber of wordsnegativenumber of sentences Length of the introductory textpositivenumber of words/100 Readabilitypositivecomprehensive readability, high score= easy to readpositivetechnical readability.

9 High score =easy to read Ambiguitynegativeof the questionnegativeof the response scale Double barrelednegativedouble versus single Complex constructionsnegativecomplex versus simple Negatively formulated questionnegativenegative versus positive Kind of information being askedno directionattitudesexperiencesopinionsbeh aviorattributionscapacitiesRetrieving relevant information from memory Complexity of the questionnegativecomplex versus simple Reference periodpositivereference period versus no referenceperiod Numerical quantitynegativenumeric versus not numeric responseJudging the retrieved information Subjective question threat, sensitivitypositivesum of 4 indicators3 Balance of the questionpositivebalance versus unbalanced Position in the questionnairenegative1st versus 2nd versus 3rd position in thequestionnaire4 Communicate the final response Number of response categoriesnegative2, 3, 4 ,5, 7 and 10 categories Offering midpointspositivemidpoint versus no midpoint offered Offering Don t know filterpositivedon t know filter versus no don tknow filter Scale labelspositivelabeled versus partly labeled responsescale 2 negative: results in low reliability; positive: results in higher reliability3 1: To personal for the respondent; 2: To threatening for the respondent; 3: Rather not answer the question.

10 4:Hard to give an honest answer to the question4 Every QUESTIONNAIRE was divided in three parts, with an equal amount of questions5 The RESEARCH question is What are the effects of child characteristics and questioncharacteristics (and their interaction effects) on the reliability of RESPONSES produced bychildren in self-administered QUESTIONNAIRE RESEARCH ? The respondent characteristics as mentioned in the satisficing theory, motivation andcognitive functioning, are difficult to measure directly within a survey. Thus, most attention isdevoted to measurable characteristics of the respondent, such as age and education ( Krosnick, 1991; Andrews & Herzog, 1986; Krosnick & Alwin, 1987; Rodgers, Andrews, &Herzog, 1989). Age and education are then used as proxy variables for cognitive the known studies on response quality in survey RESEARCH we used measurable childcharacteristics, such as age and years of education, as proxy variables for cognitive addition we used sex of the children, because boys and girls develop in different phases atthe school age.


Related search queries