Quality & Quantity (2008) 42:477–490 © Springer 2006
Order Effects in Batteries of Questions
Centre for Health Service Development, University of Wollongong, Wollongong, NSW 2522
Australia. E-mail: firstname.lastname@example.org
Abstract. Batteries of questions with identical response items are commonly used in survey
research. This paper suggests that question order has the potential to cause systematic posi-
tive or negative bias on responses to all questions in a battery. Whilst question order effects
have been studied for many decades, almost no attention has been given to this topic. The
primary aim is to draw attention to this effect, to demonstrate its possible magnitude, and
to discuss a range of mechanisms through which it might occur. These include satisﬁcing,
anchoring and cooperativeness. The effect seems apparent in the results of a recent survey.
This was a survey of Emergency Department patients presenting to Wollongong Hospital
(Australia) with apparently less urgent conditions in 2004. Two samples were taken. Ques-
tion order was ﬁxed in the ﬁrst sample (n = 104; response rate RR2 = 94%), but randomised
in the second sample (n = 46; response rate RR2 = 96%). Respondents were asked to indi-
cate whether each of 18 reasons for presenting to the ED was a ‘very important reason’,
a ‘moderately important reason’, or ‘not a reason’. The mean number of very important
reasons selected was 56% higher in the ﬁrst sample as compared to the second sample.
Key words: question order effects, survey research, questionnaire, Australia, survey design,
emergency department, context effects.
One of the most efﬁcient and common formats for gathering data in a
written survey is through ‘batteries’ of questions. In particular, series of
questions are often presented with the same response options for each
question. For example, patients at a hospital Emergency Department (ED)
were recently asked to indicate whether each of 18 reasons were ‘very
important’, ‘moderately important’ or ‘not important’ for their decision to
attend the ED (Siminski et al., 2005). The questionnaire for that survey is
contained in an Appendix A.
Survey researchers have long been aware of bias associated with ques-
tion order. The mechanisms of such bias are numerous and complicated.
The most relevant theoretical contributions will be discussed below. How-
ever, almost no attention has been given to the ways that question order
may systematically bias the responses to all questions in the same direction.
That is the topic of this paper. The aim is to draw attention to this form