Quality & Quantity 38: 577–601, 2004.
© 2004 Kluwer Academic Publishers. Printed in the Netherlands.
Ordinal Scales and Fuzzy Set Systems to Measure
Agreement: An Application to the Evaluation of
, GISELLA FACCHINETTI
Università di Modena e Reggio Emilia, Dipartimento di Economia Politica, Via J. Berengario 51,
41100 Modena, Italy. E-mails: firstname.lastname@example.org, email@example.com.
Dipartimento di Scienze
Giuridiche, Via Università 4, 41100 Modena, Italy. E-mail: firstname.lastname@example.org
Abstract. Student evaluation of university teaching activity is now compulsory in Italy and a re-
search group of the Italian Ministry of Instruction, University, and Research proposed a questionnaire
with items based on the four-point Likert scale and a traditional item-by-item analysis for the eval-
uation of classrooms, work load, course organization, lectures, and teaching aids. Three split-ballot
experiments were carried out to test the differences between the four-point and ﬁve-point Likert
scale. The traditional analysis is compared with the results of the fuzzy system set up to achieve the
same purposes. The fuzzy system yielded scores that proved to be generally higher but sometimes
also lower than those obtained using the ﬁve- or four-point Likert scale. Furthermore, an extension of
standard procedures of the fuzzy system is suggested to obtain a fuzzy item-by-item analysis, thereby
increasing the possibility of their use in social sciences.
Key words: teaching evaluation, Likert scale, neutral position, fuzzy set theory
Measurement of the intensity of a concept often involves complex steps and the
outcome is a number that sometimes represents at the most, a rough ordinal level of
the attribute. Among others, two main issues involve the measurement procedure:
the characteristics of the instrument used to detect the values and the techniques
in handling survey data, both of which depend on the speciﬁc attribute or context.
For example, different versions of the Likert scale are often employed to measure
attitudes or opinions and it generates ordinal variables, while data processes involve
indices and parameters implying that the resulting scores are real numbers. The
purpose of this paper is simply to discuss these two issues as related to teaching
evaluations. First, two alternative ordinal scales are illustrated, together with a re-
view concerning their practical applications. Second, a Fuzzy System (FS), based
This study is a part of the project “Methods and technology for innovating and re-organising
teaching activity” supported by the University of Modena and Reggio Emilia.