Combining and integrating formative and summative assessment in mathematics teacher education

Combining and integrating formative and summative assessment in mathematics teacher education Contrary to the opinion that formative and summative assessment approaches are not compatible, this article presents a theoretically grounded way in which different forms of assessment can be combined and integrated in university mathemat- ics teacher education. Two mixed-assessment approaches are demonstrated through the analysis of a case study involving a practice-based seminar accompanying a school internship. First, a formative eportfolio assessment was combined with a summative panel survey to assess the learning opportunities of mathematics pre-service teachers. Second, the formative eportfolio approach was integrated with a summative oral course examination to make statements about the learning processes and learning outcomes of the pre-service teachers. Our analyses conclude that combining and integrating the two forms of assessment present the possibility of evaluating different aspects of the pre-service teachers’ perceptions of opportunities to learn. Benefits, validation aspects and limitations of the two approaches of combining and integrating assessment forms are discussed. Keywords Formative assessment · Summative assessment · Mixed assessment · Mixed methods evaluation · Eportfolio · Teacher education · School practice · Opportunities to learn 1 Introduction subject-based learning under changing social conditions”; Kaiser 2015), for example, is to improve the seminars that Pre-service teachers in Germany frequently bemoan the lack are intended to complement school internships for pre-ser- of relation between their university studies and their later vice teachers. The newly developed seminar structure takes careers in the field, claiming that the courses they attended up practical experiences and focuses on the development of did not adequately prepare them for their work with students situation-specific aspects of teaching competence. In this (Heublein et al. 2010). Therefore, in recent years, school- paper, we focus on the seminar for the mathematics pre- practical studies in different formats (e.g., internships) play service teachers. a larger role as learning opportunities in German teacher The advantages and opportunities for practical-based education. Innovative university courses designed to accom- learning approaches have been discussed in teacher educa- pany these field experiences provide pre-service teachers tion for years (Putnam and Borko 2000). However, we as with possibilities of uniting theoretical knowledge and teach- teacher educators have to ask ourselves: How can we evalu- ing practice. These courses aim to reduce the discontinuity ate the profit that pre-service teachers gain in terms of theo- between theoretical knowledge acquired at university and retical and practical expertise in their internship and in the professional experiences in the classroom (Arnold et  al. accompanying seminar? Moreover, what are the appropriate 2014; Zeichner 2010). One aim of the University of Ham- structures and tools to use in order to assess the develop- burg’s project ProfaLe (“Professional teaching to promote ment of pre-service teachers’ professional competence in such practical-based learning approaches? It is important to monitor the development of professional competence and * Nils Frederik Buchholtz pre-service teachers’ opportunities to learn when accompa- n.f.buchholtz@ils.uio.no nying pre-service teachers and supporting them on their way to becoming professional teachers. University of Oslo, PB 1099, Blindern, 0317 Oslo, Norway University of Hamburg, Hamburg, Germany Vol.:(0123456789) 1 3 716 N. F. Buchholtz et al. Teachers’ professional competence is, according to instructional strategies” (Kaiser et al. 2015, p. 374; see also; Blömeke et al. (2015), a continuum that comprises disposi- Schoenfeld 2011). In this Perception–Interpretation–Deci- tions such as cognitions and affect-motivations as well as sion-making (PID) model, it is assumed that the availability situation-specific skills and performance. In this paper, we of situation-specific skills significantly determines whether argue that corresponding to the different aspects of teach- the transformation from disposition to performance suc- ing competence, different forms of assessment are benefi- ceeds (Blömeke and Kaiser 2016). Since this model implies cial when it comes to assessing the outcomes of pre-ser- a rather holistic view of competence, Blömeke et al. (2015, vice teachers’ learning processes. In general, two different p. 8ff) also point to the challenges of an appropriate assess- approaches to assessment can be distinguished: Summative ment: Pure cognitive-analytical approaches might lack valid- assessment (SA) measures the achievement of previously ity because relevant parts of the measured constructs can be defined standards, tasks or goals, encapsulating all collected underrepresented, and pure performance-based assessments evidence up to a given point to yield either comparative might neglect the contribution of dispositional resources. or numerical ratings (Taras 2005). Formative assessment Assuming that “the whole is greater than its parts” (p. 9), (FA), on the other hand, promotes individual development Kaiser et al. (2017) recommend working with a broader by interpreting and providing feedback according to a diag- range of combined and situated assessment formats that are nostic judgment that presents information about the candi- able to cover processes mediating the transformation of dis- dates’ continuative learning processes. Unfortunately, many positions into performance. We agree that accurately and see the distinct purposes of these two forms of assessment reliably measuring teaching competence requires a mixture as incompatible (William 2010). But in our understanding of SA and FA assessment. This assessment approach can of SA and FA, these two approaches can be combined and be described methodologically as either a combination or even be integrated to assess the development of profes- an integration of the different assessment forms. In the fol- sional teaching competence, because each assesses different lowing, we clarify the difference between combination and aspects of competence. While SA considers dispositional integration and concretize how each approach addresses aspects of competence (e.g., knowledge about study content unique aspects of the professional teaching competence of and the prevalence of perceived opportunities to learn about our pre-service teachers. different study content), FA focuses in on the contextual situatedness of the pre-service teachers’ professional actions 2.1 Combining SA and FA forms and the development of situation-specific skills. Our study aimed to traverse the classical dichotomy When evaluating pre-service teacher education measures between SA and FA in mathematics teacher education. In and the outcomes of learning, SA generally must play a this paper, we demonstrate a combination of SA and FA role, since any educator assessment aims at least to certify and an integration of FA and SA in an evaluation of the out- that certain course planning and teaching skills have been comes of a practical-based seminar designed to accompany acquired. During the internship and our accompanying semi- school-internships in mathematics teacher education. nar, pre-service teachers are provided with opportunities for cognitive learning as well as for the acquisition of situa- tion-specific skills. In our case, then, a SA might therefore 2 Theoretical considerations on using cover pre-service teachers’ cognitive knowledge dispositions mixed‑assessment approaches and the theoretical learning opportunities provided by our seminar. In the frequently cited review of research on the assessment On the other hand, FA is often brought into play as a of competencies in higher education by Blömeke et  al. pragmatic option for the support of pre-service teachers in (2015), the authors model competence as a continuum that their practice. Tillema (2010) encourages assessments aimed comprises dispositions, situation-specific skills, and perfor - at improving field-related performance: mance. This model can also describe the professional com- Supporting teachers as learners requires moving petence of mathematics teachers. beyond providing mere knowledge of results to a type Dispositions contain, for example, cognitive components of functional (i.e., performance-related) feedback that like the professional knowledge teachers acquire through can articulate developments in practice, anticipate university-based learning opportunities (Shulman 1986). Sit- professional learning needs, and monitor the learner’s uation-specific skills comprise three different aspects related progress during a course of teacher action (p. 563). to practice: “(a) Perceiving particular events in an instruc- tional setting, (b) Interpreting the perceived activities in the Tillema therefore proposes to “increase the learner classroom and (c) Decision-making, either as anticipating a control over the content to be assessed as well as over response to students’ activities or as proposing alternative the criteria by which the teaching performance will be 1 3 Combining and integrating formative and summative assessment in mathematics teacher education 717 scrutinized” (p.  567). Authors such as Bell and Cowie 2.2 Integrating SA and FA forms (2001) or Carr and Claxton (2002) also write from a socio-cultural view, understanding learning as a situated According to Wiliam and Black (1996), neither FA nor SA activity; these authors point to the situatedness of what categorically exclude each other; rather, they are seen as the they call learning dispositions. Accordingly, they favour extremes of a common continuum, the core of which is the FA approaches such as observations, interviews and self- (interpretable) evidence of performance: reports like those that appear in portfolios when it comes Any assessment must elicit evidence of performance, to the assessment of performance-related competence. which is capable of being interpreted (however inval- Thus, FA “values learning dispositions and sees [pre- idly). Whether or not these interpretations and actions service teachers’] early development as consequential on satisfy the conditions for formative functions, the fact relationships between the learner and the social and mate- that interpretable evidence has been generated means rial learning environment” (p. 18). To concretize this in that the assessment can serve a summative function. our case, a FA might yield information about a pre-service Therefore, all assessments can be summative (i.e. have teacher’s practical learning opportunities during his or her the potential to serve a summative function), but only internship—in particular, the individual’s acquisition of some have the additional capability of serving forma- situation-specific skills. Further, the FA provides us as tive functions (p. 544). evaluators with information about our pre-service teach- ers’ personal experiences with practical teaching, which Wiliam sees the integration of SA and FA as a combina- we then might use, either to provide feedback, or adap- tion of the different purposes of the assessment forms on the tively for future seminar planning. widest possible basis for evidence (Wiliam 2000, p. 11). In However, this formative approach faces methodological particular, for the use of SA as FA, further feedback from challenges. Unlike SA forms, FA forms presuppose that we the assessor is necessary—based on the interpretation of the have to give up the idea of comparing the achievements of SA results and oriented toward helping the evaluee continue pre-service teachers in a standardized way, since the learn- on from assessed to desired levels of performance (see e.g., ing progress is different for each individual pre-service Sadler 1989; Taras 2005; Hattie and Timperley 2007). On teacher. FA requires evaluators to scrutinize whether all the other hand, the idea of using FA as SA is to aggregate intended educational goals or certain measures in pre-ser- the separate results of a set of assessments designed to serve vice teacher education have been achieved. Because the formative purposes in order to get a comprehensive picture identification of evidence of performance in FA is difficult, of overall achievements (Harlen and James 1997). so far, this has been a critical issue when FA is used in The basic idea of an integration of SA and FA is therefore teacher education or professional development contexts to assess one phenomenon with different assessment forms, (Delandshere and Arens 2003). Accordingly, how mathe- for example by using small units of SA in a formative way matics (pre-service) teacher education can benefit from FA in the course of giving feedback, and integrating this FA is so far not described very well (e.g., Spanneberg 2009). into an overall SA afterwards. For our case, this means that Against this background, the basic idea of using a we are able to focus at the same time on how the pre-service combination of SA and FA is to extend the scope of the teachers develop professional teaching competence and what assessment, i.e., to both widen and deepen the assessment outcomes of the seminar and the internship they achieve. by gathering as much information as possible on multiple In particular, the FA during the internship is composed of components of the assessed phenomenon or different, but predetermined tasks affording particular situation-specific closely related phenomena. Assessment results should skills (described in Sect. 3.3), and we as lecturers provide ideally complement each other to increase the overall constructive feedback on those tasks, helping the pre-service interpretability of assessment results. For our case, the teachers to increase their teaching competence. The overall combination of SA and FA holistically addresses learning SA of our seminar—an oral examination conducted after- opportunities to increase teaching competence in order to wards—individually addresses the situation-specific skills gain information about the development of teaching com- that the pre-service teachers have acquired during their petence in the internship and the seminar. Additionally, internship. The aim behind this procedure is to increase the in favour of an increased interpretability of assessment autonomy of the pre-service teachers concerning their cer- results, we seek a mutual complementation of the find- tification of teaching competence on the one hand, while on ings from a standardized SA with the findings from an the other hand increasing the validity of each assessment of individual FA (e.g., when identifying evidence for learning situation-specific skills through the mutual corroboration of opportunities in FA) and seek ways that we might partly the results of FA and SA. overcome the challenge posed by the inability to standard- This approach is of course also challenging, since we ize FA in teacher education. as lecturers both enable learning processes and certify the 1 3 718 N. F. Buchholtz et al. outcomes of the seminar and the internship. Shavelson structure of nearly all sessions of the seminar was based (2006) highlights this “double bond” of the assessors of on the PID model (see Sect. 2). Initial points for analyses educational processes: were, for example, observations of the pre-service teachers or videotaped classroom situations. A specic fi emphasis was Significant tensions are created when the same person, placed on the heterogeneity of students and how teachers namely the teacher, is required to fulfill both forma- dealt with it. tive and summative functions. Teachers at the interface Summing up, different forms of opportunities to learn of formative and summative assessment … confront were provided: First, the content of different seminar lessons conflict daily as they gather information on student provided opportunities to learn in the form of knowledge performance—to help students close the gap between acquisition. With the help of this knowledge, the pre-service what they know/can do and what they need to know/ teachers were able to develop a certain teaching competence be able do on the one hand, and to evaluate students’ disposition. Second, observation tasks that the pre-service performance for the purpose of grading/certification teachers had to carry out within their once-weekly internship on the other hand (p. 8). during the semester provided another form of opportunity to This double bind can only be circumvented with regard learn: in this case, to perceive and interpret teacher behav- to the integration of FA and SA by not using or interpreting iour. By doing so, they were asked to reflect on necessary the same evidence for both purposes of assessment (Wil- teacher skills in certain situations. Third, within the oral iam 2000). Within the seminar, we therefore differentiated examination of the seminar, the pre-service teachers were clearly between learning situations (internship, seminar prompted to reflect on their own situation-specific skills sessions and observation tasks) and achievement situations concerning a self-chosen topic, focusing especially on the (oral examination) and made this clear to the pre-service development of these skills during the internship. Thus, teachers as well. the oral examination also reflected opportunities to learn. Whereas the observation tasks involved reflecting on the skills of others, in the oral examination, the evaluees were 3 SA and FA in the ProfaLe seminar asked to reflect on their own skills. and corresponding research questions In summer 2016 these innovations were implemented in the seminar for the first time in a group of 30 pre-service The aim of the ProfaLe seminar was to develop the profes- teachers (7 males, 23 females). Every pair of students was sional teaching competencies of pre-service teachers and to accompanied by a practicing teacher at their schools once strengthen school-practical aspects, in order to reduce the per week during the semester and every day during a practi- common feeling of discontinuity between theoretical knowl- cal phase in the semester break for 5 full weeks (Orschulik edge acquired at university and professional experiences in 2016). the field. 3.2 Implementation of SA 3.1 Structure of the seminar A special opportunity to integrate SA into the seminar The seminar was implemented across two periods of pre- was an interdisciplinary evaluation panel survey by the service teachers’ practical experiences during their master’s project ProfaLe that focused on the pre-service teachers’ studies. The first period was a full semester during which professional teaching knowledge as well as on their uni- they split into pairs and spent 1 day per week in a school and versity and school-practical learning opportunities (Doll 2 h weekly in a university seminar covering different topics et al. 2018). The survey draws on established instruments related to mathematical teaching. The second period, which from the interdisciplinary study TEDS-LT (Blömeke was held over the semester break, involved the completion et al. 2013; Buchholtz et al. 2016), a German follow-up to of a 5-week internship, during which they—in the same the international TEDS-M study released in 2008 (Tatto pairs and schools—spent every day in school. Throughout et al. 2012), and thus allows a standardized assessment of both periods, the pre-service teachers observed and accom- pre-service teachers’ professional knowledge and respec- panied experienced teachers and even had to teach small tive learning opportunities. University and school-prac- units by themselves, supervised by the teachers and once tical learning opportunities—on which we focus in this by the university lecturers. In the seminar, the pre-service paper—were surveyed in the form of an inquiry into per- teachers learned how to focus their attention on important ceived study contents, and thus they reflect dispositional pedagogical classroom events that promoted their ability aspects of teaching competence. The pre-service teachers to perceive important classroom situations, interpret them, were asked to indicate whether they had studied various and decide how they should be managed. Therefore, the content in their previous studies. The surveyed university 1 3 Combining and integrating formative and summative assessment in mathematics teacher education 719 learning opportunities refer to six different subject-specific (graphical or textual) mapping their reflections about the topics of mathematics education with a particular focus chosen topic. on aspects from the ProfaLe measures (see Table 1). The assessment was summative because the pre-service teach- 3.3 The implementation of FA ers were asked to recall a variety of different learning opportunities in the past and up to the point of the survey. The FA of the pre-service teachers’ professional teaching Further details on the survey instruments are given in the competencies was implemented in the seminar in a variety paper by Doll et al. (2018). of ways, primarily through an electronic portfolio (eport- The panel survey was conducted online and was open to folio) and corresponding feedback, as well as through the all pre-service teachers of mathematics at the University of self-responsible design of the oral examination presented Hamburg. In total, 187 pre-service mathematics teachers above. A crucial part of the eportfolios was a set of observa- took part. Participation in the online survey was voluntary tion tasks about which the pre-service teachers had to write. on the basis of an honorarium and was anonymous due to Generally, eportfolios can be used in different ways: The data privacy regulations. However, since the online survey feedback on the written observations given by university also asked about respondents’ participation in specific uni- teachers can be used by the pre-service teachers for their versity courses such as the ProfaLe seminar, we did obtain further learning processes as well as by their university data concerning the participants of our ProfaLe seminar teachers for designing appropriate further teaching (Hattie from the survey. In total, 13 of the 30 pre-service teachers and Timperley 2007). The use of eportfolios in the field of from the ProfaLe seminar participated in the online survey. teacher education in general can be of great benefit for pro- A further form of SA for assessing teaching compe- fessional development, as different case studies on the use tence in the seminar was the oral examination at the end of electronic blended learning in teacher education show of the semester. It was administered as a presentation- (Mackey 2009; Gikandi et  al. 2011; Vogel 2018). They and-reflection examination designed to reveal the out- are especially helpful when they are used in tandem with comes of pre-service teachers’ learning processes both school practice; when they provide authentic practice-related during their school internships and in the seminar. This assessment activities (e.g., realistic classroom situations); was accomplished by presenting a discussion about a self- and when they receive effective and prompt formative feed- chosen practical teaching example and included selecting back (Gikandi et al. 2011, p. 2338). Still, although the use important experiences from their own teaching activities of online FA tools like eportfolios has increased in the last or observations and linking them with the mathematics decade (Jafari and Kaufman 2006), there are also methodo- educational theory taught in the seminar. This way the logical problems related to the possibility that portfolios may pre-service teachers could stress relevant aspects within create inference for performance (Delandshere and Arens a 15-min presentation. But the oral examination did not 2003). serve only summative purposes; it also integrated forma- We decided to use an eportfolio approach to focus on the tive aspects because the pre-service teachers had to choose development of situation-specific skills of teaching compe- the topic of their oral examination by themselves, based tence. The pre-service teachers received in total 18 situa- on their written reflections from their eportfolio. Further - tional-observation tasks, which they had to address during more, the examination had to center on a specific situ- their observations of other teachers or their own teaching ational problem encountered during the internship. Prior to in school, and to work on as a written contribution to their the examination, the pre-service teachers had to hand out personal eportfolio. a mind map to the lecturers, meaning a visual presentation In the seminar, three different forms of observation tasks were used: Table 1 Opportunities to learn in mathematics teacher education Dimension Content Example Items Reliabil- Cronbach’s α ity (Rho) Mathematics education Basics of mathematics education Didactics of algebra 20 0.86 0.84 Dealing with heterogeneity Language in mathematics education 8 0.62 0.62 ICT in mathematics education Learning with apps 6 0.66 0.56 Methods of instruction Problem-solving approaches 8 0.76 0.75 Curriculum National curricula 8 0.70 0.67 Research on mathematics education Role of beliefs in mathematics learning 9 0.72 0.68 1 3 720 N. F. Buchholtz et al. 1. Six tasks in advance of a forthcoming lecture, in which also use adaptive planning to incorporate those observations the pre-service teachers had to describe and partly inter- into upcoming lectures. While the school internship took pret classroom situations. For example: “Describe when place over the semester break, the pre-service teachers had and how the teacher took up errors from the students in to choose observation tasks by themselves. In this way, the the classroom. What was done with these errors? What pre-service teachers could use their entries in the eportfolio kind of problems occurred in these teaching situations?” as a preparation for their individual oral examination (which 2. Seven subsequent tasks as a follow up of a lecture, in then served summative purposes). Thus, we were able to which the pre-service teachers not only had to describe avoid the conflict of overregulating the pre-service teachers but had to focus more on interpretation and developing through fixed observation tasks, and instead allow them to alternatives to act. For example: “Choose one of the het- develop their own sense of interpretation about their learn- erogeneity aspects that you want to look at more closely ing development—another step in this process during which in your observations. Describe the students according we were able to witness the adaptivity of FA (Tillema 2010). to this aspect, and how this aspect has influenced meas- ures of differentiation in the classroom. Evaluate what 3.4 Research questions kinds of measures were more and less beneficial for the students.” To identify the benefits of the combination and integration 3. Five tasks of students’ own choice of observation focus, of SA and FA in the case of the ProfaLe project of math- in preparation for the oral examination. ematics teacher education, we focus on the empirical results of the different assessment forms guided by two research Formulating the observation tasks required the pre-ser- questions: vice teachers to focus on one specific topic—e.g., dealing with errors—following the structure of the PID model. Thus, 1. What forms of evidence of pre-service teachers’ profes- pre-service teachers had to describe the perceived situations, sional competence and learning opportunities can we interpret them, and sometimes were prompted to develop identify from combining SA (panel survey) with FA useful responses to them. (eportfolios)? All written contributions were read by the seminar lectur- 2. Which complementary forms of evidence can we iden- ers over the following week and commented on within the tify when we integrate SA with FA in the oral examina- personal eportfolio, which only the lecturers had access to. tion? However, the written contributions were compulsory for the seminar and thus were extrinsically motivated reflections. To answer these questions, we first describe in the meth- (This duality depicts the difficult relationship between het- odological considerations how we analyzed our data, within eronomy and autonomy within this assessment format; See the scope of this paper. In the Sects. 5 and 6, we illustrate Sect. 2.2). As the reflections represent the learning processes and discuss the results of the combination and the integra- of the pre-service teachers, no rating or grading took place. tion of the different forms of assessment. The seminar lecturers provided feedback constructively and showed possibilities for the pre-service teachers to improve their instructional quality in the described situations, accord- 4 Methods for data analysis ing to the criteria already discussed in the seminar. By rec- ognizing the benefit of the feedback for the pre-service 4.1 Methods of analyzing SA data teachers’ own professional development, we hoped for a high acceptance of the feedback (Topping 1998). Of the 30 pre-service teachers in our seminar, 13 took part in The reflections in the eportfolio aimed at self-assessment the online survey, which allowed us to analyze their profes- and raising awareness of the writers’ own situation-specific sional knowledge and their university- and field-based learn- skills, so that explicit situation-specific learning opportuni- ing opportunities as a measure for dispositional teaching ties and learning processes could be reflected upon in detail competence (cf. Sect. 3.2). To analyze the learning opportu- through the given feedback. In this way, the pre-service nities, a Mokken analysis (Mokken and Lewis 1982), based teachers were able to use the feedback from the seminar lec- on the whole sample of the online survey (N = 187) was turers to optimize their observations in the subsequent weeks used. This non-parametric item response model allows us as well as to improve their own teaching (Vogel 2018). How- to check the assumption that the learning opportunities of ever, we as seminar lecturers also were able to benefit from different dimensions can be described hierarchically along this form of FA: We could not only examine the results of an ordinal scale, so that the percentage of perceived content the pre-service teachers’ reflections on the teaching prac- can be interpreted as individual progress along a study path- tices they observed and their own teaching but we could way. In Table 1, the reliability-corrected rank correlations 1 3 Combining and integrating formative and summative assessment in mathematics teacher education 721 (Spearman’s Rho) of the six dimensions, as well as Cron- was modified so that the three situation-specific skills could bach’s α, are reported. The scales showed a satisfactory reli- be identified, whereby the situation-specific skill of percep- ability in the range of 0.62–0.86. Since an analysis on the tion could be coded by utterances at the descriptive level. individual level was not possible due to data regulations, We changed the codes belonging to the category “Topic” for our analysis, the group of the 13 pre-service teachers into those that were topics of the seminar. By doing so, we from the ProfaLe Seminar was compared to a comparative were able to analyze to what extent these topics provided group of 28 pre-service mathematics teachers stemming opportunities to learn in the form of knowledge acquisition, from the whole sample. This group of pre-service teach- and sometimes of situation-specific skills. An example for a ers were enrolled in mathematics education courses at the coding process can be found in Fig. 1 in Sect. 5. same University in the same phase of study, but they did not In addition, new indicators for learning opportunities participate in any teacher education courses affected by the emerged inductively from the eportfolios, according to ProfaLe project. We used the non-parametric Mann–Whit- which the extracts also could be rated, e.g., descriptions ney U test for the comparison of medians and interquartile about the development of the personality as a teacher or the ranges of the perceived learning opportunities. professional relationship between the pre-service teachers The SA of the oral examination was carried out according and their mentors. The collection and classification of these to the following overarching criteria, which were commu- extracts showed the overall heterogeneity of the learning nicated to all pre-service teachers beforehand, in order to opportunities and situation-specific skills of the pre-service provide the possibility of comparability of the performance: teachers. Therefore, in the following section, we showcase some illustrative examples from pre-service teachers who The observations and own experiences were selected rea- have given us permission to present their work. sonably and are meaningful; An appropriate selection of theories of mathematics edu- cation has been chosen;5 Results The observations and their own experiences are compre- hensibly presented, and analyzed and evaluated with the To answer our research questions, we first present results aid of theory; of the combination of SA and FA, then the results of the Changes and learning processes are taken up; integration of SA and FA. Subsequently in the Sect. 6, we Current and relevant, in particular didactic and scientific discuss the results on a more theoretical level and also reflect literature is included; on the limitations of our approach. The proper use of scientific terminology is employed; The presentation is self-reliant, adequately prepared and 5.1 Results of the combination of SA and FA structured; Questions from the lecturers can be answered; With the results from the online survey, we can arrive at Links to different topics of the seminar can be drawn. the number of perceived university learning opportunities concerning specific study content, and thus create an indi- Each oral examination was rated by two university teach- cator for professional competence. However, we are able ers according to the criteria listed above, directly following to analyze the outcomes of learning opportunities only on its conclusion. All examinations were logged. the group level. The findings are therefore complemented and deepened by those from the FA on the individual level, 4.2 Methods of analyzing FA data which are also used to make conclusions about the applica- tion of the study contents with regard to the development of We subjected all eportfolios to a systematic process of anal- situation-specific skills. Table  3 describes the results of the ysis following suggestions from Delandshere and Arens Mann–Whitney U test for the comparison of medians and (2003) about identifying evidence of performance from port- interquartile ranges (IQRs) of the online survey. folios. We therefore combined a deductive and an inductive With regard to the perceived study contents, we find sig- approach introduced by Mayring (2015) in order to be as nificant differences from the comparison group. Thus, the open as possible for the analysis. We coded extracts from the group of pre-service teachers from our seminar felt they eportfolios according to the learning opportunities provided were presented with a significantly higher number of learn- by the ProfaLe seminar and indicators of situation-specific ing opportunities in the areas of dealing with heterogeneity skills in teaching competence that the pre-service teachers and research on mathematics education. For example, the described in their observation tasks. The deductively cre- median of 0.38 in dealing with heterogeneity means that, on ated part of the coding system (see Table 2) is based on the average, pre-service teachers from our seminar stated that work of Sherin and Van Es (2009). The category “Stance” they had previously encountered learning opportunities for 1 3 722 N. F. Buchholtz et al. Table 2 Deductive part of the Coding categories Coding coding system Actor Student Teacher Other Topic MC 1—mathematical argumentation MC 2—mathematical problem solving MC 3—mathematical modelling MC 4—using mathematical representa- tions MC 5—operate with symbolic, formal and technical elements in mathematics MC 6—Communicating Dealing with errors Mathematical concepts (“Grundvorstel- lungen”) Changing and connecting representations Language awareness in learning math- ematics Cognitive activation Problem orientation Teacher intervention Differentiation and individualized instruc- tion Form of teaching—student-centered Form of teaching—teacher-centered Classroom management Stance Describe (Situation-specific skills) Interpret Decide MC Mathematical competency three of the eight subthemes on dealing with heterogeneity When describing her perceptions concerning the subject (e.g., dyscalculia, giftedness). Despite the lack of signifi- matter of the observation task, Jennifer focuses on dealing cant differences in other dimensions, we recognized the high with errors. It becomes very clear that Jennifer observed the perception of study content related to “methods of instruc- teaching and the classroom discourse very closely, focusing tion” and the comparatively low perception of content of on both students and teacher. Her selection regarding this “Information and communication tools (ICT) in mathemat- situation involves a nice example of an occurring miscon- ics education,” which could be expected from the seminar. ception of inverse proportionality. Although not required, These findings reflect the measures taken by the ProfaLe she interprets her perceptions, taking up other topics that project, since dealing with heterogeneity and instructional were subjects of previous discussions in the seminar. A methods constituted a major focus in the seminar (whereas closer analysis of the “Interpret” and “Decide” segments the use of ICT was not in focus). In the following section, shows that Jennifer does not explain the occurring error. we combine these SA results with complementary individual However, since this was a preliminary observation task, results from the FA. Therefore, we concentrate on how the it might not yet have been possible for Jennifer to rely on seminar and the internship offered learning opportunities for broad knowledge for error analysis. Furthermore, no error the development of situation-specific skills of the pre-service interpretation was expected according to the formulation of teachers, especially concerning dealing with heterogeneity. the observation task. Nevertheless, we can recognize that A preparatory eportfolio entry on the topic of dealing she already considers the teacher’s handling of the mistake with errors from Jennifer (Fig.  1), a female pre-service as inappropriate and mentions the importance of students’ teacher (see Sect. 3.3), provides an illustrative example of understanding. This reveals a high degree of pedagogical an individual FA result. It is about observing the behaviour reflection on the learning of concepts in mathematics educa- of a teacher dealing with an occurring misconception. tion, indicating her situation-specific skills when addressing 1 3 Combining and integrating formative and summative assessment in mathematics teacher education 723 Fig. 1 ePortfolio entry from Jennifer and respective coding (translated by the authors) Table 3 Median differences in perceived study contents Group n Basics of MEd Dealing with ICT in MEd Methods of instruction Curriculum Research on MEd heterogeneity ProfaLe seminar 13 0.55 (0.30) 0.38* (0.25) 0.33 (0.42) 0.75 (0.19) 0.50 (0.25) 0.33* (0.17) Comparative group 28 0.58 (0.24) 0.25 (0.22) 0.33 (0.33) 0.75 (0.25) 0.43 (0.38) 0.22 (0.11) *p < 0.05, IQRs in brackets misconceptions, and that might reflect her theoretical knowl- pre-service teachers were used to discuss occurring errors edge in this field. of students on the basis of concrete examples and to con- Following the idea of FA, these and observations from sider how teachers can deal with misconceptions in the other pre-service teachers were used by the lecturers to classroom. Thereby our aim was to accumulate the practi- adapt teaching and learning in the course. In the next cal expertise of the pre-service teachers adaptively—as seminar session, observations and interpretations of the 1 3 724 N. F. Buchholtz et al. intended in FA—and to develop guidelines for handling ‘asterisk tasks’ as well for the children that finished mistakes in the classroom together. very quickly. These tasks do not have to be done by In summary, the online survey provided us with general all children. On the whole, I find this works out quite information about pre-service teachers’ perceptions of their well. As a result of the learning time, even the slower own opportunities to learn, while the reflections from the children have the chance to work things off. I would observation tasks provided a good learning opportunity for find it great, however, if content would be offered that training in situation-specific skills. This was the case not cognitively demands more of the faster children and only when the pre-service teachers had the opportunity to leads them in some places to get deeper into a topic, observe examples of good teaching practice but also when to making connections, etc. I have the feeling there is they observed examples that called for improvement. not so much space to explore mathematical creativ- With our coding approach in the scope of this paper, we ity. Possibly open task formats could be introduced were able to make conclusions about the skills developed so for children who already finished. (I have not yet con- far. However, our coding system requires closer examination sidered two children with learning disabilities in the if we are to make statements about the quality and appro- class. I know that it is hardly possible for a teacher priateness of the interpretations and decision-making. Nev- to provide additional meaningfully selected learning ertheless, when looking at individual cases, it is possible to material which is connected to the current teaching, show how perceived study contents are linked to other study but these two children are just employed to utilize the contents and are available not only at the knowledge level ‘leftovers.’ I do not see any substantive thread there; but are also applied in describing, interpreting and decision- no fostering plans have been consulted. I doubt that the making. In this way, the results of the SA can be deepened math teacher knows what is in it. I think these children and partially explained. should also try to follow the current teaching, at a dif- ferent level and at a different pace.) (Leonie, translated 5.2 Results of the integration of SA and FA by the authors). Leonie describes the teacher’s measures in a very detailed Regarding the overall performance in the seminar, 28 of the way, taking an appraising attitude. The entry shows her 30 pre-service teachers took the oral examination at the end development of situation-specific skills. It becomes clear of the semester. The average grade was 1.5 and the grades that she perceives the situation in her observation in a holis- ranged from 1.0 to 2.7 [grades may differ from 1.0 (best) tic manner. The questioning of the cognitive activation to 4.0 (worst)]. The topics of the oral examinations varied. and openness of the tasks shows that her situation-specific There were examinations about the handling of errors in interpretations are to some extent also based on professional the classroom, about certain measures of differentiation knowledge. Furthermore, she questions the teacher’s actions (e.g., circuit learning), cooperative learning in mathematics against the background of the increased need for differentia- education or specific aspects of teaching methods, such as tion, especially in dealing with children with special needs. classroom discourse. Although not required, she also develops alternative sugges- To illustrate how FA affected the topics in the SA for indi- tions for differentiation measures, which are characterized vidual pre-service teachers, the following describes Leonie, by ideas of open- and closed-differentiation formats as well a female pre-service teacher. One of her eportfolio entries as by collaborative learning on a common object, that take relates to the topic “measures of differentiation in mathemat- the high-performing students as well as the two students ics education” (see Example, Sect. 3) and describes how with learning disabilities into account. Overall, her descrip- the teacher Leonie observed dealt with the heterogeneity tions of the observations indicate a high degree of situation- of the class. specific skills including indications that she is already able In the class the children work with a very different to make informed decisions. speed. About 3 or 4 students work very fast and usu- During the semester, the observation tasks in the eport- ally are also correct. There is a larger mass of students folios were not graded; rather, they served as learning working ‘mediocre fast’ and some who are very slow. opportunities. Accordingly, we gave the pre-service teach- (All get the same tasks, and all have to work on them at ers learning feedback following the idea of FA. At the end the same time.) Mondays, however, there is a learning of the semester, however, the oral examination is a graded time, which forces the children to work independently certification exam that summatively assesses the pre-service on either the subjects German, English or Mathemat- teachers’ professional teaching competence. Since the topic ics in order to complete and/or practice tasks/content of the oral examination could be determined independently from the lessons. The students can use this time how- by the pre-service teachers and based on their own experi- ever they choose. Apart from that, there are sometimes ences, the oral examination also took up aspects of FA. The 1 3 Combining and integrating formative and summative assessment in mathematics teacher education 725 integration of SA and FA therefore coalesces in the sum- her observations. In preparation for the oral examination, she mative assessment of formative performance components deepened her knowledge and was able to relate her experi- (which, however, now are evaluated and graded in the exam- ences to the—however implicit—theoretical background of ination according to the criteria mentioned in Sect. 4.1). the topic by dealing with legal requirements and finding fur - We are able to integrate the findings from this kind of SA ther possibilities for die ff rentiation measures stemming from and FA on an individual basis. Identical situation-specific research on mathematics education such as, for example, aspects of the pre-service teachers’ professional competence tasks for natural differentiation (Scherer and Krauthausen can be found in their eportfolio entries, and in the topics 2010). She also describes experiences from her own lessons, for the oral examination on which the pre-service teachers in which she used several differentiation measures to deal elaborated in the mind maps, indicating that the examination with the heterogeneity of the learning group—a demonstra- assessed teaching competence that the pre-service teachers tion of her situation-specific skills. It is also important to learned during the seminar and their internship. note that she critically reflects the limitations of differen- One example of how the FA influenced the SA in that way tiation measures based on her own and observed teaching appeared in Leonie’s oral examination. In the course of her practice. experiences during the internship, she continued to deepen the topic of measures for differentiation and individualiza- tion and chose this as the topic of her oral examination. The 6 Conclusions mind map she submitted clearly illustrates the questions and problems she addressed (see Fig. 2). 6.1 Combining and integrating FA and SA By integrating the results of the analysis of her eportfolio entries (recall Sect. 5.2) with her mind map, we can see that By combining different forms of assessment in the she did not stop at developing a critical attitude according to framework of a mixed-assessment approach, we hoped to Fig. 2 Leonie’s mind map (translated by the authors) 1 3 726 N. F. Buchholtz et al. determine the best possible evidence of pre-service teach- 6.2 Addressing validity with mixed‑assessment ers’ learning opportunities and situation-specific skills. approaches With SA (online survey) data, we were able to determine comparatively that the pre-service teachers of the semi- To understand the importance of distinguishing between a nar had significantly higher perceptions of the number combination and an integration of SA and FA in terms of the of learning opportunities regarding heterogeneity and validity of the approach, it is necessary to take our assess- research in mathematics education that were available to ment to a broader theoretical level and analyze approaches to them than were held by comparable pre-service teach- creating validity in qualitative or mixed-methods evaluation ers who did not take the seminar. However, these results approaches (see also Frechtling and Sharp 1997; Gikandi alone are not a sufficient measure for the dispositional et al. 2011). Since different methods entail different weak - aspects of the pre-service teachers’ professional compe- nesses and strengths, Denzin opted for a “methodological tence since it is not clear whether collective differences triangulation”, which consists of a “complex process of play- could be attributed to enrolment in our seminar. We there- ing each method off against the other so as to maximize the fore used the data from the FA, which links the individual validity of field efforts” (Denzin 1978, p. 304). But as in learning opportunities of the pre-service teachers with assessment, the attempt to triangulate different measures can specific situational aspects of the seminar and the school lead to validity problems, because “dier ff ent methods … can internship. In this combination, the FA makes it possi- relate to different empirical phenomena and … it thus may ble to make detailed statements about the types of learn- be difficult to simply compare research results acquired by ing opportunities the pre-service teachers had and about means of different methods in order to check their validity” how their learning processes differed. Both assessment (Kelle and Buchholtz 2015, p. 332). forms provide us with indicators of the specific aspects of This pragmatic view—which is also leading the debate teaching competence of the seminar’s pre-service teach- on mixed-methods evaluations, e.g., in health science (John- ers, thus providing a more comprehensive picture of their son and Schoonenboom 2016)—can stimulate an alternative professional development. understanding of triangulation that can also be transferred to While the combination of SA and FA focuses more on the context of assessment in teacher education or education the mutual complementarity of assessment outcomes on in general. The use of different forms of assessment to elicit different aspects of a phenomenon (in our case disposi- and interpret evidence of performance can be compared with tional and situational learning opportunities), the integra- the examination of a physical object from two different view - tion of SA and FA is more likely to provide convergent points or angles. Both viewpoints provide different pictures findings about a phenomenon from different perspectives. of this object, which may or may not be useful to validate In the eportfolios, we could identify pre-service teach- each other, but regardless may yield a fuller and more com- ers’ situation-specific skills; these skills also formed the plete picture of the object if brought together. So, if SA and basis of their performance in the oral examination, so FA are mixed in any way, the following validity-related out- that the SA validly ref lected the actual learning processes comes can arise (see also Kelle and Buchholtz 2015, p. 333): the pre-service teachers had. But the integration of SA and FA in the oral examination also provided a compa- • The elicited items of evidence of performance converge, rable framework for the assessment and certification of • The elicited items of evidence relate to different aspects teaching competence for all pre-service teachers, regard- of the performance, but are complementary to each other less of the learning processes that took place during the and thus can be used to supplement each other, internship. The situation-specific skills assessed in the • The elicited items that are evidence of performance are FA could be certified by using SA criteria specified by divergent or contradictory, or the examination. By this means, a central challenge of FA • The elicited items of evidence refer to unrelated phenom- in teacher education—namely the inability to standard- ena. ize specific marks of evidence of performance—can be addressed. Furthermore, the integrated approach allowed For multiple reasons, then, it makes sense to combine or not only situation-specific skills to be assessed but also to integrate SA and FA within a mixed-assessment approach, knowledge components, so that professional teaching whether to increase or enhance the validity of the evalu- competence could be assessed with more validity. Thus, ation (Johnson and Schoonenboom 2016). An integration the two assessment approaches developed here—combi- of SA and FA may lead to convergent evidence and thus nation and integration—address different facets of the to valid interpretations, or divergent evidence and validity validity of assessments, which we discuss further below. problems. We encountered these patterns in our analysis of the outcomes of a seminar in teacher education when we integrated the pre-service teachers’ reflections on their field 1 3 Combining and integrating formative and summative assessment in mathematics teacher education 727 observations from the FA with their SA-oriented oral exami- to any form of assessment. Furthermore, our combination nations. In particular, we analyzed how their reflections of FA and SA does not allow any mutual validation of the influenced the determination and the preparation of the topic results of the assessments. Integration of the results of the of their oral examination. We identified convergent evidence, learning opportunities to strengthen the validity of results which indicated that the SA really did assess teaching com- from SA (online survey) and FA (eportfolios)—as is usually petence that was gained during the seminar and the intern- intended in mixed-method approaches—would have been ship, and thus validly assessed teaching competence. When accomplished only if the results of the FA could have been assessing a combination of SA and FA, different aspects of interpreted directly against the background of the results of the performance may yield complementary evidence of per- the SA. For this to happen, participation in the panel survey formance, or it may yield unrelated evidence. In our case, an would have to be made obligatory and non-anonymous, but online survey about pre-service teachers’ perceived learning we refrained from that due to ethical and data-protection opportunities revealed an overview about perceived study reasons. contents on the group level. We could deepen these aggre- Acknowledgements We thank Prof. Dr. Jörg Doll for providing the gated findings and partially explain them with individual data from the ProfaLe online survey. findings from the FA despite the fact that the online sur - vey and the eportfolios focused on different kinds of learn- Open Access This article is distributed under the terms of the Crea- ing opportunities. The analysis of the eportfolios revealed tive Commons Attribution 4.0 International License (http://creat iveco mmons.or g/licenses/b y/4.0/), which permits unrestricted use, distribu- complementary evidence with regard to perceived learning tion, and reproduction in any medium, provided you give appropriate opportunities and how the study contents were related and credit to the original author(s) and the source, provide a link to the applied in pre-service teachers’ teaching practices. Creative Commons license, and indicate if changes were made. Whatever way SA and FA are mixed, the main rationale behind the use of different forms of assessment is always the attempt to compensate for limitations of one form of assessment by drawing on the strengths of another form (see References also Kelle and Buchholtz 2015). The decisive factor here is whether and how SA and FA can be related to each other (or Arnold, K. H., Gröschner, A., & Hascher, T. (Eds.). (2014). Pedagogi- cal field experiences in teacher education: Theoretical founda- can be mixed) and what type of evidence of performance the tions, programmes, processes, and effects. Münster: Waxmann. respective use of an assessment form produces. The form of Bell, B., & Cowie, B. (2001). Formative assessment and science educa- the mixing needs to be described and justified based on the tion. Dordrecht: Kluwer. individual case. Blömeke, S., Bremerich-Vos, A., Kaiser, G., Nold, G., Haudeck, H., Keßler, J.-U., & Schwippert, K. (Eds.). (2013). Professionelle Kompetenzen im Studienverlauf: Weitere Ergebnisse zur Deutsch-, 6.3 Limitations of our approach Englisch- und Mathematiklehrerausbildung aus TEDS-LT. Mün- ster: Waxmann. There are certain limitations in our approach that we have Blömeke, S., Gustafsson, J. E., & Shavelson, R. (2015). Beyond dichotomies: Competence viewed as a continuum. Zeitschrift für to take into consideration when reflecting on the benefits Psychologie, 223(1), 3–13. and challenges of a combination or an integration of SA Blömeke, S., & Kaiser, G. (2016). Understanding the development of and FA. In the combination approach, the indicators of teachers’ professional competencies as personally, situationally learning opportunities are different because of the differ - and socially determined. In D. J. Clandinin & J. Husu (Eds.), International handbook of research on teacher education (Chap. ent assessment forms. An example of this aspect is how 52). Thousand Oaks: Sage. the topic heterogeneity influenced the development of Buchholtz, N., Scheiner, T., Döhrmann, M., Suhl, U., Kaiser, G., teaching competence. While the results of the SA provided & Blömeke, S. (2016). TEDS-shortM. Kurzfassung der math- aggregated but standardized data (e.g., about the perceived ematischen und mathematikdidaktischen Testinstrumente aus TEDS-M, TEDS-LT und TEDS-Telekom. Hamburg: University of learning opportunities with regard to heterogeneity), the Hamburg. analysis of the FA data is conducted on an individual basis Carr, M., & Claxton, G. (2002). Tracking the development of learn- and is still subject to subjective interpretations despite our ing dispositions. Assessment in Education: Principles, Policy and methodological approach (Delandshere and Arens 2003). Practice, 9(1), 9–37. Delandshere, G., & Arens, S. A. (2003). Examining the quality of the Even if the topic heterogeneity was a substantive part of the evidence in pre-service teacher portfolios. Journal of Teacher observation tasks, we were not able to reconstruct in every Education, 54(1), 57–73. eportfolio entry how this topic affected the development of Denzin, N. K. (1978). The research act: A theoretical introduction to pre-service teachers’ professional competence. The useful- sociological methods. New York: McGraw-Hill. Doll, J., Buchholtz, N., Kaiser, G., König, J., & Bremerich-Vos, A. ness of a mixed-assessment approach can in general only (2018). Nutzungsverläufe für fachdidaktische Studieninhalte der be estimated based on individual cases and with regard to Fächer Deutsch, Englisch und Mathematik im Lehramtsstudium: the assessment purposes. However, this limitation applies 1 3 728 N. F. Buchholtz et al. Die Bedeutung der Lehrämter und der Zusammenhang mit Putnam, R. T., & Borko, H. (2000). What do new views of knowledge Lehrinnovationen. Zeitschrift für Pädagogik. (in press). and thinking have to say about research on teacher learning? Edu- Frechtling, J., & Sharp, L. (Eds.). (1997). User-friendly handbook for cational Researcher, 29(1), 4–15. mixed method evaluations. Arlington: National Science Founda- Sadler, D. R. (1989). Formative assessment and the design of instruc- tion, Division of Research, Evaluation, and Communication. tional systems. Instructional Science, 18, 145–165. Gikandi, J. W., Morrow, D., & Davis, N. E. (2011). Online formative Scherer, P., & Krauthausen, G. (2010). Natural differentiation in math- assessment in higher education: A review of the literature. Com- ematics—The NaDiMa project. Panama Post, 3, 14–16. puter and Education, 57(1), 2333–2351. Schoenfeld, A. H. (2011). How we think: A theory of goal-oriented Harlen, W., & James, M. (1997). Assessment and learning: Differences decision making and its educational applications. New York: and relationships between formative and summative assessment. Routledge. Assessment in Education: Principles, Policy and Practice, 4(3), Shavelson, R. J. (2006). On the integration of formative assessment in 365–379. teaching and learning with implications for teacher education. In Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Paper prepared for the Stanford Education Assessment Laboratory Educational Research, 77(1), 81–112. and the University of Hawaii Curriculum Research and Develop- Heublein, U., Hutzsch, C., Schreiber, J., Sommer, D., & Besuch, ment Group. https ://web.stanf ord.edu/dept/SUSE/SEAL/Repor G. (2010). Ursachen des Studienabbruchs in Bachelor- und in ts_P aper s/On%20t he %20Int eg r at ion%20of%20F or mativ e%20 herkömmlichen Studiengängen. Hannover: HIS. (Forum Hoch-Ass essme nt_Teach er%20Ed_Final .doc. Accessed 29 May 2018. schule F2/2010). Sherin, M. G., & Van Es, E. A. (2009). Effects of video club participa- Jafari, A., & Kaufman, C. (Eds.). (2006). Handbook of research on tion on teachers’ professional vision. Journal of Teacher Educa- ePortfolios. New York: Idea Group Reference. tion, 60, 20–37. Johnson, R. B., & Schoonenboom, J. (2016). Adding qualitative and Shulman, L. S. (1986). Those who understand. Knowledge growth in mixed methods research to health intervention studies: Interacting teaching. Education Researcher, 15, 4–14. with differences. Qualitative Health Research, 26(5), 587–602. Spanneberg, R. (2009). The mathematics teaching portfolio: A reflec- Kaiser, G. (2015). Professionelles Lehrerhandeln zur Förderung fach- tive tool for developing professional growth and improving class- lichen Lernens unter sich verändernden gesellschaftlichen Bed- room practices. In B. Clarke, B. Grevholm & R. Millman (Eds.), ingungen (ProfaLe). BMBF-Qualitätsoffensive Lehrerbildung in Tasks in primary mathematics teacher education. Purpose, use Hamburg. In Presentation at the University of Hamburg. https :// and exemplars (pp. 51–65). New York: Springer. www.profale.uni-hambu r g.de/forschung/v ortraeg e/qlb---uk-jahre Taras, M. (2005). Assessment—summative and formative—some theo- stagu ng-vortr ag-29062 016.pdf. Accessed 29 May 2018. retical reflections. British Journal of Educational Studies, 53(4), Kaiser, G., Blömeke, S., König, J., Busse, A., Döhrmann, M., & Hoth, 466–478. J. (2017). Professional competencies of (prospective) mathemat- Tatto, M. T., Schwille, J., Senk, S., Ingvarson, L., Peck, R., & Rowley, ics teachers: Cognitive versus situated approaches. Educational G. (2012). Teacher education and development study in math- Studies in Mathematics, 94(2), 161–182. ematics (TEDS-M): Policy, practice, and readiness to teach pri- Kaiser, G., Busse, A., Hoth, J., König, J., & Blömeke, S. (2015). mary and secondary mathematics. Conceptual framework. East About the complexities of video-based assessments: Theoretical Lansing: Teacher Education and Development International Study and methodological approaches to overcoming shortcomings of Center, College of Education, Michigan State University. research on teachers’ competence. International Journal of Sci- Tillema, H. (2010). Formative assessment in teacher education and ence and Mathematics Education, 13, 369–387. teacher professional development. In B. McGaw, E. Baker & Kelle, U., & Buchholtz, N. (2015). The combination of qualitative P. Peterson (Eds.), International encyclopedia of education and quantitative research methods in mathematics education: A (pp. 563–571). Oxford: Reed Elsevier. “mixed methods” study on the development of the professional Topping, K. (1998). Peer assessment between students in colleges and knowledge of teachers. In A. Bikner-Ahsbahs, C. Knipping & N. universities. Review of Educational Research, 66, 249–276. Presmeg (Eds.), Approaches to qualitative research in mathemat- Vogel, R. (2018). Portfolioarbeit in der Grundschullehrerausbildung— ics education. Examples of methodology and methods (pp. 321– “Lernraum” zur Anbahnung eines professionellen mathematik- 361). Dordrecht: Springer. didaktischen und mathematischen Unterrichtshandelns. In R. Mackey, J. (2009). Virtual learning and real communities: Online Möller & R. Vogel (Eds.), Innovative Konzepte für die Grunds- professional development for teachers. In E. Stacey & P. Gerbic chullehrerausbildung im Fach Mathematik (pp. 197–218). Wies- (Eds.), Effective blended learning practices: Evidence-based per - baden: Springer. spectives in ICT-facilitated education (pp. 163–181). Hershey: Wiliam, D. (2000). Integrating formative and summative functions ICI Global. of assessment. In Paper presented to Working Group 10 of the Mayring, P. (2015). Qualitative Inhaltsanalyse. Grundlagen und Tech- 9th International Congress on Mathematics Education (ICME 9) niken. Weinheim: Beltz. in Tokyo. http://www .dylan wilia m.or g/Dylan _W ilia ms_websi Mokken, R. J., & Lewis, C. (1982). A nonparametric approach to the te/Paper s_files /ICME9 %20WGA 10%20pap er.doc. Accessed 29 analysis of dichotomous item responses. Applied Psychological May 2018. Measurement, 6, 417–430. Wiliam, D., & Black, P. (1996). Meanings and consequences: A basis Orschulik, A. (2016). Strengthening cooperation within school practi- for distinguishing formative and summative functions of assess- cal activities as a way to improve teacher education. In Poster ment? British Educational Research Journal, 22(5), 537–548. presented at the 13th International Congress on Mathematics Edu- Zeichner, K. M. (2010). Rethinking the connection between campus cation (ICME-13) in Hamburg. https ://www.profa le.uni-hambu courses and field experiences in college- and university-based rg.de/forschung/pos ter vortr aege/pos ter -orschulik -2016-icme.pdf. teacher education. Journal of Teacher Education, 61(1–2), 89–99. Accessed 15 June 2017. 1 3 http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png ZDM Springer Journals

Combining and integrating formative and summative assessment in mathematics teacher education

Free
14 pages
Loading next page...
 
/lp/springer_journal/combining-and-integrating-formative-and-summative-assessment-in-hkIOeciBqk
Publisher
Springer Berlin Heidelberg
Copyright
Copyright © 2018 by The Author(s)
Subject
Education; Mathematics Education; Mathematics, general
ISSN
1863-9690
eISSN
1863-9704
D.O.I.
10.1007/s11858-018-0948-y
Publisher site
See Article on Publisher Site

Abstract

Contrary to the opinion that formative and summative assessment approaches are not compatible, this article presents a theoretically grounded way in which different forms of assessment can be combined and integrated in university mathemat- ics teacher education. Two mixed-assessment approaches are demonstrated through the analysis of a case study involving a practice-based seminar accompanying a school internship. First, a formative eportfolio assessment was combined with a summative panel survey to assess the learning opportunities of mathematics pre-service teachers. Second, the formative eportfolio approach was integrated with a summative oral course examination to make statements about the learning processes and learning outcomes of the pre-service teachers. Our analyses conclude that combining and integrating the two forms of assessment present the possibility of evaluating different aspects of the pre-service teachers’ perceptions of opportunities to learn. Benefits, validation aspects and limitations of the two approaches of combining and integrating assessment forms are discussed. Keywords Formative assessment · Summative assessment · Mixed assessment · Mixed methods evaluation · Eportfolio · Teacher education · School practice · Opportunities to learn 1 Introduction subject-based learning under changing social conditions”; Kaiser 2015), for example, is to improve the seminars that Pre-service teachers in Germany frequently bemoan the lack are intended to complement school internships for pre-ser- of relation between their university studies and their later vice teachers. The newly developed seminar structure takes careers in the field, claiming that the courses they attended up practical experiences and focuses on the development of did not adequately prepare them for their work with students situation-specific aspects of teaching competence. In this (Heublein et al. 2010). Therefore, in recent years, school- paper, we focus on the seminar for the mathematics pre- practical studies in different formats (e.g., internships) play service teachers. a larger role as learning opportunities in German teacher The advantages and opportunities for practical-based education. Innovative university courses designed to accom- learning approaches have been discussed in teacher educa- pany these field experiences provide pre-service teachers tion for years (Putnam and Borko 2000). However, we as with possibilities of uniting theoretical knowledge and teach- teacher educators have to ask ourselves: How can we evalu- ing practice. These courses aim to reduce the discontinuity ate the profit that pre-service teachers gain in terms of theo- between theoretical knowledge acquired at university and retical and practical expertise in their internship and in the professional experiences in the classroom (Arnold et  al. accompanying seminar? Moreover, what are the appropriate 2014; Zeichner 2010). One aim of the University of Ham- structures and tools to use in order to assess the develop- burg’s project ProfaLe (“Professional teaching to promote ment of pre-service teachers’ professional competence in such practical-based learning approaches? It is important to monitor the development of professional competence and * Nils Frederik Buchholtz pre-service teachers’ opportunities to learn when accompa- n.f.buchholtz@ils.uio.no nying pre-service teachers and supporting them on their way to becoming professional teachers. University of Oslo, PB 1099, Blindern, 0317 Oslo, Norway University of Hamburg, Hamburg, Germany Vol.:(0123456789) 1 3 716 N. F. Buchholtz et al. Teachers’ professional competence is, according to instructional strategies” (Kaiser et al. 2015, p. 374; see also; Blömeke et al. (2015), a continuum that comprises disposi- Schoenfeld 2011). In this Perception–Interpretation–Deci- tions such as cognitions and affect-motivations as well as sion-making (PID) model, it is assumed that the availability situation-specific skills and performance. In this paper, we of situation-specific skills significantly determines whether argue that corresponding to the different aspects of teach- the transformation from disposition to performance suc- ing competence, different forms of assessment are benefi- ceeds (Blömeke and Kaiser 2016). Since this model implies cial when it comes to assessing the outcomes of pre-ser- a rather holistic view of competence, Blömeke et al. (2015, vice teachers’ learning processes. In general, two different p. 8ff) also point to the challenges of an appropriate assess- approaches to assessment can be distinguished: Summative ment: Pure cognitive-analytical approaches might lack valid- assessment (SA) measures the achievement of previously ity because relevant parts of the measured constructs can be defined standards, tasks or goals, encapsulating all collected underrepresented, and pure performance-based assessments evidence up to a given point to yield either comparative might neglect the contribution of dispositional resources. or numerical ratings (Taras 2005). Formative assessment Assuming that “the whole is greater than its parts” (p. 9), (FA), on the other hand, promotes individual development Kaiser et al. (2017) recommend working with a broader by interpreting and providing feedback according to a diag- range of combined and situated assessment formats that are nostic judgment that presents information about the candi- able to cover processes mediating the transformation of dis- dates’ continuative learning processes. Unfortunately, many positions into performance. We agree that accurately and see the distinct purposes of these two forms of assessment reliably measuring teaching competence requires a mixture as incompatible (William 2010). But in our understanding of SA and FA assessment. This assessment approach can of SA and FA, these two approaches can be combined and be described methodologically as either a combination or even be integrated to assess the development of profes- an integration of the different assessment forms. In the fol- sional teaching competence, because each assesses different lowing, we clarify the difference between combination and aspects of competence. While SA considers dispositional integration and concretize how each approach addresses aspects of competence (e.g., knowledge about study content unique aspects of the professional teaching competence of and the prevalence of perceived opportunities to learn about our pre-service teachers. different study content), FA focuses in on the contextual situatedness of the pre-service teachers’ professional actions 2.1 Combining SA and FA forms and the development of situation-specific skills. Our study aimed to traverse the classical dichotomy When evaluating pre-service teacher education measures between SA and FA in mathematics teacher education. In and the outcomes of learning, SA generally must play a this paper, we demonstrate a combination of SA and FA role, since any educator assessment aims at least to certify and an integration of FA and SA in an evaluation of the out- that certain course planning and teaching skills have been comes of a practical-based seminar designed to accompany acquired. During the internship and our accompanying semi- school-internships in mathematics teacher education. nar, pre-service teachers are provided with opportunities for cognitive learning as well as for the acquisition of situa- tion-specific skills. In our case, then, a SA might therefore 2 Theoretical considerations on using cover pre-service teachers’ cognitive knowledge dispositions mixed‑assessment approaches and the theoretical learning opportunities provided by our seminar. In the frequently cited review of research on the assessment On the other hand, FA is often brought into play as a of competencies in higher education by Blömeke et  al. pragmatic option for the support of pre-service teachers in (2015), the authors model competence as a continuum that their practice. Tillema (2010) encourages assessments aimed comprises dispositions, situation-specific skills, and perfor - at improving field-related performance: mance. This model can also describe the professional com- Supporting teachers as learners requires moving petence of mathematics teachers. beyond providing mere knowledge of results to a type Dispositions contain, for example, cognitive components of functional (i.e., performance-related) feedback that like the professional knowledge teachers acquire through can articulate developments in practice, anticipate university-based learning opportunities (Shulman 1986). Sit- professional learning needs, and monitor the learner’s uation-specific skills comprise three different aspects related progress during a course of teacher action (p. 563). to practice: “(a) Perceiving particular events in an instruc- tional setting, (b) Interpreting the perceived activities in the Tillema therefore proposes to “increase the learner classroom and (c) Decision-making, either as anticipating a control over the content to be assessed as well as over response to students’ activities or as proposing alternative the criteria by which the teaching performance will be 1 3 Combining and integrating formative and summative assessment in mathematics teacher education 717 scrutinized” (p.  567). Authors such as Bell and Cowie 2.2 Integrating SA and FA forms (2001) or Carr and Claxton (2002) also write from a socio-cultural view, understanding learning as a situated According to Wiliam and Black (1996), neither FA nor SA activity; these authors point to the situatedness of what categorically exclude each other; rather, they are seen as the they call learning dispositions. Accordingly, they favour extremes of a common continuum, the core of which is the FA approaches such as observations, interviews and self- (interpretable) evidence of performance: reports like those that appear in portfolios when it comes Any assessment must elicit evidence of performance, to the assessment of performance-related competence. which is capable of being interpreted (however inval- Thus, FA “values learning dispositions and sees [pre- idly). Whether or not these interpretations and actions service teachers’] early development as consequential on satisfy the conditions for formative functions, the fact relationships between the learner and the social and mate- that interpretable evidence has been generated means rial learning environment” (p. 18). To concretize this in that the assessment can serve a summative function. our case, a FA might yield information about a pre-service Therefore, all assessments can be summative (i.e. have teacher’s practical learning opportunities during his or her the potential to serve a summative function), but only internship—in particular, the individual’s acquisition of some have the additional capability of serving forma- situation-specific skills. Further, the FA provides us as tive functions (p. 544). evaluators with information about our pre-service teach- ers’ personal experiences with practical teaching, which Wiliam sees the integration of SA and FA as a combina- we then might use, either to provide feedback, or adap- tion of the different purposes of the assessment forms on the tively for future seminar planning. widest possible basis for evidence (Wiliam 2000, p. 11). In However, this formative approach faces methodological particular, for the use of SA as FA, further feedback from challenges. Unlike SA forms, FA forms presuppose that we the assessor is necessary—based on the interpretation of the have to give up the idea of comparing the achievements of SA results and oriented toward helping the evaluee continue pre-service teachers in a standardized way, since the learn- on from assessed to desired levels of performance (see e.g., ing progress is different for each individual pre-service Sadler 1989; Taras 2005; Hattie and Timperley 2007). On teacher. FA requires evaluators to scrutinize whether all the other hand, the idea of using FA as SA is to aggregate intended educational goals or certain measures in pre-ser- the separate results of a set of assessments designed to serve vice teacher education have been achieved. Because the formative purposes in order to get a comprehensive picture identification of evidence of performance in FA is difficult, of overall achievements (Harlen and James 1997). so far, this has been a critical issue when FA is used in The basic idea of an integration of SA and FA is therefore teacher education or professional development contexts to assess one phenomenon with different assessment forms, (Delandshere and Arens 2003). Accordingly, how mathe- for example by using small units of SA in a formative way matics (pre-service) teacher education can benefit from FA in the course of giving feedback, and integrating this FA is so far not described very well (e.g., Spanneberg 2009). into an overall SA afterwards. For our case, this means that Against this background, the basic idea of using a we are able to focus at the same time on how the pre-service combination of SA and FA is to extend the scope of the teachers develop professional teaching competence and what assessment, i.e., to both widen and deepen the assessment outcomes of the seminar and the internship they achieve. by gathering as much information as possible on multiple In particular, the FA during the internship is composed of components of the assessed phenomenon or different, but predetermined tasks affording particular situation-specific closely related phenomena. Assessment results should skills (described in Sect. 3.3), and we as lecturers provide ideally complement each other to increase the overall constructive feedback on those tasks, helping the pre-service interpretability of assessment results. For our case, the teachers to increase their teaching competence. The overall combination of SA and FA holistically addresses learning SA of our seminar—an oral examination conducted after- opportunities to increase teaching competence in order to wards—individually addresses the situation-specific skills gain information about the development of teaching com- that the pre-service teachers have acquired during their petence in the internship and the seminar. Additionally, internship. The aim behind this procedure is to increase the in favour of an increased interpretability of assessment autonomy of the pre-service teachers concerning their cer- results, we seek a mutual complementation of the find- tification of teaching competence on the one hand, while on ings from a standardized SA with the findings from an the other hand increasing the validity of each assessment of individual FA (e.g., when identifying evidence for learning situation-specific skills through the mutual corroboration of opportunities in FA) and seek ways that we might partly the results of FA and SA. overcome the challenge posed by the inability to standard- This approach is of course also challenging, since we ize FA in teacher education. as lecturers both enable learning processes and certify the 1 3 718 N. F. Buchholtz et al. outcomes of the seminar and the internship. Shavelson structure of nearly all sessions of the seminar was based (2006) highlights this “double bond” of the assessors of on the PID model (see Sect. 2). Initial points for analyses educational processes: were, for example, observations of the pre-service teachers or videotaped classroom situations. A specic fi emphasis was Significant tensions are created when the same person, placed on the heterogeneity of students and how teachers namely the teacher, is required to fulfill both forma- dealt with it. tive and summative functions. Teachers at the interface Summing up, different forms of opportunities to learn of formative and summative assessment … confront were provided: First, the content of different seminar lessons conflict daily as they gather information on student provided opportunities to learn in the form of knowledge performance—to help students close the gap between acquisition. With the help of this knowledge, the pre-service what they know/can do and what they need to know/ teachers were able to develop a certain teaching competence be able do on the one hand, and to evaluate students’ disposition. Second, observation tasks that the pre-service performance for the purpose of grading/certification teachers had to carry out within their once-weekly internship on the other hand (p. 8). during the semester provided another form of opportunity to This double bind can only be circumvented with regard learn: in this case, to perceive and interpret teacher behav- to the integration of FA and SA by not using or interpreting iour. By doing so, they were asked to reflect on necessary the same evidence for both purposes of assessment (Wil- teacher skills in certain situations. Third, within the oral iam 2000). Within the seminar, we therefore differentiated examination of the seminar, the pre-service teachers were clearly between learning situations (internship, seminar prompted to reflect on their own situation-specific skills sessions and observation tasks) and achievement situations concerning a self-chosen topic, focusing especially on the (oral examination) and made this clear to the pre-service development of these skills during the internship. Thus, teachers as well. the oral examination also reflected opportunities to learn. Whereas the observation tasks involved reflecting on the skills of others, in the oral examination, the evaluees were 3 SA and FA in the ProfaLe seminar asked to reflect on their own skills. and corresponding research questions In summer 2016 these innovations were implemented in the seminar for the first time in a group of 30 pre-service The aim of the ProfaLe seminar was to develop the profes- teachers (7 males, 23 females). Every pair of students was sional teaching competencies of pre-service teachers and to accompanied by a practicing teacher at their schools once strengthen school-practical aspects, in order to reduce the per week during the semester and every day during a practi- common feeling of discontinuity between theoretical knowl- cal phase in the semester break for 5 full weeks (Orschulik edge acquired at university and professional experiences in 2016). the field. 3.2 Implementation of SA 3.1 Structure of the seminar A special opportunity to integrate SA into the seminar The seminar was implemented across two periods of pre- was an interdisciplinary evaluation panel survey by the service teachers’ practical experiences during their master’s project ProfaLe that focused on the pre-service teachers’ studies. The first period was a full semester during which professional teaching knowledge as well as on their uni- they split into pairs and spent 1 day per week in a school and versity and school-practical learning opportunities (Doll 2 h weekly in a university seminar covering different topics et al. 2018). The survey draws on established instruments related to mathematical teaching. The second period, which from the interdisciplinary study TEDS-LT (Blömeke was held over the semester break, involved the completion et al. 2013; Buchholtz et al. 2016), a German follow-up to of a 5-week internship, during which they—in the same the international TEDS-M study released in 2008 (Tatto pairs and schools—spent every day in school. Throughout et al. 2012), and thus allows a standardized assessment of both periods, the pre-service teachers observed and accom- pre-service teachers’ professional knowledge and respec- panied experienced teachers and even had to teach small tive learning opportunities. University and school-prac- units by themselves, supervised by the teachers and once tical learning opportunities—on which we focus in this by the university lecturers. In the seminar, the pre-service paper—were surveyed in the form of an inquiry into per- teachers learned how to focus their attention on important ceived study contents, and thus they reflect dispositional pedagogical classroom events that promoted their ability aspects of teaching competence. The pre-service teachers to perceive important classroom situations, interpret them, were asked to indicate whether they had studied various and decide how they should be managed. Therefore, the content in their previous studies. The surveyed university 1 3 Combining and integrating formative and summative assessment in mathematics teacher education 719 learning opportunities refer to six different subject-specific (graphical or textual) mapping their reflections about the topics of mathematics education with a particular focus chosen topic. on aspects from the ProfaLe measures (see Table 1). The assessment was summative because the pre-service teach- 3.3 The implementation of FA ers were asked to recall a variety of different learning opportunities in the past and up to the point of the survey. The FA of the pre-service teachers’ professional teaching Further details on the survey instruments are given in the competencies was implemented in the seminar in a variety paper by Doll et al. (2018). of ways, primarily through an electronic portfolio (eport- The panel survey was conducted online and was open to folio) and corresponding feedback, as well as through the all pre-service teachers of mathematics at the University of self-responsible design of the oral examination presented Hamburg. In total, 187 pre-service mathematics teachers above. A crucial part of the eportfolios was a set of observa- took part. Participation in the online survey was voluntary tion tasks about which the pre-service teachers had to write. on the basis of an honorarium and was anonymous due to Generally, eportfolios can be used in different ways: The data privacy regulations. However, since the online survey feedback on the written observations given by university also asked about respondents’ participation in specific uni- teachers can be used by the pre-service teachers for their versity courses such as the ProfaLe seminar, we did obtain further learning processes as well as by their university data concerning the participants of our ProfaLe seminar teachers for designing appropriate further teaching (Hattie from the survey. In total, 13 of the 30 pre-service teachers and Timperley 2007). The use of eportfolios in the field of from the ProfaLe seminar participated in the online survey. teacher education in general can be of great benefit for pro- A further form of SA for assessing teaching compe- fessional development, as different case studies on the use tence in the seminar was the oral examination at the end of electronic blended learning in teacher education show of the semester. It was administered as a presentation- (Mackey 2009; Gikandi et  al. 2011; Vogel 2018). They and-reflection examination designed to reveal the out- are especially helpful when they are used in tandem with comes of pre-service teachers’ learning processes both school practice; when they provide authentic practice-related during their school internships and in the seminar. This assessment activities (e.g., realistic classroom situations); was accomplished by presenting a discussion about a self- and when they receive effective and prompt formative feed- chosen practical teaching example and included selecting back (Gikandi et al. 2011, p. 2338). Still, although the use important experiences from their own teaching activities of online FA tools like eportfolios has increased in the last or observations and linking them with the mathematics decade (Jafari and Kaufman 2006), there are also methodo- educational theory taught in the seminar. This way the logical problems related to the possibility that portfolios may pre-service teachers could stress relevant aspects within create inference for performance (Delandshere and Arens a 15-min presentation. But the oral examination did not 2003). serve only summative purposes; it also integrated forma- We decided to use an eportfolio approach to focus on the tive aspects because the pre-service teachers had to choose development of situation-specific skills of teaching compe- the topic of their oral examination by themselves, based tence. The pre-service teachers received in total 18 situa- on their written reflections from their eportfolio. Further - tional-observation tasks, which they had to address during more, the examination had to center on a specific situ- their observations of other teachers or their own teaching ational problem encountered during the internship. Prior to in school, and to work on as a written contribution to their the examination, the pre-service teachers had to hand out personal eportfolio. a mind map to the lecturers, meaning a visual presentation In the seminar, three different forms of observation tasks were used: Table 1 Opportunities to learn in mathematics teacher education Dimension Content Example Items Reliabil- Cronbach’s α ity (Rho) Mathematics education Basics of mathematics education Didactics of algebra 20 0.86 0.84 Dealing with heterogeneity Language in mathematics education 8 0.62 0.62 ICT in mathematics education Learning with apps 6 0.66 0.56 Methods of instruction Problem-solving approaches 8 0.76 0.75 Curriculum National curricula 8 0.70 0.67 Research on mathematics education Role of beliefs in mathematics learning 9 0.72 0.68 1 3 720 N. F. Buchholtz et al. 1. Six tasks in advance of a forthcoming lecture, in which also use adaptive planning to incorporate those observations the pre-service teachers had to describe and partly inter- into upcoming lectures. While the school internship took pret classroom situations. For example: “Describe when place over the semester break, the pre-service teachers had and how the teacher took up errors from the students in to choose observation tasks by themselves. In this way, the the classroom. What was done with these errors? What pre-service teachers could use their entries in the eportfolio kind of problems occurred in these teaching situations?” as a preparation for their individual oral examination (which 2. Seven subsequent tasks as a follow up of a lecture, in then served summative purposes). Thus, we were able to which the pre-service teachers not only had to describe avoid the conflict of overregulating the pre-service teachers but had to focus more on interpretation and developing through fixed observation tasks, and instead allow them to alternatives to act. For example: “Choose one of the het- develop their own sense of interpretation about their learn- erogeneity aspects that you want to look at more closely ing development—another step in this process during which in your observations. Describe the students according we were able to witness the adaptivity of FA (Tillema 2010). to this aspect, and how this aspect has influenced meas- ures of differentiation in the classroom. Evaluate what 3.4 Research questions kinds of measures were more and less beneficial for the students.” To identify the benefits of the combination and integration 3. Five tasks of students’ own choice of observation focus, of SA and FA in the case of the ProfaLe project of math- in preparation for the oral examination. ematics teacher education, we focus on the empirical results of the different assessment forms guided by two research Formulating the observation tasks required the pre-ser- questions: vice teachers to focus on one specific topic—e.g., dealing with errors—following the structure of the PID model. Thus, 1. What forms of evidence of pre-service teachers’ profes- pre-service teachers had to describe the perceived situations, sional competence and learning opportunities can we interpret them, and sometimes were prompted to develop identify from combining SA (panel survey) with FA useful responses to them. (eportfolios)? All written contributions were read by the seminar lectur- 2. Which complementary forms of evidence can we iden- ers over the following week and commented on within the tify when we integrate SA with FA in the oral examina- personal eportfolio, which only the lecturers had access to. tion? However, the written contributions were compulsory for the seminar and thus were extrinsically motivated reflections. To answer these questions, we first describe in the meth- (This duality depicts the difficult relationship between het- odological considerations how we analyzed our data, within eronomy and autonomy within this assessment format; See the scope of this paper. In the Sects. 5 and 6, we illustrate Sect. 2.2). As the reflections represent the learning processes and discuss the results of the combination and the integra- of the pre-service teachers, no rating or grading took place. tion of the different forms of assessment. The seminar lecturers provided feedback constructively and showed possibilities for the pre-service teachers to improve their instructional quality in the described situations, accord- 4 Methods for data analysis ing to the criteria already discussed in the seminar. By rec- ognizing the benefit of the feedback for the pre-service 4.1 Methods of analyzing SA data teachers’ own professional development, we hoped for a high acceptance of the feedback (Topping 1998). Of the 30 pre-service teachers in our seminar, 13 took part in The reflections in the eportfolio aimed at self-assessment the online survey, which allowed us to analyze their profes- and raising awareness of the writers’ own situation-specific sional knowledge and their university- and field-based learn- skills, so that explicit situation-specific learning opportuni- ing opportunities as a measure for dispositional teaching ties and learning processes could be reflected upon in detail competence (cf. Sect. 3.2). To analyze the learning opportu- through the given feedback. In this way, the pre-service nities, a Mokken analysis (Mokken and Lewis 1982), based teachers were able to use the feedback from the seminar lec- on the whole sample of the online survey (N = 187) was turers to optimize their observations in the subsequent weeks used. This non-parametric item response model allows us as well as to improve their own teaching (Vogel 2018). How- to check the assumption that the learning opportunities of ever, we as seminar lecturers also were able to benefit from different dimensions can be described hierarchically along this form of FA: We could not only examine the results of an ordinal scale, so that the percentage of perceived content the pre-service teachers’ reflections on the teaching prac- can be interpreted as individual progress along a study path- tices they observed and their own teaching but we could way. In Table 1, the reliability-corrected rank correlations 1 3 Combining and integrating formative and summative assessment in mathematics teacher education 721 (Spearman’s Rho) of the six dimensions, as well as Cron- was modified so that the three situation-specific skills could bach’s α, are reported. The scales showed a satisfactory reli- be identified, whereby the situation-specific skill of percep- ability in the range of 0.62–0.86. Since an analysis on the tion could be coded by utterances at the descriptive level. individual level was not possible due to data regulations, We changed the codes belonging to the category “Topic” for our analysis, the group of the 13 pre-service teachers into those that were topics of the seminar. By doing so, we from the ProfaLe Seminar was compared to a comparative were able to analyze to what extent these topics provided group of 28 pre-service mathematics teachers stemming opportunities to learn in the form of knowledge acquisition, from the whole sample. This group of pre-service teach- and sometimes of situation-specific skills. An example for a ers were enrolled in mathematics education courses at the coding process can be found in Fig. 1 in Sect. 5. same University in the same phase of study, but they did not In addition, new indicators for learning opportunities participate in any teacher education courses affected by the emerged inductively from the eportfolios, according to ProfaLe project. We used the non-parametric Mann–Whit- which the extracts also could be rated, e.g., descriptions ney U test for the comparison of medians and interquartile about the development of the personality as a teacher or the ranges of the perceived learning opportunities. professional relationship between the pre-service teachers The SA of the oral examination was carried out according and their mentors. The collection and classification of these to the following overarching criteria, which were commu- extracts showed the overall heterogeneity of the learning nicated to all pre-service teachers beforehand, in order to opportunities and situation-specific skills of the pre-service provide the possibility of comparability of the performance: teachers. Therefore, in the following section, we showcase some illustrative examples from pre-service teachers who The observations and own experiences were selected rea- have given us permission to present their work. sonably and are meaningful; An appropriate selection of theories of mathematics edu- cation has been chosen;5 Results The observations and their own experiences are compre- hensibly presented, and analyzed and evaluated with the To answer our research questions, we first present results aid of theory; of the combination of SA and FA, then the results of the Changes and learning processes are taken up; integration of SA and FA. Subsequently in the Sect. 6, we Current and relevant, in particular didactic and scientific discuss the results on a more theoretical level and also reflect literature is included; on the limitations of our approach. The proper use of scientific terminology is employed; The presentation is self-reliant, adequately prepared and 5.1 Results of the combination of SA and FA structured; Questions from the lecturers can be answered; With the results from the online survey, we can arrive at Links to different topics of the seminar can be drawn. the number of perceived university learning opportunities concerning specific study content, and thus create an indi- Each oral examination was rated by two university teach- cator for professional competence. However, we are able ers according to the criteria listed above, directly following to analyze the outcomes of learning opportunities only on its conclusion. All examinations were logged. the group level. The findings are therefore complemented and deepened by those from the FA on the individual level, 4.2 Methods of analyzing FA data which are also used to make conclusions about the applica- tion of the study contents with regard to the development of We subjected all eportfolios to a systematic process of anal- situation-specific skills. Table  3 describes the results of the ysis following suggestions from Delandshere and Arens Mann–Whitney U test for the comparison of medians and (2003) about identifying evidence of performance from port- interquartile ranges (IQRs) of the online survey. folios. We therefore combined a deductive and an inductive With regard to the perceived study contents, we find sig- approach introduced by Mayring (2015) in order to be as nificant differences from the comparison group. Thus, the open as possible for the analysis. We coded extracts from the group of pre-service teachers from our seminar felt they eportfolios according to the learning opportunities provided were presented with a significantly higher number of learn- by the ProfaLe seminar and indicators of situation-specific ing opportunities in the areas of dealing with heterogeneity skills in teaching competence that the pre-service teachers and research on mathematics education. For example, the described in their observation tasks. The deductively cre- median of 0.38 in dealing with heterogeneity means that, on ated part of the coding system (see Table 2) is based on the average, pre-service teachers from our seminar stated that work of Sherin and Van Es (2009). The category “Stance” they had previously encountered learning opportunities for 1 3 722 N. F. Buchholtz et al. Table 2 Deductive part of the Coding categories Coding coding system Actor Student Teacher Other Topic MC 1—mathematical argumentation MC 2—mathematical problem solving MC 3—mathematical modelling MC 4—using mathematical representa- tions MC 5—operate with symbolic, formal and technical elements in mathematics MC 6—Communicating Dealing with errors Mathematical concepts (“Grundvorstel- lungen”) Changing and connecting representations Language awareness in learning math- ematics Cognitive activation Problem orientation Teacher intervention Differentiation and individualized instruc- tion Form of teaching—student-centered Form of teaching—teacher-centered Classroom management Stance Describe (Situation-specific skills) Interpret Decide MC Mathematical competency three of the eight subthemes on dealing with heterogeneity When describing her perceptions concerning the subject (e.g., dyscalculia, giftedness). Despite the lack of signifi- matter of the observation task, Jennifer focuses on dealing cant differences in other dimensions, we recognized the high with errors. It becomes very clear that Jennifer observed the perception of study content related to “methods of instruc- teaching and the classroom discourse very closely, focusing tion” and the comparatively low perception of content of on both students and teacher. Her selection regarding this “Information and communication tools (ICT) in mathemat- situation involves a nice example of an occurring miscon- ics education,” which could be expected from the seminar. ception of inverse proportionality. Although not required, These findings reflect the measures taken by the ProfaLe she interprets her perceptions, taking up other topics that project, since dealing with heterogeneity and instructional were subjects of previous discussions in the seminar. A methods constituted a major focus in the seminar (whereas closer analysis of the “Interpret” and “Decide” segments the use of ICT was not in focus). In the following section, shows that Jennifer does not explain the occurring error. we combine these SA results with complementary individual However, since this was a preliminary observation task, results from the FA. Therefore, we concentrate on how the it might not yet have been possible for Jennifer to rely on seminar and the internship offered learning opportunities for broad knowledge for error analysis. Furthermore, no error the development of situation-specific skills of the pre-service interpretation was expected according to the formulation of teachers, especially concerning dealing with heterogeneity. the observation task. Nevertheless, we can recognize that A preparatory eportfolio entry on the topic of dealing she already considers the teacher’s handling of the mistake with errors from Jennifer (Fig.  1), a female pre-service as inappropriate and mentions the importance of students’ teacher (see Sect. 3.3), provides an illustrative example of understanding. This reveals a high degree of pedagogical an individual FA result. It is about observing the behaviour reflection on the learning of concepts in mathematics educa- of a teacher dealing with an occurring misconception. tion, indicating her situation-specific skills when addressing 1 3 Combining and integrating formative and summative assessment in mathematics teacher education 723 Fig. 1 ePortfolio entry from Jennifer and respective coding (translated by the authors) Table 3 Median differences in perceived study contents Group n Basics of MEd Dealing with ICT in MEd Methods of instruction Curriculum Research on MEd heterogeneity ProfaLe seminar 13 0.55 (0.30) 0.38* (0.25) 0.33 (0.42) 0.75 (0.19) 0.50 (0.25) 0.33* (0.17) Comparative group 28 0.58 (0.24) 0.25 (0.22) 0.33 (0.33) 0.75 (0.25) 0.43 (0.38) 0.22 (0.11) *p < 0.05, IQRs in brackets misconceptions, and that might reflect her theoretical knowl- pre-service teachers were used to discuss occurring errors edge in this field. of students on the basis of concrete examples and to con- Following the idea of FA, these and observations from sider how teachers can deal with misconceptions in the other pre-service teachers were used by the lecturers to classroom. Thereby our aim was to accumulate the practi- adapt teaching and learning in the course. In the next cal expertise of the pre-service teachers adaptively—as seminar session, observations and interpretations of the 1 3 724 N. F. Buchholtz et al. intended in FA—and to develop guidelines for handling ‘asterisk tasks’ as well for the children that finished mistakes in the classroom together. very quickly. These tasks do not have to be done by In summary, the online survey provided us with general all children. On the whole, I find this works out quite information about pre-service teachers’ perceptions of their well. As a result of the learning time, even the slower own opportunities to learn, while the reflections from the children have the chance to work things off. I would observation tasks provided a good learning opportunity for find it great, however, if content would be offered that training in situation-specific skills. This was the case not cognitively demands more of the faster children and only when the pre-service teachers had the opportunity to leads them in some places to get deeper into a topic, observe examples of good teaching practice but also when to making connections, etc. I have the feeling there is they observed examples that called for improvement. not so much space to explore mathematical creativ- With our coding approach in the scope of this paper, we ity. Possibly open task formats could be introduced were able to make conclusions about the skills developed so for children who already finished. (I have not yet con- far. However, our coding system requires closer examination sidered two children with learning disabilities in the if we are to make statements about the quality and appro- class. I know that it is hardly possible for a teacher priateness of the interpretations and decision-making. Nev- to provide additional meaningfully selected learning ertheless, when looking at individual cases, it is possible to material which is connected to the current teaching, show how perceived study contents are linked to other study but these two children are just employed to utilize the contents and are available not only at the knowledge level ‘leftovers.’ I do not see any substantive thread there; but are also applied in describing, interpreting and decision- no fostering plans have been consulted. I doubt that the making. In this way, the results of the SA can be deepened math teacher knows what is in it. I think these children and partially explained. should also try to follow the current teaching, at a dif- ferent level and at a different pace.) (Leonie, translated 5.2 Results of the integration of SA and FA by the authors). Leonie describes the teacher’s measures in a very detailed Regarding the overall performance in the seminar, 28 of the way, taking an appraising attitude. The entry shows her 30 pre-service teachers took the oral examination at the end development of situation-specific skills. It becomes clear of the semester. The average grade was 1.5 and the grades that she perceives the situation in her observation in a holis- ranged from 1.0 to 2.7 [grades may differ from 1.0 (best) tic manner. The questioning of the cognitive activation to 4.0 (worst)]. The topics of the oral examinations varied. and openness of the tasks shows that her situation-specific There were examinations about the handling of errors in interpretations are to some extent also based on professional the classroom, about certain measures of differentiation knowledge. Furthermore, she questions the teacher’s actions (e.g., circuit learning), cooperative learning in mathematics against the background of the increased need for differentia- education or specific aspects of teaching methods, such as tion, especially in dealing with children with special needs. classroom discourse. Although not required, she also develops alternative sugges- To illustrate how FA affected the topics in the SA for indi- tions for differentiation measures, which are characterized vidual pre-service teachers, the following describes Leonie, by ideas of open- and closed-differentiation formats as well a female pre-service teacher. One of her eportfolio entries as by collaborative learning on a common object, that take relates to the topic “measures of differentiation in mathemat- the high-performing students as well as the two students ics education” (see Example, Sect. 3) and describes how with learning disabilities into account. Overall, her descrip- the teacher Leonie observed dealt with the heterogeneity tions of the observations indicate a high degree of situation- of the class. specific skills including indications that she is already able In the class the children work with a very different to make informed decisions. speed. About 3 or 4 students work very fast and usu- During the semester, the observation tasks in the eport- ally are also correct. There is a larger mass of students folios were not graded; rather, they served as learning working ‘mediocre fast’ and some who are very slow. opportunities. Accordingly, we gave the pre-service teach- (All get the same tasks, and all have to work on them at ers learning feedback following the idea of FA. At the end the same time.) Mondays, however, there is a learning of the semester, however, the oral examination is a graded time, which forces the children to work independently certification exam that summatively assesses the pre-service on either the subjects German, English or Mathemat- teachers’ professional teaching competence. Since the topic ics in order to complete and/or practice tasks/content of the oral examination could be determined independently from the lessons. The students can use this time how- by the pre-service teachers and based on their own experi- ever they choose. Apart from that, there are sometimes ences, the oral examination also took up aspects of FA. The 1 3 Combining and integrating formative and summative assessment in mathematics teacher education 725 integration of SA and FA therefore coalesces in the sum- her observations. In preparation for the oral examination, she mative assessment of formative performance components deepened her knowledge and was able to relate her experi- (which, however, now are evaluated and graded in the exam- ences to the—however implicit—theoretical background of ination according to the criteria mentioned in Sect. 4.1). the topic by dealing with legal requirements and finding fur - We are able to integrate the findings from this kind of SA ther possibilities for die ff rentiation measures stemming from and FA on an individual basis. Identical situation-specific research on mathematics education such as, for example, aspects of the pre-service teachers’ professional competence tasks for natural differentiation (Scherer and Krauthausen can be found in their eportfolio entries, and in the topics 2010). She also describes experiences from her own lessons, for the oral examination on which the pre-service teachers in which she used several differentiation measures to deal elaborated in the mind maps, indicating that the examination with the heterogeneity of the learning group—a demonstra- assessed teaching competence that the pre-service teachers tion of her situation-specific skills. It is also important to learned during the seminar and their internship. note that she critically reflects the limitations of differen- One example of how the FA influenced the SA in that way tiation measures based on her own and observed teaching appeared in Leonie’s oral examination. In the course of her practice. experiences during the internship, she continued to deepen the topic of measures for differentiation and individualiza- tion and chose this as the topic of her oral examination. The 6 Conclusions mind map she submitted clearly illustrates the questions and problems she addressed (see Fig. 2). 6.1 Combining and integrating FA and SA By integrating the results of the analysis of her eportfolio entries (recall Sect. 5.2) with her mind map, we can see that By combining different forms of assessment in the she did not stop at developing a critical attitude according to framework of a mixed-assessment approach, we hoped to Fig. 2 Leonie’s mind map (translated by the authors) 1 3 726 N. F. Buchholtz et al. determine the best possible evidence of pre-service teach- 6.2 Addressing validity with mixed‑assessment ers’ learning opportunities and situation-specific skills. approaches With SA (online survey) data, we were able to determine comparatively that the pre-service teachers of the semi- To understand the importance of distinguishing between a nar had significantly higher perceptions of the number combination and an integration of SA and FA in terms of the of learning opportunities regarding heterogeneity and validity of the approach, it is necessary to take our assess- research in mathematics education that were available to ment to a broader theoretical level and analyze approaches to them than were held by comparable pre-service teach- creating validity in qualitative or mixed-methods evaluation ers who did not take the seminar. However, these results approaches (see also Frechtling and Sharp 1997; Gikandi alone are not a sufficient measure for the dispositional et al. 2011). Since different methods entail different weak - aspects of the pre-service teachers’ professional compe- nesses and strengths, Denzin opted for a “methodological tence since it is not clear whether collective differences triangulation”, which consists of a “complex process of play- could be attributed to enrolment in our seminar. We there- ing each method off against the other so as to maximize the fore used the data from the FA, which links the individual validity of field efforts” (Denzin 1978, p. 304). But as in learning opportunities of the pre-service teachers with assessment, the attempt to triangulate different measures can specific situational aspects of the seminar and the school lead to validity problems, because “dier ff ent methods … can internship. In this combination, the FA makes it possi- relate to different empirical phenomena and … it thus may ble to make detailed statements about the types of learn- be difficult to simply compare research results acquired by ing opportunities the pre-service teachers had and about means of different methods in order to check their validity” how their learning processes differed. Both assessment (Kelle and Buchholtz 2015, p. 332). forms provide us with indicators of the specific aspects of This pragmatic view—which is also leading the debate teaching competence of the seminar’s pre-service teach- on mixed-methods evaluations, e.g., in health science (John- ers, thus providing a more comprehensive picture of their son and Schoonenboom 2016)—can stimulate an alternative professional development. understanding of triangulation that can also be transferred to While the combination of SA and FA focuses more on the context of assessment in teacher education or education the mutual complementarity of assessment outcomes on in general. The use of different forms of assessment to elicit different aspects of a phenomenon (in our case disposi- and interpret evidence of performance can be compared with tional and situational learning opportunities), the integra- the examination of a physical object from two different view - tion of SA and FA is more likely to provide convergent points or angles. Both viewpoints provide different pictures findings about a phenomenon from different perspectives. of this object, which may or may not be useful to validate In the eportfolios, we could identify pre-service teach- each other, but regardless may yield a fuller and more com- ers’ situation-specific skills; these skills also formed the plete picture of the object if brought together. So, if SA and basis of their performance in the oral examination, so FA are mixed in any way, the following validity-related out- that the SA validly ref lected the actual learning processes comes can arise (see also Kelle and Buchholtz 2015, p. 333): the pre-service teachers had. But the integration of SA and FA in the oral examination also provided a compa- • The elicited items of evidence of performance converge, rable framework for the assessment and certification of • The elicited items of evidence relate to different aspects teaching competence for all pre-service teachers, regard- of the performance, but are complementary to each other less of the learning processes that took place during the and thus can be used to supplement each other, internship. The situation-specific skills assessed in the • The elicited items that are evidence of performance are FA could be certified by using SA criteria specified by divergent or contradictory, or the examination. By this means, a central challenge of FA • The elicited items of evidence refer to unrelated phenom- in teacher education—namely the inability to standard- ena. ize specific marks of evidence of performance—can be addressed. Furthermore, the integrated approach allowed For multiple reasons, then, it makes sense to combine or not only situation-specific skills to be assessed but also to integrate SA and FA within a mixed-assessment approach, knowledge components, so that professional teaching whether to increase or enhance the validity of the evalu- competence could be assessed with more validity. Thus, ation (Johnson and Schoonenboom 2016). An integration the two assessment approaches developed here—combi- of SA and FA may lead to convergent evidence and thus nation and integration—address different facets of the to valid interpretations, or divergent evidence and validity validity of assessments, which we discuss further below. problems. We encountered these patterns in our analysis of the outcomes of a seminar in teacher education when we integrated the pre-service teachers’ reflections on their field 1 3 Combining and integrating formative and summative assessment in mathematics teacher education 727 observations from the FA with their SA-oriented oral exami- to any form of assessment. Furthermore, our combination nations. In particular, we analyzed how their reflections of FA and SA does not allow any mutual validation of the influenced the determination and the preparation of the topic results of the assessments. Integration of the results of the of their oral examination. We identified convergent evidence, learning opportunities to strengthen the validity of results which indicated that the SA really did assess teaching com- from SA (online survey) and FA (eportfolios)—as is usually petence that was gained during the seminar and the intern- intended in mixed-method approaches—would have been ship, and thus validly assessed teaching competence. When accomplished only if the results of the FA could have been assessing a combination of SA and FA, different aspects of interpreted directly against the background of the results of the performance may yield complementary evidence of per- the SA. For this to happen, participation in the panel survey formance, or it may yield unrelated evidence. In our case, an would have to be made obligatory and non-anonymous, but online survey about pre-service teachers’ perceived learning we refrained from that due to ethical and data-protection opportunities revealed an overview about perceived study reasons. contents on the group level. We could deepen these aggre- Acknowledgements We thank Prof. Dr. Jörg Doll for providing the gated findings and partially explain them with individual data from the ProfaLe online survey. findings from the FA despite the fact that the online sur - vey and the eportfolios focused on different kinds of learn- Open Access This article is distributed under the terms of the Crea- ing opportunities. The analysis of the eportfolios revealed tive Commons Attribution 4.0 International License (http://creat iveco mmons.or g/licenses/b y/4.0/), which permits unrestricted use, distribu- complementary evidence with regard to perceived learning tion, and reproduction in any medium, provided you give appropriate opportunities and how the study contents were related and credit to the original author(s) and the source, provide a link to the applied in pre-service teachers’ teaching practices. Creative Commons license, and indicate if changes were made. Whatever way SA and FA are mixed, the main rationale behind the use of different forms of assessment is always the attempt to compensate for limitations of one form of assessment by drawing on the strengths of another form (see References also Kelle and Buchholtz 2015). The decisive factor here is whether and how SA and FA can be related to each other (or Arnold, K. H., Gröschner, A., & Hascher, T. (Eds.). (2014). Pedagogi- cal field experiences in teacher education: Theoretical founda- can be mixed) and what type of evidence of performance the tions, programmes, processes, and effects. Münster: Waxmann. respective use of an assessment form produces. The form of Bell, B., & Cowie, B. (2001). Formative assessment and science educa- the mixing needs to be described and justified based on the tion. Dordrecht: Kluwer. individual case. Blömeke, S., Bremerich-Vos, A., Kaiser, G., Nold, G., Haudeck, H., Keßler, J.-U., & Schwippert, K. (Eds.). (2013). Professionelle Kompetenzen im Studienverlauf: Weitere Ergebnisse zur Deutsch-, 6.3 Limitations of our approach Englisch- und Mathematiklehrerausbildung aus TEDS-LT. Mün- ster: Waxmann. There are certain limitations in our approach that we have Blömeke, S., Gustafsson, J. E., & Shavelson, R. (2015). Beyond dichotomies: Competence viewed as a continuum. Zeitschrift für to take into consideration when reflecting on the benefits Psychologie, 223(1), 3–13. and challenges of a combination or an integration of SA Blömeke, S., & Kaiser, G. (2016). Understanding the development of and FA. In the combination approach, the indicators of teachers’ professional competencies as personally, situationally learning opportunities are different because of the differ - and socially determined. In D. J. Clandinin & J. Husu (Eds.), International handbook of research on teacher education (Chap. ent assessment forms. An example of this aspect is how 52). Thousand Oaks: Sage. the topic heterogeneity influenced the development of Buchholtz, N., Scheiner, T., Döhrmann, M., Suhl, U., Kaiser, G., teaching competence. While the results of the SA provided & Blömeke, S. (2016). TEDS-shortM. Kurzfassung der math- aggregated but standardized data (e.g., about the perceived ematischen und mathematikdidaktischen Testinstrumente aus TEDS-M, TEDS-LT und TEDS-Telekom. Hamburg: University of learning opportunities with regard to heterogeneity), the Hamburg. analysis of the FA data is conducted on an individual basis Carr, M., & Claxton, G. (2002). Tracking the development of learn- and is still subject to subjective interpretations despite our ing dispositions. Assessment in Education: Principles, Policy and methodological approach (Delandshere and Arens 2003). Practice, 9(1), 9–37. Delandshere, G., & Arens, S. A. (2003). Examining the quality of the Even if the topic heterogeneity was a substantive part of the evidence in pre-service teacher portfolios. Journal of Teacher observation tasks, we were not able to reconstruct in every Education, 54(1), 57–73. eportfolio entry how this topic affected the development of Denzin, N. K. (1978). The research act: A theoretical introduction to pre-service teachers’ professional competence. The useful- sociological methods. New York: McGraw-Hill. Doll, J., Buchholtz, N., Kaiser, G., König, J., & Bremerich-Vos, A. ness of a mixed-assessment approach can in general only (2018). Nutzungsverläufe für fachdidaktische Studieninhalte der be estimated based on individual cases and with regard to Fächer Deutsch, Englisch und Mathematik im Lehramtsstudium: the assessment purposes. However, this limitation applies 1 3 728 N. F. Buchholtz et al. Die Bedeutung der Lehrämter und der Zusammenhang mit Putnam, R. T., & Borko, H. (2000). What do new views of knowledge Lehrinnovationen. Zeitschrift für Pädagogik. (in press). and thinking have to say about research on teacher learning? Edu- Frechtling, J., & Sharp, L. (Eds.). (1997). User-friendly handbook for cational Researcher, 29(1), 4–15. mixed method evaluations. Arlington: National Science Founda- Sadler, D. R. (1989). Formative assessment and the design of instruc- tion, Division of Research, Evaluation, and Communication. tional systems. Instructional Science, 18, 145–165. Gikandi, J. W., Morrow, D., & Davis, N. E. (2011). Online formative Scherer, P., & Krauthausen, G. (2010). Natural differentiation in math- assessment in higher education: A review of the literature. Com- ematics—The NaDiMa project. Panama Post, 3, 14–16. puter and Education, 57(1), 2333–2351. Schoenfeld, A. H. (2011). How we think: A theory of goal-oriented Harlen, W., & James, M. (1997). Assessment and learning: Differences decision making and its educational applications. New York: and relationships between formative and summative assessment. Routledge. Assessment in Education: Principles, Policy and Practice, 4(3), Shavelson, R. J. (2006). On the integration of formative assessment in 365–379. teaching and learning with implications for teacher education. In Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Paper prepared for the Stanford Education Assessment Laboratory Educational Research, 77(1), 81–112. and the University of Hawaii Curriculum Research and Develop- Heublein, U., Hutzsch, C., Schreiber, J., Sommer, D., & Besuch, ment Group. https ://web.stanf ord.edu/dept/SUSE/SEAL/Repor G. (2010). Ursachen des Studienabbruchs in Bachelor- und in ts_P aper s/On%20t he %20Int eg r at ion%20of%20F or mativ e%20 herkömmlichen Studiengängen. Hannover: HIS. (Forum Hoch-Ass essme nt_Teach er%20Ed_Final .doc. Accessed 29 May 2018. schule F2/2010). Sherin, M. G., & Van Es, E. A. (2009). Effects of video club participa- Jafari, A., & Kaufman, C. (Eds.). (2006). Handbook of research on tion on teachers’ professional vision. Journal of Teacher Educa- ePortfolios. New York: Idea Group Reference. tion, 60, 20–37. Johnson, R. B., & Schoonenboom, J. (2016). Adding qualitative and Shulman, L. S. (1986). Those who understand. Knowledge growth in mixed methods research to health intervention studies: Interacting teaching. Education Researcher, 15, 4–14. with differences. Qualitative Health Research, 26(5), 587–602. Spanneberg, R. (2009). The mathematics teaching portfolio: A reflec- Kaiser, G. (2015). Professionelles Lehrerhandeln zur Förderung fach- tive tool for developing professional growth and improving class- lichen Lernens unter sich verändernden gesellschaftlichen Bed- room practices. In B. Clarke, B. Grevholm & R. Millman (Eds.), ingungen (ProfaLe). BMBF-Qualitätsoffensive Lehrerbildung in Tasks in primary mathematics teacher education. Purpose, use Hamburg. In Presentation at the University of Hamburg. https :// and exemplars (pp. 51–65). New York: Springer. www.profale.uni-hambu r g.de/forschung/v ortraeg e/qlb---uk-jahre Taras, M. (2005). Assessment—summative and formative—some theo- stagu ng-vortr ag-29062 016.pdf. Accessed 29 May 2018. retical reflections. British Journal of Educational Studies, 53(4), Kaiser, G., Blömeke, S., König, J., Busse, A., Döhrmann, M., & Hoth, 466–478. J. (2017). Professional competencies of (prospective) mathemat- Tatto, M. T., Schwille, J., Senk, S., Ingvarson, L., Peck, R., & Rowley, ics teachers: Cognitive versus situated approaches. Educational G. (2012). Teacher education and development study in math- Studies in Mathematics, 94(2), 161–182. ematics (TEDS-M): Policy, practice, and readiness to teach pri- Kaiser, G., Busse, A., Hoth, J., König, J., & Blömeke, S. (2015). mary and secondary mathematics. Conceptual framework. East About the complexities of video-based assessments: Theoretical Lansing: Teacher Education and Development International Study and methodological approaches to overcoming shortcomings of Center, College of Education, Michigan State University. research on teachers’ competence. International Journal of Sci- Tillema, H. (2010). Formative assessment in teacher education and ence and Mathematics Education, 13, 369–387. teacher professional development. In B. McGaw, E. Baker & Kelle, U., & Buchholtz, N. (2015). The combination of qualitative P. Peterson (Eds.), International encyclopedia of education and quantitative research methods in mathematics education: A (pp. 563–571). Oxford: Reed Elsevier. “mixed methods” study on the development of the professional Topping, K. (1998). Peer assessment between students in colleges and knowledge of teachers. In A. Bikner-Ahsbahs, C. Knipping & N. universities. Review of Educational Research, 66, 249–276. Presmeg (Eds.), Approaches to qualitative research in mathemat- Vogel, R. (2018). Portfolioarbeit in der Grundschullehrerausbildung— ics education. Examples of methodology and methods (pp. 321– “Lernraum” zur Anbahnung eines professionellen mathematik- 361). Dordrecht: Springer. didaktischen und mathematischen Unterrichtshandelns. In R. Mackey, J. (2009). Virtual learning and real communities: Online Möller & R. Vogel (Eds.), Innovative Konzepte für die Grunds- professional development for teachers. In E. Stacey & P. Gerbic chullehrerausbildung im Fach Mathematik (pp. 197–218). Wies- (Eds.), Effective blended learning practices: Evidence-based per - baden: Springer. spectives in ICT-facilitated education (pp. 163–181). Hershey: Wiliam, D. (2000). Integrating formative and summative functions ICI Global. of assessment. In Paper presented to Working Group 10 of the Mayring, P. (2015). Qualitative Inhaltsanalyse. Grundlagen und Tech- 9th International Congress on Mathematics Education (ICME 9) niken. Weinheim: Beltz. in Tokyo. http://www .dylan wilia m.or g/Dylan _W ilia ms_websi Mokken, R. J., & Lewis, C. (1982). A nonparametric approach to the te/Paper s_files /ICME9 %20WGA 10%20pap er.doc. Accessed 29 analysis of dichotomous item responses. Applied Psychological May 2018. Measurement, 6, 417–430. Wiliam, D., & Black, P. (1996). Meanings and consequences: A basis Orschulik, A. (2016). Strengthening cooperation within school practi- for distinguishing formative and summative functions of assess- cal activities as a way to improve teacher education. In Poster ment? British Educational Research Journal, 22(5), 537–548. presented at the 13th International Congress on Mathematics Edu- Zeichner, K. M. (2010). Rethinking the connection between campus cation (ICME-13) in Hamburg. https ://www.profa le.uni-hambu courses and field experiences in college- and university-based rg.de/forschung/pos ter vortr aege/pos ter -orschulik -2016-icme.pdf. teacher education. Journal of Teacher Education, 61(1–2), 89–99. Accessed 15 June 2017. 1 3

Journal

ZDMSpringer Journals

Published: May 29, 2018

References

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off