Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Exploring how teachers suggest informing designs for learning

Exploring how teachers suggest informing designs for learning Received 16 February 2021 Purpose – An issue that has received limited attention is how teachers make informed decisions when Revised 8 June 2021 designing learning activities, which makes effective use of resources and technologies. The aim of this paper is Accepted 6 July 2021 to explore how teachers suggest informing their designs for learning. Design/methodology/approach – The study is based on documentation from workshops where adult education teachers were encouraged to suggest how a design for learning can draw on what is known, how the design can be evaluated and how the design and lessons learned can be shared. The data was analyzed using thematic analysis. Findings – Most teachers did not explicitly suggest how to inform their designs for learning, although some suggested to draw on learning theories. The teachers were able to provide many examples of how their designs for learning could be evaluated based on student perceptions and assessment. They primarily suggested to discuss and share their designs and lessons learned at the school. Practical implications – The workshop format could be adapted and used in practice. Originality/value – Relatively little attention has been paid to how teachers inform their designs for learning. A key challenge for teachers is to understand design for learning as an informed practice, which could be integrated as part of their daily work. Keywords Design for learning, Digital technologies, Informed design, Evaluation, Sharing Paper type Research paper Introduction Learning design aims to support teachers to make pedagogically informed decisions when designing learning activities, which makes effective use of resources and technologies (Conole, 2013). Another aim of learning design is to make the design process more explicit and shareable (Agostinho et al., 2011). Learning design approaches help teachers to shift from focusing on content to learning activities and student experiences (Conole, 2013). However, learning itself cannot be designed, we can only attempt to design for learning (Kali et al.,2011). Therefore, the term design for learning is preferred in this paper. Many models attempt to guide teachers to take advantage of digital technologies when designing for learning (Bower and Vlachopoulos, 2018). An issue that has received limited attention is how to inform designs for learning (McKenney et al., 2015), although there are exceptions. For example, Michos and Hernandez- Leo (2018) studied teacher communities that discussed designs for learning. They showed how data and visualizations can foster in-depth reflection. Teachers could be informed by different resources, such as research and best practices, by reflecting on how such resources could be applied in their teaching context (Oliver and Conole, 2003). Being informed is also about reflecting and acting on analyses and evaluations of designs for learning to improve student learning and the quality of teaching (Zwozdiak-Myers, 2018). Thus, informing designs for learning could be based on resources, but also reflections and evaluations of previous designs. The International Journal of © Stefan Hrastinski. Published by Emerald Publishing Limited. This article is published under the Information and Learning Technology Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full pp. 381-392 Emerald Publishing Limited attribution to the original publication and authors. The full terms of this licence may be seen at http:// 2056-4880 creativecommons.org/licences/by/4.0/legalcode DOI 10.1108/IJILT-02-2021-0036 Laurillard (2012) argues that teachers should use what is known and iteratively improve IJILT designs for learning, which lead us to questions, such as the following: How can teachers 38,4 inform their designs for learning by drawing on what is known, such as research findings and best practices? How can teachers evaluate their designs for learning to improve the designs and their knowledge? How can teachers share their designs and what they learn? This paper aims to explore how teachers suggest informing their designs for learning. Teachers are suggested to inform their design ideas based on what is currently known and continually improve their design ideas by conducting evaluations. Moreover, teachers are suggested to share their designs for learning and lessons learned to support the improvement of what communities of teachers can accomplish (Scardamalia and Bereiter, 1994). The study of this paper is based on workshops where adult education teachers were encouraged to suggest how their designs for learning could be informed and evaluated and how the designs and lessons learned could be shared. The adult education teachers were teaching courses corresponding to K12 education aimed at adult students that need to complement their previous studies, for example, to obtain an upper secondary diploma, to achieve eligibility requirements for university studies or Swedish for immigrants. The paper is guided by the following research questions: RQ1. How did the teachers suggest drawing on what is known when designing for learning? RQ2. How did the teachers suggest evaluating their designs for learning? RQ3. How did the teachers suggest sharing their designs for learning and lessons learned? Theoretical background In this section, a background is provided to drawing on what is known, evaluation of design for learning and sharing designs for learning and lessons learned, all of which could be considered to be key aspects when regarding design for learning as an informed practice. Drawing on what is known Designs for learning are often based on implicit assumptions and personal reflections (Bates, 2015; Oliver et al., 2017; Price and Kirkwood, 2014). Teachers are faced with many complex variables and need to make decisions on how to design and teach a course to the best of their knowledge and based on “what is known” (Bates, 2015). Different types of resources could inform design for learning, which are discussed below. These are learning theory, research findings, best practices and teacher knowledge. Learning theory. There are many different theories of learning, which at first might seem conflicting but might be complementary (Sfard, 1998). Many learning design models consider learning theory in some way. While some learning design models are flexible (Young and Perovic, 2016), others are underlined by certain learning theories, typically socioconstructivist theories (Bower and Vlachopoulos, 2018). It can be noted that many learning design models consider learning theory in some way. While some learning design models are flexible (Young and Perovic, 2016), others are underlined by certain learning theories, typically socioconstructivist theories (Bower and Vlachopoulos, 2018). Research findings. It has been argued that teaching should be a research-based profession (Hargreaves, 2007). More realistically, it can be claimed that research should be regarded as one factor that could inform teachers’ work (Biesta, 2007). Encouraging teachers to find research on relevant issues is sensible if they are encouraged to use it critically and thoughtfully and by reflecting on how it can be used to inform their design work (Oliver and Conole, 2003). Although models that aim to connect teaching and research have been proposed over the years (Atkinson, 2001), it is unusual that designs for learning are based on Informing research (Price and Kirkwood, 2014). Many teachers need support in curating and tailoring designs for research to increase their use of research (Joram et al., 2020). learning Best practices. The term best practice suggests that there is a best approach that is the optimal solution to complex problems (Coffield and Edward, 2009). Hargreaves (2004) argues that “‘good practice’ and ‘best practice’ are [often] treated as synonyms, although clearly ‘best practice’ suggests a practice that has been compared with others and has proved itself better than other ‘good’ practices” (p. 72). It might be worthwhile sharing teaching experiences and designs even though they have not been found “better” than other practices. What might be preferable or “best” is often a judgment that needs to be made by teachers in ways that are relevant in their contextualized settings (Biesta, 2007). Teacher knowledge. Teacher knowledge is based on reflective thinking, described as “active, persistent and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it and the further conclusion to which it tends” (Dewey, 1933, p. 9). Shulman (1986) introduced the idea of pedagogical content knowledge (PCK). Instead of focusing on subject matter knowledge or pedagogy, he suggested that it is important to consider the relationship between the two. PCK is about “the ways of representing and formulating the subject that make it comprehensible to others” (Shulman, 1986, p. 9). In their extension of PCK, Mishra and Koehler (2006) argue that teachers also need technological knowledge, which in turn interacts with pedagogical and content knowledge. Based on the fact that teacher knowledge is complex and situated, it is argued that teachers need to adapt what they do to different students and contexts (Wilson et al., 1987). In Figure 1, different types of resources that could inform designs for learning are summarized. These resources range from being more personal and contextual, which are typically very relevant to the individual teacher, to more collective and abstract, which are of interest to many but need to be adapted critically and thoughtfully (Oliver and Conole, 2003). Evaluation of design for learning Evaluation can simply be defined as “determining the effectiveness.” This has little meaning until we answer the question: “In terms of what?” (Kirkpatrick and Craig, 1967). Course evaluations are often teaching and teacher-focused. They have a fire alarm function rather than focusing on course development (Edstr€om, 2008). The complex nature of digital education calls for using multiple methods and sources of data to understand student learning, satisfaction and participation. Quantitative analyses can yield useful information on learning outcomes and related variables. Qualitative data can help us better understand observed differences and student perceptions (Gunawardena et al., 2000). Bennett et al. (2017) investigated teachers’ design work and found that more than half of the interviewed teachers described that they identified future changes, typically after the course was completed, but some also made notes of their ideas during the course. These reflective activities contributed to developing teacher knowledge and iteratively refining their courses over multiple offerings. Thus, we can distinguish between two types of evaluation: formative evaluation while teachers teach and summative evaluation after they have taught. Hrastinski (2021) explored how 31 blended and online courses were evaluated by Learning Best pracces theory Figure 1. COLLECTIVE AND PERSONAL AND Examples of how ABSTRACT CONTEXTUAL designs for learning Teacher Research could be informed knowledge findings experienced practitioners as part of an edited book. They used course evaluations, IJILT questionnaires as a complement to standardized course evaluations, data from digital 38,4 platforms, assessment results, focus groups, written reflections from students and class discussions with students. These teachers can be assumed to be early adopters of technology (Rogers, 2010). A question that needs to be further explored is how we can engage, not only the early adopters and those with a special interest, but most time-pressed teachers in iterative evaluation and improvement of their designs for learning. Sharing designs for learning and lessons learned One of the aims of design for learning is to share the designs (Conole and Wills, 2013). There is also a long tradition to develop digital technologies to support codesign and collaboration between teachers (Hernandez-Leo et al.,2014). Different types of structured and semistructured representations, such as pedagogical patterns (Bergin et al.,2012), have been suggested as ways to share designs for learning, although these have not been adopted by most teachers (Oliver et al., 2017). Less focus has been put on how to share lessons learned (Conole and Wills, 2013; Laurillard, 2012). Teachers typically enjoy sharing experiences with colleagues, especially within their subject discipline (Conole and Wills, 2013; Oliver et al., 2017). Prestridge (2019) has developed a categorization of teachers’ use of social mediated spaces for their professional learning, which indicates different ways teachers could share design ideas and lessons learned in digital settings. The categorization includes four categories: info-seekers that seek to find new knowledge or ideas and resources from social mediated spaces; info-networkers that also share the ideas and resources with colleagues; self-seeking contributors that share knowledge, ideas or curriculum materials to get feedback, ask specific questions or acquire additional information; and vocationalists that engage as a member of a community of learning that aims to contribute with new knowledge. Collectively sharing and improving ideas is an important dimension of situated learning and knowledge development through awareness and reflection (van der Linden and McKenney, 2020; Scardamalia and Bereiter, 2003). Method Research setting The empirical data is based on workshops with adult education teachers. The purpose of the workshop for each participant was to suggest a design for learning, how the design drew on what is known, how to evaluate the design and how to share the design and lessons learned. The workshops were facilitated by the author as part of a teacher conference on digital learning in a Swedish region. The introduction to the workshop was intentionally short in order not to bias the results. The workshop instructions are presented in Table 1. The process of the workshops was as follows: (1) introduction by the facilitator, (2) individual work, (3) pairs of teachers provided feedback to each other, (4) individual work and (5) brief presentations of some of the designs for learning. All designs for learning were documented Background Describe the module, e.g. name, purpose, level, scope/credits, approximate number of participants Design Describe and motivate how the module will be designed Why and how should digital technologies be used?* Evaluate How should the module be evaluated?* Share How can lessons learned be shared with colleagues?* Locally, nationally, internationally? Note(s): *Motivate based on resources and experience. Examples of resources and experience: course evaluations, research papers, books, your experience, experience of colleagues, blogs and social media and Table 1. Workshop instructions best practices by using a digital form. Each participant individually submitted answers according to the Informing themes/question outlined in Table 1. designs for Two workshops were conducted for 60 min with nine participants in each workshop (see learning Table 2). There were 11 females and seven males. The years of teaching experience ranged from 1.5 to 40 years (M 5 17, SD 5 10). The teachers represented many different subject disciplines. Two of the teachers (F8a and F8b) worked together in the workshop. The study complies with research ethics rules and guidelines (Swedish Research Council, 2017). It was voluntary to participate in this study and the data has been anonymized. All teachers who completed the digital form chose to be part of the study. Data collection and analysis The documentation from the workshops (i.e. the text submitted by using a digital form) was analyzed using thematic analysis, which can be described as a method for identifying, analyzing and reporting themes within data (Braun and Clarke, 2006). The process of coding is an essential part of thematic analysis (Miles and Huberman, 1994), where data are organized into meaningful themes (Braun and Clarke, 2006). All the texts from the digital forms were anonymized and exported to text format. The data were imported and analyzed by using the open-source software RQDA that is used for qualitative data analysis (Huang, 2014). A semantic approach was adopted since the documentation from the workshops was in written form. This means that the explicit meanings of the data (i.e. what the participants had written) were analyzed (Braun and Clarke, 2006). The themes were deductive and based on the research questions. The subthemes were identified inductively. Braun and Clarke (2006) provide guidelines for doing thematic analysis. Qualitative analysis guidelines are not rules and should be applied flexibly to fit the research questions and data (Patton, 1990). In Table 3, the phases of the thematic analysis are described. The description of the process has been adapted from Braun and Clarke (2006). Qualitative analysis is a recursive process where movement between the phases is typically necessary, which was also the case in this study (Braun and Clarke, 2006). Results The themes and subthemes, which are described in more detail below, are presented in Figure 2. Teacher Workshop Gender Teaching experience (years) Subject discipline M1 1 Male 30 Philosophy F2 1 Female 10 Health M3 1 Male 17 Mathematics F4 1 Female 17 Literature M5 1 Male 1.5 Mathematics F6 1 Female 3 Psychiatry M7 1 Male 29 English F8a 1 Female 2 Health F8b 1 Female 20 Health F9 2 Female 13 History F10 2 Female 9 Swedish for immigrants M11 2 Male 9 Swedish M12 2 Male 20 Biology F13 2 Female 13 Study skills F14 2 Female 40 Chemistry Table 2. F15 2 Female 21 Swedish F16 2 Female 30 Health Participants of the M17 2 Male 13 Digital skills workshops IJILT Phase Description of the process 38,4 Familiarizing yourself with the Data were read and reread, and initial ideas were noted data Generating initial codes Initial codes were generated. Data relevant to each code were collated Collating codes into deductive The codes were collated into deductive themes based on the research themes questions Inductively searching for The subthemes were inductively identified subthemes Reviewing themes It was checked if the coded extracts work in relation to the themes, subthemes and the entire data set Defining and naming themes Names and definitions of each sub theme were generated Table 3. Producing the report Selection of extracts to quote and production of the paper Phases of the thematic analysis Source(s): Adapted from Braun and Clarke (2006),p. 87 RQ1. Drawing on what is Learning known theory Student perceptions RQ2. Evaluation of design for learning Assessment Collegial learning RQ3. Sharing designs for Figure 2. learning and lessons Subthemes organized learned Digital sharing of according to research designs questions Drawing on what is known There were few examples where the teachers explicitly described how they informed their design, despite being encouraged to do so. Only one subtheme was identified: learning theory. In the learning theory theme, the teachers suggested using different theoretical perspectives. This can be linked to how repetition is important for learning. Behaviorism - repetition. (M5) The circle model [genre pedagogy], joint reading and processing of central parts on smartboards. (F10) Given the socio-cultural perspective, it is important that students get the chance to create knowledge together. This can lead to different views. (F6) Vygotsky maintains that students need inspiration, motivation and encouragement in their learning. (F8) Quick feedback is known to be important for learning, feasible with digital tools. (F8) There was one exception, a teacher explicitly acknowledging teacher experience. My experience is that students also have to learn how to produce mathematics (writing on a paper) Informing to gain knowledge. (F6) designs for learning Evaluation of design for learning The teachers suggested different ways to evaluate their designs. Two subthemes were identified: student perceptions and assessment. Three teachers suggested conducting a discussion with the students. [T]he group evaluates orally: How did it feel? What was positive? What could have been done differently? (M5) Three teachers suggested designing a survey. Survey.. . With questions about how the student experienced the way of working and what they have learned. (M7) [S]urvey on the different parts (recorded lessons, test questions and collaboration tasks). (M11) Two teachers proposed using a standardized course evaluation survey. One of these teachers suggested complementing the survey with open questions that the students were to complete in writing. Another teacher exemplified what kind of open questions to use. Finally, the student evaluates in writing with questions of the type: How does it feel now? Would you be able to give a speech again? Would you be able to give a speech outside the school?” (M5) Two teachers suggested conducting an evaluation together with the students by asking specific questions using the smartboard or a smartphone app. Quick evaluation on the smartboard by the students. Did you learn imperatives? Can you use it? (F9) One teacher suggested conducting individual informal discussions with students during the course and a more formal discussion on the course and their learning after the course. The second subtheme was how to use assessment to evaluate the design for learning. Two teachers suggested analyzing the quality of student texts. Two teachers suggested following student development over time to see if their abilities improved. For example, in one course on study skills, it was suggested to “check that the students de facto find information faster” (F13). Two teachers suggested using automated quizzes to give the students and teacher fast feedback. One teacher suggested conducting a pretest–posttest where “students answer a new quiz with a similar design but new questions” (F6) to investigate if student results improved following the intervention. Similarly, one teacher suggested comparing student results with previous years. One teacher wanted to conduct informal individual discussions with students throughout the course and a more formal discussion after the course with a focus on the course design and student learning. [D]iscussion in conjunction with feedback on examination, as this is the first part of the course, so you can adjust the upcoming and a more thorough evaluation after the course has been completed. (M17) Sharing designs for learning and lessons learned The teachers suggested different ways to share their designs for learning and lessons learned. Two subthemes were identified: collegial learning and digital sharing of designs for learning. Six teachers suggested different ways to share experiences at the school. Pedagogical meetings among colleagues. (M7) Teachers in different subjects at the school can be involved. (F15) Me and the other responsible teacher will probably show other colleagues who are interested in a IJILT similar distance structure. (M17) 38,4 Four teachers proposed sharing experiences in their teacher team: “conversations at the subject meeting at work” (F4) and “[w]e share within the working group” (F8). As evident above, most suggestions in the theme collegial learning concerned experiences at the school. However, two teachers suggested sharing experiences in regional networks: “collaboration between municipalities within established networks” (F15). The second subtheme was digital sharing of designs for learning. Five teachers provided examples of how to share their design with their colleagues by sharing files in shared folders and learning management systems at the school. The following is an example of a more systematic approach. The idea is that I will build together and try out a material that will later be handed over to other [Swedish for immigrants] colleagues so that they have something to start from and be able to work with in future classes. (F13) Two teachers suggested to share the design on a blog and one teacher suggested YouTube. Discussion The first research question investigated how teachers suggested drawing on what is known when designing for learning. There were some examples where teachers were inspired by different learning theories: behaviorism, genre pedagogy and sociocultural learning theory. This finding can be compared with current learning design models that sometimes are underpinned by learning theories, typically socioconstructivist theories (Bower and Vlachopoulos, 2018). Most teachers who used learning theory as a foundation gave exclusivity to one learning theory, which might serve the interest of certain students but might disadvantage others (Sfard, 1998). Here, learning design models, such as the ABC model, which encourage basing designs on different, complementary “learning types” (Young and Perovic, 2016) might be useful to encourage teachers to consider alternative or complementary approaches. In line with previous research, the teachers did not draw on research findings. Teachers might need support in curating and tailoring research to increase their use of research (Joram et al., 2020). Thus, it seems necessary to support teachers in how to identify research findings that could inform their designs for learning. There was only one example where teacher experience was explicitly recognized. These findings can be compared with Agostinho et al. (2018) that identified the types of support the teachers reported using in their design work: colleagues, literature, workshops and seminars, conferences, institutional support services and enrolment in postgraduate study. It is notable that few teachers explicitly described how they used resources to inform their design, despite being encouraged to do so. This finding is aligned with previous research, which found that teachers followed a top-down approach where they initially considered the learning outcomes, scope of content, ideas for activities and assessment strategy (Bennett et al., 2017), but typically do not draw on external sources of knowledge, such as research findings (Price and Kirkwood, 2014). The second research question investigated how teachers suggested evaluating their designs for learning. We can distinguish between two types of evaluation: formative evaluation while teachers teach and summative evaluation after they have taught. The teachers provided many different examples of how to evaluate their designs, by drawing on student perceptions and assessment. They suggested investigating student perceptions by discussions with students, in some cases complemented by using a smartboard or app to ask specific questions as a basis for the discussion, surveys, complementing standardized course evaluation surveys with open questions and individual discussions with students during and after the course. The teachers proposed drawing on assessment in different ways: analyzing Informing the quality of student texts, following student development over time, automated quizzes, designs for pretest–posttest of student results and comparing student results with previous years. These learning findings can be compared with a study that explored how university teachers evaluated 31 blended and online courses. They used course evaluations, questionnaires, data from digital platforms, assessment results, focus groups, written reflections from students and class discussions with students (Hrastinski, 2021). Thus, there are many different methods that teachers can use when evaluating designs for learning. The complex nature of evaluating digital education calls for using multiple methods and sources of data to understand student learning, satisfaction and participation (Gunawardena et al., 2000). There seems to be a need to develop a more thorough understanding of the benefits and limitations of using the identified evaluation methods. The third research question investigated how teachers suggested sharing their designs for learning and lessons learned. They proposed drawing on collegial learning to share and discuss experiences, mainly at the school and in teacher teams. The teachers also suggested sharing designs digitally, primarily with colleagues at the school, but there were also occasional suggestions to share designs by using blogs and YouTube. Thus, the teachers seemed willing to move beyond information consumption toward sharing designs for learning and lessons learned (Prestridge, 2019), but preferably at their school. Although researchers envision different ways that teachers could share their designs and lessons learned, ranging from design representations to journal papers (Bergin et al., 2012; Bernstein and Bass, 2005), the findings of this paper are aligned with research suggesting that teachers typically want to discuss their ideas, experiences and designs with colleagues, preferably from their subject discipline (Conole and Wills, 2013; Oliver et al., 2017). In light of these findings, it might be more realistic to accept that most teachers prefer to share designs for learning and lessons learned at their school. If a school encourages at least some teachers to take on more outbound roles, the school will have an influx of ideas, designs for learning and lessons learned, which could be adapted by colleagues to their local context. Limitations and further research A limitation of this study is that it is based on workshops with teachers. The teachers suggested a design for learning, how to draw on what is known, how to evaluate the design and how to share the design and lessons learned. What the teachers plan to do will not necessarily correspond with what they do in practice. It would be valuable to explore similar research questions in “real” design and teaching practice. That said, it is also important that research explores teacher ideas and suggestions, which can guide future research and practice. It also needs to be acknowledged that this study was based on a specific type of adult education corresponding to K12 education aimed at adult students that need to complement their previous studies, for example, to obtain an upper secondary diploma, to achieve eligibility requirements for university studies or Swedish for immigrants. This paper and related studies have identified different ways that teachers draw on what is known to inform their designs for learning (Agostinho et al., 2018) and how they evaluate their designs for learning (Hrastinski, 2021). It was surprising that other types of resources, such as best practices and experiences of colleagues were not acknowledged. Previous research has found that teachers enjoy sharing experiences with other teachers (Conole and Wills, 2013), but a question that remains is to what extent and how such experiences are then used to inform teachers’ design work. There are opportunities to develop more systematic approaches to how designs for learning could be informed and evaluated. One such example is how learning analytics could be used to inform designs for learning (Michos and Hernandez-Leo, 2020; Wiley et al., 2020). However, as found in this paper and related research (Agostinho et al., 2018; Hrastinski, 2021), there are many other resources and methods for IJILT evaluation that could support informing designs for learning, which also needs to be further 38,4 developed and investigated in future research. Conclusions This paper explored how adult teachers suggested informing and evaluating their designs for learning and how they suggested sharing these designs and lessons learned, all of which could be considered to be key elements of informed design for learning. It was found that teachers typically referred to learning theories to inform their designs for learning. However, other types of resources, such as research findings, best practices and experiences of colleagues, were not explicitly suggested. Teachers seem to need support to identify research and to understand how research findings are relevant in their setting (Joram et al., 2020). The teachers provided many examples of how to evaluate their design based on student perceptions and assessment. They suggested discussing their experiences with colleagues and sharing their designs digitally, although most preferred doing this locally at the school. In conclusion, the teachers were aware of how their designs for learning could be evaluated and how to share the designs and lessons learned at the school. Teachers seem to need support to understand how to inform their designs. They could also be encouraged to share their designs and lessons learned beyond their school when it makes sense. A key challenge for future research is to put more focus on design for learning as an informed practice, to support the development of teacher knowledge and the improvement of future designs for learning. References Agostinho, S., Bennett, S., Lockyer, L. and Harper, B. (2011), “The future of learning design”, Learning, Media and Technology, Vol. 36 No. 2, pp. 97-99. Agostinho, S., Lockyer, L. and Bennett, S. (2018), “Identifying the characteristics of support Australian university teachers use in their design work: implications for the learning design field”, Australasian Journal of Educational Technology, Vol. 34 No. 2, pp. 1-15. Atkinson, M.P. (2001), “The scholarship of teaching and learning: reconceptualizing scholarship and transforming the academy”, Social Forces, Vol. 79 No. 4, pp. 1217-1229. Bates, T. (2015), Teaching in a Digital Age, BC Campus, Victoria BC. Bennett, S., Agostinho, S. and Lockyer, L. (2017), “The Process of designing for learning: understanding university teachers’ design work”, Educational Technology Research and Development, Vol. 65 No. 1, pp. 125-145. Bergin, J., Eckstein, J., Volter, M., Sipos, M., Wallingford, E., Marquardt, K., Chandler, J., Sharp, H. and Manns, M.L. (2012), Pedagogical Patterns: Advice for Educators, Joseph Bergin Software Tools. Bernstein, D. and Bass, R. (2005), “The scholarship of teaching and learning”, Academe, Vol. 91 No. 4, pp. 37-43. Biesta, G. (2007), “Why ‘what works’ won’t work: evidence-based practice and the democratic deficit in educational research”, Educational Theory, Vol. 57 No. 1, pp. 1-22. Bower, M. and Vlachopoulos, P. (2018), “A critical analysis of technology-enhanced learning design frameworks”, British Journal of Educational Technology, Vol. 49 No. 6, pp. 981-997. Braun, V. and Clarke, V. (2006), “Using thematic analysis in psychology”, Qualitative Research in Psychology, Vol. 3 No. 2, pp. 77-101. Coffield, F. and Edward, S. (2009), “Rolling out ‘good’, ‘best’ and ‘excellent’ practice. What next? Perfect practice?”, British Educational Research Journal, Vol. 35 No. 3, pp. 371-390. Conole, G. (2013), Designing for Learning in an Open World, Springer, New York, NY. Informing Conole, G. and Wills, S. (2013), “Representing learning designs – making design explicit and designs for shareable”, Educational Media International, Vol. 50 No. 1, pp. 24-38. learning Dewey, J. (1933), How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process, DC Heath, Boston. Edstrom, K. (2008), “Doing course evaluation as if learning matters most”, Higher Education Research and Development, Vol. 27 No. 2, pp. 95-106. Gunawardena, C.N., Lowe, C. and Carabajal, K. (2000), “Evaluating online learning: models and methods”, Society for Information Technology and Teacher Education International Conference, San Diego. Hargreaves, D.H. (2004), Learning for Life: The Foundations for Lifelong Learning, Policy Press, Bristol. Hargreaves, D.H. (2007), “Teaching as a research-based profession: possibilities and prospects, (the teacher training agency lecture 1996)”, Educational Research and Evidence-Based Practice, pp. 3-17. Hernandez-Leo, D., Moreno, P., Chacon, J. and Blat, J. (2014), “LdShake support for team-based learning design”, Computers in Human Behavior, Vol. 37, pp. 402-412. Hrastinski, S. (2021), “Teachers as developers of local evidence to improve digital course design”, Interactive Learning Environments, Vol. 29 No. 4, pp. 648-654, doi: 10.1080/10494820.2019. Huang, R. (2014), RQDA: R-Based Qualitative Data Analysis, R Package Version 0.2–8. Joram, E., Gabriele, A.J. and Walton, K. (2020), “What influences teachers’‘buy-in’ of research? Teachers’ beliefs about the applicability of educational research to their practice”, Teaching and Teacher Education, Vol. 88, 102980. Kali, Y., Goodyear, P. and Markauskaite, L. (2011), “Researching design practices and design cognition: contexts, experiences and pedagogical knowledge-in-pieces”, Learning, Media and Technology, Vol. 36 No. 2, pp. 129-149. Kirkpatrick, D.L. and Craig, R. (1967), “Evaluation of training”, in Browning, P.L. (Ed.), Evaluation of Short-Term Training in Rehabilitation, Oregon University, Washington, pp. 35-56. Laurillard, D. (2012), Teaching as a Design Science: Building Pedagogical Patterns for Learning and Technology, Routledge, New York. McKenney, S., Kali, Y., Markauskaite, L. and Voogt, J. (2015), “Teacher design knowledge for technology enhanced learning: an ecological framework for investigating assets and needs”, Instructional Science, Vol. 43 No. 2, pp. 181-202. Michos, K. and Hernandez-Leo, D. (2018), “Supporting awareness in communities of learning design practice”, Computers in Human Behavior, Vol. 85, pp. 255-270. Michos, K. and Hernandez-Leo, D. (2020), “CIDA: a collective inquiry framework to study and support teachers as designers in technological environments”, Computers and Education, Vol. 143, Miles, M.B. and Huberman, A.M. (1994), Qualitative Data Analysis, Sage, Thousand Oaks. Mishra, P. and Koehler, M.J. (2006), “Technological pedagogical content knowledge: a framework for teacher knowledge”, Teachers College Record, Vol. 108 No. 6, pp. 1017-1054. Oliver, M. and Conole, G. (2003), “Evidence-based practice and e-learning in Higher Education: can we and should we?”, Research Papers in Education, Vol. 18 No. 4, pp. 385-397. Oliver, M., Avramides, K., Clark, W., Hunter, J., Luckin, R., Hansen, C. and Wasson, B. (2017), “Sharing teacher knowledge at scale: teacher inquiry, learning design and the representation of teachers’ practice”, Teacher Development, Vol. 22 No. 4, pp. 587-606. Patton, M.Q. (1990), Qualitative Evaluation and Research Methods, Sage, Newbury Park, CA. Prestridge, S. (2019), “Categorising teachers’ use of social media for their professional learning: a self- IJILT generating professional learning paradigm”, Computers and Education, Vol. 129, pp. 143-158. 38,4 Price, L. and Kirkwood, A. (2014), “Using technology for teaching and learning in higher education: a critical review of the role of evidence in informing practice”, Higher Education Research and Development, Vol. 33 No. 3, pp. 549-564. Rogers, E.M. (2010), Diffusion of Innovations, Simon and Schuster, New York. Scardamalia, M. and Bereiter, C. (1994), “Computer support for knowledge-building communities”, Journal of the Learning Sciences, Vol. 3, pp. 265-283. Scardamalia, M. and Bereiter, C. (2003), “Knowledge Building”, Encyclopedia of Education, Macmillan Reference, New York, NY, pp. 1370-1373. Sfard, A. (1998), “On two metaphors for learning and the dangers of choosing just one”, Educational Researcher, Vol. 27 No. 2, pp. 4-13. Shulman, L.S. (1986), “Those who understand: knowledge growth in teaching”, Educational Researcher, Vol. 15 No. 2, pp. 4-14. Swedish Research Council (2017), Good Research Practice, Swedish Research Council, Stockholm. van der Linden, S. and McKenney, S. (2020), “Uniting epistemological perspectives to support contextualized knowledge development”, Educational Technology Research and Development, Vol. 68 No. 2, pp. 703-727. Wiley, K.J., Dimitriadis, Y., Bradford, A. and Linn, M.C. (2020), “From theory to action: developing and evaluating learning analytics for learning design”, Proceedings of the Tenth International Conference on Learning Analytics and Knowledge. Wilson, S.M., Shulman, L.S. and Richert, A.E. (1987), “150 different ways of knowing: representations of knowledge in teaching”, in Calderhead, J. (Ed.), Exploring Teachers’ Thinking, Mansell, pp. 104-124. Young, C. and Perovic, N. (2016), “Rapid and creative course design: as easy as ABC?”, Procedia-Social and Behavioral Sciences, Vol. 228, pp. 390-395. Zwozdiak-Myers, P. (2018), The Teacher’s Reflective Practice Handbook: Becoming an Extended Professional through Capturing Evidence-Informed Practice, Routledge, New York. Corresponding author Stefan Hrastinski can be contacted at: stefanhr@kth.se For instructions on how to order reprints of this article, please visit our website: www.emeraldgrouppublishing.com/licensing/reprints.htm Or contact us for further details: permissions@emeraldinsight.com http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png The International Journal of Information and Learning Technology Emerald Publishing

Exploring how teachers suggest informing designs for learning

Loading next page...
 
/lp/emerald-publishing/exploring-how-teachers-suggest-informing-designs-for-learning-bnkLEvN8sv
Publisher
Emerald Publishing
Copyright
© Stefan Hrastinski
ISSN
2056-4880
DOI
10.1108/ijilt-02-2021-0036
Publisher site
See Article on Publisher Site

Abstract

Received 16 February 2021 Purpose – An issue that has received limited attention is how teachers make informed decisions when Revised 8 June 2021 designing learning activities, which makes effective use of resources and technologies. The aim of this paper is Accepted 6 July 2021 to explore how teachers suggest informing their designs for learning. Design/methodology/approach – The study is based on documentation from workshops where adult education teachers were encouraged to suggest how a design for learning can draw on what is known, how the design can be evaluated and how the design and lessons learned can be shared. The data was analyzed using thematic analysis. Findings – Most teachers did not explicitly suggest how to inform their designs for learning, although some suggested to draw on learning theories. The teachers were able to provide many examples of how their designs for learning could be evaluated based on student perceptions and assessment. They primarily suggested to discuss and share their designs and lessons learned at the school. Practical implications – The workshop format could be adapted and used in practice. Originality/value – Relatively little attention has been paid to how teachers inform their designs for learning. A key challenge for teachers is to understand design for learning as an informed practice, which could be integrated as part of their daily work. Keywords Design for learning, Digital technologies, Informed design, Evaluation, Sharing Paper type Research paper Introduction Learning design aims to support teachers to make pedagogically informed decisions when designing learning activities, which makes effective use of resources and technologies (Conole, 2013). Another aim of learning design is to make the design process more explicit and shareable (Agostinho et al., 2011). Learning design approaches help teachers to shift from focusing on content to learning activities and student experiences (Conole, 2013). However, learning itself cannot be designed, we can only attempt to design for learning (Kali et al.,2011). Therefore, the term design for learning is preferred in this paper. Many models attempt to guide teachers to take advantage of digital technologies when designing for learning (Bower and Vlachopoulos, 2018). An issue that has received limited attention is how to inform designs for learning (McKenney et al., 2015), although there are exceptions. For example, Michos and Hernandez- Leo (2018) studied teacher communities that discussed designs for learning. They showed how data and visualizations can foster in-depth reflection. Teachers could be informed by different resources, such as research and best practices, by reflecting on how such resources could be applied in their teaching context (Oliver and Conole, 2003). Being informed is also about reflecting and acting on analyses and evaluations of designs for learning to improve student learning and the quality of teaching (Zwozdiak-Myers, 2018). Thus, informing designs for learning could be based on resources, but also reflections and evaluations of previous designs. The International Journal of © Stefan Hrastinski. Published by Emerald Publishing Limited. This article is published under the Information and Learning Technology Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full pp. 381-392 Emerald Publishing Limited attribution to the original publication and authors. The full terms of this licence may be seen at http:// 2056-4880 creativecommons.org/licences/by/4.0/legalcode DOI 10.1108/IJILT-02-2021-0036 Laurillard (2012) argues that teachers should use what is known and iteratively improve IJILT designs for learning, which lead us to questions, such as the following: How can teachers 38,4 inform their designs for learning by drawing on what is known, such as research findings and best practices? How can teachers evaluate their designs for learning to improve the designs and their knowledge? How can teachers share their designs and what they learn? This paper aims to explore how teachers suggest informing their designs for learning. Teachers are suggested to inform their design ideas based on what is currently known and continually improve their design ideas by conducting evaluations. Moreover, teachers are suggested to share their designs for learning and lessons learned to support the improvement of what communities of teachers can accomplish (Scardamalia and Bereiter, 1994). The study of this paper is based on workshops where adult education teachers were encouraged to suggest how their designs for learning could be informed and evaluated and how the designs and lessons learned could be shared. The adult education teachers were teaching courses corresponding to K12 education aimed at adult students that need to complement their previous studies, for example, to obtain an upper secondary diploma, to achieve eligibility requirements for university studies or Swedish for immigrants. The paper is guided by the following research questions: RQ1. How did the teachers suggest drawing on what is known when designing for learning? RQ2. How did the teachers suggest evaluating their designs for learning? RQ3. How did the teachers suggest sharing their designs for learning and lessons learned? Theoretical background In this section, a background is provided to drawing on what is known, evaluation of design for learning and sharing designs for learning and lessons learned, all of which could be considered to be key aspects when regarding design for learning as an informed practice. Drawing on what is known Designs for learning are often based on implicit assumptions and personal reflections (Bates, 2015; Oliver et al., 2017; Price and Kirkwood, 2014). Teachers are faced with many complex variables and need to make decisions on how to design and teach a course to the best of their knowledge and based on “what is known” (Bates, 2015). Different types of resources could inform design for learning, which are discussed below. These are learning theory, research findings, best practices and teacher knowledge. Learning theory. There are many different theories of learning, which at first might seem conflicting but might be complementary (Sfard, 1998). Many learning design models consider learning theory in some way. While some learning design models are flexible (Young and Perovic, 2016), others are underlined by certain learning theories, typically socioconstructivist theories (Bower and Vlachopoulos, 2018). It can be noted that many learning design models consider learning theory in some way. While some learning design models are flexible (Young and Perovic, 2016), others are underlined by certain learning theories, typically socioconstructivist theories (Bower and Vlachopoulos, 2018). Research findings. It has been argued that teaching should be a research-based profession (Hargreaves, 2007). More realistically, it can be claimed that research should be regarded as one factor that could inform teachers’ work (Biesta, 2007). Encouraging teachers to find research on relevant issues is sensible if they are encouraged to use it critically and thoughtfully and by reflecting on how it can be used to inform their design work (Oliver and Conole, 2003). Although models that aim to connect teaching and research have been proposed over the years (Atkinson, 2001), it is unusual that designs for learning are based on Informing research (Price and Kirkwood, 2014). Many teachers need support in curating and tailoring designs for research to increase their use of research (Joram et al., 2020). learning Best practices. The term best practice suggests that there is a best approach that is the optimal solution to complex problems (Coffield and Edward, 2009). Hargreaves (2004) argues that “‘good practice’ and ‘best practice’ are [often] treated as synonyms, although clearly ‘best practice’ suggests a practice that has been compared with others and has proved itself better than other ‘good’ practices” (p. 72). It might be worthwhile sharing teaching experiences and designs even though they have not been found “better” than other practices. What might be preferable or “best” is often a judgment that needs to be made by teachers in ways that are relevant in their contextualized settings (Biesta, 2007). Teacher knowledge. Teacher knowledge is based on reflective thinking, described as “active, persistent and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it and the further conclusion to which it tends” (Dewey, 1933, p. 9). Shulman (1986) introduced the idea of pedagogical content knowledge (PCK). Instead of focusing on subject matter knowledge or pedagogy, he suggested that it is important to consider the relationship between the two. PCK is about “the ways of representing and formulating the subject that make it comprehensible to others” (Shulman, 1986, p. 9). In their extension of PCK, Mishra and Koehler (2006) argue that teachers also need technological knowledge, which in turn interacts with pedagogical and content knowledge. Based on the fact that teacher knowledge is complex and situated, it is argued that teachers need to adapt what they do to different students and contexts (Wilson et al., 1987). In Figure 1, different types of resources that could inform designs for learning are summarized. These resources range from being more personal and contextual, which are typically very relevant to the individual teacher, to more collective and abstract, which are of interest to many but need to be adapted critically and thoughtfully (Oliver and Conole, 2003). Evaluation of design for learning Evaluation can simply be defined as “determining the effectiveness.” This has little meaning until we answer the question: “In terms of what?” (Kirkpatrick and Craig, 1967). Course evaluations are often teaching and teacher-focused. They have a fire alarm function rather than focusing on course development (Edstr€om, 2008). The complex nature of digital education calls for using multiple methods and sources of data to understand student learning, satisfaction and participation. Quantitative analyses can yield useful information on learning outcomes and related variables. Qualitative data can help us better understand observed differences and student perceptions (Gunawardena et al., 2000). Bennett et al. (2017) investigated teachers’ design work and found that more than half of the interviewed teachers described that they identified future changes, typically after the course was completed, but some also made notes of their ideas during the course. These reflective activities contributed to developing teacher knowledge and iteratively refining their courses over multiple offerings. Thus, we can distinguish between two types of evaluation: formative evaluation while teachers teach and summative evaluation after they have taught. Hrastinski (2021) explored how 31 blended and online courses were evaluated by Learning Best pracces theory Figure 1. COLLECTIVE AND PERSONAL AND Examples of how ABSTRACT CONTEXTUAL designs for learning Teacher Research could be informed knowledge findings experienced practitioners as part of an edited book. They used course evaluations, IJILT questionnaires as a complement to standardized course evaluations, data from digital 38,4 platforms, assessment results, focus groups, written reflections from students and class discussions with students. These teachers can be assumed to be early adopters of technology (Rogers, 2010). A question that needs to be further explored is how we can engage, not only the early adopters and those with a special interest, but most time-pressed teachers in iterative evaluation and improvement of their designs for learning. Sharing designs for learning and lessons learned One of the aims of design for learning is to share the designs (Conole and Wills, 2013). There is also a long tradition to develop digital technologies to support codesign and collaboration between teachers (Hernandez-Leo et al.,2014). Different types of structured and semistructured representations, such as pedagogical patterns (Bergin et al.,2012), have been suggested as ways to share designs for learning, although these have not been adopted by most teachers (Oliver et al., 2017). Less focus has been put on how to share lessons learned (Conole and Wills, 2013; Laurillard, 2012). Teachers typically enjoy sharing experiences with colleagues, especially within their subject discipline (Conole and Wills, 2013; Oliver et al., 2017). Prestridge (2019) has developed a categorization of teachers’ use of social mediated spaces for their professional learning, which indicates different ways teachers could share design ideas and lessons learned in digital settings. The categorization includes four categories: info-seekers that seek to find new knowledge or ideas and resources from social mediated spaces; info-networkers that also share the ideas and resources with colleagues; self-seeking contributors that share knowledge, ideas or curriculum materials to get feedback, ask specific questions or acquire additional information; and vocationalists that engage as a member of a community of learning that aims to contribute with new knowledge. Collectively sharing and improving ideas is an important dimension of situated learning and knowledge development through awareness and reflection (van der Linden and McKenney, 2020; Scardamalia and Bereiter, 2003). Method Research setting The empirical data is based on workshops with adult education teachers. The purpose of the workshop for each participant was to suggest a design for learning, how the design drew on what is known, how to evaluate the design and how to share the design and lessons learned. The workshops were facilitated by the author as part of a teacher conference on digital learning in a Swedish region. The introduction to the workshop was intentionally short in order not to bias the results. The workshop instructions are presented in Table 1. The process of the workshops was as follows: (1) introduction by the facilitator, (2) individual work, (3) pairs of teachers provided feedback to each other, (4) individual work and (5) brief presentations of some of the designs for learning. All designs for learning were documented Background Describe the module, e.g. name, purpose, level, scope/credits, approximate number of participants Design Describe and motivate how the module will be designed Why and how should digital technologies be used?* Evaluate How should the module be evaluated?* Share How can lessons learned be shared with colleagues?* Locally, nationally, internationally? Note(s): *Motivate based on resources and experience. Examples of resources and experience: course evaluations, research papers, books, your experience, experience of colleagues, blogs and social media and Table 1. Workshop instructions best practices by using a digital form. Each participant individually submitted answers according to the Informing themes/question outlined in Table 1. designs for Two workshops were conducted for 60 min with nine participants in each workshop (see learning Table 2). There were 11 females and seven males. The years of teaching experience ranged from 1.5 to 40 years (M 5 17, SD 5 10). The teachers represented many different subject disciplines. Two of the teachers (F8a and F8b) worked together in the workshop. The study complies with research ethics rules and guidelines (Swedish Research Council, 2017). It was voluntary to participate in this study and the data has been anonymized. All teachers who completed the digital form chose to be part of the study. Data collection and analysis The documentation from the workshops (i.e. the text submitted by using a digital form) was analyzed using thematic analysis, which can be described as a method for identifying, analyzing and reporting themes within data (Braun and Clarke, 2006). The process of coding is an essential part of thematic analysis (Miles and Huberman, 1994), where data are organized into meaningful themes (Braun and Clarke, 2006). All the texts from the digital forms were anonymized and exported to text format. The data were imported and analyzed by using the open-source software RQDA that is used for qualitative data analysis (Huang, 2014). A semantic approach was adopted since the documentation from the workshops was in written form. This means that the explicit meanings of the data (i.e. what the participants had written) were analyzed (Braun and Clarke, 2006). The themes were deductive and based on the research questions. The subthemes were identified inductively. Braun and Clarke (2006) provide guidelines for doing thematic analysis. Qualitative analysis guidelines are not rules and should be applied flexibly to fit the research questions and data (Patton, 1990). In Table 3, the phases of the thematic analysis are described. The description of the process has been adapted from Braun and Clarke (2006). Qualitative analysis is a recursive process where movement between the phases is typically necessary, which was also the case in this study (Braun and Clarke, 2006). Results The themes and subthemes, which are described in more detail below, are presented in Figure 2. Teacher Workshop Gender Teaching experience (years) Subject discipline M1 1 Male 30 Philosophy F2 1 Female 10 Health M3 1 Male 17 Mathematics F4 1 Female 17 Literature M5 1 Male 1.5 Mathematics F6 1 Female 3 Psychiatry M7 1 Male 29 English F8a 1 Female 2 Health F8b 1 Female 20 Health F9 2 Female 13 History F10 2 Female 9 Swedish for immigrants M11 2 Male 9 Swedish M12 2 Male 20 Biology F13 2 Female 13 Study skills F14 2 Female 40 Chemistry Table 2. F15 2 Female 21 Swedish F16 2 Female 30 Health Participants of the M17 2 Male 13 Digital skills workshops IJILT Phase Description of the process 38,4 Familiarizing yourself with the Data were read and reread, and initial ideas were noted data Generating initial codes Initial codes were generated. Data relevant to each code were collated Collating codes into deductive The codes were collated into deductive themes based on the research themes questions Inductively searching for The subthemes were inductively identified subthemes Reviewing themes It was checked if the coded extracts work in relation to the themes, subthemes and the entire data set Defining and naming themes Names and definitions of each sub theme were generated Table 3. Producing the report Selection of extracts to quote and production of the paper Phases of the thematic analysis Source(s): Adapted from Braun and Clarke (2006),p. 87 RQ1. Drawing on what is Learning known theory Student perceptions RQ2. Evaluation of design for learning Assessment Collegial learning RQ3. Sharing designs for Figure 2. learning and lessons Subthemes organized learned Digital sharing of according to research designs questions Drawing on what is known There were few examples where the teachers explicitly described how they informed their design, despite being encouraged to do so. Only one subtheme was identified: learning theory. In the learning theory theme, the teachers suggested using different theoretical perspectives. This can be linked to how repetition is important for learning. Behaviorism - repetition. (M5) The circle model [genre pedagogy], joint reading and processing of central parts on smartboards. (F10) Given the socio-cultural perspective, it is important that students get the chance to create knowledge together. This can lead to different views. (F6) Vygotsky maintains that students need inspiration, motivation and encouragement in their learning. (F8) Quick feedback is known to be important for learning, feasible with digital tools. (F8) There was one exception, a teacher explicitly acknowledging teacher experience. My experience is that students also have to learn how to produce mathematics (writing on a paper) Informing to gain knowledge. (F6) designs for learning Evaluation of design for learning The teachers suggested different ways to evaluate their designs. Two subthemes were identified: student perceptions and assessment. Three teachers suggested conducting a discussion with the students. [T]he group evaluates orally: How did it feel? What was positive? What could have been done differently? (M5) Three teachers suggested designing a survey. Survey.. . With questions about how the student experienced the way of working and what they have learned. (M7) [S]urvey on the different parts (recorded lessons, test questions and collaboration tasks). (M11) Two teachers proposed using a standardized course evaluation survey. One of these teachers suggested complementing the survey with open questions that the students were to complete in writing. Another teacher exemplified what kind of open questions to use. Finally, the student evaluates in writing with questions of the type: How does it feel now? Would you be able to give a speech again? Would you be able to give a speech outside the school?” (M5) Two teachers suggested conducting an evaluation together with the students by asking specific questions using the smartboard or a smartphone app. Quick evaluation on the smartboard by the students. Did you learn imperatives? Can you use it? (F9) One teacher suggested conducting individual informal discussions with students during the course and a more formal discussion on the course and their learning after the course. The second subtheme was how to use assessment to evaluate the design for learning. Two teachers suggested analyzing the quality of student texts. Two teachers suggested following student development over time to see if their abilities improved. For example, in one course on study skills, it was suggested to “check that the students de facto find information faster” (F13). Two teachers suggested using automated quizzes to give the students and teacher fast feedback. One teacher suggested conducting a pretest–posttest where “students answer a new quiz with a similar design but new questions” (F6) to investigate if student results improved following the intervention. Similarly, one teacher suggested comparing student results with previous years. One teacher wanted to conduct informal individual discussions with students throughout the course and a more formal discussion after the course with a focus on the course design and student learning. [D]iscussion in conjunction with feedback on examination, as this is the first part of the course, so you can adjust the upcoming and a more thorough evaluation after the course has been completed. (M17) Sharing designs for learning and lessons learned The teachers suggested different ways to share their designs for learning and lessons learned. Two subthemes were identified: collegial learning and digital sharing of designs for learning. Six teachers suggested different ways to share experiences at the school. Pedagogical meetings among colleagues. (M7) Teachers in different subjects at the school can be involved. (F15) Me and the other responsible teacher will probably show other colleagues who are interested in a IJILT similar distance structure. (M17) 38,4 Four teachers proposed sharing experiences in their teacher team: “conversations at the subject meeting at work” (F4) and “[w]e share within the working group” (F8). As evident above, most suggestions in the theme collegial learning concerned experiences at the school. However, two teachers suggested sharing experiences in regional networks: “collaboration between municipalities within established networks” (F15). The second subtheme was digital sharing of designs for learning. Five teachers provided examples of how to share their design with their colleagues by sharing files in shared folders and learning management systems at the school. The following is an example of a more systematic approach. The idea is that I will build together and try out a material that will later be handed over to other [Swedish for immigrants] colleagues so that they have something to start from and be able to work with in future classes. (F13) Two teachers suggested to share the design on a blog and one teacher suggested YouTube. Discussion The first research question investigated how teachers suggested drawing on what is known when designing for learning. There were some examples where teachers were inspired by different learning theories: behaviorism, genre pedagogy and sociocultural learning theory. This finding can be compared with current learning design models that sometimes are underpinned by learning theories, typically socioconstructivist theories (Bower and Vlachopoulos, 2018). Most teachers who used learning theory as a foundation gave exclusivity to one learning theory, which might serve the interest of certain students but might disadvantage others (Sfard, 1998). Here, learning design models, such as the ABC model, which encourage basing designs on different, complementary “learning types” (Young and Perovic, 2016) might be useful to encourage teachers to consider alternative or complementary approaches. In line with previous research, the teachers did not draw on research findings. Teachers might need support in curating and tailoring research to increase their use of research (Joram et al., 2020). Thus, it seems necessary to support teachers in how to identify research findings that could inform their designs for learning. There was only one example where teacher experience was explicitly recognized. These findings can be compared with Agostinho et al. (2018) that identified the types of support the teachers reported using in their design work: colleagues, literature, workshops and seminars, conferences, institutional support services and enrolment in postgraduate study. It is notable that few teachers explicitly described how they used resources to inform their design, despite being encouraged to do so. This finding is aligned with previous research, which found that teachers followed a top-down approach where they initially considered the learning outcomes, scope of content, ideas for activities and assessment strategy (Bennett et al., 2017), but typically do not draw on external sources of knowledge, such as research findings (Price and Kirkwood, 2014). The second research question investigated how teachers suggested evaluating their designs for learning. We can distinguish between two types of evaluation: formative evaluation while teachers teach and summative evaluation after they have taught. The teachers provided many different examples of how to evaluate their designs, by drawing on student perceptions and assessment. They suggested investigating student perceptions by discussions with students, in some cases complemented by using a smartboard or app to ask specific questions as a basis for the discussion, surveys, complementing standardized course evaluation surveys with open questions and individual discussions with students during and after the course. The teachers proposed drawing on assessment in different ways: analyzing Informing the quality of student texts, following student development over time, automated quizzes, designs for pretest–posttest of student results and comparing student results with previous years. These learning findings can be compared with a study that explored how university teachers evaluated 31 blended and online courses. They used course evaluations, questionnaires, data from digital platforms, assessment results, focus groups, written reflections from students and class discussions with students (Hrastinski, 2021). Thus, there are many different methods that teachers can use when evaluating designs for learning. The complex nature of evaluating digital education calls for using multiple methods and sources of data to understand student learning, satisfaction and participation (Gunawardena et al., 2000). There seems to be a need to develop a more thorough understanding of the benefits and limitations of using the identified evaluation methods. The third research question investigated how teachers suggested sharing their designs for learning and lessons learned. They proposed drawing on collegial learning to share and discuss experiences, mainly at the school and in teacher teams. The teachers also suggested sharing designs digitally, primarily with colleagues at the school, but there were also occasional suggestions to share designs by using blogs and YouTube. Thus, the teachers seemed willing to move beyond information consumption toward sharing designs for learning and lessons learned (Prestridge, 2019), but preferably at their school. Although researchers envision different ways that teachers could share their designs and lessons learned, ranging from design representations to journal papers (Bergin et al., 2012; Bernstein and Bass, 2005), the findings of this paper are aligned with research suggesting that teachers typically want to discuss their ideas, experiences and designs with colleagues, preferably from their subject discipline (Conole and Wills, 2013; Oliver et al., 2017). In light of these findings, it might be more realistic to accept that most teachers prefer to share designs for learning and lessons learned at their school. If a school encourages at least some teachers to take on more outbound roles, the school will have an influx of ideas, designs for learning and lessons learned, which could be adapted by colleagues to their local context. Limitations and further research A limitation of this study is that it is based on workshops with teachers. The teachers suggested a design for learning, how to draw on what is known, how to evaluate the design and how to share the design and lessons learned. What the teachers plan to do will not necessarily correspond with what they do in practice. It would be valuable to explore similar research questions in “real” design and teaching practice. That said, it is also important that research explores teacher ideas and suggestions, which can guide future research and practice. It also needs to be acknowledged that this study was based on a specific type of adult education corresponding to K12 education aimed at adult students that need to complement their previous studies, for example, to obtain an upper secondary diploma, to achieve eligibility requirements for university studies or Swedish for immigrants. This paper and related studies have identified different ways that teachers draw on what is known to inform their designs for learning (Agostinho et al., 2018) and how they evaluate their designs for learning (Hrastinski, 2021). It was surprising that other types of resources, such as best practices and experiences of colleagues were not acknowledged. Previous research has found that teachers enjoy sharing experiences with other teachers (Conole and Wills, 2013), but a question that remains is to what extent and how such experiences are then used to inform teachers’ design work. There are opportunities to develop more systematic approaches to how designs for learning could be informed and evaluated. One such example is how learning analytics could be used to inform designs for learning (Michos and Hernandez-Leo, 2020; Wiley et al., 2020). However, as found in this paper and related research (Agostinho et al., 2018; Hrastinski, 2021), there are many other resources and methods for IJILT evaluation that could support informing designs for learning, which also needs to be further 38,4 developed and investigated in future research. Conclusions This paper explored how adult teachers suggested informing and evaluating their designs for learning and how they suggested sharing these designs and lessons learned, all of which could be considered to be key elements of informed design for learning. It was found that teachers typically referred to learning theories to inform their designs for learning. However, other types of resources, such as research findings, best practices and experiences of colleagues, were not explicitly suggested. Teachers seem to need support to identify research and to understand how research findings are relevant in their setting (Joram et al., 2020). The teachers provided many examples of how to evaluate their design based on student perceptions and assessment. They suggested discussing their experiences with colleagues and sharing their designs digitally, although most preferred doing this locally at the school. In conclusion, the teachers were aware of how their designs for learning could be evaluated and how to share the designs and lessons learned at the school. Teachers seem to need support to understand how to inform their designs. They could also be encouraged to share their designs and lessons learned beyond their school when it makes sense. A key challenge for future research is to put more focus on design for learning as an informed practice, to support the development of teacher knowledge and the improvement of future designs for learning. References Agostinho, S., Bennett, S., Lockyer, L. and Harper, B. (2011), “The future of learning design”, Learning, Media and Technology, Vol. 36 No. 2, pp. 97-99. Agostinho, S., Lockyer, L. and Bennett, S. (2018), “Identifying the characteristics of support Australian university teachers use in their design work: implications for the learning design field”, Australasian Journal of Educational Technology, Vol. 34 No. 2, pp. 1-15. Atkinson, M.P. (2001), “The scholarship of teaching and learning: reconceptualizing scholarship and transforming the academy”, Social Forces, Vol. 79 No. 4, pp. 1217-1229. Bates, T. (2015), Teaching in a Digital Age, BC Campus, Victoria BC. Bennett, S., Agostinho, S. and Lockyer, L. (2017), “The Process of designing for learning: understanding university teachers’ design work”, Educational Technology Research and Development, Vol. 65 No. 1, pp. 125-145. Bergin, J., Eckstein, J., Volter, M., Sipos, M., Wallingford, E., Marquardt, K., Chandler, J., Sharp, H. and Manns, M.L. (2012), Pedagogical Patterns: Advice for Educators, Joseph Bergin Software Tools. Bernstein, D. and Bass, R. (2005), “The scholarship of teaching and learning”, Academe, Vol. 91 No. 4, pp. 37-43. Biesta, G. (2007), “Why ‘what works’ won’t work: evidence-based practice and the democratic deficit in educational research”, Educational Theory, Vol. 57 No. 1, pp. 1-22. Bower, M. and Vlachopoulos, P. (2018), “A critical analysis of technology-enhanced learning design frameworks”, British Journal of Educational Technology, Vol. 49 No. 6, pp. 981-997. Braun, V. and Clarke, V. (2006), “Using thematic analysis in psychology”, Qualitative Research in Psychology, Vol. 3 No. 2, pp. 77-101. Coffield, F. and Edward, S. (2009), “Rolling out ‘good’, ‘best’ and ‘excellent’ practice. What next? Perfect practice?”, British Educational Research Journal, Vol. 35 No. 3, pp. 371-390. Conole, G. (2013), Designing for Learning in an Open World, Springer, New York, NY. Informing Conole, G. and Wills, S. (2013), “Representing learning designs – making design explicit and designs for shareable”, Educational Media International, Vol. 50 No. 1, pp. 24-38. learning Dewey, J. (1933), How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process, DC Heath, Boston. Edstrom, K. (2008), “Doing course evaluation as if learning matters most”, Higher Education Research and Development, Vol. 27 No. 2, pp. 95-106. Gunawardena, C.N., Lowe, C. and Carabajal, K. (2000), “Evaluating online learning: models and methods”, Society for Information Technology and Teacher Education International Conference, San Diego. Hargreaves, D.H. (2004), Learning for Life: The Foundations for Lifelong Learning, Policy Press, Bristol. Hargreaves, D.H. (2007), “Teaching as a research-based profession: possibilities and prospects, (the teacher training agency lecture 1996)”, Educational Research and Evidence-Based Practice, pp. 3-17. Hernandez-Leo, D., Moreno, P., Chacon, J. and Blat, J. (2014), “LdShake support for team-based learning design”, Computers in Human Behavior, Vol. 37, pp. 402-412. Hrastinski, S. (2021), “Teachers as developers of local evidence to improve digital course design”, Interactive Learning Environments, Vol. 29 No. 4, pp. 648-654, doi: 10.1080/10494820.2019. Huang, R. (2014), RQDA: R-Based Qualitative Data Analysis, R Package Version 0.2–8. Joram, E., Gabriele, A.J. and Walton, K. (2020), “What influences teachers’‘buy-in’ of research? Teachers’ beliefs about the applicability of educational research to their practice”, Teaching and Teacher Education, Vol. 88, 102980. Kali, Y., Goodyear, P. and Markauskaite, L. (2011), “Researching design practices and design cognition: contexts, experiences and pedagogical knowledge-in-pieces”, Learning, Media and Technology, Vol. 36 No. 2, pp. 129-149. Kirkpatrick, D.L. and Craig, R. (1967), “Evaluation of training”, in Browning, P.L. (Ed.), Evaluation of Short-Term Training in Rehabilitation, Oregon University, Washington, pp. 35-56. Laurillard, D. (2012), Teaching as a Design Science: Building Pedagogical Patterns for Learning and Technology, Routledge, New York. McKenney, S., Kali, Y., Markauskaite, L. and Voogt, J. (2015), “Teacher design knowledge for technology enhanced learning: an ecological framework for investigating assets and needs”, Instructional Science, Vol. 43 No. 2, pp. 181-202. Michos, K. and Hernandez-Leo, D. (2018), “Supporting awareness in communities of learning design practice”, Computers in Human Behavior, Vol. 85, pp. 255-270. Michos, K. and Hernandez-Leo, D. (2020), “CIDA: a collective inquiry framework to study and support teachers as designers in technological environments”, Computers and Education, Vol. 143, Miles, M.B. and Huberman, A.M. (1994), Qualitative Data Analysis, Sage, Thousand Oaks. Mishra, P. and Koehler, M.J. (2006), “Technological pedagogical content knowledge: a framework for teacher knowledge”, Teachers College Record, Vol. 108 No. 6, pp. 1017-1054. Oliver, M. and Conole, G. (2003), “Evidence-based practice and e-learning in Higher Education: can we and should we?”, Research Papers in Education, Vol. 18 No. 4, pp. 385-397. Oliver, M., Avramides, K., Clark, W., Hunter, J., Luckin, R., Hansen, C. and Wasson, B. (2017), “Sharing teacher knowledge at scale: teacher inquiry, learning design and the representation of teachers’ practice”, Teacher Development, Vol. 22 No. 4, pp. 587-606. Patton, M.Q. (1990), Qualitative Evaluation and Research Methods, Sage, Newbury Park, CA. Prestridge, S. (2019), “Categorising teachers’ use of social media for their professional learning: a self- IJILT generating professional learning paradigm”, Computers and Education, Vol. 129, pp. 143-158. 38,4 Price, L. and Kirkwood, A. (2014), “Using technology for teaching and learning in higher education: a critical review of the role of evidence in informing practice”, Higher Education Research and Development, Vol. 33 No. 3, pp. 549-564. Rogers, E.M. (2010), Diffusion of Innovations, Simon and Schuster, New York. Scardamalia, M. and Bereiter, C. (1994), “Computer support for knowledge-building communities”, Journal of the Learning Sciences, Vol. 3, pp. 265-283. Scardamalia, M. and Bereiter, C. (2003), “Knowledge Building”, Encyclopedia of Education, Macmillan Reference, New York, NY, pp. 1370-1373. Sfard, A. (1998), “On two metaphors for learning and the dangers of choosing just one”, Educational Researcher, Vol. 27 No. 2, pp. 4-13. Shulman, L.S. (1986), “Those who understand: knowledge growth in teaching”, Educational Researcher, Vol. 15 No. 2, pp. 4-14. Swedish Research Council (2017), Good Research Practice, Swedish Research Council, Stockholm. van der Linden, S. and McKenney, S. (2020), “Uniting epistemological perspectives to support contextualized knowledge development”, Educational Technology Research and Development, Vol. 68 No. 2, pp. 703-727. Wiley, K.J., Dimitriadis, Y., Bradford, A. and Linn, M.C. (2020), “From theory to action: developing and evaluating learning analytics for learning design”, Proceedings of the Tenth International Conference on Learning Analytics and Knowledge. Wilson, S.M., Shulman, L.S. and Richert, A.E. (1987), “150 different ways of knowing: representations of knowledge in teaching”, in Calderhead, J. (Ed.), Exploring Teachers’ Thinking, Mansell, pp. 104-124. Young, C. and Perovic, N. (2016), “Rapid and creative course design: as easy as ABC?”, Procedia-Social and Behavioral Sciences, Vol. 228, pp. 390-395. Zwozdiak-Myers, P. (2018), The Teacher’s Reflective Practice Handbook: Becoming an Extended Professional through Capturing Evidence-Informed Practice, Routledge, New York. Corresponding author Stefan Hrastinski can be contacted at: stefanhr@kth.se For instructions on how to order reprints of this article, please visit our website: www.emeraldgrouppublishing.com/licensing/reprints.htm Or contact us for further details: permissions@emeraldinsight.com

Journal

The International Journal of Information and Learning TechnologyEmerald Publishing

Published: Aug 12, 2021

Keywords: Design for learning; Digital technologies; Informed design; Evaluation; Sharing

References