Making the point: the place of gesture and annotation in teaching STEM subjects using pen-enabled Tablet PCs

Making the point: the place of gesture and annotation in teaching STEM subjects using pen-enabled... Abstract The teaching of STEM subjects, and engineering and mathematics in particular, involves the use of a wide range of representational forms, including equations, diagrams, sketches and graphs, supported by speech and gestures. In the traditional face-to-face ‘board’ based classroom, the integration of writing, speech and gesture has been a key feature of pedagogical delivery approaches. The pen-enabled Tablet PC (penTPC), used in conjunction with a data projector, allows for the maintenance of a handwritten approach in teaching environments where traditional boards are unavailable or limited. However, it has been suggested that the use of the digital interface imposes restrictions on lecturer movement and gesture, compared to traditional board environments. This article examines the adaptations made by lecturers in using the penTPC in a classroom environment. The study suggests that the use of penTPC technology does not preclude the use of gesture, and that the augmented capability for annotation in conjunction with other digital representations can enhance teaching, particularly of STEM-based discipline subjects. 1. Introduction Gestures … are writing in the air, and written signs frequently are simply gestures that have been fixed. (Vygotsky, 1978, p. 107) This article is concerned with the effective use of the pen-enabled Tablet PC (penTPC) as a technology for the teaching of mathematically based subjects, specifically as it may be used by a teacher in the context of a classroom environment. While the lecture as a generic pedagogic device has been widely criticised (Bates, 2015), Fox & Artemeva (2011) have defended its use in the mathematically intensive disciplines where the characteristic combination of writing and talking is seen as providing essential components for developing students-capability. Greiffenhagen (2008) describes the mathematics lecture as where ‘an experienced mathematician demonstrates mathematical expertise to novices as an important part of their progressive induction into professionally competent autonomous mathematical practice’. An important element of this approach is the use of handwritten modes for the development of material, which establishes a pacing suiting the student acquisition of procedural skills and concepts (Artemeva & Fox, 2011). While the penTPC clearly supports the handwriting component essential in developing traditional mathematical thinking, other components are also involved in face-to-face communication. Gesture has been suggested as playing an important role in teaching and learning (Roth, 2001), with a number of authors noting the importance of gesture in the specific context of developing mathematical thinking (Arzarello, 2006; Arzarello et al., 2008; Radford, 2008; Goldin, 2010; Fox & Artemeva, 2011; Alibali et al., 2014). Thus, along with additional affordances of the penTPC, attention needs to be given to the constraints that the device may impose on the way gesture is used (Thomas & Hong, 2013). If, as Yoon, Thomas and Dreyfus (2011) argue, gestures can help to develop advanced mathematical insights ‘by supporting the creation of virtual mathematical constructs’ (p. 891) then it is important to investigate the constraints on use of gesture that may arise in the penTPC environment. Gestures in the broad sense have been classified in a range of forms, from gesticulation, emblems, pantomime, to sign language, in a succession in which concurrent speech becomes progressively less important in maintaining meaning (McNeill, 2005, p. 5). As a prominent form of gesture, gesticulation has been a major focus for many researchers, such that the generic term gesture is often used in place of gesticulation (McNeill, 2005, p. 5, 2006, p. 3; Roth, 2001, p. 369). McNeill (1992) identified four major types of gesture: iconic (referencing a concrete object); metaphoric (referencing an abstract idea); beat (a rhythmic emphasis); and deictic (pointing, positional). McNeill later (2005, p. 38) proposed that it is better to think of gestures in terms of having dimensions (rather than being of distinct types), with any one gesture potentially exhibiting some or all of the various dimensions to differing extents. This is the approach used here in examining lecturer use of gestures. While using a penTPC the lecturer is holding a writing instrument in the form of a digital pen. Thus the lecturer has options to make a gesture in a conventional way, in a transient form in the air with hands, or to give it concrete form as a written mark on the screen. These written marks have been termed both ‘writing gestures’ (Alibali et al., 2014, p. 76) and ‘attentional marks’ (Anderson et al., 2004, p. 797), with each term assigning emphasis to different aspects of their generation. Classifications of written, oral, and gesture may be used to identify primary registers of communication, or as termed by Bosch & Chevallard (1999, p. 96), and translated in Arzarello (2006, p. 270): trace (written), oral and gesture. Sabena (2008) defined gestures in a mathematical setting as including ‘all those movements of hands and arms that subjects (students and teachers) perform during their mathematical activities and which are not a significative part of any other action’ (such as writing with a pen). Thus the term attentional marks (rather than writing gestures) will be preferred here, locating them clearly as permanent marks within the trace register. Nevertheless, attentional marks share some characteristics of gestures, in that they may at times be dependent for full meaning on other co-existing modes, including previously created functional writing and accompanying speech. However, as Anderson et al. (2005) noted, there is a critical difference between the ephemeral nature of gestures and the persistence of attentional marks. It has also been suggested that there are strong linkages between diagrams and gesture, with diagrams suggested as a refined extension of an iconic gesture, in written form. De Freitas & Sinclair (2011) noted the relationship between gesture and diagrams, expanding on the work of Châtelet (2000), and suggesting that diagrams lock or capture gestures. Bunt et al. (2009) describe how ‘mathematicians make liberal use of sketches, mathematical expressions and annotations to render abstract mathematical concepts more concrete’, also noting that these representational forms should be viewed as dynamic objects, as ‘terms in an expression are crossed out, content is added to sketches, and new insights lead to new annotations’. It may be useful to consider attentional marks within a wider context of annotations, just as gesticulation may be considered as a particular form of gesture. Thus annotations may range from non-descript marks (analogous to gesticulations), through universally recognizable signs, such as ticks, circles, underlines and highlights (emblems), to elements within fully structured symbolic languages, such as algebraic equations (sign languages). While annotations may be created concurrently with speech, and some may be dependent on that speech for full meaning, others annotations may establish or maintain meaning independently of speech, and beyond the instant of their creation. Furthermore, just as gestures may be considered as having dimensions (rather than being of distinct types), annotations may have multiple dimensions, potentially serving multiple purposes. Thus the addition or development of functional content in graphs, diagrams and symbolic expressions (such as adding tangents to a curve, labelling points on a diagram, or cancelling terms in an algebraic expression) adds functional content in conjunction with giving timely emphasis to the sequencing of that content. In a board-based classroom, the teaching of mathematically intensive subjects has developed into a standard and well recognized pedagogical form (Fox & Artemeva, 2011, p. 83; Shulman, 2005, pp. 53–54). In substituting a penTPC for a board, lecturers are required to adapt this traditional approach according to the affordances and limitations of the technology. Fox & Artemeva (2011) described traditional classroom presentation as a ‘cinematic art’ that has many elements, including the use of gesture as a natural and spontaneous component of teaching and ‘enacted in part through the physical positioning of the professor’. There are significant interactions between body movements (including position) and forms of gesture and writing, and these are impacted by a change from the use of board to penTPC. Thus whole body movements (including positioning within the classroom) are also an important feature examined in this study. While the use of the penTPC may affect communication in the classroom by constraining the range of movements and gestures used by the lecturer, it also provides other affordances, with increased capabilities for introduction of other digital forms, and interactions with those forms through direct annotation. In recent years, the use of software-based methods for problem solving and data visualization has had a significant impact on STEM content knowledge. New discipline procedures and conceptual understandings have been developed that would not be possible without the use of new computing technologies. The capabilities of software to generate visual representations of data have also altered approaches to the interpretation of data and decision making (Hodge & Taylor, 2002; Campbell & Latulippe, 2015; Goeser & Ruiz, 2015). The capability to integrate software output and diagrammatic representations into digital teaching environments, and to annotate directly on them, is arguably now an essential element of STEM education, and an element for which the use of digital pen devices such as the penTPC provide critical affordances. Singer & Smith (2013) have noted the value of using multiple forms of representations in instruction, including ‘realistic (picture or text), diagrammatic (free body diagram) and symbolic (mathematical)’ representations (p. 470). Oviatt (2013) stresses the importance of using interfaces that support ‘expression of non-linguistic representations, including diagrams, symbols, numbers, and informal marking” (p. 61).’. The capability of the penTPC to provide support for these features is also of interest here. While recognizing that detailed analysis might cover a complex range of interacting resources (as a semiotic bundle, as described by Arzarello (2006)), this study focused on the specific elements as identified by Fox & Artemeva (2011, p. 80) as occurring in the board-based classroom, but extended to include additional elements available in the penTPC environment. Table 1 provides a comparative framework of elements, with column 1 relating to board environments, and column 2 the penTPC environment. Thus the study sought to identify the broad effect on the nature of multimodal communication, in particular in the use of annotation and the relationship with gesture, resulting from the adoption of the penTPC technology by lecturers and as used in the specific context of the teaching of mathematically intensive engineering courses. Table 1. Components of the mathematical narrative in board-based and penTPC class sessions (after Fox & Artemeva (2011)) Element code  Board classroom—co-occurring elements  Tablet PC—co-occurring elements  A  Writing mathematical symbolism and text on a board; drawing graphs/diagrams  Writing mathematical symbolism and text on the penTPC; drawing graphs/diagrams  B  Referring to problem sets and textbook chapters  Referring to problem sets and textbook chapters  C  Verbalizing what is being written while writing on the board  Verbalizing what is being written while writing on the penTPC  D  Talking about what has been written on the board  Talking about what has been written on the penTPC  E  Moving in space  Moving in space (away from the penTPC, and with the penTPC)  F  Gesturing (including pointing) to indicate relationships, signal references, highlight key issues, and so on  Annotating (and where possible, gesturing) to indicate relationships, signal references, highlight key issues, and so on  G  Consulting/reading lecture notes  Consulting/reading lecture notes  H  Stepping back from the board, pausing the action for reflection  Looking up from the penTPC, pausing the action for reflection  I  Checking student understanding by quickly glancing at the class  Checking student understanding by quickly glancing at the class  J  Turning to students and asking questions, and talking with students  Looking up from the penTPC and asking questions, and talking with students  K    Inclusion of detailed graphic images/diagrams in presented material for subsequent development and annotation  L    Use of graphic images or video for development of conceptual understanding  M    Dynamic use of software to produce tables, diagrams, visualizations; copy and paste into pages for subsequent annotation  Element code  Board classroom—co-occurring elements  Tablet PC—co-occurring elements  A  Writing mathematical symbolism and text on a board; drawing graphs/diagrams  Writing mathematical symbolism and text on the penTPC; drawing graphs/diagrams  B  Referring to problem sets and textbook chapters  Referring to problem sets and textbook chapters  C  Verbalizing what is being written while writing on the board  Verbalizing what is being written while writing on the penTPC  D  Talking about what has been written on the board  Talking about what has been written on the penTPC  E  Moving in space  Moving in space (away from the penTPC, and with the penTPC)  F  Gesturing (including pointing) to indicate relationships, signal references, highlight key issues, and so on  Annotating (and where possible, gesturing) to indicate relationships, signal references, highlight key issues, and so on  G  Consulting/reading lecture notes  Consulting/reading lecture notes  H  Stepping back from the board, pausing the action for reflection  Looking up from the penTPC, pausing the action for reflection  I  Checking student understanding by quickly glancing at the class  Checking student understanding by quickly glancing at the class  J  Turning to students and asking questions, and talking with students  Looking up from the penTPC and asking questions, and talking with students  K    Inclusion of detailed graphic images/diagrams in presented material for subsequent development and annotation  L    Use of graphic images or video for development of conceptual understanding  M    Dynamic use of software to produce tables, diagrams, visualizations; copy and paste into pages for subsequent annotation  2. Study approach The study involved seven lecturers who had been using penTPCs in teaching STEM disciplines in a university environment, drawn from a group who participated in pilot projects which provided them with access to a penTPC for teaching, who consented to be studied, and where timetabling constraints allowed access to class sessions. While essentially a convenience sample, the lecturers involved had varying levels of experience in use of a penTPC, from those in their first semester of use, to those with over 3 years of experience, and the lecture sessions surveyed covered a range of subjects and levels within mathematics/engineering disciplines. One of the lecturers was no longer using a penTPC in the classroom, and had reverted to using a board. While the number of lecturers involved was small, it was considered sufficient to give useful insights into current practices within the university (Tang & Davis, 1995; Nielsen, 2000). The study was approved by the institutional ethics committee. The study examined the method of delivery used in lectures involving use of the penTPC as the primary classroom presentation device for handwritten material. Video analysis of lectures was conducted with specific focus on the use of gesture and annotation in the classroom situation. The features of the penTPC class session were identified and compared with those of the conventional board classroom, focussing on the co-occurring multimodal elements that were employed by the lecturers. The activities observed in lecture session are first described briefly for each lecture session (Cases), and then analysed collectively, in relation to key elements of the board classroom approach (Fox & Artemeva, 2011, p. 80). The opinions of lecturers involved in the use of the penTPC were also surveyed in a questionnaire as part of a wider study (Maclaren et al., in press). The perceptions of the particular lecturers in this study of the affordances of the penTPC and adaptations required for their use, as recorded in questionnaire comments, are also referenced here and related to their observed practices. These give an indication of the extent to which lecturers have consciously adapted their approach to the affordances of the different environment. While the analysis and conclusions are drawn from, and relate to, the particular context of this study, there are many aspects that will be familiar to those working in similar environments and that may be used to inform developments in the introduction and use of penTPC technology. 3. Observations 3.1 Case 1: undergraduate (yr 1) engineering mathematics lecture This lecturer was in the first semester of using a penTPC, teaching a first year undergraduate engineering mathematics class. The environment was a medium size tiered lecture theatre, seating 172 (Fig. 1a). Approximately 80 students were present. There was a single data projection screen (DPSc) located at the front, centre of the room. Fig. 1. View largeDownload slide (a) Lecture theatre seating 172; (b) lecturer raises head to gesture. Fig. 1. View largeDownload slide (a) Lecture theatre seating 172; (b) lecturer raises head to gesture. The lecturer conducted the session using the penTPC as the only presentation technology, with all material handwritten live on the penTPC using Microsoft OneNote software. The lecturer remained standing at the lectern for the duration of the lecture, almost exclusively facing the class, using a lectern-mounted microphone. Lecturer activity alternated between hand writing on the penTPC (developing diagrams in conjunction with equations) with commentary, and looking up to face and talk to the class and interact with students (Fig. 1b). The lecturer asked questions of the students and also responded to student initiated questions, looking up as necessary. The majority of the session involved the exposition of theoretical concepts and example problems using both diagrammatic and symbolic representations written on the penTPC (and projected). The lecturer did not use any diagrams or figures other than those hand drawn, or any other software, in the observed session, and used a single coloured pen throughout. In terms of the content used and developed, there was nothing that was significantly different from what might have been developed on a whiteboard. On occasions the lecturer scrolled back to previous material, adding attentional marks (such as circling) to indicate relevant material (rather than deictic gestures towards DPSc content). While talking to the class, the lecturer used hand gestures in a ‘traditional’ way i.e. gestures were made with the hands involving beat dimensions for emphasis, and iconic dimensions (e.g. moving one hand to and fro in association with saying ‘move object from one place to another place’). In survey comments, the lecturer was very positive about the use of the penTPC, valuing the ability to remain at a desk, with a microphone, and to ‘not need to jump in front of the board’. The lecturer also valued the fact that the lecture notes were clear and visible to all students especially given the large class (and room) size. The lecturer also appreciated being always orientated facing the students and expressed the view that in using the penTPC ‘the teaching becomes more effective as students feel free and can participate in discussions’. The lecturer valued the fact that notes did not need to be erased to make space for new notes, and that notes could be made available to students after the class. 3.2 Case 2: undergraduate (yr 1) engineering mathematics lecture (2) This lecturer was in the second semester of using a penTPC, and also teaching a first year undergraduate engineering mathematics class. The environment was a medium size tiered lecture theatre, seating 236 (Fig. 2a). Approximately 90 students were present. There was a large DPSc extending to approximately 3 metres above floor, with a strip of white board along the front of room, partially obscured when the DPSc was lowered. Fig. 2. View largeDownload slide (a) Lecture theatre seating 236; (b) lecturer writes on penTPC at lectern; (c) lecturer gestures towards point of interest on DPSc. Fig. 2. View largeDownload slide (a) Lecture theatre seating 236; (b) lecturer writes on penTPC at lectern; (c) lecturer gestures towards point of interest on DPSc. The lecturer conducted the observed session using the penTPC and single DPSc as the only presentation technology. The lecturer was confident and fluent in the use of the penTPC. Some material was handwritten live on the penTPC (within Microsoft OneNote), and other material was presented in the form of prepared static slides. The lecturer alternated between working at the lectern using the penTPC (Fig. 2b) and walking away from the lectern, in front of the screen, to talk to the class. While writing on the penTPC at the lectern the lecturer followed the pattern of dynamically developing both diagrammatic and symbolic representations in conjunction with dialogue, as a mathematical narrative. Rather than using a deictic gesture or mark to indicate location, location was associated with the position of appearance of specific content. For example, in referring to a region on a graph as ‘over here’, the location of ‘here’ was only made explicit for students by the appearance of the label being written at that place. Material was scrolled vertically so that new information was generally being added in the blank space immediately below previously written material. There was occasional use of horizontal lines, short diagonal lines or boxes; rather than being attentional marks for emphasis, these marks served as separators, to distinguish one block of content from another (for example, between sample problems). Most of the material was written in the penTPC using black ink, with occasional use of a different colour to distinguish key points of interest (e.g. blue lines to highlight turning points on a hand drawn line graph). The lecturer displayed slides using PDF reader software and did not annotate this material using the penTPC. When projecting slides, the lecturer would frequently move in front of the screen to discuss the material being presented, often pointing directly at material displayed on the screen while commenting (e.g. ‘going up and down’) (Fig. 2c). In this lecture theatre only the bottom half of the DPSc is within reach, so while items of interest on the lower section of the screen were pointed at directly, with a hand placed contiguous to the screen material, items at the top of the screen were referenced by positional statements e.g. ‘the first one’, referring to a slide displaying three theorems. Other than these pointing (deictic) gestures, the lecturer was quite restrained in the use of hand gestures while standing. In the lecturer survey, the lecturer rated the whiteboard as a preferred technology, but stated that the penTPC was almost always used for teaching and that it ‘works as well as a whiteboard’. In describing advantages of the penTPC, it was described as ‘no better than a whiteboard, except that the notes are kept online and can be exported to PDF and other formats’. Comment was made that white board space in many rooms was limited and ‘ineffective’, and was obstructed once the DPSc was lowered, and these issues being a driver for using the penTPC. As in Case 1, the handwritten material observed being developed using the penTPC was not significantly different from what might have been developed on a board. 3.3 Case 3: undergraduate (yr 3) engineering design lecture This lecturer was in second semester of using a penTPC, teaching engineering design to third year undergraduate students. The teaching space was a large flat-floor classroom seating up to 60 (with 30 students present), with individual table and chairs, set in rows. The room contained a lecturer desk at the front, side of the room, and a whiteboard that was partially obscured when the small DPSc was lowered (Fig. 3a). Fig. 3. View largeDownload slide (a) Flat-floor classroom seating 60; (b) lecturer writes on penTPC sitting at desk; (c) lecturer references table of images within OneNote page; (d) lecturer stands to point to specific location on projected image; (e) while seated lecturer annotates graph on penTPC and also points towards DPSc. Fig. 3. View largeDownload slide (a) Flat-floor classroom seating 60; (b) lecturer writes on penTPC sitting at desk; (c) lecturer references table of images within OneNote page; (d) lecturer stands to point to specific location on projected image; (e) while seated lecturer annotates graph on penTPC and also points towards DPSc. The lecturer conducted the session solely with the penTPC using Microsoft OneNote, mostly sitting at a desk at the front, to the side of the DPSc facing the students (Fig. 3b). The lecturer moved naturally between sitting (while writing, annotating, and talking) and standing and moving about the room (using a range of body movements and gestures). The lecturer would occasionally indicate towards the screen (deictic gesture) while sitting, but would often get up and move to the screen to point to a specific point of interest on the screen (Fig. 3e), or to talk to the students. The positioning and small size of the DPSc allowed pointing with a hand directly proximate to items of interest on the screen. The lecturer (as in the previous cases) developed a mathematical narrative involving dynamic development of diagrammatic and symbolic material. However, this lecturer had incorporated prepared digital material on the OneNote pages, including engineering tables, standard formulae and diagrams, and referenced and added to this material during the session (Fig. 3c). For example, while referencing tables, the lecturer used attentional marks on the penTPC to indicate the relevant position in the table (drawing lines to indicate appropriate rows and circling relevant values) and on other occasions standing and pointing to the position on the DPSc (Fig. 3d). Underlining (often doubled) was used to emphasize important results. The lecturer used the zooming and scrolling capabilities of the hardware/software to focus on salient aspects of the content (such as zooming in on relevant sections of a table). The lecturer made the digital notes available to the students after the class, and in commencing the observed session, displayed and reviewed notes developed in the previous session. Thus in integrating other digital material, the lecturer was using capabilities of the penTPC that are not available in a board-based environment. In feedback, the lecturer commented that facing the students while drawing and talking was considered to be an important element of classroom presentation that was facilitated by the penTPC. The lecturer also valued the capability with the penTPC to project material at a large scale (via the DPSc), to integrate the handwritten OneNote material with other media, and to produce a record of the material as developed in class. 3.4 Case 4: undergraduate (yr 3) engineering statics This lecturer was in the third semester of using a penTPC, teaching engineering mechanics to first year undergraduate students in a large tiered lecture theatre seating 360 (Fig. 4a). Approximately 200 students were present. The lectern was at the front, centre of theatre. There were dual Data Projector Screens (and a larger single screen option). There were no wall-mounted whiteboards, but one small portable whiteboard was located to the side and beneath one DPSc. Fig. 4. View largeDownload slide (a) Large lecture theatre seating 360; (b) lecturer stands to write, holding penTPC. Fig. 4. View largeDownload slide (a) Large lecture theatre seating 360; (b) lecturer stands to write, holding penTPC. The lecturer conducted the session using the dual data projectors. One DPSc displayed PowerPoint slides (driven from the lectern PC) containing theory or textbook example questions, which were not annotated during the session. The other projector was used to display handwritten material created live on the penTPC using Microsoft OneNote. The lecturer would commonly walk to the side of the lectern while elaborating on slide material, and used hand/arm gestures freely. The lecturer used the penTPC to develop solutions to problems mostly standing with the penTPC resting on the lectern, and occasionally picking the penTPC up in one hand and moving to the side of the lectern (as enabled by the 2 m cable) to write with the other hand (Fig. 4b). Once again, the session proceeded as a mathematical narrative; the lecturer hand-wrote equations integrated with diagrams and verbal commentary. The lecturer would frequently look up to interact with the class, asking questions and clarifying points, and gesturing (beat and iconic gestures) with a hand (or hands, if not holding the penTPC). The lecturer used attentional marks (e.g. circling with highlighter ink, drawing a box around a result) to emphasise particular items, and linked associated objects with lines. The lecturer used a range of colours for emphasis and to distinguish different features. On occasion, colour was chosen with deliberate association to content (for example, switching to a blue pen and drawing wavy lines to indicate water in a diagram). Pen strokes were clear and of broad width, appropriate to the large venue. The lecturer scrolled the (OneNote) pages to add additional functional material, and on occasion scrolled back to refer to previously developed material. The features of penTPC use that lecturer valued were: facing the class; markers that did not fade; the capability to use annotations and remove them; the capability to bring in images. The lecturer commented that the penTPC served as ‘a whiteboard on steroids’. The lack of whiteboards in rooms was stated as being a key influence on adopting the penTPC approach. 3.5 Case 5: undergraduate (yr 2) mathematics This lecturer was in the first semester of using a penTPC, and teaching second year undergraduate engineering mathematics in a medium sized tiered lecture theatre (seating 236), with approximately 100 students present. A lectern was situated at the front, side of the theatre. There was a single DPSc and a strip of fixed whiteboard that was partially obscured when the screen was lowered. The lecturer used PowerPoint, running off the penTPC, projected on the single DPSc. PowerPoint presentation slides had been prepared in two forms, with some containing extensive content and others allowing additional blank space for annotation and development of additional material. When developing material, the lecturer used the penTPC on the (standing-height lectern), and spoke facing the students while writing. Problem solutions were developed as a sequential mathematical narrative, in an equivalent manner to that of a board environment. On occasion the lecturer would look up to talk, remaining at the lectern, and gesture (Fig. 5a). At times, previously written material was referred back to, and identified with attentional marks (circling). Fig. 5. View largeDownload slide (a) Lecturer lifts head to talk and gesture while at lectern; (b) new material is added and attention is drawn to key material with annotations such as circling; (c) lecturer moves to side to talk about material, and gesture. Fig. 5. View largeDownload slide (a) Lecturer lifts head to talk and gesture while at lectern; (b) new material is added and attention is drawn to key material with annotations such as circling; (c) lecturer moves to side to talk about material, and gesture. Some PowerPoint presentation slides had extensive content, often including diagrams, some of which were complex representations of 3D figures that are hard to draw live by hand. These slides were annotated mainly using attentional marks (circling, underlines, etc.) on the pre-written material to indicate points of particular focus, concurrent with speech (e.g. ‘this plane’ while circling the appropriate surface-plane on the figure on the penTPC) (Fig. 5b). Some annotations added functional information (e.g. an additional line as a functional component of a diagram). When expanding on a topic or asking questions the lecturer would often move into the space between the lectern and screen to address the students while facing them, occasionally glancing and/or gesturing towards the screen (Fig. 5c). Feedback from this lecturer included seeing the penTPC environment as having value in allowing the maintenance of a stance facing students while writing, and in facilitating the use of prepared slide material in conjunction with writing activity without the need to switch from whiteboard to DPSc. 3.6 Case 6: undergraduate (yr 3) electrical engineering This lecturer was in their fifth semester of using a penTPC, teaching a third year undergraduate paper in control engineering. The room used for the session was a medium size flat-floor classroom (seating 50) with a lecturer desk at the front to the side of the room. There was a single DPSc which partially obscured a whiteboard when lowered. The lecturer conducted the session using the penTPC and Microsoft OneNote to display prepared material and add live handwritten material. The lecturer set up the penTPC on a desk at the front (to the side of the normal lecturer desk) and sat facing the class while writing/annotating in OneNote and displaying output on the DPSc. The lecturer had earlier inserted PowerPoint slides (originally developed in previous years) as printouts in a OneNote page, and this material was further developed in the session (Fig. 6a). Slides included photographs and software-created graphics (Fig. 6c). The lecturer used the touch capability of the penTPC to scroll through the material, and zoom in and out on elements of interest. Fig. 6. View largeDownload slide (a) PowerPoint slide printed into OneNote and then annotated live in the class session; (b) lecturer gestures in circular motion (‘cows walk around crater in contours’); (c) lecturer annotates software output using penTPC. Fig. 6. View largeDownload slide (a) PowerPoint slide printed into OneNote and then annotated live in the class session; (b) lecturer gestures in circular motion (‘cows walk around crater in contours’); (c) lecturer annotates software output using penTPC. The lecturer took advantage of the capability of OneNote to provide an unconstrained writing area in scrolling to the right of the slide material to expand on the material presented in the slides. Different coloured inks were used with specific meaning assigned to different colours in some situations. This use of colour in the functional writing added emphasis that would not otherwise be as apparent. The lecturer freely interspersed writing/annotating with looking up and gesturing (e.g. drawing circular contour lines on an image, and then making iconic circular hand gestures) (Fig. 6b). Labels were added and linked to relevant points on a diagram with arrows. Thus many annotations served to add functional information, but also as attentional marks to add emphasis within the sequence of the narrative. From time to time the lecturer also would stand and move to the centre-front of the class to discuss material, freely using a range of gestures. In questionnaire response and other comments, the lecturer had expressed valuing: sitting, facing the students; ability to use a range of representational forms and colour; using a handwritten approach allowed dynamic development of material, in ways that was less formal (than text), and maintained a student-friendly pace; capability to make completed notes available to students as both a static and dynamic recording. 3.7 Case 7: undergraduate (yr 3) statistics While this lecturer was no longer using a penTPC, having personally decided there were too many disadvantages with the technology, the session is included to provide a comparative analysis. The session was a third year undergraduate statistics class in a ‘flexible blended learning space’ (flat-floor classroom) seating 36, with approximately 25 students present. The room has a lectern at the left-front, a single DPSc connected to a lectern desktop PC, and two portable whiteboards placed to the right-hand side of the DPSc (Fig. 7a). Room furniture consists of trapezoidal tables, arranged in groups seating up to six, with chairs on castors. Fig. 7. View largeDownload slide (a) Room with DPSc and portable whiteboard; (b) lecturer points to items of interest on DPSc; (c) lecturer writes, facing the board. Fig. 7. View largeDownload slide (a) Room with DPSc and portable whiteboard; (b) lecturer points to items of interest on DPSc; (c) lecturer writes, facing the board. The lecturer used the lectern PC to project use of statistical software (i.e. showing command inputs and text and graphical outputs) onto the DPSc. The two portable whiteboards were used to develop theory and examples. The lecturer was very active, moving between the lectern PC (to enter software commands), the DPSc (to point to output) and the whiteboards to develop theory and examples. The board writing was mostly with a single-colour of markers (with occasional use of a second colour, for emphasis), the lecturer writing with back to the class, talking towards the board (Fig. 7c). The lecturer’s body frequently obscured material as it was being added. The lecturer frequently pointed to features of specific interest on hand-drawn diagrams, or used annotations to highlight existing features, or add new features, along with verbal commentary. There was ready transition between pointing to existing diagrammatic features, and switching to using the in-hand marker to add new features or emphasis. When referring to material on the DPSc the lecturer used whole arm movements to point or touch contiguous to the output of interest (Fig. 7b), but written annotations were not applied (as this capability was not available). On occasion some screen material was elaborated on by reproducing material on the board. Again, gaze (and voice) were mostly directed to the board/screen, and at times the images on the DPSc were partially obscured and or shadowed by the lecturer’s body. The reasons given by the lecturer for abandoning use of the penTPC include: technical-reliability issues; regarding the technology as forming a barrier between lecturer and students, with the need to ‘have to look down’ to use it; that the ‘text moves around, unlike writing on a board which stays put until erased’. 4. Analysis and comment Characteristic elements of a mathematical lesson, as identified by Fox & Artemeva (2011) are shown in Table 1, with Column 1 showing the forms characteristic of the board classroom, and column 2 the equivalent (and additional) forms in the penTPC environment. The alphabetic code assigned to the rows in Table 1 (e.g. [A]) are used to identify these elements in the following discussion. The basic process of developing a mathematical narrative, involving writing mathematical symbols, texts, graphs and diagrams [A] while verbalizing [B], was observed to take similar form in both board and penTPC environments. In both environments lecturers would verbalize while they were writing [C], and stop writing to talk about what they had written [D]. However, there were differences in how they moved in space [E], and in the use of gestures [F]. While in the process of writing [A] lecturers using the penTPC were more static with regard to whole body positioning. Lecturers did not move across the room as a board user would as they wrote progressively across a board, but were ‘tied to the tablet’ (Bonnington et al., 2007, p. 8). Nevertheless, the sessions examined here showed that lecturers could (and would, according to preference) move freely, stepping away from the device, when talking about what had been written [D]. In doing so the lecturer would interact with the displayed material as they would in with board-based material or slides displayed on a DPSc, potentially using a full range of gestures [F], but with limitations, as discussed below. In the penTPC environment, whole body movement was not necessary to transition between writing and other interactions; rather than stopping writing and turning (as with a board technology), the lecturers just stopped writing and looked up. Thus the activities of talking [D], some forms of gesturing [F], consulting of notes [G], pausing for reflection [H], checking student understanding [I] and asking questions [J] were possible without needing to turn or move away from the device (although that option was available). In board-based teaching, it has been noted that 75% of the lecturer’s time might be spent with body facing the board (Fox & Artemeva, 2011, pp. 95–96). Most lecturers in this study saw it as a key advantage of the penTPC mode for their body to be facing the students (rather than the board) while writing and talking [A][C], allowing them to simply glance up to talk [D] and interact with students [I][J]. However, in Case 7 here the lecturer regarded the penTPC as a barrier in the relationship with students, seeing ‘looking down to write’, as an activity that disrupted the normal flow of a lesson. As observed, the lecturer was very active in class, in both body movement and gesture. This style was also recorded by Bonnington et al. (2007, p. 8), who noted a lecturer’s observation that his ‘distinctive dramatic style involving much arm-waving and walking around to emphasise points’ was severely constrained in the penTPC environment. It may be that for some lecturers, the penTPC requires too a radical a change to their accustomed movements for it to be comfortably adopted. A critical difference between board and penTPC environments is in the capability for use of deictic (pointing) gestures [F], while talking about what had been [D], or was in the process of being [A][C], written. While writing at a board, a lecturer can simply point to an object for emphasis (as well as circle or underline), but when working at a penTPC the lecturer needs either to use an attentional mark or to move away from the device to point to the object on the projected image [E][F]. Furthermore, depending on the size and positioning of the DPSc, lecturers were not always able to position a hand directly adjacent to the indicated object, so that pointing, or deictic gestures were less precise, or constrained, compared to a board environment. Thus in the penTPC environment, the use of annotation, in the form of attentional marks, becomes an essential mechanism for indicating relationships, signalling references, and highlighting key issues [F]. For the lecturer working on the Tablet screen, the focal point of the writing operation is directly that of the tip of the stylus (as directed by the hand/arm), with touch actions also operating directly on the on-screen representation of objects, as if in direct physical contact. Thus while the lecturer maintains, in their sight, the linkage between their physical and mental focus, the student only explicitly views the results of lecturer actions (on the Tablet PC screen) in changes as projected on the DPSc, without the physical cues of arm/hand pointing to the location of activity. The student is thus very reliant on visual cues on the screen to focus attention on the point of interest, and the lecturer needs to give conscious attention to the creation of these cues. While developing symbolic content following a standard left-to-right, top-to-bottom sequence, the position of new material was generally able to be readily anticipated, and was clearly apparent. However, for material not entered in such a sequence (for example, when a lecturer added a label to an existing chart, or new point on an existing graph) it was important that this added material had sufficient prominence, through use of colour and/or size and dynamic development, to ensure the location of the point (the ‘here’ in the verbal commentary) was immediately obvious. As described by Bunt et al. (2009, pp. 229, 230) (as referenced earlier) the development of a mathematical narrative involves the simultaneous development of dynamic objects with writing occurring in sequence with verbal commentary. As observed here (in Case 1), commentary stating ‘from point a to point b’ was accompanied with the marking, and labelling, of these points on a previously drawn graph axis. Subsequently the distance between these points was also identified, with a double-headed arrow joining the points and new label added. Thus, the timely addition of these points provides what we will term sequential emphasis; however, this writing also provides functional information that remains relevant after the commentary moves on, and even in a static form as written notes. There were only a few instances of use of purely attentional marks: at times underlines were used when referring back to previously written material, to emphasis words or terms that were again significant in the procedural flow; on one occasion a circle was used to highlight a symbol in an equation written earlier (requiring scrolling the screen to reveal, circle, and scrolling back to continue the narrative). In addition, some lines/points on a diagram were drawn over with multiple strokes, to emphasize their location as relevant to the ongoing narrative. However, in most cases, it was the direct initial appearance of new content that gave emphasis to its sequence in the narrative. As noted the penTPC environment provides capabilities to include a range of different graphic material in digital form directly into the writing and viewing space [K]. Not all lecturers made use of this affordance, with some simply using the penTPC as a basic writing slate, while others inserted additional material into presentations, closely integrating it into the mathematical narrative. For example, in Case 6, rather than rely purely on iconic gestures with the hands to suggest a volcanic crater, with circular motions to suggest contour lines within it (Fig. 6b), the lecturer included a photograph of a crater, and drew contour lines directly on the image (Fig. 6a). Graphic output from software (showing mathematic representations of craters) was also annotated, with both attentional and functional purpose (Fig. 6c). Similarly, in Case 5, the lecturer displayed a full table of values, and annotated (circled or highlighted) relevant values for the ongoing calculations. Thus these annotations had both short-term attentional purpose, as well as long-term functional purpose in providing a permanent record of the origin of values that were used later in procedural developments. In this study, annotations such as arrows were observed being used to dynamically connect equivalent items in the different representational forms (images, tables, diagrams, symbolic forms), and in enabling different representational forms to be integrated (such as dynamically adding symbolic equations to diagrams). While some use of annotation in way this may be observed in the board environment, the extended range of representational forms [K][L][M] available in the digital penTPC environment provides additional opportunities for annotation, and unlike transient gestures and speech, was observed here serving a contextual, attentional purpose, while adding persistent functional information. That annotation may be used for emphasis (attentional marks) and to communicate functional content has been noted previously (Ambikairajah et al., 2006; Alibali et al., 2014). Thus, rather than regarding annotations as being of distinct types, it may be appropriate to regard them also as having a range of dimensions, following McNeill’s approach to types of gesture (McNeill, 2005, p. 38). An initial suggestion for categorizing dimensional forms of annotations is shown as Table 2. Table 2. Forms of annotation, distinguished by suggested dimensions/purpose. Note that anotations may have more than one purpose. As with gestures, there may be an associated progression of formality, from simple marks (equivalent to gesticulations), through conventional signs (emblems), iconic or representational content (pantomime), through to functional content structured according to formal rules (sign language) Annotation dimension  Description/purpose  Examples  Attentional focus (simple mark)  Directions attention to point in written narrative  Dot or indistinct mark  Conceptual emphasis (emblem)  Focus or re-focus on key objects (equations, terms)  Box, highlight  Sequential emphasis (emblem)  Call attentions to an existing object at critically relevant time in the narrative  Circled item ‘this’;  Sequential emphasis (content)  Addition of content object at critically relevant time in the narrative  Addition of written content, point ‘a’, or tangent on curve  Closure/completion (emblem)  Indicates completion of a task, or separates coherent sections  Double diagonal lines//horizontal line -___________  Linking (emblem)  Connects different objects, or the same object in different representations  Arrows, lines connecting value, symbol in an expression or calculation with value in a table  Associative or cohesive (emblem)  Collects objects having a common property together  Enclosing objects within line; use of common colour for ‘like’ items  Iconic representational (pantomime)  Representational drawing or graphic  Wheeled cart drawn in mechanics problem  Iconic association–metaphoric form (pantomime)  Format of object reflects a property of object  Spring drawn as zigzag lines Use of blue colour, wavy lines to represent water on a diagram  Structured functional content  Diagram, graph, algebraic working, text (or part of)  Equation, mathematical working  Annotation dimension  Description/purpose  Examples  Attentional focus (simple mark)  Directions attention to point in written narrative  Dot or indistinct mark  Conceptual emphasis (emblem)  Focus or re-focus on key objects (equations, terms)  Box, highlight  Sequential emphasis (emblem)  Call attentions to an existing object at critically relevant time in the narrative  Circled item ‘this’;  Sequential emphasis (content)  Addition of content object at critically relevant time in the narrative  Addition of written content, point ‘a’, or tangent on curve  Closure/completion (emblem)  Indicates completion of a task, or separates coherent sections  Double diagonal lines//horizontal line -___________  Linking (emblem)  Connects different objects, or the same object in different representations  Arrows, lines connecting value, symbol in an expression or calculation with value in a table  Associative or cohesive (emblem)  Collects objects having a common property together  Enclosing objects within line; use of common colour for ‘like’ items  Iconic representational (pantomime)  Representational drawing or graphic  Wheeled cart drawn in mechanics problem  Iconic association–metaphoric form (pantomime)  Format of object reflects a property of object  Spring drawn as zigzag lines Use of blue colour, wavy lines to represent water on a diagram  Structured functional content  Diagram, graph, algebraic working, text (or part of)  Equation, mathematical working  Classroom sessions in mathematical disciplines have traditionally been structured around having large areas of board space remaining visible and displaying unchanged content during at least part of a session (but erased at some point). The use of penTPC software means all content is retained (even after a session), but a smaller proportion may remain in the field of view displayed by the DPSc at any one time, and other material requires scrolling or zooming to access. Zooming and panning was used as an additional mechanism for providing emphasis, by centring and enlarging critical information on the DPSc. From the perspective of the lecturer, this mechanism has some characteristics of gesture, in that a hand movement initiates a communication, with direct results of that action manifest in the display. However, the hand action is not directly visible to the students, who only observe the resulting change in display. As such, these (and zooming/panning) might arguably be termed as technology-mediated deictic gestures (Table 3). Table 3. Technology mediated gestures and transient annotations Technology mediated gestures (deictic)   Involve hand actions by the lecturer, with features of a deictic gesture from their perspective, but with only the effects of those actions viewable to students on the DPSc (and not the hand movements themselves).         Lecturer action      Effect on DPSc            Pinch touch gesture          Screen expands to give overview          Stretch touch gesture          Focus directed to detail of content          Finger scroll          Focus directed to previous content    Transient annotations (deictic)  Mouse/pointer actions have features of an annotation, and involve use of a tool, but are transient and do not appear in a static digital record of a session.     Lecturer action   Effect on DPSc            Mouse or pen-hovered over (points to) specific content (or pen-as-laser setting used)—directs attention          Transient appearance/movement of mouse-pointer position          Pen-as-pointer setting to annotate—directs attention          Annotation appears, but disappears after a few seconds)  Technology mediated gestures (deictic)   Involve hand actions by the lecturer, with features of a deictic gesture from their perspective, but with only the effects of those actions viewable to students on the DPSc (and not the hand movements themselves).         Lecturer action      Effect on DPSc            Pinch touch gesture          Screen expands to give overview          Stretch touch gesture          Focus directed to detail of content          Finger scroll          Focus directed to previous content    Transient annotations (deictic)  Mouse/pointer actions have features of an annotation, and involve use of a tool, but are transient and do not appear in a static digital record of a session.     Lecturer action   Effect on DPSc            Mouse or pen-hovered over (points to) specific content (or pen-as-laser setting used)—directs attention          Transient appearance/movement of mouse-pointer position          Pen-as-pointer setting to annotate—directs attention          Annotation appears, but disappears after a few seconds)  When using a mouse, the position of attention is indicated on-screen by a mouse pointer icon. This pointer icon can also be activated in the penTPC environment by hovering the pen over the screen. Thus rather than creating persistent attentional marks, focus can be directed to particular locations by the lecturer, by their positioning, or pointing, with a pen or mouse. While the position of a standard mouse pointer may not always be obvious on a DPSC, there are techniques and settings to increase its visibility (for example, increasing the cursor/pointer size, using the CTRL key to reveal mouse/pen position, or using the laser pointer tool in PowerPoint). In addition, the pen-as-pointer tool in OneNote can be used to make a transient annotation (a mark that disappears after a short interval). These actions have features of a technology mediated gesture, but may better classified as transient annotations, in the trace register (Table 3). While the use of specialized pointer icons was not observed in this study, it is an issue for further investigation. The affordance of the penTPC environment for ready use of colour in annotation has been previously noted (Fister & McCarthy, 2008; Wilson & Maclaren, 2013), although it has been observed elsewhere that it is not consistently used in a meaningful way (Anderson et al., 2004, p. 573). While most lecturers here tended to use a single colour, or occasionally a second colour for emphasis, some lecturers here used a range of colours systematically, with concurrent attentional and functional purposes. For example, systematic use of colour in circling, highlighting or writing objects gave not just transient attention to particular objects, but provided ongoing focus on the common functional properties of the like-coloured objects. Wang & Chu (2013) proposed that even non-representational beat gestures help convey meaning. While annotations may take on some of the role of iconic and deictic gestures, the beat gesture is not so readily transferred into a written form (with multiple underline strokes perhaps being one option). However, the lecturers did not appear to be constrained in the use of hand created beat gestures, simply looking up from the penTPC and moving hands (sometimes while holding pen, or glasses) in beat gestures to provide emphasis when talking about material. 5. Conclusions and further developments It had been suggested that the use of the penTPC may affect communication in the classroom by constraining the range of movements and gestures used by the lecturer. However, the examination of the use of penTPCs here has revealed that, while some changes were observed in the sequencing and features of movements, a wide range of communication elements used by the board-based lecturer were still exhibited, with use of gestures remaining as a core component in lecturer presentations. Thus the core elements of ‘chalk talk’ as described by Fox & Artemeva (2011) were observed in use, albeit in modified form, by lecturers using the penTPC (Table 1, Rows [A]–[J]). It is in the reduced use of deictic gesture and expanded use of annotation that the most significant changes accompanying the introduction of the penTPC environment were manifest. It is apparent that annotation may provide a powerful way of conveying meaning that extends beyond that available through simple gesture. While this study has been focused on the use of gesture and annotation as used by the lecturer in the classroom, the penTPC also readily enables classroom generated notes to later be made available to students as a permanent record of the session. Most lecturers here regarded this capability to provide a permanent digital record of all material that could be scrolled and accessed within and outside class sessions as an advantage of the penTPC approach, and made active use of the capability. The persistent nature of annotations (as opposed to the non-recorded ephemeral nature of gestures) can have a significant impact (both positive and negative) on the future use of recorded notes. The nature of the annotations (and gestures) made in class may also impact on any notes that students make themselves. These aspects are outside the scope of this article, and are discussed elsewhere (In press, 2017). While the adoption of the technology has not been without technical issues, the majority of lecturers who have piloted the use of the penTPC determined that the benefits outweighed the disadvantages. In some cases, while there was initial resistance fostered by a view that the technology was only required because of the absence of suitable whiteboards in classrooms, lecturers became appreciative of the potential benefits of the penTPC approach. Some lecturers may find the restrictions on their established natural movement and use of gesture a challenge. The successful use of a penTPC approach requires adoption of different techniques, and a willingness to do so depends on how the individual lecturer perceives and weighs the relative advantages and disadvantages. Unlike the established form of chalk talk as described by Artemeva & Fox (2011; Fox & Artemeva, 2011), the precise form of delivery of the mathematical narrative in the penTPC environment is still evolving. It is apparent that the augmented capability for annotation in conjunction with other digital representations in the penTPC environment can provide opportunities to enhance teaching, particularly of STEM based discipline subjects. However, this study observed a wide range in current levels of utilisation of these affordances. The increasing adoption of the penTPC in the study university gives importance to the establishment of guidelines for their effective use, which will be a focus of ongoing work. Peter Maclaren is a Principal Lecturer and Academic Advisor in the Centre for Learning and Teaching at the Auckland University of Technology (AUT). He has a background in engineering (B.E., University of Auckland) and regional planning. He taught applied mathematics at AUT for some 15 years before becoming involved with the development of online learning environments. He has a Masters in Educational Technology completed online through the University of Southern Queensland. His current professional and research interest (and PhD topic) is in developing pedagogies in STEM disciplines that take advantage of the particular affordances of pen-enabled Tablet PC technology. David Wilson is an Associate Professor in Electrical Engineering at AUT. Prior to joining AUT he was on the faculty at Karlstad University in Sweden following a position at the Swiss Federal Institute of Technology (ETH) in Zürich, Switzerland. His main research interests are modelling, optimisation and control of industrial processes, development of embedded controllers, and effective graduate teaching strategies for mathematically intensive engineering subjects. Currently he is a director of the research-based Industrial Information and Control Centre (I2C2) where he manages multi-faceted research projects for international and national clients. He is a director of Inverse Problems Ltd which is a startup company developing and marketing small high-performance embedded controllers. This company recently won the inaugural AUT Enterprises Innovation Challenge for 2012. Sergiy Klymchuk is an Associate Professor of mathematics in the School of Engineering, Computer and Mathematical Sciences, AUT, New Zealand. He has been teaching university mathematics in different countries since 1980. His PhD (1988) was in differential equations and recent research interests are in mathematics education. He is an author of more than 200 publications including the Counterexamples in Calculus book that received an Outstanding Academic Title Award from the Choice magazine of the American Library Association in 2010, Paradoxes and Sophisms in Calculus book that made the cover of the 2014 Publications Catalogue of the Mathematical Association of America, and Money Puzzles book on popular mathematics that has been published in nine countries. References Alibali M. W, Nathan M. J, Wolfgram M. S, Church R. B, Jacobs S. A, Johnson Martinez C, Knuth E. J. ( 2014) How teachers link ideas in mathematics instruction using speech and gesture: a corpus analysis. Cogn. Instr , 32, 65– 100. Google Scholar CrossRef Search ADS   Ambikairajah E, Epps J, Sheng M, Celler B. ( 2007) Signal processing education using the TabletPC and electronic whiteboard. IEEE Signal Processing Magazine 24.1 (2007): 130–33. Anderson R, Anderson R, McDowell L. ( 2005) Best Practices for Lecturing with Digital Ink . Washington, DC: University of Washington. Anderson R, Hoyer C, Wolfman S. A, Anderson R. ( 2004) A study of digital ink in lecture presentation. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 567–574. Artemeva N, Fox J. ( 2011) The Writings on the Board: The Global and the Local in Teaching Undergraduate Mathematics Through Chalk Talk. Writ. Commun. doi:10.1177/0741088311419630 Arzarello F. ( 2006) Semiosis as a multimodal process. RELIME Rev. Latinoam. Investig. En Matemática Educ., 9, 267– 300. Arzarello F, Paola D, Robutti O, Sabena C. ( 2008) Gestures as semiotic resources in the mathematics classroom. Educ. Stud. Math , 70, 97– 109. Google Scholar CrossRef Search ADS   Bates A. W. ( 2015). Teaching in a Digital Age: Guidelines for Designing Teaching and Learning . Vanvouver BC: Tony Bates Associates Ltd. Bonnington P, Oates G, Parnell S, Paterson J, Stratton W. ( 2007) A report on the use of tablet technology and screen recording software in tertiary mathematics courses. Vis. Change New Century Proc. Calafate Delta , 7, 19– 32. Bosch M, Chevallard Y. ( 1999) La sensibilité de l’activité mathématique aux ostensifs: objet d’étude et problématique. Rech. En Didact. Mathématiques , 19, 77– 123. Bunt A, Terry M, Lank E. ( 2009) Friend or foe?: examining CAS use in mathematics research. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, pp. 229– 238. Campbell S, Latulippe D. R. ( 2015) Towards improved learning of fluid mechanics via integration of a commercial software package into an undergraduate course. in Proceedings of the Canadian Engineering Education Association . Hamilton, Ontario: Canadian Engineering Education Association. Retrieved from http://library.queensu.ca/ojs/index.php/PCEEA/article/viewFile/5767/pdf Châtelet G. 2000. Les enjeux du mobile. Seuil, Paris (1993); English translation by Shore R., Zagha M.: Figuring Space: Philosophy, Mathematics, and Physics . Dordrecht: Kluwer Academic Publishers. De Freitas E, Sinclair N. ( 2011) Diagram, gesture, agency: theorizing embodiment in the mathematics classroom. Educ. Stud. Math ., 80, 133– 152. Google Scholar CrossRef Search ADS   Fister K. R, McCarthy M. L. ( 2008) Mathematics instruction and the tablet PC. Int. J. Math. Educ. Sci. Technol , 39, 285– 292. Google Scholar CrossRef Search ADS   Fox J, Artemeva N. ( 2011) The cinematic art of teaching university mathematics: chalk talk as embodied practice. Multimodal Commun ., 1, 83– 103. Goeser P. T, Ruiz S. ( 2015) The Development of MATLAB Functions for Effective Use and Improvement of Student Learning in a Thermodynamics Course. Presented at the ASEE Southeast Section Conference, University of Florida, Gainsville, FL: American Society for Engineering Education, Southeastern Section. Retrieved from http://www.softwareeducationsupport.com/ASEE%20SE%20Conference%20Proceedings/Conference%20Files/ASEE2015/Papers2015/141.pdf Goldin G. A. ( 2010) Perspectives on representation in mathematical learning and problem solving 2nd Edition. In L. D. English (Ed.), Handbook of International Research in Mathematics Education (pp. 176–201). New York, NY: Routledge. Greiffenhagen C. ( 2008) Video analysis of mathematical practice? Different attempts to ‘open up’ mathematics for sociological investigation. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research. Hodge B. K, Taylor R. P. ( 2002) Piping-system solutions using Mathcad. Comput. Appl. Eng. Educ , 10, 59– 78. Google Scholar CrossRef Search ADS   Maclaren P, Wilson D. I, Klymchuk S. ( in press) I see what you are doing: Student views on lecturer use of Tablet PCs in the classroom. Australasian Journal of Educational Technology. Maclaren P, Wilson D. I, Klymchuk S. ( 2017) Lecturer Use of Pen-enabled Tablet PC Technology in the STEM Classroom: Implications for Notetaking (Manuscript in preparation). McNeill D. ( 1992) Hand and Mind: What Gestures Reveal about Thought . Chicago: University of Chicago Press. McNeill D. ( 2005) Gesture and Thought . Chicago: University of Chicago Press. Google Scholar CrossRef Search ADS   McNeill D. ( 2006) Gesture and thought. The Summer Institute on Verbal and Non-Verbal Communication and the Biometrical Principle. Vietri sul Mare, Italy. Retrieved from http://mcneilllab.uchicago.edu/pdfs/dmcn_vietri_sul_mare.pdf Nielsen J. ( 2000) Why you only need to test with 5 users [WWW Document]. Nielsen Norman Group. URL https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/ (accessed 12 June 2016). Oviatt S. ( 2013) The Design of Future Educational Interfaces . New York: Routledge, Taylor & Francis Group. Radford L. ( 2008) Why do gestures matter? Sensuous cognition and the palpability of mathematical meanings. Educ. Stud. Math ., 70, 111– 126. Google Scholar CrossRef Search ADS   Roth W. M. ( 2001) Gestures: their role in teaching and learning. Rev. Educ. Res ., 71, 365– 392. Google Scholar CrossRef Search ADS   Sabena C. ( 2008). On the semiotics of gestures. Semiotics in Mathematics Education: Epistemology, History, Classroom, and Culture  ( Radford L., Chubring G., Seeger F. eds). Taipei: Sense, Rotterdam. Shulman L. ( 2005) Signature pedagogies in the professions. Daedalus , 134, 52– 59. Google Scholar CrossRef Search ADS   Singer S, Smith K. A. ( 2013) Discipline-based education research: understanding and improving learning in undergraduate science and engineering: discipline-based education research. J. Eng. Educ ., 102, 468– 471. Google Scholar CrossRef Search ADS   Tang K. C, Davis A. ( 1995) Critical factors in the determination of focus group size. Fam. Pract ., 12, 474– 475. Google Scholar CrossRef Search ADS PubMed  Thomas M. O, Hong Y. Y. ( 2013) Teacher integration of technology into Mathematics Learning. Int. J. Technol. Math Cambridge. Educ ., 20( 2), 69– 84. Vygotsky L. ( 1978) Mind in Society: The Development of Higher Psychological Processes . Cambridge, MA: Harvard University Press. Wang L, Chu M. ( 2013) The role of beat gesture and pitch accent in semantic processing: an ERP study. Neuropsychologia , 51, 2847– 2855. Google Scholar CrossRef Search ADS PubMed  Wilson D. I, Maclaren P. ( 2013) From Chalk Talk to Tablet Talk: Pedagogies for Control Engineering. Advances in Control Education, Advances in Control Education. Presented at the 10th IFAC Symposium Advances in Control Education, 2013, International Federation of Automatic Control, University of Sheffield, Sheffield, United Kingdom, pp. 144–149. Yoon C, Thomas M. O. J, Dreyfus T. ( 2011) Gestures and insight in advanced mathematical thinking. Int. J. Math. Educ. Sci. Technol ., 42, 891– 901. Google Scholar CrossRef Search ADS   © The Author 2017. Published by Oxford University Press on behalf of The Institute of Mathematics and its Applications. All rights reserved. For permissions, please email: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices) For permissions, please e-mail: journals. permissions@oup.com http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Teaching Mathematics and Its Applications: International Journal of the IMA Oxford University Press

Making the point: the place of gesture and annotation in teaching STEM subjects using pen-enabled Tablet PCs

Loading next page...
 
/lp/ou_press/making-the-point-the-place-of-gesture-and-annotation-in-teaching-stem-8LJ5db7yFG
Publisher
Institute of Mathematics and its Applications
Copyright
© The Author 2017. Published by Oxford University Press on behalf of The Institute of Mathematics and its Applications. All rights reserved. For permissions, please email: journals.permissions@oup.com
ISSN
0268-3679
eISSN
1471-6976
D.O.I.
10.1093/teamat/hrx002
Publisher site
See Article on Publisher Site

Abstract

Abstract The teaching of STEM subjects, and engineering and mathematics in particular, involves the use of a wide range of representational forms, including equations, diagrams, sketches and graphs, supported by speech and gestures. In the traditional face-to-face ‘board’ based classroom, the integration of writing, speech and gesture has been a key feature of pedagogical delivery approaches. The pen-enabled Tablet PC (penTPC), used in conjunction with a data projector, allows for the maintenance of a handwritten approach in teaching environments where traditional boards are unavailable or limited. However, it has been suggested that the use of the digital interface imposes restrictions on lecturer movement and gesture, compared to traditional board environments. This article examines the adaptations made by lecturers in using the penTPC in a classroom environment. The study suggests that the use of penTPC technology does not preclude the use of gesture, and that the augmented capability for annotation in conjunction with other digital representations can enhance teaching, particularly of STEM-based discipline subjects. 1. Introduction Gestures … are writing in the air, and written signs frequently are simply gestures that have been fixed. (Vygotsky, 1978, p. 107) This article is concerned with the effective use of the pen-enabled Tablet PC (penTPC) as a technology for the teaching of mathematically based subjects, specifically as it may be used by a teacher in the context of a classroom environment. While the lecture as a generic pedagogic device has been widely criticised (Bates, 2015), Fox & Artemeva (2011) have defended its use in the mathematically intensive disciplines where the characteristic combination of writing and talking is seen as providing essential components for developing students-capability. Greiffenhagen (2008) describes the mathematics lecture as where ‘an experienced mathematician demonstrates mathematical expertise to novices as an important part of their progressive induction into professionally competent autonomous mathematical practice’. An important element of this approach is the use of handwritten modes for the development of material, which establishes a pacing suiting the student acquisition of procedural skills and concepts (Artemeva & Fox, 2011). While the penTPC clearly supports the handwriting component essential in developing traditional mathematical thinking, other components are also involved in face-to-face communication. Gesture has been suggested as playing an important role in teaching and learning (Roth, 2001), with a number of authors noting the importance of gesture in the specific context of developing mathematical thinking (Arzarello, 2006; Arzarello et al., 2008; Radford, 2008; Goldin, 2010; Fox & Artemeva, 2011; Alibali et al., 2014). Thus, along with additional affordances of the penTPC, attention needs to be given to the constraints that the device may impose on the way gesture is used (Thomas & Hong, 2013). If, as Yoon, Thomas and Dreyfus (2011) argue, gestures can help to develop advanced mathematical insights ‘by supporting the creation of virtual mathematical constructs’ (p. 891) then it is important to investigate the constraints on use of gesture that may arise in the penTPC environment. Gestures in the broad sense have been classified in a range of forms, from gesticulation, emblems, pantomime, to sign language, in a succession in which concurrent speech becomes progressively less important in maintaining meaning (McNeill, 2005, p. 5). As a prominent form of gesture, gesticulation has been a major focus for many researchers, such that the generic term gesture is often used in place of gesticulation (McNeill, 2005, p. 5, 2006, p. 3; Roth, 2001, p. 369). McNeill (1992) identified four major types of gesture: iconic (referencing a concrete object); metaphoric (referencing an abstract idea); beat (a rhythmic emphasis); and deictic (pointing, positional). McNeill later (2005, p. 38) proposed that it is better to think of gestures in terms of having dimensions (rather than being of distinct types), with any one gesture potentially exhibiting some or all of the various dimensions to differing extents. This is the approach used here in examining lecturer use of gestures. While using a penTPC the lecturer is holding a writing instrument in the form of a digital pen. Thus the lecturer has options to make a gesture in a conventional way, in a transient form in the air with hands, or to give it concrete form as a written mark on the screen. These written marks have been termed both ‘writing gestures’ (Alibali et al., 2014, p. 76) and ‘attentional marks’ (Anderson et al., 2004, p. 797), with each term assigning emphasis to different aspects of their generation. Classifications of written, oral, and gesture may be used to identify primary registers of communication, or as termed by Bosch & Chevallard (1999, p. 96), and translated in Arzarello (2006, p. 270): trace (written), oral and gesture. Sabena (2008) defined gestures in a mathematical setting as including ‘all those movements of hands and arms that subjects (students and teachers) perform during their mathematical activities and which are not a significative part of any other action’ (such as writing with a pen). Thus the term attentional marks (rather than writing gestures) will be preferred here, locating them clearly as permanent marks within the trace register. Nevertheless, attentional marks share some characteristics of gestures, in that they may at times be dependent for full meaning on other co-existing modes, including previously created functional writing and accompanying speech. However, as Anderson et al. (2005) noted, there is a critical difference between the ephemeral nature of gestures and the persistence of attentional marks. It has also been suggested that there are strong linkages between diagrams and gesture, with diagrams suggested as a refined extension of an iconic gesture, in written form. De Freitas & Sinclair (2011) noted the relationship between gesture and diagrams, expanding on the work of Châtelet (2000), and suggesting that diagrams lock or capture gestures. Bunt et al. (2009) describe how ‘mathematicians make liberal use of sketches, mathematical expressions and annotations to render abstract mathematical concepts more concrete’, also noting that these representational forms should be viewed as dynamic objects, as ‘terms in an expression are crossed out, content is added to sketches, and new insights lead to new annotations’. It may be useful to consider attentional marks within a wider context of annotations, just as gesticulation may be considered as a particular form of gesture. Thus annotations may range from non-descript marks (analogous to gesticulations), through universally recognizable signs, such as ticks, circles, underlines and highlights (emblems), to elements within fully structured symbolic languages, such as algebraic equations (sign languages). While annotations may be created concurrently with speech, and some may be dependent on that speech for full meaning, others annotations may establish or maintain meaning independently of speech, and beyond the instant of their creation. Furthermore, just as gestures may be considered as having dimensions (rather than being of distinct types), annotations may have multiple dimensions, potentially serving multiple purposes. Thus the addition or development of functional content in graphs, diagrams and symbolic expressions (such as adding tangents to a curve, labelling points on a diagram, or cancelling terms in an algebraic expression) adds functional content in conjunction with giving timely emphasis to the sequencing of that content. In a board-based classroom, the teaching of mathematically intensive subjects has developed into a standard and well recognized pedagogical form (Fox & Artemeva, 2011, p. 83; Shulman, 2005, pp. 53–54). In substituting a penTPC for a board, lecturers are required to adapt this traditional approach according to the affordances and limitations of the technology. Fox & Artemeva (2011) described traditional classroom presentation as a ‘cinematic art’ that has many elements, including the use of gesture as a natural and spontaneous component of teaching and ‘enacted in part through the physical positioning of the professor’. There are significant interactions between body movements (including position) and forms of gesture and writing, and these are impacted by a change from the use of board to penTPC. Thus whole body movements (including positioning within the classroom) are also an important feature examined in this study. While the use of the penTPC may affect communication in the classroom by constraining the range of movements and gestures used by the lecturer, it also provides other affordances, with increased capabilities for introduction of other digital forms, and interactions with those forms through direct annotation. In recent years, the use of software-based methods for problem solving and data visualization has had a significant impact on STEM content knowledge. New discipline procedures and conceptual understandings have been developed that would not be possible without the use of new computing technologies. The capabilities of software to generate visual representations of data have also altered approaches to the interpretation of data and decision making (Hodge & Taylor, 2002; Campbell & Latulippe, 2015; Goeser & Ruiz, 2015). The capability to integrate software output and diagrammatic representations into digital teaching environments, and to annotate directly on them, is arguably now an essential element of STEM education, and an element for which the use of digital pen devices such as the penTPC provide critical affordances. Singer & Smith (2013) have noted the value of using multiple forms of representations in instruction, including ‘realistic (picture or text), diagrammatic (free body diagram) and symbolic (mathematical)’ representations (p. 470). Oviatt (2013) stresses the importance of using interfaces that support ‘expression of non-linguistic representations, including diagrams, symbols, numbers, and informal marking” (p. 61).’. The capability of the penTPC to provide support for these features is also of interest here. While recognizing that detailed analysis might cover a complex range of interacting resources (as a semiotic bundle, as described by Arzarello (2006)), this study focused on the specific elements as identified by Fox & Artemeva (2011, p. 80) as occurring in the board-based classroom, but extended to include additional elements available in the penTPC environment. Table 1 provides a comparative framework of elements, with column 1 relating to board environments, and column 2 the penTPC environment. Thus the study sought to identify the broad effect on the nature of multimodal communication, in particular in the use of annotation and the relationship with gesture, resulting from the adoption of the penTPC technology by lecturers and as used in the specific context of the teaching of mathematically intensive engineering courses. Table 1. Components of the mathematical narrative in board-based and penTPC class sessions (after Fox & Artemeva (2011)) Element code  Board classroom—co-occurring elements  Tablet PC—co-occurring elements  A  Writing mathematical symbolism and text on a board; drawing graphs/diagrams  Writing mathematical symbolism and text on the penTPC; drawing graphs/diagrams  B  Referring to problem sets and textbook chapters  Referring to problem sets and textbook chapters  C  Verbalizing what is being written while writing on the board  Verbalizing what is being written while writing on the penTPC  D  Talking about what has been written on the board  Talking about what has been written on the penTPC  E  Moving in space  Moving in space (away from the penTPC, and with the penTPC)  F  Gesturing (including pointing) to indicate relationships, signal references, highlight key issues, and so on  Annotating (and where possible, gesturing) to indicate relationships, signal references, highlight key issues, and so on  G  Consulting/reading lecture notes  Consulting/reading lecture notes  H  Stepping back from the board, pausing the action for reflection  Looking up from the penTPC, pausing the action for reflection  I  Checking student understanding by quickly glancing at the class  Checking student understanding by quickly glancing at the class  J  Turning to students and asking questions, and talking with students  Looking up from the penTPC and asking questions, and talking with students  K    Inclusion of detailed graphic images/diagrams in presented material for subsequent development and annotation  L    Use of graphic images or video for development of conceptual understanding  M    Dynamic use of software to produce tables, diagrams, visualizations; copy and paste into pages for subsequent annotation  Element code  Board classroom—co-occurring elements  Tablet PC—co-occurring elements  A  Writing mathematical symbolism and text on a board; drawing graphs/diagrams  Writing mathematical symbolism and text on the penTPC; drawing graphs/diagrams  B  Referring to problem sets and textbook chapters  Referring to problem sets and textbook chapters  C  Verbalizing what is being written while writing on the board  Verbalizing what is being written while writing on the penTPC  D  Talking about what has been written on the board  Talking about what has been written on the penTPC  E  Moving in space  Moving in space (away from the penTPC, and with the penTPC)  F  Gesturing (including pointing) to indicate relationships, signal references, highlight key issues, and so on  Annotating (and where possible, gesturing) to indicate relationships, signal references, highlight key issues, and so on  G  Consulting/reading lecture notes  Consulting/reading lecture notes  H  Stepping back from the board, pausing the action for reflection  Looking up from the penTPC, pausing the action for reflection  I  Checking student understanding by quickly glancing at the class  Checking student understanding by quickly glancing at the class  J  Turning to students and asking questions, and talking with students  Looking up from the penTPC and asking questions, and talking with students  K    Inclusion of detailed graphic images/diagrams in presented material for subsequent development and annotation  L    Use of graphic images or video for development of conceptual understanding  M    Dynamic use of software to produce tables, diagrams, visualizations; copy and paste into pages for subsequent annotation  2. Study approach The study involved seven lecturers who had been using penTPCs in teaching STEM disciplines in a university environment, drawn from a group who participated in pilot projects which provided them with access to a penTPC for teaching, who consented to be studied, and where timetabling constraints allowed access to class sessions. While essentially a convenience sample, the lecturers involved had varying levels of experience in use of a penTPC, from those in their first semester of use, to those with over 3 years of experience, and the lecture sessions surveyed covered a range of subjects and levels within mathematics/engineering disciplines. One of the lecturers was no longer using a penTPC in the classroom, and had reverted to using a board. While the number of lecturers involved was small, it was considered sufficient to give useful insights into current practices within the university (Tang & Davis, 1995; Nielsen, 2000). The study was approved by the institutional ethics committee. The study examined the method of delivery used in lectures involving use of the penTPC as the primary classroom presentation device for handwritten material. Video analysis of lectures was conducted with specific focus on the use of gesture and annotation in the classroom situation. The features of the penTPC class session were identified and compared with those of the conventional board classroom, focussing on the co-occurring multimodal elements that were employed by the lecturers. The activities observed in lecture session are first described briefly for each lecture session (Cases), and then analysed collectively, in relation to key elements of the board classroom approach (Fox & Artemeva, 2011, p. 80). The opinions of lecturers involved in the use of the penTPC were also surveyed in a questionnaire as part of a wider study (Maclaren et al., in press). The perceptions of the particular lecturers in this study of the affordances of the penTPC and adaptations required for their use, as recorded in questionnaire comments, are also referenced here and related to their observed practices. These give an indication of the extent to which lecturers have consciously adapted their approach to the affordances of the different environment. While the analysis and conclusions are drawn from, and relate to, the particular context of this study, there are many aspects that will be familiar to those working in similar environments and that may be used to inform developments in the introduction and use of penTPC technology. 3. Observations 3.1 Case 1: undergraduate (yr 1) engineering mathematics lecture This lecturer was in the first semester of using a penTPC, teaching a first year undergraduate engineering mathematics class. The environment was a medium size tiered lecture theatre, seating 172 (Fig. 1a). Approximately 80 students were present. There was a single data projection screen (DPSc) located at the front, centre of the room. Fig. 1. View largeDownload slide (a) Lecture theatre seating 172; (b) lecturer raises head to gesture. Fig. 1. View largeDownload slide (a) Lecture theatre seating 172; (b) lecturer raises head to gesture. The lecturer conducted the session using the penTPC as the only presentation technology, with all material handwritten live on the penTPC using Microsoft OneNote software. The lecturer remained standing at the lectern for the duration of the lecture, almost exclusively facing the class, using a lectern-mounted microphone. Lecturer activity alternated between hand writing on the penTPC (developing diagrams in conjunction with equations) with commentary, and looking up to face and talk to the class and interact with students (Fig. 1b). The lecturer asked questions of the students and also responded to student initiated questions, looking up as necessary. The majority of the session involved the exposition of theoretical concepts and example problems using both diagrammatic and symbolic representations written on the penTPC (and projected). The lecturer did not use any diagrams or figures other than those hand drawn, or any other software, in the observed session, and used a single coloured pen throughout. In terms of the content used and developed, there was nothing that was significantly different from what might have been developed on a whiteboard. On occasions the lecturer scrolled back to previous material, adding attentional marks (such as circling) to indicate relevant material (rather than deictic gestures towards DPSc content). While talking to the class, the lecturer used hand gestures in a ‘traditional’ way i.e. gestures were made with the hands involving beat dimensions for emphasis, and iconic dimensions (e.g. moving one hand to and fro in association with saying ‘move object from one place to another place’). In survey comments, the lecturer was very positive about the use of the penTPC, valuing the ability to remain at a desk, with a microphone, and to ‘not need to jump in front of the board’. The lecturer also valued the fact that the lecture notes were clear and visible to all students especially given the large class (and room) size. The lecturer also appreciated being always orientated facing the students and expressed the view that in using the penTPC ‘the teaching becomes more effective as students feel free and can participate in discussions’. The lecturer valued the fact that notes did not need to be erased to make space for new notes, and that notes could be made available to students after the class. 3.2 Case 2: undergraduate (yr 1) engineering mathematics lecture (2) This lecturer was in the second semester of using a penTPC, and also teaching a first year undergraduate engineering mathematics class. The environment was a medium size tiered lecture theatre, seating 236 (Fig. 2a). Approximately 90 students were present. There was a large DPSc extending to approximately 3 metres above floor, with a strip of white board along the front of room, partially obscured when the DPSc was lowered. Fig. 2. View largeDownload slide (a) Lecture theatre seating 236; (b) lecturer writes on penTPC at lectern; (c) lecturer gestures towards point of interest on DPSc. Fig. 2. View largeDownload slide (a) Lecture theatre seating 236; (b) lecturer writes on penTPC at lectern; (c) lecturer gestures towards point of interest on DPSc. The lecturer conducted the observed session using the penTPC and single DPSc as the only presentation technology. The lecturer was confident and fluent in the use of the penTPC. Some material was handwritten live on the penTPC (within Microsoft OneNote), and other material was presented in the form of prepared static slides. The lecturer alternated between working at the lectern using the penTPC (Fig. 2b) and walking away from the lectern, in front of the screen, to talk to the class. While writing on the penTPC at the lectern the lecturer followed the pattern of dynamically developing both diagrammatic and symbolic representations in conjunction with dialogue, as a mathematical narrative. Rather than using a deictic gesture or mark to indicate location, location was associated with the position of appearance of specific content. For example, in referring to a region on a graph as ‘over here’, the location of ‘here’ was only made explicit for students by the appearance of the label being written at that place. Material was scrolled vertically so that new information was generally being added in the blank space immediately below previously written material. There was occasional use of horizontal lines, short diagonal lines or boxes; rather than being attentional marks for emphasis, these marks served as separators, to distinguish one block of content from another (for example, between sample problems). Most of the material was written in the penTPC using black ink, with occasional use of a different colour to distinguish key points of interest (e.g. blue lines to highlight turning points on a hand drawn line graph). The lecturer displayed slides using PDF reader software and did not annotate this material using the penTPC. When projecting slides, the lecturer would frequently move in front of the screen to discuss the material being presented, often pointing directly at material displayed on the screen while commenting (e.g. ‘going up and down’) (Fig. 2c). In this lecture theatre only the bottom half of the DPSc is within reach, so while items of interest on the lower section of the screen were pointed at directly, with a hand placed contiguous to the screen material, items at the top of the screen were referenced by positional statements e.g. ‘the first one’, referring to a slide displaying three theorems. Other than these pointing (deictic) gestures, the lecturer was quite restrained in the use of hand gestures while standing. In the lecturer survey, the lecturer rated the whiteboard as a preferred technology, but stated that the penTPC was almost always used for teaching and that it ‘works as well as a whiteboard’. In describing advantages of the penTPC, it was described as ‘no better than a whiteboard, except that the notes are kept online and can be exported to PDF and other formats’. Comment was made that white board space in many rooms was limited and ‘ineffective’, and was obstructed once the DPSc was lowered, and these issues being a driver for using the penTPC. As in Case 1, the handwritten material observed being developed using the penTPC was not significantly different from what might have been developed on a board. 3.3 Case 3: undergraduate (yr 3) engineering design lecture This lecturer was in second semester of using a penTPC, teaching engineering design to third year undergraduate students. The teaching space was a large flat-floor classroom seating up to 60 (with 30 students present), with individual table and chairs, set in rows. The room contained a lecturer desk at the front, side of the room, and a whiteboard that was partially obscured when the small DPSc was lowered (Fig. 3a). Fig. 3. View largeDownload slide (a) Flat-floor classroom seating 60; (b) lecturer writes on penTPC sitting at desk; (c) lecturer references table of images within OneNote page; (d) lecturer stands to point to specific location on projected image; (e) while seated lecturer annotates graph on penTPC and also points towards DPSc. Fig. 3. View largeDownload slide (a) Flat-floor classroom seating 60; (b) lecturer writes on penTPC sitting at desk; (c) lecturer references table of images within OneNote page; (d) lecturer stands to point to specific location on projected image; (e) while seated lecturer annotates graph on penTPC and also points towards DPSc. The lecturer conducted the session solely with the penTPC using Microsoft OneNote, mostly sitting at a desk at the front, to the side of the DPSc facing the students (Fig. 3b). The lecturer moved naturally between sitting (while writing, annotating, and talking) and standing and moving about the room (using a range of body movements and gestures). The lecturer would occasionally indicate towards the screen (deictic gesture) while sitting, but would often get up and move to the screen to point to a specific point of interest on the screen (Fig. 3e), or to talk to the students. The positioning and small size of the DPSc allowed pointing with a hand directly proximate to items of interest on the screen. The lecturer (as in the previous cases) developed a mathematical narrative involving dynamic development of diagrammatic and symbolic material. However, this lecturer had incorporated prepared digital material on the OneNote pages, including engineering tables, standard formulae and diagrams, and referenced and added to this material during the session (Fig. 3c). For example, while referencing tables, the lecturer used attentional marks on the penTPC to indicate the relevant position in the table (drawing lines to indicate appropriate rows and circling relevant values) and on other occasions standing and pointing to the position on the DPSc (Fig. 3d). Underlining (often doubled) was used to emphasize important results. The lecturer used the zooming and scrolling capabilities of the hardware/software to focus on salient aspects of the content (such as zooming in on relevant sections of a table). The lecturer made the digital notes available to the students after the class, and in commencing the observed session, displayed and reviewed notes developed in the previous session. Thus in integrating other digital material, the lecturer was using capabilities of the penTPC that are not available in a board-based environment. In feedback, the lecturer commented that facing the students while drawing and talking was considered to be an important element of classroom presentation that was facilitated by the penTPC. The lecturer also valued the capability with the penTPC to project material at a large scale (via the DPSc), to integrate the handwritten OneNote material with other media, and to produce a record of the material as developed in class. 3.4 Case 4: undergraduate (yr 3) engineering statics This lecturer was in the third semester of using a penTPC, teaching engineering mechanics to first year undergraduate students in a large tiered lecture theatre seating 360 (Fig. 4a). Approximately 200 students were present. The lectern was at the front, centre of theatre. There were dual Data Projector Screens (and a larger single screen option). There were no wall-mounted whiteboards, but one small portable whiteboard was located to the side and beneath one DPSc. Fig. 4. View largeDownload slide (a) Large lecture theatre seating 360; (b) lecturer stands to write, holding penTPC. Fig. 4. View largeDownload slide (a) Large lecture theatre seating 360; (b) lecturer stands to write, holding penTPC. The lecturer conducted the session using the dual data projectors. One DPSc displayed PowerPoint slides (driven from the lectern PC) containing theory or textbook example questions, which were not annotated during the session. The other projector was used to display handwritten material created live on the penTPC using Microsoft OneNote. The lecturer would commonly walk to the side of the lectern while elaborating on slide material, and used hand/arm gestures freely. The lecturer used the penTPC to develop solutions to problems mostly standing with the penTPC resting on the lectern, and occasionally picking the penTPC up in one hand and moving to the side of the lectern (as enabled by the 2 m cable) to write with the other hand (Fig. 4b). Once again, the session proceeded as a mathematical narrative; the lecturer hand-wrote equations integrated with diagrams and verbal commentary. The lecturer would frequently look up to interact with the class, asking questions and clarifying points, and gesturing (beat and iconic gestures) with a hand (or hands, if not holding the penTPC). The lecturer used attentional marks (e.g. circling with highlighter ink, drawing a box around a result) to emphasise particular items, and linked associated objects with lines. The lecturer used a range of colours for emphasis and to distinguish different features. On occasion, colour was chosen with deliberate association to content (for example, switching to a blue pen and drawing wavy lines to indicate water in a diagram). Pen strokes were clear and of broad width, appropriate to the large venue. The lecturer scrolled the (OneNote) pages to add additional functional material, and on occasion scrolled back to refer to previously developed material. The features of penTPC use that lecturer valued were: facing the class; markers that did not fade; the capability to use annotations and remove them; the capability to bring in images. The lecturer commented that the penTPC served as ‘a whiteboard on steroids’. The lack of whiteboards in rooms was stated as being a key influence on adopting the penTPC approach. 3.5 Case 5: undergraduate (yr 2) mathematics This lecturer was in the first semester of using a penTPC, and teaching second year undergraduate engineering mathematics in a medium sized tiered lecture theatre (seating 236), with approximately 100 students present. A lectern was situated at the front, side of the theatre. There was a single DPSc and a strip of fixed whiteboard that was partially obscured when the screen was lowered. The lecturer used PowerPoint, running off the penTPC, projected on the single DPSc. PowerPoint presentation slides had been prepared in two forms, with some containing extensive content and others allowing additional blank space for annotation and development of additional material. When developing material, the lecturer used the penTPC on the (standing-height lectern), and spoke facing the students while writing. Problem solutions were developed as a sequential mathematical narrative, in an equivalent manner to that of a board environment. On occasion the lecturer would look up to talk, remaining at the lectern, and gesture (Fig. 5a). At times, previously written material was referred back to, and identified with attentional marks (circling). Fig. 5. View largeDownload slide (a) Lecturer lifts head to talk and gesture while at lectern; (b) new material is added and attention is drawn to key material with annotations such as circling; (c) lecturer moves to side to talk about material, and gesture. Fig. 5. View largeDownload slide (a) Lecturer lifts head to talk and gesture while at lectern; (b) new material is added and attention is drawn to key material with annotations such as circling; (c) lecturer moves to side to talk about material, and gesture. Some PowerPoint presentation slides had extensive content, often including diagrams, some of which were complex representations of 3D figures that are hard to draw live by hand. These slides were annotated mainly using attentional marks (circling, underlines, etc.) on the pre-written material to indicate points of particular focus, concurrent with speech (e.g. ‘this plane’ while circling the appropriate surface-plane on the figure on the penTPC) (Fig. 5b). Some annotations added functional information (e.g. an additional line as a functional component of a diagram). When expanding on a topic or asking questions the lecturer would often move into the space between the lectern and screen to address the students while facing them, occasionally glancing and/or gesturing towards the screen (Fig. 5c). Feedback from this lecturer included seeing the penTPC environment as having value in allowing the maintenance of a stance facing students while writing, and in facilitating the use of prepared slide material in conjunction with writing activity without the need to switch from whiteboard to DPSc. 3.6 Case 6: undergraduate (yr 3) electrical engineering This lecturer was in their fifth semester of using a penTPC, teaching a third year undergraduate paper in control engineering. The room used for the session was a medium size flat-floor classroom (seating 50) with a lecturer desk at the front to the side of the room. There was a single DPSc which partially obscured a whiteboard when lowered. The lecturer conducted the session using the penTPC and Microsoft OneNote to display prepared material and add live handwritten material. The lecturer set up the penTPC on a desk at the front (to the side of the normal lecturer desk) and sat facing the class while writing/annotating in OneNote and displaying output on the DPSc. The lecturer had earlier inserted PowerPoint slides (originally developed in previous years) as printouts in a OneNote page, and this material was further developed in the session (Fig. 6a). Slides included photographs and software-created graphics (Fig. 6c). The lecturer used the touch capability of the penTPC to scroll through the material, and zoom in and out on elements of interest. Fig. 6. View largeDownload slide (a) PowerPoint slide printed into OneNote and then annotated live in the class session; (b) lecturer gestures in circular motion (‘cows walk around crater in contours’); (c) lecturer annotates software output using penTPC. Fig. 6. View largeDownload slide (a) PowerPoint slide printed into OneNote and then annotated live in the class session; (b) lecturer gestures in circular motion (‘cows walk around crater in contours’); (c) lecturer annotates software output using penTPC. The lecturer took advantage of the capability of OneNote to provide an unconstrained writing area in scrolling to the right of the slide material to expand on the material presented in the slides. Different coloured inks were used with specific meaning assigned to different colours in some situations. This use of colour in the functional writing added emphasis that would not otherwise be as apparent. The lecturer freely interspersed writing/annotating with looking up and gesturing (e.g. drawing circular contour lines on an image, and then making iconic circular hand gestures) (Fig. 6b). Labels were added and linked to relevant points on a diagram with arrows. Thus many annotations served to add functional information, but also as attentional marks to add emphasis within the sequence of the narrative. From time to time the lecturer also would stand and move to the centre-front of the class to discuss material, freely using a range of gestures. In questionnaire response and other comments, the lecturer had expressed valuing: sitting, facing the students; ability to use a range of representational forms and colour; using a handwritten approach allowed dynamic development of material, in ways that was less formal (than text), and maintained a student-friendly pace; capability to make completed notes available to students as both a static and dynamic recording. 3.7 Case 7: undergraduate (yr 3) statistics While this lecturer was no longer using a penTPC, having personally decided there were too many disadvantages with the technology, the session is included to provide a comparative analysis. The session was a third year undergraduate statistics class in a ‘flexible blended learning space’ (flat-floor classroom) seating 36, with approximately 25 students present. The room has a lectern at the left-front, a single DPSc connected to a lectern desktop PC, and two portable whiteboards placed to the right-hand side of the DPSc (Fig. 7a). Room furniture consists of trapezoidal tables, arranged in groups seating up to six, with chairs on castors. Fig. 7. View largeDownload slide (a) Room with DPSc and portable whiteboard; (b) lecturer points to items of interest on DPSc; (c) lecturer writes, facing the board. Fig. 7. View largeDownload slide (a) Room with DPSc and portable whiteboard; (b) lecturer points to items of interest on DPSc; (c) lecturer writes, facing the board. The lecturer used the lectern PC to project use of statistical software (i.e. showing command inputs and text and graphical outputs) onto the DPSc. The two portable whiteboards were used to develop theory and examples. The lecturer was very active, moving between the lectern PC (to enter software commands), the DPSc (to point to output) and the whiteboards to develop theory and examples. The board writing was mostly with a single-colour of markers (with occasional use of a second colour, for emphasis), the lecturer writing with back to the class, talking towards the board (Fig. 7c). The lecturer’s body frequently obscured material as it was being added. The lecturer frequently pointed to features of specific interest on hand-drawn diagrams, or used annotations to highlight existing features, or add new features, along with verbal commentary. There was ready transition between pointing to existing diagrammatic features, and switching to using the in-hand marker to add new features or emphasis. When referring to material on the DPSc the lecturer used whole arm movements to point or touch contiguous to the output of interest (Fig. 7b), but written annotations were not applied (as this capability was not available). On occasion some screen material was elaborated on by reproducing material on the board. Again, gaze (and voice) were mostly directed to the board/screen, and at times the images on the DPSc were partially obscured and or shadowed by the lecturer’s body. The reasons given by the lecturer for abandoning use of the penTPC include: technical-reliability issues; regarding the technology as forming a barrier between lecturer and students, with the need to ‘have to look down’ to use it; that the ‘text moves around, unlike writing on a board which stays put until erased’. 4. Analysis and comment Characteristic elements of a mathematical lesson, as identified by Fox & Artemeva (2011) are shown in Table 1, with Column 1 showing the forms characteristic of the board classroom, and column 2 the equivalent (and additional) forms in the penTPC environment. The alphabetic code assigned to the rows in Table 1 (e.g. [A]) are used to identify these elements in the following discussion. The basic process of developing a mathematical narrative, involving writing mathematical symbols, texts, graphs and diagrams [A] while verbalizing [B], was observed to take similar form in both board and penTPC environments. In both environments lecturers would verbalize while they were writing [C], and stop writing to talk about what they had written [D]. However, there were differences in how they moved in space [E], and in the use of gestures [F]. While in the process of writing [A] lecturers using the penTPC were more static with regard to whole body positioning. Lecturers did not move across the room as a board user would as they wrote progressively across a board, but were ‘tied to the tablet’ (Bonnington et al., 2007, p. 8). Nevertheless, the sessions examined here showed that lecturers could (and would, according to preference) move freely, stepping away from the device, when talking about what had been written [D]. In doing so the lecturer would interact with the displayed material as they would in with board-based material or slides displayed on a DPSc, potentially using a full range of gestures [F], but with limitations, as discussed below. In the penTPC environment, whole body movement was not necessary to transition between writing and other interactions; rather than stopping writing and turning (as with a board technology), the lecturers just stopped writing and looked up. Thus the activities of talking [D], some forms of gesturing [F], consulting of notes [G], pausing for reflection [H], checking student understanding [I] and asking questions [J] were possible without needing to turn or move away from the device (although that option was available). In board-based teaching, it has been noted that 75% of the lecturer’s time might be spent with body facing the board (Fox & Artemeva, 2011, pp. 95–96). Most lecturers in this study saw it as a key advantage of the penTPC mode for their body to be facing the students (rather than the board) while writing and talking [A][C], allowing them to simply glance up to talk [D] and interact with students [I][J]. However, in Case 7 here the lecturer regarded the penTPC as a barrier in the relationship with students, seeing ‘looking down to write’, as an activity that disrupted the normal flow of a lesson. As observed, the lecturer was very active in class, in both body movement and gesture. This style was also recorded by Bonnington et al. (2007, p. 8), who noted a lecturer’s observation that his ‘distinctive dramatic style involving much arm-waving and walking around to emphasise points’ was severely constrained in the penTPC environment. It may be that for some lecturers, the penTPC requires too a radical a change to their accustomed movements for it to be comfortably adopted. A critical difference between board and penTPC environments is in the capability for use of deictic (pointing) gestures [F], while talking about what had been [D], or was in the process of being [A][C], written. While writing at a board, a lecturer can simply point to an object for emphasis (as well as circle or underline), but when working at a penTPC the lecturer needs either to use an attentional mark or to move away from the device to point to the object on the projected image [E][F]. Furthermore, depending on the size and positioning of the DPSc, lecturers were not always able to position a hand directly adjacent to the indicated object, so that pointing, or deictic gestures were less precise, or constrained, compared to a board environment. Thus in the penTPC environment, the use of annotation, in the form of attentional marks, becomes an essential mechanism for indicating relationships, signalling references, and highlighting key issues [F]. For the lecturer working on the Tablet screen, the focal point of the writing operation is directly that of the tip of the stylus (as directed by the hand/arm), with touch actions also operating directly on the on-screen representation of objects, as if in direct physical contact. Thus while the lecturer maintains, in their sight, the linkage between their physical and mental focus, the student only explicitly views the results of lecturer actions (on the Tablet PC screen) in changes as projected on the DPSc, without the physical cues of arm/hand pointing to the location of activity. The student is thus very reliant on visual cues on the screen to focus attention on the point of interest, and the lecturer needs to give conscious attention to the creation of these cues. While developing symbolic content following a standard left-to-right, top-to-bottom sequence, the position of new material was generally able to be readily anticipated, and was clearly apparent. However, for material not entered in such a sequence (for example, when a lecturer added a label to an existing chart, or new point on an existing graph) it was important that this added material had sufficient prominence, through use of colour and/or size and dynamic development, to ensure the location of the point (the ‘here’ in the verbal commentary) was immediately obvious. As described by Bunt et al. (2009, pp. 229, 230) (as referenced earlier) the development of a mathematical narrative involves the simultaneous development of dynamic objects with writing occurring in sequence with verbal commentary. As observed here (in Case 1), commentary stating ‘from point a to point b’ was accompanied with the marking, and labelling, of these points on a previously drawn graph axis. Subsequently the distance between these points was also identified, with a double-headed arrow joining the points and new label added. Thus, the timely addition of these points provides what we will term sequential emphasis; however, this writing also provides functional information that remains relevant after the commentary moves on, and even in a static form as written notes. There were only a few instances of use of purely attentional marks: at times underlines were used when referring back to previously written material, to emphasis words or terms that were again significant in the procedural flow; on one occasion a circle was used to highlight a symbol in an equation written earlier (requiring scrolling the screen to reveal, circle, and scrolling back to continue the narrative). In addition, some lines/points on a diagram were drawn over with multiple strokes, to emphasize their location as relevant to the ongoing narrative. However, in most cases, it was the direct initial appearance of new content that gave emphasis to its sequence in the narrative. As noted the penTPC environment provides capabilities to include a range of different graphic material in digital form directly into the writing and viewing space [K]. Not all lecturers made use of this affordance, with some simply using the penTPC as a basic writing slate, while others inserted additional material into presentations, closely integrating it into the mathematical narrative. For example, in Case 6, rather than rely purely on iconic gestures with the hands to suggest a volcanic crater, with circular motions to suggest contour lines within it (Fig. 6b), the lecturer included a photograph of a crater, and drew contour lines directly on the image (Fig. 6a). Graphic output from software (showing mathematic representations of craters) was also annotated, with both attentional and functional purpose (Fig. 6c). Similarly, in Case 5, the lecturer displayed a full table of values, and annotated (circled or highlighted) relevant values for the ongoing calculations. Thus these annotations had both short-term attentional purpose, as well as long-term functional purpose in providing a permanent record of the origin of values that were used later in procedural developments. In this study, annotations such as arrows were observed being used to dynamically connect equivalent items in the different representational forms (images, tables, diagrams, symbolic forms), and in enabling different representational forms to be integrated (such as dynamically adding symbolic equations to diagrams). While some use of annotation in way this may be observed in the board environment, the extended range of representational forms [K][L][M] available in the digital penTPC environment provides additional opportunities for annotation, and unlike transient gestures and speech, was observed here serving a contextual, attentional purpose, while adding persistent functional information. That annotation may be used for emphasis (attentional marks) and to communicate functional content has been noted previously (Ambikairajah et al., 2006; Alibali et al., 2014). Thus, rather than regarding annotations as being of distinct types, it may be appropriate to regard them also as having a range of dimensions, following McNeill’s approach to types of gesture (McNeill, 2005, p. 38). An initial suggestion for categorizing dimensional forms of annotations is shown as Table 2. Table 2. Forms of annotation, distinguished by suggested dimensions/purpose. Note that anotations may have more than one purpose. As with gestures, there may be an associated progression of formality, from simple marks (equivalent to gesticulations), through conventional signs (emblems), iconic or representational content (pantomime), through to functional content structured according to formal rules (sign language) Annotation dimension  Description/purpose  Examples  Attentional focus (simple mark)  Directions attention to point in written narrative  Dot or indistinct mark  Conceptual emphasis (emblem)  Focus or re-focus on key objects (equations, terms)  Box, highlight  Sequential emphasis (emblem)  Call attentions to an existing object at critically relevant time in the narrative  Circled item ‘this’;  Sequential emphasis (content)  Addition of content object at critically relevant time in the narrative  Addition of written content, point ‘a’, or tangent on curve  Closure/completion (emblem)  Indicates completion of a task, or separates coherent sections  Double diagonal lines//horizontal line -___________  Linking (emblem)  Connects different objects, or the same object in different representations  Arrows, lines connecting value, symbol in an expression or calculation with value in a table  Associative or cohesive (emblem)  Collects objects having a common property together  Enclosing objects within line; use of common colour for ‘like’ items  Iconic representational (pantomime)  Representational drawing or graphic  Wheeled cart drawn in mechanics problem  Iconic association–metaphoric form (pantomime)  Format of object reflects a property of object  Spring drawn as zigzag lines Use of blue colour, wavy lines to represent water on a diagram  Structured functional content  Diagram, graph, algebraic working, text (or part of)  Equation, mathematical working  Annotation dimension  Description/purpose  Examples  Attentional focus (simple mark)  Directions attention to point in written narrative  Dot or indistinct mark  Conceptual emphasis (emblem)  Focus or re-focus on key objects (equations, terms)  Box, highlight  Sequential emphasis (emblem)  Call attentions to an existing object at critically relevant time in the narrative  Circled item ‘this’;  Sequential emphasis (content)  Addition of content object at critically relevant time in the narrative  Addition of written content, point ‘a’, or tangent on curve  Closure/completion (emblem)  Indicates completion of a task, or separates coherent sections  Double diagonal lines//horizontal line -___________  Linking (emblem)  Connects different objects, or the same object in different representations  Arrows, lines connecting value, symbol in an expression or calculation with value in a table  Associative or cohesive (emblem)  Collects objects having a common property together  Enclosing objects within line; use of common colour for ‘like’ items  Iconic representational (pantomime)  Representational drawing or graphic  Wheeled cart drawn in mechanics problem  Iconic association–metaphoric form (pantomime)  Format of object reflects a property of object  Spring drawn as zigzag lines Use of blue colour, wavy lines to represent water on a diagram  Structured functional content  Diagram, graph, algebraic working, text (or part of)  Equation, mathematical working  Classroom sessions in mathematical disciplines have traditionally been structured around having large areas of board space remaining visible and displaying unchanged content during at least part of a session (but erased at some point). The use of penTPC software means all content is retained (even after a session), but a smaller proportion may remain in the field of view displayed by the DPSc at any one time, and other material requires scrolling or zooming to access. Zooming and panning was used as an additional mechanism for providing emphasis, by centring and enlarging critical information on the DPSc. From the perspective of the lecturer, this mechanism has some characteristics of gesture, in that a hand movement initiates a communication, with direct results of that action manifest in the display. However, the hand action is not directly visible to the students, who only observe the resulting change in display. As such, these (and zooming/panning) might arguably be termed as technology-mediated deictic gestures (Table 3). Table 3. Technology mediated gestures and transient annotations Technology mediated gestures (deictic)   Involve hand actions by the lecturer, with features of a deictic gesture from their perspective, but with only the effects of those actions viewable to students on the DPSc (and not the hand movements themselves).         Lecturer action      Effect on DPSc            Pinch touch gesture          Screen expands to give overview          Stretch touch gesture          Focus directed to detail of content          Finger scroll          Focus directed to previous content    Transient annotations (deictic)  Mouse/pointer actions have features of an annotation, and involve use of a tool, but are transient and do not appear in a static digital record of a session.     Lecturer action   Effect on DPSc            Mouse or pen-hovered over (points to) specific content (or pen-as-laser setting used)—directs attention          Transient appearance/movement of mouse-pointer position          Pen-as-pointer setting to annotate—directs attention          Annotation appears, but disappears after a few seconds)  Technology mediated gestures (deictic)   Involve hand actions by the lecturer, with features of a deictic gesture from their perspective, but with only the effects of those actions viewable to students on the DPSc (and not the hand movements themselves).         Lecturer action      Effect on DPSc            Pinch touch gesture          Screen expands to give overview          Stretch touch gesture          Focus directed to detail of content          Finger scroll          Focus directed to previous content    Transient annotations (deictic)  Mouse/pointer actions have features of an annotation, and involve use of a tool, but are transient and do not appear in a static digital record of a session.     Lecturer action   Effect on DPSc            Mouse or pen-hovered over (points to) specific content (or pen-as-laser setting used)—directs attention          Transient appearance/movement of mouse-pointer position          Pen-as-pointer setting to annotate—directs attention          Annotation appears, but disappears after a few seconds)  When using a mouse, the position of attention is indicated on-screen by a mouse pointer icon. This pointer icon can also be activated in the penTPC environment by hovering the pen over the screen. Thus rather than creating persistent attentional marks, focus can be directed to particular locations by the lecturer, by their positioning, or pointing, with a pen or mouse. While the position of a standard mouse pointer may not always be obvious on a DPSC, there are techniques and settings to increase its visibility (for example, increasing the cursor/pointer size, using the CTRL key to reveal mouse/pen position, or using the laser pointer tool in PowerPoint). In addition, the pen-as-pointer tool in OneNote can be used to make a transient annotation (a mark that disappears after a short interval). These actions have features of a technology mediated gesture, but may better classified as transient annotations, in the trace register (Table 3). While the use of specialized pointer icons was not observed in this study, it is an issue for further investigation. The affordance of the penTPC environment for ready use of colour in annotation has been previously noted (Fister & McCarthy, 2008; Wilson & Maclaren, 2013), although it has been observed elsewhere that it is not consistently used in a meaningful way (Anderson et al., 2004, p. 573). While most lecturers here tended to use a single colour, or occasionally a second colour for emphasis, some lecturers here used a range of colours systematically, with concurrent attentional and functional purposes. For example, systematic use of colour in circling, highlighting or writing objects gave not just transient attention to particular objects, but provided ongoing focus on the common functional properties of the like-coloured objects. Wang & Chu (2013) proposed that even non-representational beat gestures help convey meaning. While annotations may take on some of the role of iconic and deictic gestures, the beat gesture is not so readily transferred into a written form (with multiple underline strokes perhaps being one option). However, the lecturers did not appear to be constrained in the use of hand created beat gestures, simply looking up from the penTPC and moving hands (sometimes while holding pen, or glasses) in beat gestures to provide emphasis when talking about material. 5. Conclusions and further developments It had been suggested that the use of the penTPC may affect communication in the classroom by constraining the range of movements and gestures used by the lecturer. However, the examination of the use of penTPCs here has revealed that, while some changes were observed in the sequencing and features of movements, a wide range of communication elements used by the board-based lecturer were still exhibited, with use of gestures remaining as a core component in lecturer presentations. Thus the core elements of ‘chalk talk’ as described by Fox & Artemeva (2011) were observed in use, albeit in modified form, by lecturers using the penTPC (Table 1, Rows [A]–[J]). It is in the reduced use of deictic gesture and expanded use of annotation that the most significant changes accompanying the introduction of the penTPC environment were manifest. It is apparent that annotation may provide a powerful way of conveying meaning that extends beyond that available through simple gesture. While this study has been focused on the use of gesture and annotation as used by the lecturer in the classroom, the penTPC also readily enables classroom generated notes to later be made available to students as a permanent record of the session. Most lecturers here regarded this capability to provide a permanent digital record of all material that could be scrolled and accessed within and outside class sessions as an advantage of the penTPC approach, and made active use of the capability. The persistent nature of annotations (as opposed to the non-recorded ephemeral nature of gestures) can have a significant impact (both positive and negative) on the future use of recorded notes. The nature of the annotations (and gestures) made in class may also impact on any notes that students make themselves. These aspects are outside the scope of this article, and are discussed elsewhere (In press, 2017). While the adoption of the technology has not been without technical issues, the majority of lecturers who have piloted the use of the penTPC determined that the benefits outweighed the disadvantages. In some cases, while there was initial resistance fostered by a view that the technology was only required because of the absence of suitable whiteboards in classrooms, lecturers became appreciative of the potential benefits of the penTPC approach. Some lecturers may find the restrictions on their established natural movement and use of gesture a challenge. The successful use of a penTPC approach requires adoption of different techniques, and a willingness to do so depends on how the individual lecturer perceives and weighs the relative advantages and disadvantages. Unlike the established form of chalk talk as described by Artemeva & Fox (2011; Fox & Artemeva, 2011), the precise form of delivery of the mathematical narrative in the penTPC environment is still evolving. It is apparent that the augmented capability for annotation in conjunction with other digital representations in the penTPC environment can provide opportunities to enhance teaching, particularly of STEM based discipline subjects. However, this study observed a wide range in current levels of utilisation of these affordances. The increasing adoption of the penTPC in the study university gives importance to the establishment of guidelines for their effective use, which will be a focus of ongoing work. Peter Maclaren is a Principal Lecturer and Academic Advisor in the Centre for Learning and Teaching at the Auckland University of Technology (AUT). He has a background in engineering (B.E., University of Auckland) and regional planning. He taught applied mathematics at AUT for some 15 years before becoming involved with the development of online learning environments. He has a Masters in Educational Technology completed online through the University of Southern Queensland. His current professional and research interest (and PhD topic) is in developing pedagogies in STEM disciplines that take advantage of the particular affordances of pen-enabled Tablet PC technology. David Wilson is an Associate Professor in Electrical Engineering at AUT. Prior to joining AUT he was on the faculty at Karlstad University in Sweden following a position at the Swiss Federal Institute of Technology (ETH) in Zürich, Switzerland. His main research interests are modelling, optimisation and control of industrial processes, development of embedded controllers, and effective graduate teaching strategies for mathematically intensive engineering subjects. Currently he is a director of the research-based Industrial Information and Control Centre (I2C2) where he manages multi-faceted research projects for international and national clients. He is a director of Inverse Problems Ltd which is a startup company developing and marketing small high-performance embedded controllers. This company recently won the inaugural AUT Enterprises Innovation Challenge for 2012. Sergiy Klymchuk is an Associate Professor of mathematics in the School of Engineering, Computer and Mathematical Sciences, AUT, New Zealand. He has been teaching university mathematics in different countries since 1980. His PhD (1988) was in differential equations and recent research interests are in mathematics education. He is an author of more than 200 publications including the Counterexamples in Calculus book that received an Outstanding Academic Title Award from the Choice magazine of the American Library Association in 2010, Paradoxes and Sophisms in Calculus book that made the cover of the 2014 Publications Catalogue of the Mathematical Association of America, and Money Puzzles book on popular mathematics that has been published in nine countries. References Alibali M. W, Nathan M. J, Wolfgram M. S, Church R. B, Jacobs S. A, Johnson Martinez C, Knuth E. J. ( 2014) How teachers link ideas in mathematics instruction using speech and gesture: a corpus analysis. Cogn. Instr , 32, 65– 100. Google Scholar CrossRef Search ADS   Ambikairajah E, Epps J, Sheng M, Celler B. ( 2007) Signal processing education using the TabletPC and electronic whiteboard. IEEE Signal Processing Magazine 24.1 (2007): 130–33. Anderson R, Anderson R, McDowell L. ( 2005) Best Practices for Lecturing with Digital Ink . Washington, DC: University of Washington. Anderson R, Hoyer C, Wolfman S. A, Anderson R. ( 2004) A study of digital ink in lecture presentation. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. pp. 567–574. Artemeva N, Fox J. ( 2011) The Writings on the Board: The Global and the Local in Teaching Undergraduate Mathematics Through Chalk Talk. Writ. Commun. doi:10.1177/0741088311419630 Arzarello F. ( 2006) Semiosis as a multimodal process. RELIME Rev. Latinoam. Investig. En Matemática Educ., 9, 267– 300. Arzarello F, Paola D, Robutti O, Sabena C. ( 2008) Gestures as semiotic resources in the mathematics classroom. Educ. Stud. Math , 70, 97– 109. Google Scholar CrossRef Search ADS   Bates A. W. ( 2015). Teaching in a Digital Age: Guidelines for Designing Teaching and Learning . Vanvouver BC: Tony Bates Associates Ltd. Bonnington P, Oates G, Parnell S, Paterson J, Stratton W. ( 2007) A report on the use of tablet technology and screen recording software in tertiary mathematics courses. Vis. Change New Century Proc. Calafate Delta , 7, 19– 32. Bosch M, Chevallard Y. ( 1999) La sensibilité de l’activité mathématique aux ostensifs: objet d’étude et problématique. Rech. En Didact. Mathématiques , 19, 77– 123. Bunt A, Terry M, Lank E. ( 2009) Friend or foe?: examining CAS use in mathematics research. Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, pp. 229– 238. Campbell S, Latulippe D. R. ( 2015) Towards improved learning of fluid mechanics via integration of a commercial software package into an undergraduate course. in Proceedings of the Canadian Engineering Education Association . Hamilton, Ontario: Canadian Engineering Education Association. Retrieved from http://library.queensu.ca/ojs/index.php/PCEEA/article/viewFile/5767/pdf Châtelet G. 2000. Les enjeux du mobile. Seuil, Paris (1993); English translation by Shore R., Zagha M.: Figuring Space: Philosophy, Mathematics, and Physics . Dordrecht: Kluwer Academic Publishers. De Freitas E, Sinclair N. ( 2011) Diagram, gesture, agency: theorizing embodiment in the mathematics classroom. Educ. Stud. Math ., 80, 133– 152. Google Scholar CrossRef Search ADS   Fister K. R, McCarthy M. L. ( 2008) Mathematics instruction and the tablet PC. Int. J. Math. Educ. Sci. Technol , 39, 285– 292. Google Scholar CrossRef Search ADS   Fox J, Artemeva N. ( 2011) The cinematic art of teaching university mathematics: chalk talk as embodied practice. Multimodal Commun ., 1, 83– 103. Goeser P. T, Ruiz S. ( 2015) The Development of MATLAB Functions for Effective Use and Improvement of Student Learning in a Thermodynamics Course. Presented at the ASEE Southeast Section Conference, University of Florida, Gainsville, FL: American Society for Engineering Education, Southeastern Section. Retrieved from http://www.softwareeducationsupport.com/ASEE%20SE%20Conference%20Proceedings/Conference%20Files/ASEE2015/Papers2015/141.pdf Goldin G. A. ( 2010) Perspectives on representation in mathematical learning and problem solving 2nd Edition. In L. D. English (Ed.), Handbook of International Research in Mathematics Education (pp. 176–201). New York, NY: Routledge. Greiffenhagen C. ( 2008) Video analysis of mathematical practice? Different attempts to ‘open up’ mathematics for sociological investigation. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research. Hodge B. K, Taylor R. P. ( 2002) Piping-system solutions using Mathcad. Comput. Appl. Eng. Educ , 10, 59– 78. Google Scholar CrossRef Search ADS   Maclaren P, Wilson D. I, Klymchuk S. ( in press) I see what you are doing: Student views on lecturer use of Tablet PCs in the classroom. Australasian Journal of Educational Technology. Maclaren P, Wilson D. I, Klymchuk S. ( 2017) Lecturer Use of Pen-enabled Tablet PC Technology in the STEM Classroom: Implications for Notetaking (Manuscript in preparation). McNeill D. ( 1992) Hand and Mind: What Gestures Reveal about Thought . Chicago: University of Chicago Press. McNeill D. ( 2005) Gesture and Thought . Chicago: University of Chicago Press. Google Scholar CrossRef Search ADS   McNeill D. ( 2006) Gesture and thought. The Summer Institute on Verbal and Non-Verbal Communication and the Biometrical Principle. Vietri sul Mare, Italy. Retrieved from http://mcneilllab.uchicago.edu/pdfs/dmcn_vietri_sul_mare.pdf Nielsen J. ( 2000) Why you only need to test with 5 users [WWW Document]. Nielsen Norman Group. URL https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/ (accessed 12 June 2016). Oviatt S. ( 2013) The Design of Future Educational Interfaces . New York: Routledge, Taylor & Francis Group. Radford L. ( 2008) Why do gestures matter? Sensuous cognition and the palpability of mathematical meanings. Educ. Stud. Math ., 70, 111– 126. Google Scholar CrossRef Search ADS   Roth W. M. ( 2001) Gestures: their role in teaching and learning. Rev. Educ. Res ., 71, 365– 392. Google Scholar CrossRef Search ADS   Sabena C. ( 2008). On the semiotics of gestures. Semiotics in Mathematics Education: Epistemology, History, Classroom, and Culture  ( Radford L., Chubring G., Seeger F. eds). Taipei: Sense, Rotterdam. Shulman L. ( 2005) Signature pedagogies in the professions. Daedalus , 134, 52– 59. Google Scholar CrossRef Search ADS   Singer S, Smith K. A. ( 2013) Discipline-based education research: understanding and improving learning in undergraduate science and engineering: discipline-based education research. J. Eng. Educ ., 102, 468– 471. Google Scholar CrossRef Search ADS   Tang K. C, Davis A. ( 1995) Critical factors in the determination of focus group size. Fam. Pract ., 12, 474– 475. Google Scholar CrossRef Search ADS PubMed  Thomas M. O, Hong Y. Y. ( 2013) Teacher integration of technology into Mathematics Learning. Int. J. Technol. Math Cambridge. Educ ., 20( 2), 69– 84. Vygotsky L. ( 1978) Mind in Society: The Development of Higher Psychological Processes . Cambridge, MA: Harvard University Press. Wang L, Chu M. ( 2013) The role of beat gesture and pitch accent in semantic processing: an ERP study. Neuropsychologia , 51, 2847– 2855. Google Scholar CrossRef Search ADS PubMed  Wilson D. I, Maclaren P. ( 2013) From Chalk Talk to Tablet Talk: Pedagogies for Control Engineering. Advances in Control Education, Advances in Control Education. Presented at the 10th IFAC Symposium Advances in Control Education, 2013, International Federation of Automatic Control, University of Sheffield, Sheffield, United Kingdom, pp. 144–149. Yoon C, Thomas M. O. J, Dreyfus T. ( 2011) Gestures and insight in advanced mathematical thinking. Int. J. Math. Educ. Sci. Technol ., 42, 891– 901. Google Scholar CrossRef Search ADS   © The Author 2017. Published by Oxford University Press on behalf of The Institute of Mathematics and its Applications. All rights reserved. For permissions, please email: journals.permissions@oup.com This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices) For permissions, please e-mail: journals. permissions@oup.com

Journal

Teaching Mathematics and Its Applications: International Journal of the IMAOxford University Press

Published: Mar 1, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off